Friday, 15 March 2019

Use Terraform in the Oracle Cloud with Stacks

I already described how to use Terraform with the Oracle Cloud Infrastructure (OCI) from on-premise. But with the Resource Manager Stacks, Oracle offers a smart alternative to use Terraform with OCI.

The Resource Manager can be found in the Hamburger menu on the left side.

Click on 'Create Stack' to do so.

The Terraform files are needed in a single zip file, so we need to create some. Find my examples on GitHub.
resource "oci_core_virtual_network" "VCN-Demo" {
  cidr_block     = "${var.VCN_SEDEMO_CIDR}"
  compartment_id = "${var.compartment_ocid}"
  display_name   = "VCN-Demo"
  dns_label      = "demovcn"

resource "oci_core_subnet" "SN_Bastion" {
  availability_domain = "${lookup(data.oci_identity_availability_domains.ADs.availability_domains[0],"name")}"
  cidr_block          = "${var.SN_BASTION_CIDR}"
  display_name        = "SN_Bastion"
  compartment_id      = "${var.compartment_ocid}"
  vcn_id              = "${}"
  dns_label           = "snbastion"

I like to start with a Virtual Cloud Network (VCN) and a subnet, as these require minimal parameters.
data "oci_identity_availability_domains" "ADs" {
  compartment_id = "${var.compartment_ocid}"

To avoid hard coding the ocid's of the Availbility Domains (ADs), I query the ADs from the compartment.
variable "compartment_ocid" {}
variable "region" {}

#########   CIDR  #################

variable "VCN_SEDEMO_CIDR" {
  default = ""

variable "SN_BASTION_CIDR" {
  default = ""

As with a standard Terraform script, I query the compartment_ocid and region from the environment and set two CIDR blocks for the VCN and the subnet.

provider "oci" {
  region = "${var.region}"

Now for the, this one gets rather short. As we are already logged into oci, we do not need to handle the key file or user_ocid.

Pack those four files into a zip archive and upload them to the create Stack dialog.

We cannot source the variables via shell, but we can add these to the create dialog and we are done here.

To use the newly created Stack, click on its name or from the menu choose Edit.

Here you can do the usual Terraform plan, apply and destroy actions. Click on Plan and confirm the dialog.

In the Jobs list, you will see a new Terraform Job. Click on its name to see the details. It should succeed and on the bottom you will see the Terraform output.
After that, use Terraform Actions | Apply to create the resources, which should result in a

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

So this is a smart option to use Terraform with OCI without the need of a client software installation or a seperate vm to run the scripts, which could save time and money.

Friday, 15 June 2018

Oracle Cloud: howto access the public web from private networks via Squid proxy

Building up on my howto guide 'Run a bastion host with ssh-forwarding on Oracle Cloud Infrastructure', this guide is for the opposite direction. Say you have a server in a private subnet, only accessible through the bastion host. If that host needs access to the public internet, a proxy in a public subnet of the VCN (Virtual Compute Network) could be a solution.

The situation is the same as in the former guide. Two hosts, one named bastion in a public subnet and one named webserver in a private subnet. The webserver host has no access to the internet, but needs to access some information from a public URL, so a proxy on the bastionhost needs to be set up.

  • Install and setup Squid on a host in the public subnet
  • Create an ingress rule for the public subnet to allow traffic to the Squid port
  • Create an egress rule to allow egress traffic for the private subnet
  • Open the Squid port on the Linux firewall of the Squid host
  • Set http_proxy to the Squid host:port
Step-by-step guide

This guide uses Apache Squid, but only uses it as a proxy server. So first install it via yum:

[root@bastion ~] yum install squid

Start Squid and check it's status:

[root@bastion ~] systemctl start squid
[root@bastion ~] systemctl status squid
● squid.service - Squid caching proxy
   Loaded: loaded (/usr/lib/systemd/system/squid.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-06-14 12:34:49 GMT; 10s ago
  Process: 11307 ExecStart=/usr/sbin/squid $SQUID_OPTS -f $SQUID_CONF (code=exited, status=0/SUCCESS)
  Process: 11301 ExecStartPre=/usr/libexec/squid/ (code=exited, status=0/SUCCESS)
 Main PID: 11309 (squid)
   CGroup: /system.slice/squid.service
           ├─11309 /usr/sbin/squid -f /etc/squid/squid.conf
           ├─11311 (squid-1) -f /etc/squid/squid.conf
           └─11312 (logfile-daemon) /var/log/squid/access.log

Jun 14 12:34:49 bastion systemd[1]: Starting Squid caching proxy...
Jun 14 12:34:49 bastion squid[11309]: Squid Parent: will start 1 kids
Jun 14 12:34:49 bastion squid[11309]: Squid Parent: (squid-1) process 11311...ed
Jun 14 12:34:49 bastion systemd[1]: Started Squid caching proxy.
Hint: Some lines were ellipsized, use -l to show in full.

If everything is fine, enable automatic Squid startup at boot. If you get errors, run a complete yum update and try again. That fixed my problems with startup.

[root@bastion ~] systemctl enable squid
Created symlink from /etc/systemd/system/ to /usr/lib/systemd/system/squid.service.

To test it from the system Squid is running on, set the HTTP proxy to the Squid default port, which is 3128.

[opc@bastion ~]$ export http_proxy= 
[opc@bastion ~]$ export https_proxy=

Then open a page with curl or lynx, eg.

[opc@bastion ~]$ lynx

That should return the text version of that page.

Check that the proxy has been used in the squid access log.

[root@bastion ~]# tail -f /var/log/squid/access.log
1528980628.887     45 TCP_MISS/200 3125 GET - HIER_DIRECT/ text/html
1528980661.842      5 TCP_MISS/301 314 GET - HIER_DIRECT/ -

Now that the squid proxy works and is accessible locally, create a security role on OCI to give the webserver host access to the Squid proxy on the bastionhost.

Find the public subnet in your VCN and click on the security list.

Click on 'Edit all rules' and add the network CIDR block of the webserver subnet (alternatively only the host IP address).

After clicking on 'Save Security List Rules', these should be listed in the Security List.

Also make sure, that in the private networks security list egress traffic is allowed at least for port 3128 to the private network, or like in this case allow all egress traffic. In the default security list, egress traffic is allowed. If you have created your own security list, you might need to add an egress rule.

firewall-cmd --new-zone=webapp --permanent
firewall-cmd --reload
firewall-cmd --zone=webapp --add-source= --permanent
firewall-cmd --zone=webapp --add-port=3128/tcp --permanent
firewall-cmd --reload

The last step is to open the Squid port on the public host in the Linux firewall.

[opc@webserver ~]$ curl
[opc@webserver ~]$ export http_proxy=
[opc@webserver ~]$ curl
<!doctype html><html itemscope="" itemtype="" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description">....

Now check, if everything works. Try to open any public URL, eg. with curl. That should not work. Wait for the timeout or CTRL-C. Set the proxy to the Squid server and port and try again. That should work now, if not double-check all settings.
So now, the server in the private network with no access to the public internet can access the web through the Squid proxy in a public subnet of your VPN.

Sunday, 10 June 2018

How to run Python from Notepad++

Notepad++ can easily be configured to run Python. This is practically the same, as my former 'Running Oracle Cloud Stack Manager CLI from Notepad++' howto. But there are some minor updates.
Most important, all required plugins are now available for 64Bit Notepad++. so install the Notepad++ version of choice.

Install Plugin Manager

The nppPluginManager is now on GitHub. Just follow the steps (copy PluginManager.dll to the plugins and gpup.exe to the updater directory) from the Restart Notepad++

Install NppExec

Open Plugin Manager (Plugins|Plugin Manager|Show Plugin Manager), pick the NppExec Plugin from the list of available Plugins and click 'Install'. After restart, it should be available.

4. Save a command for Python and run it

With some meaningful Python code opened, hit F6 in Notepad++. Enter the following command for running Python (adjust the path to your environment) and save it.

c:\dev\Python\Python36\python.exe "$(FULL_CURRENT_PATH)" 

Clicking OK opens a console window and runs Python with your last saved code version.

5. Re-run Python while editing

Via CTRL-F6, the last command can be re-run, don't forget to CTRL-S first to save your code.

Friday, 8 June 2018

Run a bastion host with ssh-forwarding on Oracle Cloud Infrastructure

This short howto guide will show how to setup ssh forwarding for Oracle Linux on OCI and access from Windows via PuTTY.
The idea of a bastion host is to have a single entry point into your network. The bastion host is the only one that is accessible from outside the network, there is no sensible data on it. All applications and data reside on private hosts, that are not accessible from the outside. To access the applications, ssh into the bastion host and from there ssh to the specific private host. Firewall rules will block all unwanted traffic.
There are several options to do the key handling. One obvious one would be, to install all private keys on the bastion host. But if there would be a successful attack onto the bastion host, the attacker would have access to the private hosts.
Better would be, to use ssh forwarding. Here, the keys will remain on the administrators PC, out of reach for an attacker on the bastion host.

Consider a VCN (Virtual Compute Network) with two subnets, one public and one private.

These contain two compute instances. The bastionhost in the bastion-net with a public IP address and the applicationhost in the private-server-net with no public access.

To access SSH through the firewall, port 22 (ssh) needs to be added to the security lists. By using the Default Security List, this is enabled by default.

login as: opc
Authenticating with public key "putty-LASC-key"
[opc@bastionhost ~]$ sudo su
[root@bastionhost opc]# vi /etc/ssh/sshd_config

To enable ssh forwarding from the bastionhost, edit  /etc/ssh/sshd_config.

Uncomment the line containing 'AllowAgentForwarding yes' and save the file.

systemctl restart sshd.service

Restart the sshd to make the changes effective.

Check both boxes under 'Authentication parameters' to enable SSH forwarding in PuTTY.

To tell PuTTY, which ssh-key to forward, start Pageant which comes with your PuTTY installation. Add the private key for the private host. Close your PuTTY SSH session and open a new one to the bastion host.

Now it is possible to ssh from the bastionhost to the applicationhost without the need for private keys on the bastionhost because these are forwarded from the adminstration PC.

Friday, 1 June 2018

Run fnproject on Oracle Linux 7.5

A great thing about fn is that it runs on your (Linux-)Desktop. So for a local installation, Docker needs to be installed first. On Oracle Linux this is an easy task, as everything needed is already available in the yum-repositories. Just make sure, that all needed repositories are enabled.

[root@localhost] yum-config-manager --enable ol7_preview

Docker 17.12.1 is found in the ol7_preview repository, so enable that one.

[root@localhost] yum-config-manager --enable ol7_addons

Docker depends on container-selinux, which is found in the ol7_addons repository.

[root@localhost ~]# yum install docker-engine
 Package                      Arch              Version                            Repository              Size
 docker-engine                x86_64            17.12.1.ol-1.0.3.el7               ol7_preview             31 M
Installing for dependencies:
 container-selinux            noarch            2:2.21-1.el7                       ol7_addons              28 k

Transaction Summary
  docker-engine.x86_64 0:17.12.1.ol-1.0.3.el7                                                                   

Dependency Installed:
  container-selinux.noarch 2:2.21-1.el7                                                                         


Now that we have the right repositories enabled, just yum install docker-engine to get docker.

[root@localhost ~]# systemctl start docker

Start Docker (and systemctl enable docker, if you want to start it automatically). Then follow the Post-installation steps for Docker.

[oracle@localhost ~]$ sudo usermod -aG docker $USER

The docker group is already available, but the non-root user to run Docker needs to be added (and don't forget to logout/logon again, if you already are that user).

[oracle@localhost ~]$ docker run hello-world

Hello from Docker!

Check, that Docker is running correctly, if you like.
Now that Docker is running, just follow the fn installation guide to get fn.

[oracle@localhost ~]$ curl -LSs | sh

fn version 0.4.113

       / ____/___
      / /_  / __ \
     / __/ / / / /
    /_/   /_/ /_/`

This only takes seconds.

[oracle@localhost ~]$ fn start
Unable to find image 'fnproject/fnserver:latest' locally
latest: Pulling from fnproject/fnserver
ff3a5c916c92: Pull complete 
Digest: sha256:b1a330d24f07c8297fe8e9ba275eda6e04bf814a20b221be87bb492e9da2ff61
Status: Downloaded newer image for fnproject/fnserver:latest
time="2018-06-01T14:45:37Z" level=info msg="Setting log level to" level=info
time="2018-06-01T14:45:37Z" level=info msg="datastore dialed" datastore=sqlite3 max_idle_connections=256
time="2018-06-01T14:45:37Z" level=info msg="agent starting cfg={MinDockerVersion:17.10.0-ce DockerNetworks: FreezeIdle:50ms EjectIdle:1s HotPoll:200ms HotLauncherTimeout:1h0m0s AsyncChewPoll:1m0s CallEndTimeout:10m0s MaxCallEndStacking:8192 MaxResponseSize:0 MaxRequestSize:0 MaxLogSize:1048576 MaxTotalCPU:0 MaxTotalMemory:0 MaxFsSize:0 PreForkPoolSize:0 PreForkImage:busybox PreForkCmd:tail -f /dev/null PreForkUseOnce:0 PreForkNetworks: EnableNBResourceTracker:false MaxTmpFsInodes:0 DisableReadOnlyRootFs:false}"
time="2018-06-01T14:45:37Z" level=info msg="no docker auths from config files found (this is fine)" error="open /root/.dockercfg: no such file or directory"
time="2018-06-01T14:45:37Z" level=info msg="available memory" availMemory=13503231591 cgroupLimit=9223372036854771712 headRoom=1500359065 totalMemory=15003590656
time="2018-06-01T14:45:37Z" level=info msg="sync and async ram reservations" ramAsync=10802585273 ramAsyncHWMark=8642068218 ramSync=2700646318
time="2018-06-01T14:45:37Z" level=info msg="available cpu" availCPU=1000 totalCPU=1000
time="2018-06-01T14:45:37Z" level=info msg="sync and async cpu reservations" cpuAsync=800 cpuAsyncHWMark=640 cpuSync=200
time="2018-06-01T14:45:37Z" level=warning msg="Severaly Limited CPU: cpuAsync < 1000m (1 CPU)"
time="2018-06-01T14:45:37Z" level=info msg="Fn serving on `:8080`" type=full

       / ____/___
      / /_  / __ \
     / __/ / / / /
    /_/   /_/ /_/

The whole installation is straightforward and only takes minutes (or seconds, if you already have Docker installed), then fn is ready to run. For first steps follow the official tutorial.

Friday, 3 November 2017

Oracle Cloud Infrastructure and Terraform first steps on Windows

Oracle has released the 'Terraform provider for Oracle Cloud Infrastructure'. This tutorial is written to help with the first steps with Terraform and OCI on Windows.
In contrast to procedural tools like Oracle's ocicli, Terraform implements a declarative approach. That means, the user declares how the resulting architecture should look like and not how to get there. To do all the footwork, Terraform needs to know how to work with the target environment. That is what the Terraform provider for OCI is for.
To start with Terraform on OCI, you need to install Terraform first from and put it in your Path.

Check it with terraform -version. Next get the Terraform Provider for OCI from According to Oracle's documentation on GitHub, the windows_amd64 folder found in the file must be placed in %APPDATA%/terraform.d/plugins/. A %APPDATA%/terraform.rc would only be needed for compatibility with Terraform 0.9, so that can be skipped if not needed.

Now for the first steps with Terraform and OCI, we need at least the Tenancy OCID and the User OCID from our OCI environment as well as an private/public API key pair, the hash of the public key and the region name. For creating a VCN, we also need the compartment OCID.

The name of the region is on the top and the Tenancy OCID can be copied from the bottom of the the OCI welcome page.

The User OCID is found on the users settings page.

The Compartment OCID is found under Identity/Compartments. Make sure to pick the right compartment.

The API key we need is best created with openssl. Under Windows, get Git Bash with Git for Windows from and install it. Open Git Bash and run

$ mkdir ~/.oraclebmc (or any directory that is fine for you)
$ openssl genrsa -out ~/.oraclebmc/bmcs_api_key.pem 2048
$ chmod 0700 ~/.oraclebmc
$ chmod 0600 ~/.oraclebmc/bmcs_api_key.pem
$ openssl rsa -pubout -in ~/.oraclebmc/bmcs_api_key.pem -out ~/.oraclebmc/bmcs_api_key_public.pem
$ cat ~/.oraclebmc/bmcs_api_key_public.pem
$ openssl rsa -pubout -outform DER -in ~/.oraclebmc/bmcs_api_key.pem | openssl md5
$ openssl rsa -pubout -outform DER -in ~/.oraclebmc/bmcs_api_key.pem | openssl md5 > ~/.oraclebmc/bmcs_api_key_fingerprint

That should result in the following files:

The public key needs to be copied to the OCI UI, so copy the output from the cat command above ...

... click on Add Public Key and paste your key.

After adding the key, the OCI displays your keys fingerprint information. That should be the same as in bmcs_api_key.pem. If not, retake the steps from the first openssl command.

We need a few environment variables to pass these parameters to terraform. Create a working directory (eg. D:\Terraform-OCI), there create a batch file such as terraform-env.cmd and set the following

setx TF_VAR_tenancy_ocid "<your tenancy ocid>"
setx TF_VAR_user_ocid "<your user ocid>"
setx TF_VAR_fingerprint "<your api key fingerprint from bmcs_api_key.pem>"
setx TF_VAR_private_key_path "d:\Arnes\oraclebmc\bmcs_api_key.pem"
setx TF_VAR_compartment_ocid "<your compartment ocid>"
setx TF_VAR_region "us-phoenix-1"

Setx requires to close the shell and open a new one. Now we need to pass these variables to Terraform, so create a with the following:

variable "tenancy_ocid" {}
variable "user_ocid" {}
variable "fingerprint" {}
variable "private_key_path" {}
variable "region" {}

variable "compartment_ocid" {}

variable "VPC-CIDR" {
  default = ""

The first six variables are for Terraform to take our according environment variables. Every environment variable beginning with TF_VAR_ will automatically be mapped. We don't need the  CIDR variable right now, but later.
Next we need to pass the mandatory variables to the OCI provider. Create a with the following:

provider "oci" {
  tenancy_ocid     = "${var.tenancy_ocid}"
  user_ocid        = "${var.user_ocid}"
  fingerprint      = "${var.fingerprint}"
  private_key_path = "${var.private_key_path}"
  region           = "${var.region}"

Now we are ready for a first run to see if everything is set up correctly. In the working directory run a terraform init.

The OCI provider has been correctly initialized. With terraform plan we can check what Terraform is planning to do: 

Nothing of course, as we don't gave it any instructions on what to build. So let's create something. First we need to create a Virtual Cloud Network (VCN), so create a with the following

resource "oci_core_virtual_network" "Arne-VCN" {
  cidr_block     = "${var.VPC-CIDR}"
  compartment_id = "${var.compartment_ocid}"
  display_name   = "Arne-VCN"
  dns_label      = "arnevcn"

That will define a simple VCN using the CIDR block defined in VPC-CIDR. To see what Terraform will do with that, re-run terraform plan.

This time, Terraform displays, that it wants to create the VCN we defined. Looks good, so let Terraform execute that plan with terraform apply.

That should be done in seconds. Double-check that with the OCI web UI.

If everything worked, the newly created VCN should be shown here. Re-running terraform plan should show no changes. 
For a little more complex example, add the following to to add an internet gateway and a routing roule to the VCN.

resource "oci_core_internet_gateway" "Arne-IGW" {
  compartment_id = "${var.compartment_ocid}"
  display_name   = "Arne-IGW"
  vcn_id         = "${}"

resource "oci_core_route_table" "Arne-RT" {
  compartment_id = "${var.compartment_ocid}"
  vcn_id         = "${}"
  display_name   = "Arne-RT"

  route_rules {
    cidr_block        = ""
    network_entity_id = "${}"

Terraform plan should show, that the VCN won't be touched, as it already exists. But the internet gateway and the routing table with the rule will be created.
This is a very simple example, but the basic setup for using Terraform with OCI is done.

Monday, 29 May 2017

Oracle Cloud Compute: Workaround for Docker / XFS issue with Oracle Linux

With the latest Oracle Linux 7.2 UEK4 images on the Oracle Cloud, the following error occurs when running Docker containers.

docker: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/66c96d19f86d1262f2472cda6639dacfde8f13d0bebff9e434b74b9cebdcc32f-init/merged: invalid argument.

This is related to the ftype parameter of the xfs filesystem, which is set at formatting. It can be verified by running

xfs_info /

and the output should look like this:

This is a screenshot from a fresh OL 7.2 UEK4 instance after a full yum update. As you can see, ftype is zero. Unfortunately, that setting is not compatible with the overlayfs from Docker and cannot be changed after formatting.
For more infos on this issue see To solve this, /var/lib/docker must be moved to a different volume with xfs and ftype=1 set or just use a different file system.

This example shows, how to solve this with an EXT4 volume.

On the Compute dashboard, go to the Storage Tab and click on Create Storage Volume.

Set the settings to your like and create the volume.

Klick on the hamburger menu next to your newly created volume and choose Attach to instance.

Choose your Compute instance and attach the volume.

To mount the volume, just follow the steps from the Oracle documentation: Mounting a Storage Volume on a Linux Instance.

First find out the device name via

ls /dev/xvd*

The output should look like this

The first device, the boot volume, is always /dev/xvdb, the next one is /dev/xvdc etc. In my example, the new volume is disk #2, so it is /dev/xvdc.
So the next step is a

fdisk /dev/xvdc

and go through the following steps:

In short, select n for new partition, p for primary, leave 1 as the default partition number and accept both sector numbers (dependent on the size of your volume). Then write the configuration with w.

Next create a file system, eg.

sudo mkfs -t ext4 /dev/xvdc

and the result should look like this

Now we need a mount point, which would be /u01 according to Oracle's styleguides. By default, the OL image already comes with a /u01 directory as mount point, so we can use this.
As we want this to be a permanent mount, add this to /etc/fstab by

sudo vi /etc/fstab

and add the following line

/dev/xvdc /u01 ext4 defaults 0 0

so it should look like this:

Save and restart the instance to check if everything worked. After reboot, check if the file system is mounted, eg. with

mount | grep /u01; ls /u01

and the output should look like this:

Now create a location where to move the directory /var/lib/docker, for example /u01/var/lib. Use any method you like to move the docker directory to the new location. I prefer to use tar, so I have an additional backup, eg.

cd /var/lib
tar cvfz docker.tgz docker
cd /u01/var/lib/
tar xvfz /var/lib/docker.tgz
mv /var/lib/docker /var/lib/docker.bak
cd /var/lib
ln -s /u01/var/lib/docker/ docker

Then set a symlink to your newly created docker lib directory and restart the docker daemon and try it with any image.

Should run now, that's it.