Tuesday, 24 March 2020

How to run Minecraft on Oracle Cloud Infrastructure, the even easier way


There is an Update to this post: Run a free Minecraft Server in the Oracle Cloud (3rd. Update)

---8<-----------------------------------------------------------------------------

This is an update to my older post 'How to run a Minecraft Server in the Oracle Cloud (OCI Update)' which has already been an update to an even older posting covering the older Oracle Cloud Infrastructure Classic (OCI-C). As the OCI is constantly enhancing, I try to keep up describing the easiest way to run your Minecraft server.
Start at the OCI console startpage and choose 'Create a VM instance'.
Give it a name and stick with the defaults.
Scroll down and adjust the shape, if needed. If this is your first project on OCI, you probably don't have a Virtual Cloud Network. The VM wizard will notice this and offers you, to create such. Give it a name and make sure that 'assign a public IP address' is checked.
Do not forget to upload your ssh public key. For the rest, leave the defaults and click 'Create'
You will get a long list of what OCI will do for you. Wait until it shows 'Running' and copy your public IP address.
The next steps are the same as in OCI-C. Set up a ssh connection with your favorite ssh client. I use PuTTY on Windows. Paste the IP adress and save the session.
Enter the Auto-login username 'opc'.
Under Connection/SSH/Auth choose you private key file. Then save again and open your session.
And here we are, hello OCI. Pull the Minecraft server directly from its publisher via (check for the version you prefer)

wget https://launcher.mojang.com/v1/objects/bb2b6b1aefcd70dfd1892149ac3a215f6c636b07/server.jar

Install the Java version you want to use, e.g.

sudo yum install java-11-openjdk

Probably this will pull some other updates. Then start the server:

java -jar server.jar

After the first start, the server terminates immediately, because you have to accept the EULA first.
It created a new file eula.txt. Open it in vi and change the content to eula=true.
Before we start the server again, lets open the port in the Linux firewall.

sudo firewall-cmd --permanent --zone=public --add-port=25565/tcp
sudo firewall-cmd --reload

As the Minecraft server has an interactive console, it should be accessible even when the SSH console has been closed in between. I prefer to the screen command for that, which can be installed by.

sudo yum install screen.x86_64

Then open a new screen and start the server with

screen -S minecraft
java -jar server.jar

With the server running, we are done with SSH. Close the screen session via CRTL-A D or just close your PuTTY window. If you want to reconnect to your Minecraft server from a new SSH session, just type

screen -d -r minecraft

Back in the OCI web UI, we also need to configure the firewall rules. From the Server VM page, click on the subnet.

Now our Minecraft server is up and running and the Linux firewall is open, so it could serve requests. But we first need to open the Minecraft port on Linux too, because by default on OCI all ports except for ssh are closed for security reasons.
Back to the OCI console, open the Hamburger menu and to to Networking/Virtual Cloud Networks.
There you shoud find a VCN with the name you gave it when creating the instance. Click on the name to open the settings.
Click on Security Lists on the left under Resources.
Click on the Default Security List to open the settings.
From the Ingress Rules you can see, that any incoming TCP traffic, except for ssh, is blocked. So we need to add a rule for our minecraft server here.
By clicking on 'Add Ingress Rule' we can define our rule for CIDR 0.0.0.0/0 (all internet) and the Minecraft server port 25565.
The rule should be listed under Ingress Rules. Now we are done with the OCI setup.
Start Minecraft and configure a new Server. Give it a name and enter the public IP address of your instance.
Your Oracle Cloud instance will be listed in Minecrafts server list, notice the good ping. Double click to start your game.
Have fun with the Oracle Cloud and Minecraft!

BTW: this tutorial is meant to demonstrate the usage of OCI, not to do a perfect Minecraft setup. There are many tutorials to do this, like this one.

Thursday, 6 June 2019

How to run a Minecraft Server in the Oracle Cloud (OCI Update)


Update: I made a newer and shorter version of this tutorial utilizing new OCI UI features. So please read no further and go to my updated posting under: https://arnes-stuff.blogspot.com/2020/08/run-free-minecraft-server-in-oracle.html

---8<-----------------------------------------------------------------------------

This is an update to my older post 'How to run a Minecraft Server in the Oracle Cloud' covering the older Oracle Cloud Infrastructure Classic (OCI-C). This posting describes how to do the same on the newer Oracle Cloud Infrastructure (OCI).
This time start at the OCI startpage and choose 'Create a virtual cloud network', because we need a VCN first.
For an easy start, select 'Create a virtual cloud network plus related resources', which will give you a complete starter virtual cloud network. Just give it a name, scroll down and confirm.
This will give us a VCN with three subnets in three different availability domains (three physical data centers). This is more than what we need, but as subnets are not charged, it does not hurt either.
Now that we have our networking up and ready, let us move to the compute menu. Click on the hamburger menu and choose Compute|Instances.
There click on 'Create Instance'
You can leave all the defaults. Your VCN should be chosen automatically.
But do not forget to upload your ssh public key.
Wait for your machine to be running, then copy the public IP address.
The next steps are the same as in OCI-C. Set up a ssh connection with your favorite ssh client. I use PuTTY on Windows. Paste the IP adress and save the session.
Enter the Auto-login username 'opc'.

Under Connection/SSH/Auth choose you private key file. Then save again and open your session.

Pull the Minecraft server directly from its publisher via

wget https://launcher.mojang.com/v1/objects/808be3869e2ca6b62378f9f4b33c946621620019/server.jar

Install the Java version you want to use, e.g.

sudo yum install java-1.8.0-openjdk.x86_64

Then start the server:

java -jar minecraft_server.1.9.2.jar

After the first start, the server terminates immediately, because you have to accept the EULA first.
It created a new file eula.txt. Open it in vi and change the content to eula=true.
Before we start the server again, lets open the port in the Linux firewall.

sudo firewall-cmd --permanent --zone=public --add-port=25565/tcp
sudo firewall-cmd --reload

As the Minecraft server has an interactive console, it should be accessible even when the SSH console has been closed in between. I prefer to the screen command for that, which can be installed by.

sudo yum install screen.x86_64

Then open a new screen and start the server with

screen -S minecraft
java -jar server.jar

With the server running, we are done with SSH. Close the screen session via CRTL-A D or just close your PuTTY window. If you want to reconnect to your Minecraft server from a new SSH session, just type

screen -d -r minecraft

Back in the OCI web UI, we also need to configure the firewall rules. From the Server VM page, click on the subnet.
On the subnet page, click on the security lists.
Here, click on the default security list
From the Ingress Rules you can see, that any incoming TCP traffic, except for ssh, is blocked. So we need to add a rule for our minecraft server here.
By clicking on 'Add Ingress Rule' we can define our rule for CIDR 0.0.0.0/0 (all internet) and the Minecraft server port 25565.
The rule should be listed under Ingress Rules. Now we are done with the OCI setup.
Start Minecraft and configure a new Server. Give it a name and enter the public IP address of your instance.
Your Oracle Cloud instance will be listed in Minecrafts server list, notice the good ping. Double click to start your game.
Have fun with the Oracle Cloud and Minecraft!

BTW: this tutorial is meant to demonstrate the usage of OCI, not to do a perfect Minecraft setup. There are many tutorials to do this, like this one.

Friday, 15 March 2019

Use Terraform in the Oracle Cloud with Stacks

I already described how to use Terraform with the Oracle Cloud Infrastructure (OCI) from on-premise. But with the Resource Manager Stacks, Oracle offers a smart alternative to use Terraform with OCI.


The Resource Manager can be found in the Hamburger menu on the left side.


Click on 'Create Stack' to do so.


The Terraform files are needed in a single zip file, so we need to create some. Find my examples on GitHub.

network.tf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
resource "oci_core_virtual_network" "VCN-Demo" {
  cidr_block     = "${var.VCN_SEDEMO_CIDR}"
  compartment_id = "${var.compartment_ocid}"
  display_name   = "VCN-Demo"
  dns_label      = "demovcn"
}

resource "oci_core_subnet" "SN_Bastion" {
  availability_domain = "${lookup(data.oci_identity_availability_domains.ADs.availability_domains[0],"name")}"
  cidr_block          = "${var.SN_BASTION_CIDR}"
  display_name        = "SN_Bastion"
  compartment_id      = "${var.compartment_ocid}"
  vcn_id              = "${oci_core_virtual_network.VCN-SE-Demo.id}"
  dns_label           = "snbastion"
}

I like to start with a Virtual Cloud Network (VCN) and a subnet, as these require minimal parameters.

data.tf
1
2
3
data "oci_identity_availability_domains" "ADs" {
  compartment_id = "${var.compartment_ocid}"
}

To avoid hard coding the ocid's of the Availbility Domains (ADs), I query the ADs from the compartment.

variables.tf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
variable "compartment_ocid" {}
variable "region" {}

#########   CIDR  #################

variable "VCN_SEDEMO_CIDR" {
  default = "10.0.0.0/16"
}

variable "SN_BASTION_CIDR" {
  default = "10.0.1.0/24"
}

As with a standard Terraform script, I query the compartment_ocid and region from the environment and set two CIDR blocks for the VCN and the subnet.

provider.tf

1
2
3
provider "oci" {
  region = "${var.region}"
}

Now for the provider.tf, this one gets rather short. As we are already logged into oci, we do not need to handle the key file or user_ocid.


Pack those four files into a zip archive and upload them to the create Stack dialog.


We cannot source the variables via shell, but we can add these to the create dialog and we are done here.


To use the newly created Stack, click on its name or from the menu choose Edit.


Here you can do the usual Terraform plan, apply and destroy actions. Click on Plan and confirm the dialog.


In the Jobs list, you will see a new Terraform Job. Click on its name to see the details. It should succeed and on the bottom you will see the Terraform output.
After that, use Terraform Actions | Apply to create the resources, which should result in a


1
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

So this is a smart option to use Terraform with OCI without the need of a client software installation or a seperate vm to run the scripts, which could save time and money.

Friday, 15 June 2018

Oracle Cloud: howto access the public web from private networks via Squid proxy

Building up on my howto guide 'Run a bastion host with ssh-forwarding on Oracle Cloud Infrastructure', this guide is for the opposite direction. Say you have a server in a private subnet, only accessible through the bastion host. If that host needs access to the public internet, a proxy in a public subnet of the VCN (Virtual Compute Network) could be a solution.

The situation is the same as in the former guide. Two hosts, one named bastion in a public subnet and one named webserver in a private subnet. The webserver host has no access to the internet, but needs to access some information from a public URL, so a proxy on the bastionhost needs to be set up.

Summary
  • Install and setup Squid on a host in the public subnet
  • Create an ingress rule for the public subnet to allow traffic to the Squid port
  • Create an egress rule to allow egress traffic for the private subnet
  • Open the Squid port on the Linux firewall of the Squid host
  • Set http_proxy to the Squid host:port
Step-by-step guide

This guide uses Apache Squid, but only uses it as a proxy server. So first install it via yum:

[root@bastion ~] yum install squid

Start Squid and check it's status:

[root@bastion ~] systemctl start squid
[root@bastion ~] systemctl status squid
● squid.service - Squid caching proxy
   Loaded: loaded (/usr/lib/systemd/system/squid.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-06-14 12:34:49 GMT; 10s ago
  Process: 11307 ExecStart=/usr/sbin/squid $SQUID_OPTS -f $SQUID_CONF (code=exited, status=0/SUCCESS)
  Process: 11301 ExecStartPre=/usr/libexec/squid/cache_swap.sh (code=exited, status=0/SUCCESS)
 Main PID: 11309 (squid)
   CGroup: /system.slice/squid.service
           ├─11309 /usr/sbin/squid -f /etc/squid/squid.conf
           ├─11311 (squid-1) -f /etc/squid/squid.conf
           └─11312 (logfile-daemon) /var/log/squid/access.log

Jun 14 12:34:49 bastion systemd[1]: Starting Squid caching proxy...
Jun 14 12:34:49 bastion squid[11309]: Squid Parent: will start 1 kids
Jun 14 12:34:49 bastion squid[11309]: Squid Parent: (squid-1) process 11311...ed
Jun 14 12:34:49 bastion systemd[1]: Started Squid caching proxy.
Hint: Some lines were ellipsized, use -l to show in full.

If everything is fine, enable automatic Squid startup at boot. If you get errors, run a complete yum update and try again. That fixed my problems with startup.

[root@bastion ~] systemctl enable squid
Created symlink from /etc/systemd/system/multi-user.target.wants/squid.service to /usr/lib/systemd/system/squid.service.

To test it from the system Squid is running on, set the HTTP proxy to the Squid default port, which is 3128.

[opc@bastion ~]$ export http_proxy=http://127.0.0.1:3128 
[opc@bastion ~]$ export https_proxy=http://127.0.0.1:3128

Then open a page with curl or lynx, eg.

[opc@bastion ~]$ lynx https://www.oracle.com

That should return the text version of that page.


Check that the proxy has been used in the squid access log.

[root@bastion ~]# tail -f /var/log/squid/access.log
1528980628.887     45 127.0.0.1 TCP_MISS/200 3125 GET http://www.google.com/ - HIER_DIRECT/172.217.5.228 text/html
1528980661.842      5 127.0.0.1 TCP_MISS/301 314 GET http://www.oracle.com/ - HIER_DIRECT/23.15.129.79 -

Now that the squid proxy works and is accessible locally, create a security role on OCI to give the webserver host access to the Squid proxy on the bastionhost.


Find the public subnet in your VCN and click on the security list.


Click on 'Edit all rules' and add the network CIDR block of the webserver subnet (alternatively only the host IP address).


After clicking on 'Save Security List Rules', these should be listed in the Security List.


Also make sure, that in the private networks security list egress traffic is allowed at least for port 3128 to the private network, or like in this case allow all egress traffic. In the default security list, egress traffic is allowed. If you have created your own security list, you might need to add an egress rule.

firewall-cmd --new-zone=webapp --permanent
firewall-cmd --reload
firewall-cmd --zone=webapp --add-source=10.11.111.132/32 --permanent
firewall-cmd --zone=webapp --add-port=3128/tcp --permanent
firewall-cmd --reload

The last step is to open the Squid port on the public host in the Linux firewall.

[opc@webserver ~]$ curl www.google.com
^C
[opc@webserver ~]$ export http_proxy=10.11.111.148:3128
[opc@webserver ~]$ curl www.google.com
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description">....

Now check, if everything works. Try to open any public URL, eg. with curl. That should not work. Wait for the timeout or CTRL-C. Set the proxy to the Squid server and port and try again. That should work now, if not double-check all settings.
So now, the server in the private network with no access to the public internet can access the web through the Squid proxy in a public subnet of your VPN.

Sunday, 10 June 2018

How to run Python from Notepad++

Notepad++ can easily be configured to run Python. This is practically the same, as my former 'Running Oracle Cloud Stack Manager CLI from Notepad++' howto. But there are some minor updates.
Most important, all required plugins are now available for 64Bit Notepad++. so install the Notepad++ version of choice.

Install Plugin Manager

The nppPluginManager is now on GitHub. Just follow the steps (copy PluginManager.dll to the plugins and gpup.exe to the updater directory) from the README.md. Restart Notepad++

Install NppExec

Open Plugin Manager (Plugins|Plugin Manager|Show Plugin Manager), pick the NppExec Plugin from the list of available Plugins and click 'Install'. After restart, it should be available.


4. Save a command for Python and run it

With some meaningful Python code opened, hit F6 in Notepad++. Enter the following command for running Python (adjust the path to your environment) and save it.

c:\dev\Python\Python36\python.exe "$(FULL_CURRENT_PATH)" 


Clicking OK opens a console window and runs Python with your last saved code version.

5. Re-run Python while editing

Via CTRL-F6, the last command can be re-run, don't forget to CTRL-S first to save your code.


Friday, 8 June 2018

Run a bastion host with ssh-forwarding on Oracle Cloud Infrastructure

This short howto guide will show how to setup ssh forwarding for Oracle Linux on OCI and access from Windows via PuTTY.
The idea of a bastion host is to have a single entry point into your network. The bastion host is the only one that is accessible from outside the network, there is no sensible data on it. All applications and data reside on private hosts, that are not accessible from the outside. To access the applications, ssh into the bastion host and from there ssh to the specific private host. Firewall rules will block all unwanted traffic.
There are several options to do the key handling. One obvious one would be, to install all private keys on the bastion host. But if there would be a successful attack onto the bastion host, the attacker would have access to the private hosts.
Better would be, to use ssh forwarding. Here, the keys will remain on the administrators PC, out of reach for an attacker on the bastion host.


Consider a VCN (Virtual Compute Network) with two subnets, one public and one private.


These contain two compute instances. The bastionhost in the bastion-net with a public IP address and the applicationhost in the private-server-net with no public access.


To access SSH through the firewall, port 22 (ssh) needs to be added to the security lists. By using the Default Security List, this is enabled by default.

login as: opc
Authenticating with public key "putty-LASC-key"
[opc@bastionhost ~]$ sudo su
[root@bastionhost opc]# vi /etc/ssh/sshd_config

To enable ssh forwarding from the bastionhost, edit  /etc/ssh/sshd_config.


Uncomment the line containing 'AllowAgentForwarding yes' and save the file.

systemctl restart sshd.service

Restart the sshd to make the changes effective.


Check both boxes under 'Authentication parameters' to enable SSH forwarding in PuTTY.


To tell PuTTY, which ssh-key to forward, start Pageant which comes with your PuTTY installation. Add the private key for the private host. Close your PuTTY SSH session and open a new one to the bastion host.


Now it is possible to ssh from the bastionhost to the applicationhost without the need for private keys on the bastionhost because these are forwarded from the adminstration PC.


Friday, 1 June 2018

Run fnproject on Oracle Linux 7.5

A great thing about fn is that it runs on your (Linux-)Desktop. So for a local installation, Docker needs to be installed first. On Oracle Linux this is an easy task, as everything needed is already available in the yum-repositories. Just make sure, that all needed repositories are enabled.

[root@localhost] yum-config-manager --enable ol7_preview

Docker 17.12.1 is found in the ol7_preview repository, so enable that one.

[root@localhost] yum-config-manager --enable ol7_addons

Docker depends on container-selinux, which is found in the ol7_addons repository.

[root@localhost ~]# yum install docker-engine
...
================================================================================================================
 Package                      Arch              Version                            Repository              Size
================================================================================================================
Installing:
 docker-engine                x86_64            17.12.1.ol-1.0.3.el7               ol7_preview             31 M
Installing for dependencies:
 container-selinux            noarch            2:2.21-1.el7                       ol7_addons              28 k

Transaction Summary
================================================================================================================
...
Installed:
  docker-engine.x86_64 0:17.12.1.ol-1.0.3.el7                                                                   

Dependency Installed:
  container-selinux.noarch 2:2.21-1.el7                                                                         

Complete!


Now that we have the right repositories enabled, just yum install docker-engine to get docker.

[root@localhost ~]# systemctl start docker

Start Docker (and systemctl enable docker, if you want to start it automatically). Then follow the Post-installation steps for Docker.

[oracle@localhost ~]$ sudo usermod -aG docker $USER

The docker group is already available, but the non-root user to run Docker needs to be added (and don't forget to logout/logon again, if you already are that user).

[oracle@localhost ~]$ docker run hello-world

Hello from Docker!

Check, that Docker is running correctly, if you like.
Now that Docker is running, just follow the fn installation guide to get fn.

[oracle@localhost ~]$ curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

fn version 0.4.113

        ______
       / ____/___
      / /_  / __ \
     / __/ / / / /
    /_/   /_/ /_/`

This only takes seconds.

[oracle@localhost ~]$ fn start
Unable to find image 'fnproject/fnserver:latest' locally
latest: Pulling from fnproject/fnserver
ff3a5c916c92: Pull complete 
...
Digest: sha256:b1a330d24f07c8297fe8e9ba275eda6e04bf814a20b221be87bb492e9da2ff61
Status: Downloaded newer image for fnproject/fnserver:latest
time="2018-06-01T14:45:37Z" level=info msg="Setting log level to" level=info
time="2018-06-01T14:45:37Z" level=info msg="datastore dialed" datastore=sqlite3 max_idle_connections=256
time="2018-06-01T14:45:37Z" level=info msg="agent starting cfg={MinDockerVersion:17.10.0-ce DockerNetworks: FreezeIdle:50ms EjectIdle:1s HotPoll:200ms HotLauncherTimeout:1h0m0s AsyncChewPoll:1m0s CallEndTimeout:10m0s MaxCallEndStacking:8192 MaxResponseSize:0 MaxRequestSize:0 MaxLogSize:1048576 MaxTotalCPU:0 MaxTotalMemory:0 MaxFsSize:0 PreForkPoolSize:0 PreForkImage:busybox PreForkCmd:tail -f /dev/null PreForkUseOnce:0 PreForkNetworks: EnableNBResourceTracker:false MaxTmpFsInodes:0 DisableReadOnlyRootFs:false}"
time="2018-06-01T14:45:37Z" level=info msg="no docker auths from config files found (this is fine)" error="open /root/.dockercfg: no such file or directory"
time="2018-06-01T14:45:37Z" level=info msg="available memory" availMemory=13503231591 cgroupLimit=9223372036854771712 headRoom=1500359065 totalMemory=15003590656
time="2018-06-01T14:45:37Z" level=info msg="sync and async ram reservations" ramAsync=10802585273 ramAsyncHWMark=8642068218 ramSync=2700646318
time="2018-06-01T14:45:37Z" level=info msg="available cpu" availCPU=1000 totalCPU=1000
time="2018-06-01T14:45:37Z" level=info msg="sync and async cpu reservations" cpuAsync=800 cpuAsyncHWMark=640 cpuSync=200
time="2018-06-01T14:45:37Z" level=warning msg="Severaly Limited CPU: cpuAsync < 1000m (1 CPU)"
time="2018-06-01T14:45:37Z" level=info msg="Fn serving on `:8080`" type=full

        ______
       / ____/___
      / /_  / __ \
     / __/ / / / /
    /_/   /_/ /_/
        v0.3.460

The whole installation is straightforward and only takes minutes (or seconds, if you already have Docker installed), then fn is ready to run. For first steps follow the official tutorial.