If you have ever wanted to deploy and configure a Docker host with a single command from your own terminal you have come to the right place. Docker Machine enables you to provision and manage Docker systems in the cloud directly from your own terminal.
Using the Docker Machine tools installed locally you can deploy a new cloud server through the UpCloud API in seconds without ever opening a web browser. This guide shows what you need to get started on Ubuntu Linux but Docker Machine is also available on a number of other operating systems including macOS and Windows.
Enable API permissions
Before you begin, deploying a new server with the Docker Machine onto your UpCloud account requires you to enable the API permissions, these can be found in your user account details at the UpCloud control panel.
The first step is to create a new subaccount just for the Docker Machine to communicate with the API. This can be done in the My Account menu under User Accounts tab. We strongly recommend limiting the connections to specific IP address or address range for security reasons. You can find out more about the UpCloud accounts and permissions at an article about group accounts and subaccount management.
Option 1. Quick start for testing
If you wish to quickly test provisioning cloud servers with the Docker Machine and do not mind missing the remote Docker commands, you can download and include the required binaries to your PATH without installing the Docker environment.
Download the Docker Machine binary.
curl -L https://github.com/docker/machine/releases/download/v0.13.0/docker-machine-`uname -s`-`uname -m` -o ~/docker-machine
The crucial part in allowing the Docker Machine to communicate with a cloud provider API comes in a form of the Machine drivers. The Docker Machine driver for this was written and published by a fellow UpCloud user torras on GitHub. It utilizes the Go API client for UpCloud which can be found at our GitHub library.
Download the UpCloud driver.
curl -LO https://github.com/torras/docker-machine-driver-upcloud/releases/download/0/docker-machine-driver-upcloud
Make the two binary files executable with the next command.
chmod +x ~/docker-machine ~/docker-machine-driver-upcloud
Then move the binaries to an appropriate directory that is included in your PATH.
sudo mv ~/docker-machine* /usr/local/bin/
That is all you need to be able to run docker-machine commands using the UpCloud driver to deploy, manage, and delete servers straight from your own command line. Jump forward to the instructions on how to provision a docker host using the Docker Machine.
Option 2. Setting up Docker Machine environment
To get the most out of what Docker Machine has to offer, you might want to set up a proper environment to enable all features and easy updates. This includes installing the Docker Engine, the Docker Machine, a Go programming environment, as well as a few pre-requisites to these packages.
Start by making sure you have the following tools installed and ready. Applications like curl, gcc, and git may be included in your distribution, simply verify that they can be found as these will be needed later on.
sudo apt-get install curl gcc git
You will also need to have the next two packages to securely download the Docker Engine. Depending on your system these might already be installed.
sudo apt-get install apt-transport-https ca-certificates software-properties-common
Once you have the pre-requisites installed, continue below with the next step.
Installing Docker CE
To install Docker with the following instructions, you will need a 64-bit version of Ubuntu 14.04 or newer. For other operating systems and Linux distributions, you can find install instructions at Docker documentation. Docker itself will not be used locally in this guide but the command line tool is required to run Docker queries on the remote servers. Download and add the Docker project repository key.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
You can verify that the key is valid with the following command.
sudo apt-key fingerprint 0EBFCD88
After successfully executing the above command, you should see a printout as below.
pub 4096R/0EBFCD88 2017-02-22 Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88 uid Docker Release (CE deb) <[email protected]> sub 4096R/F273FCD8 2017-02-22
You can then add the Docker repository to your sources list.
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
Then update the sources.
sudo apt-get update
Lastly, install the latest version of Docker with the following command.
sudo apt-get install docker-ce
Once installed, Docker is ready to run. Usually, you might wish to add your username to the docker users group to avoid having to use sudo on every docker run command, but with Docker Machine the commands are run remotely and this is not necessary.
Installing Docker Machine
Next, download the Docker Machine binary and copy it to an appropriate directory in your PATH environmental variable.
curl -L https://github.com/docker/machine/releases/download/v0.13.0/docker-machine-`uname -s`-`uname -m` -o ~/docker-machine
Make sure the program is executable.
chmod +x ~/docker-machine
Then copy it to the /usr/local/bin/ directory.
sudo cp ~/docker-machine /usr/local/bin/
The docker-machine command-line tool lets you deploy, manage and delete Docker servers both locally and remotely. Next, continue below with installing the Go programming language packages to download and compile the Docker Machine driver for UpCloud.
Go is an open source programming language that was used to write the Docker Machine driver for UpCloud. The Go binaries get frequent updates and the download command below might not use the latest version. You can check for possible newer versions on their download page and change the version number in the package URL as necessary.
curl -O https://storage.googleapis.com/golang/go1.9.3.linux-amd64.tar.gz
Unpack the downloaded binaries to an appropriate location e.g. with the next command.
sudo tar -C /usr/local -xzf go1.9.3.linux-amd64.tar.gz
Then create a new directory to save any Go sources into. You are of course free to choose the location of the Go home directory. If you wish to store the related files somewhere else, set the GOPATH environmental variable accordingly.
mkdir -p ~/go/bin
export GOPATH=$HOME/go export GOBIN=$GOPATH/bin export PATH=$PATH:/usr/local/go/bin:$GOBIN
For future use, you should also save the above lines in your profile.
echo 'export GOPATH=$HOME/go' | tee -a ~/.profile echo 'export GOBIN=$GOPATH/bin' | tee -a ~/.profile echo 'export PATH=$PATH:/usr/local/go/bin:$GOBIN' | tee -a ~/.profile
This way you will avoid having to reset the paths every time you log in or open a new terminal.
Go should now be available and configured. Test the install by checking the version number.
You should see an output as shown below.
go version go1.9.3 linux/amd64
The version number should match the package name you downloaded, 1.9.3 for 64-bit Linux in this case as above. With Go packages installed, you are ready to download and compile the Docker Machine driver for UpCloud.
Installing Docker Machine UpCloud driver
The Machine drivers are an important part of how the Docker Machine functions with a cloud provider API. The Docker Machine driver for this was written and published by a fellow UpCloud user torras on GitHub. It utilizes the Go API client for UpCloud which can be found at our GitHub library.
Download and build the driver using the Go command below.
go get -u github.com/torras/docker-machine-driver-upcloud
If you receive an error during the build, use the command underneath to update the import paths in the driver and then rerun the previous command.
sed -i -e 's/Jalle19\/upcloud-go-sdk/UpCloudLtd\/upcloud-go-api/' $GOPATH/src/github.com/torras/docker-machine-driver-upcloud/driver/upcloud.go
Make sure the driver is set as executable.
chmod +x $GOBIN/docker-machine-driver-upcloud
Then copy it to the same directory as the Docker Machine binary.
sudo cp $GOBIN/docker-machine-driver-upcloud /usr/local/bin/
That is it, you should now be able to run docker-machine commands using the UpCloud driver to deploy, manage and delete servers straight from your own command line. Continue with the instructions underneath on how to deploy a Docker host using Docker Machine.
Provisioning Docker host
Check that Docker Machine responds to commands and can find the required driver using the command below. This prints the usage options for deploying a new server, including the UpCloud specific parameters with their default values.
docker-machine create --driver upcloud
To actually deploy a server, you will need to choose a few parameters such as the plan, zone, and template with which the server will be created. The following UpCloud specific options can be used to customize the server.
docker-machine create --driver upcloud --upcloud-user "username" --upcloud-passwd "password" --upcloud-plan "1xCPU-1GB" / "1xCPU-2GB" / "2xCPU-4GB" / "4xCPU-8GB" --upcloud-zone "uk-lon1" / "de-fra1" / "us-chi1" / "fi-hel2" / "nl-ams1" / "sg-sin1" --upcloud-template "01000000-0000-4000-8000-000030060200" (Ubuntu Server 16.04) --upcloud-ssh-user "root" --upcloud-use-private-network --upcloud-userdata "/path/to/cloud-init" <server_name>
The template option takes an UpCloud storage disk UUID as a parameter. Debian based systems work well together with Docker Machine commands but you can also use other distributions or even create your own template. You can find a list of the public templates at the UpCloud Control Panel under the Disks menu, the following public templates are tested and recommended.
Debian GNU/Linux 8.10 (Jessie) 01000000-0000-4000-8000-000020030100 Debian GNU/Linux 9.3 (Stretch) 01000000-0000-4000-8000-000020040100 Ubuntu Server 14.04 LTS (Trusty Tahr) 01000000-0000-4000-8000-000030040200 Ubuntu Server 16.04 LTS (Xenial Xerus) 01000000-0000-4000-8000-000030060200
Test your Docker Machine setup by creating a new host using the command below. Replace the <username> and <password> with your API subaccount details to authenticate to the API and enter a hostname by replacing the <server_name>.
docker-machine create --driver upcloud --upcloud-user "<username>" --upcloud-passwd "<password>" --upcloud-plan "1xCPU-1GB" --upcloud-zone "uk-lon1" --upcloud-template "01000000-0000-4000-8000-000030060200" <server_name>
The deployment will only take a moment and when the server is running a Docker Engine will be installed. In case something goes wrong while provisioning, you can restart the process with the following command. This might be necessary if the system image you used to deploy the Docker Machine does not include all the tools Docker expected.
docker-machine provision <server_name>
Once the server is up and running, you can see it on a list of provisioned Docker hosts with the next command.
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS docker - upcloud Running tcp://126.96.36.199:2376 v18.01.0-ce
Also as usual when your cloud Docker host is being deployed, you will receive an email about the server creation to your UpCloud account with the root password if you wish to log in manually using SSH to check on the server.
Deploying containers to Docker hosts remotely
After the newly provisioned host replies that it is ready, you can employ the docker run command to deploy containers, granted that you installed the Docker binaries. First, check the server environment with the following command. Replace the <server_name> with yours.
docker-machine env <server_name>
Then change focus to the new machine with the command shown in the environment output like the example below.
eval $(docker-machine env <server_name>)
Any Docker commands will now be executed on the focused server instead of running them locally. Deploy a simple container to test the setup with the command below.
docker run hello-world
You should see the latest container image getting downloaded and run ending with a hello message from Docker confirming that the installation is working correctly.
If you wish to verify that the container was indeed executed on the remote host you can use the Docker Machine to open an SSH connection to the server using a command like below.
docker-machine ssh <server_name>
Once connected, check the containers deployed to the server using the docker ps command with the --all parameter to show exited containers as well.
docker ps --all
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c8aa3c18b365 hello-world "/hello" 39 seconds ago Exited (0) 37 seconds ago laughing_newton
When you are finished checking out the server, close the SSH connection and return to your local terminal.
Congratulations, you now have a fully featured Docker Machine command centre at your disposal. You can continue working with the host you deployed or simply delete it if you no longer have a need for the server.
docker-machine rm <server_name>
Deploying Docker ready cloud servers does not get much easier than with Docker Machine. The simplicity and ease of use will help you save time and money whether you wish to quickly test a container you are developing or build an on-demand scalable cluster. Using Docker Machine also allows you to create and manage Docker Swarm clusters of which you can find out more at the Docker documentation.