Terraform and UpCloud
- 1 - Terraform and UpCloud
- 2 - Terraform and UpCloud Part 2
- 3 - Ansible and UpCloud
- 4 - Combine Ansible & Terraform
- 5 - Etcd and SkyDNS on UpCloud.
- 6 - Etcd cluster for Kubernetes
- 7 - Firewall with ansible
This post is outdated and a new more up-to-date post can be found here!
Me and my colleges are currently working on moving all our infrastructure from a few different providers into a single one. In the end we are aiming for a kubernetes cluster which can take care of itself on multiple providers, but initially, one provider is where we start.
We where researching providers to use mainly because the ones that we have our current infrastructure on is based in USA, due to a bunch of new laws which might compromise the integrity of our customers, we decided that we wanted to move to European providers.
During our research we came across UpCloud, UpCloud is a Finnish company (which is in Europe for those who didn’t know) and have great prices for good virtual machines.
Creating a new machine on UpCloud is easy and fast, so we decided to try it out! A few minutes after we created our accounts for testing, we got an email from the server provider, an email with quite a personal touch, something that always makes one happy, so we responded and started communicating with the company.
During our chat, we decided that UpCloud was a perfect fit for our cluster, and we started to test their machines and their provisioning capabilities.
UpCloud does not have a CaaS (Containers as a Service) option. This would of course have been nice, making it easier to just push our containers directly to a managed cloud, but nevertheless, provisioning servers and installing stuff on the servers is not a huge issue. Initially we started out with creating a Docker Swarm cluster. This worked well and all, but since we have discussed that we intended to move over to kubernetes, we decided to test that out.
So, during our research and testing phase, we started using Terraform. Terraform is a tool which “codifies APIs to configuration files”, this basically means that it allows the developers to write configuration files which then, when ran through terraform, can create, update and remove resources on remote machines. It talks to the providers API through providers, and luckily, UpCloud does have their own terraform provider.
Now, the provider from UpCloud is still a work in progress, the documentation is slim but due to it being open source, it’s easy to check what is available, and honestly, everything we could need in the aspect of provisioning new machines was available through the provider and their API.
This post will only go through the very basics of how to provision new machines through terraform, and in later posts, I will try to go through how to set firewall rules and tag the new servers and at the end show how we setup our K8S stage and production clusters.
Terraform configuration files
Terraform have a quite basic configuration language, even though it is basic it’s still quite powerful.
A full documentation overview can be found at the terraform documentation page, but I will try to explain all the parts when we get to it.
All terraform scripts (.tf) in a directory will be used when terraform is ran, they are parsed in alphabetic order, so I personally name them after the order I wish them to run: 01_variables.tf, 02_master.tf etc. You can use your own naming standard if you wish, I just find this very self-explanatory and easy.
There is two main commands that are used in terraform and the one that is used the most (while developing the scripts) is the plan command. Running terraform plan will show you all the resources that terraform will work with when you finally deploy the scripts, so run it quite often to see what will happen (or at the very least, before running the apply command!).
It’s customary to keep all variables inside its own file, I use a tf file for this, and put it as the first file to use (01_variables.tf, yes, I actually use a none-0-indexed conversion in this case even though it might feel wrong in some aspects!).
Defining a variable is quite simple:
variable "variable_name" {
type = "string"
default = "This is a string variable."
}
To use it inside the script later on all that has to be done is:
"${var.variable_name}"
All defined variables uses the var prefix, and if you omit the default in the var creation, terraform will treat it as a input value, that is, it will ask the user for the value. I like to use this for passwords and the likes, cause I don’t personally like to keep them in files.
The most important part of terraform scripts is the Resources, each resource is a piece of the script that will be created on the provider. It could be a full server, a firewall rule or even just a simple server tag. But it’s something that will be created.
A resource uses the following basic definition:
resource "provider_name" "resource_name" {
# Resource data.
}
The provider_name and resource_name should combined create a unique identifier, so all new resources should be named to something unique.
Now, you wont have to create a new resource for each new provisioned machine that is identical, that would be a pain and end up in huge files with a whole lot of copy paste problems. Instead of creating a new resource, the count meta variable can be used inside the resource declaration:
resource "provider_name" "mini_server" {
count = 1000
}
The above example will create 1000 instances of the mini_server resource. Maybe a bit too much, but yeah, you get the idea!
I will leave the rest of the terraform syntax until when it’s actually used in this post, but for now you should at least get the basic idea of how the system works.
Install terraform
Terraform is a single binary file, you could - if you wanted - build it from source, but I will not go through that process in this post, but rather just give pointers on how to install:
Go to the terraform download page, find your operating system and architecture and download the zip file.
Open the zip and extract the terraform binary from it.
On windows, you should probably place the terraform.exe file somewhere inside one of your Program Files directories, for example C:/Program Files/Terraform/terraform.exe, this path should also be added to the PATH environment variable to be able to reach terraform from anywhere.
On Linux-ish distributions, it can be extracted directly into any of the directories that will allow for usage of the program right away, I put mine in the /usr/local/bin directory, but anywhere you prefer should work just fine.
After that, terraform is “installed” on your computer.
Install the UpCloud provider
As I mentioned before, terraform uses providers to convert the API endpoints into the config files, UpCloud have their own provider, but it have to be installed to be pssible to use.
The UpCloud provider is written in Go, so the first step in this part is to install Go.
When you have installed go, make sure that the $GOPATH env variable is set, because we will use it later. If it is not set, check the go env GOPATH command output and set your $GOPATH to the same.
Go is quite nice when it comes to install dependencies, all you really have to do is tell go to get a package directly from a VCS as github, in this specific case, it’s hosted on github, so just call the following to install the dependency to the go dependencies directory:
go get github.com/UpCloudLtd/terraform-provider-upcloud
go install github.com/UpCloudLtd/terraform-provider-upcloud
Now the terraform provider for UpCloud will be installed in the $GOPATH/bin/ directory. To allow terraform to automatically load the provider, we should symlink it into the terraform plugin directory.
The directory might not exist yet, hence we make sure it does before we link the plugin.
# Create a new directory for the plugins (.terraform.d is the default dir for terraform)
mkdir -p $HOME/.terraform.d/plugins
# Link the plugin to the new directory
ln -s $GOPATH/bin/terraform-provider-upcloud $HOME/.terraform.d/plugins/terraform-provider-upcloud
And that’s it, now we have the provider installed and ready to use!
Create the first resource
A provider is only supposed to be loaded once, if multiple times, terraform will throw an error and the scripts wont run. So I add my provider declarations inside my 01_variables.tf file:
provider "upcloud" {}
If you wish, you could set the login credentials of your API user in the upcloud provider clause, but I don’t, I prefer to let terraform ask me for the password and username instead, if you do too, just leave it as above.
When we have declared the provider, we can start setting up resources. Our first resource will be a small server instance (the smallest there is on UpCloud!).
# The `upcloud_server` string is required, that is defined in the
# provider to let terraform know what kind of resource it is,
# in this case it is a "server" resource.
resource "upcloud_server" "my_first_resource" {
count = 1
zone = "de-fra1" # This is the zone identifier on upcloud (this one is Frankfurt1)
hostname = "my-first-resource"
cpu = "1" # 1 cpu
mem = "1024" # 1024 (1gb) ram
login { # In this clause, we can define how to log in to the server.
# Personally I hate using passwords, so we use a SSH key instead!
user = "my-name"
keys = [
"my-public-ssh-key"
]
create_password = false
# If you wish to use passwords, set the above to true and add the following:
# password_delivery = "sms" # which will send the generated password to your phone!
}
storage_devices = [ # In this clause we can define the storage devices of the machine
# this includes the Operating system device and all.
{ # Each device is in it's own object inside the array
tier = "maxiops" # This is the disk tier, "maxiops" is a fast drive, which is always nice for the OS.
size = 25 # 25 GB, all sizes is defined in GB.
action = "clone" # In this case we set it to clone, as we want to use a template,
# if we wanted to create a new one, we could use "create" instead.
storage = "Ubuntu Server 16.04 LTS (Xenial Xerus)" # A template name or identifier can be used.
# In this case, it's the latest LTS ubuntu version.
}
]
}
# We can here define some output that we want terraform to print to the terminal when done
# in this case, it will print the public IP address of the server (check the upcloud docs to see all possible values)
output "server_ip" {
value = "${upcloud_server.my_first_resource.ipv4_address}"
}
When the configuration script is created, enter the directory through the terminal and initialize terraform in the directory:
terraform init
When initializing terraform in the directory, a .terraform directory will be created there. Inside this, some local terraform settings is stored.
Feel free to check it out, but don’t edit it if you don’t know what you are doing. If you accidentally do, just remove the directory and run init again to go back to start!
Planning
When we have initialized the project, we should check so that everything is correct. This is done with the previously mentioned terraform plan command.
When executing it, we should get some output which I will try to describe below…
If you decided to go the same approach as me when it comes to credentials, the first two things that will happen is that the upcloud provider asks for a password and a username, this is your API enabled upcloud user. For safety, I would recommend creating a new user inside your account and give it access to the stuff that you wish it have, then enable the api access on only that user.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ upcloud_server.my_first_resource
id: <computed>
cpu: "1"
hostname: "my-first-resource"
ipv4: "true"
ipv4_address: <computed>
ipv4_address_private: <computed>
ipv6: "true"
ipv6_address: <computed>
login.#: "1"
login.2025795319.create_password: "false"
login.2025795319.keys.#: "1"
login.2025795319.keys.0: "my-public-ssh-key"
login.2025795319.password_delivery: "none"
login.2025795319.user: "my-name"
mem: "1024"
private_networking: "true"
storage_devices.#: "1"
storage_devices.0.action: "clone"
storage_devices.0.address: <computed>
storage_devices.0.id: <computed>
storage_devices.0.size: "25"
storage_devices.0.storage: "Ubuntu Server 16.04 LTS (Xenial Xerus)"
storage_devices.0.tier: "maxiops"
storage_devices.0.title: <computed>
title: <computed>
zone: "de-fra1"
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
This is the output from the terraform plan command. What it does is to describe what will happen when the apply command is executed.
So from top:
Terraform tells us that + means create, that is, any resource which is prefixed with a + sign, will be created on the provider. The upcloud_server.my_first_resource is prefixed with a + and that is correct, it should create a new resource for us!
After the resource name we can see a bunch of set values, the id is set to <computed>, which is correct, that means that the upcloud api will create the server id for us, something that we can’t and really should not be able to control. The hostname was defined in the resource definition and is correctly set to my-first-resource, when the server is provisioned, it will have that hostname.
We told upcloud that we wanted a public v4 IP, and due to not defining anything else, it also got a ipV6 address. We did not though define the actual ip to use, but rather let upcloud assign one automatically.
After IP addresses we can see our first user. It’s set to not use passwords, the name is my_user and the ssh key is set to the value we entered in the 01_variables.tf file (the above of course is not a valid ssh key). Our CPU and Memory is set to the correct values (1 cpu and 1gb of ram) (as you might see, they are not in the order we defined them, but rather in alphabetic order), the private network is enabled and our storage device is set to the values we defined in our resource declaration. The zone is “de-fra1” and that is also correct.
Now, if we want to provision the resource, we just run terraform apply. It takes a little bit for UpCloud to provision the new server, so we have to wait for the output… When it’s ready it will print the output we defined in the resource script, that is, the ipv4 address of the newly created server.
Open a terminal, write ssh my-name@<the ip from output> and you will be able to log on to the server that we just created!
If you wish to get rid of the server, just type terraform destroy and ALL the resources you have defined in the configuration will be removed.
Final words
As you might see, the terraform script does what we could have done through the UpCloud web interface. If you prefer it that way, go for it, but personally, I want as much as possible automated, that way the issues that comes in to play with the human factor is minimized; and that is a good thing in my humble opinion.
I really enjoy using terraform, it’s simple but powerful, and I will keep on using it - especially for initial provisioning - until I find something that is even better. I hope that this post gave some clarity to what terraform is and how to use it with UpCloud and as always, comment if you have any questions, critique or just want to spam my comment system (j/k!!!)!
In the next part of this series, we will check out some other parts of the UpCloud provider for creating firewall rules and tag servers, how to change already provisioned resources and I will then try to move forward to initialization scripts on the actual machines, to set up basic software. At the end of the series, the intention is that we should have the basic understanding of the whole process of setting up a new Kubernetes cluster using all automated configuration and a simple command line command to execute it all at once!