Ansible and UpCloud

This post is outdated and a new more up-to-date post can be found here!

In this post, I’ll introduce you to ansible and how to run ansible scripts on your development computer to install software on your remote servers, the post is a follow-up on the Terraform posts and we are still focusing on UpCloud.

The php-fpm version used in this writeup is php 7.0, if you use another version some of the commands have to be changed. I always recommend using the latest stable (7.3), but to keep this tutorial simple, I leave that part out.

What is ansible

Ansible is a tool used to deploy software (and provision servers if one wish to use that approach) on remote servers. The main reason to use it, is to make all the configuration of server software simple in configuration files and to be able to run the scripts for multiple machines at once.
This is exactly what we need, so Ansible should be a good product for us!

Installing ansible

Ansible requires python 2.7, so install that or let your package manager install it as a dependency if it can do that. In this part I will focus on linux-like environments and use Ubuntu as the system to make it easy for all of us.

apt-get install ansible

The Ansible playbook

Ansible files are written in the YAML language. If you have no earlier experience with yaml I’d recommend that you go read up on it a bit before continuing reading this post!

We will use something called playbooks in this tutorial, and we will stick to one single playbook and one single server. Due to the fact that ansible requires python running on the remote machine we start with a server that we deploy manually. Later on, we will install python in the terraform provision script, but this post is all about ansible for now!

Deploy a new ubuntu server and ssh into it.

ssh root@server-ip
apt-get update 
apt-get install python -y

Now we have a server with all the dependencies that is needed for the playbook to be able to run.

When working with playbooks, we need a file that includes all the servers that we want to work with, this is called an inventory file and to make it easy, we call it hosts for now.
The hosts file should be in the same directory as the playbook files will reside in and should contain the following:

[machines]
<server-public-ip>

When the hosts file is created, we can start looking on the ansible playbook.

The playbook is a yaml file containing all the tasks that should be run on the server, it always start with a --- at the top of the file and then the base is defined:

---
- name: My playbook
  hosts: machines
  remote_user: root
  become: true

In the above snippet we set the name of the playbook to My playbook, it could be whatever you want to call the playbook, it’s just a name.
After the name, we define which of the machines in the hosts file that should be included in the script, the [machines] bit in the hosts file indicates on which servers that are included in that specific configuration. You can add multiple groups like that, and you will when we later create more playbooks and stuff that should be installed on multiple servers that behaves differently.
We then define the user that will be doing the work on the remote machine, in this case we use the root user, just to make it simple, we also tell the script that we want to become the root user during the tasks.

The first task

I personally prefer to keep my playbooks and surrounding files inside a directory for each server type, for example, my SkyDns server resides in the ansible/skydns directory, in which I have all configurations for skydns, the etcd instances and such and the playbook for the software installation.
This is of course a personal preference, and one can do as they wish! Each of my playbooks contains all the setup that the specific server requires, even though it sometimes makes the files quite big.
A lot of code editors such as most IDEs and text editors as sublime, atom or vs-code have support for yaml files, which makes it a lot more easy to see what the files contains and allows for a better syntax highlighting than a standard text editor as notepad.

For the example, we will install applications to run a webserver with php and nginx, we will set up UFW and we will go through a few of the good stuff that ansible can give us. Due to this post being a part of a series, we will use the previouse terraform server script as the server that we work against, and we will pretend that its public ip address is 257.0.0.1 to keep it easy (that is a 100% invalid ipv4 address, I know).

First thing first, we update the hosts file with the webserver IP.

[webservers]
257.0.0.1

Now, the host file have a single entry under the webservers group, to make ansible use the webservers group when we deploy the scripts, we have to update the playbook directives to use it.

---
- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true

Each - name: ... part of a playbook can be seen as a group, you can use multiple in a single playbook or you can split them up into many if you prefer. The first task we will create is the firewall task, installation of UFW, then we will run a few commands to set up the rules.

  tasks:
    - name: Install UFW
      apt:
        name: ufw
        state: present

The apt directive tells Ansible to install the package using the apt package manager, the package goes under name and the state indicates that the package have to be present for the task to be successful.
Some distros have UFW installed by default, but if it is already present nothing will be done, so all good!

  - name: Setup rules
    command: "{{ item }}"
    with_items:
      - ufw default deny incoming
      - ufw allow ssh
      - ufw allow http
      - ufw allow https
      - ufw --force enable

In the above task we use a special Ansible feature. The Command we use uses a template string, the {{ item }} is specific, as you can add a with_items list so that ansible runs each of the things in the items list after each other. This allows for multi-command input in a single task, which makes this kind of stuff a whole lot easier to work with, it would be quite annoying to run each command in a task for itself.

Now UFW is installed and enabled, the 80, 443 and 22 ports are open and we are ready to install php and nginx.

  - name: Install php
    apt:
      name: php-fpm
      state: present
  - name: install nginx
    apt:
      name: nginx
      state: present

The above commands install php and nginx, it uses the same command as when we installed UFW so further description is not really needed.

Php fpm (FastCGI Process Manager) is a php fast-cgi version which is often used on servers, it’s easy to set up with nginx hence fitting for the example.

The php.ini file that php comes with should probably be modified, but that is a tutorial in itself, so we skip that for now, nginx though could use a default configuration that fits our server, and why not do that with ansibles template system jinja2!

PHP FPM have in earlier version had a setting enabled cgi.fix_pathinfo which made it quite insecure, it might be worth checking if the version of fpm you run have this or not, or either way, you might want to set it to cgi.fix_pathinfo=0 to be sure.

Being able to use templates when provisioning a server is something wonderful. You always want to be able to reuse files, so templates is really the way to go. Our nginx config will be quite simple, but here goes:

user www-data; # The user that runs the webserver.
events {}      # Events directive is required, else server wont start.

http {

  server {
    # Redirect all port 80 to port 443 for TLS only!
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://$host$request_uri;
  }

  server {
    listen      443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    ssl_certificate     {{ cert_path }};
    ssl_certificate_key {{ key_path }};

    server_name {{ servername }};

    root /var/www/html;
    index index.html index.php;

    # This is the php fast-cgi settings.
    location ~\.php$ {
        try_files $uri =404; # This directive makes sure that the file exist, else it will return a 404 - not found.
        fastcgi_pass  unix:/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include       fastcgi_params;
    }
  }

}

We can also create an example php file that can be added to the server to show that it all works as intended:

<?php
echo 'Hi! You succeeded in provisioning a server with nginx and php with the use of ansible!';

And thats a very basic setup that should work for this example. As i mentioned before, i like to place the configuration files in the same directory as the playbook, jinja2 scripts are usualy saved with the intended name with the extra j2 file type, so now, in the directory we have the following:

ansible /
  webserver /
    playbook.yml
    nginx.conf.j2
    index.php

Copying configuration is quite simple but before we copy the file, we update the header of the file to include some variables that we want injected into the nginx configuration!

- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true
  vars:
    servername: my.domain.tdl
    cert_path: /etc/ssl/certs/my.domain.tdl.crt
    key_path: /etc/ssl/certs/my.domain.tdl.key

The variables we defined will - through the use of jinja2 be injected into the scripts when we move them to the server.

The cert_path and key_path is to the TLS certificate and key that the server uses, and due to this being a ansible specific tutorial I will skip the part with creation of tls certificates.

Moving a file that already exist uses the template task directive, there is a lot of stuff that you can do through the directive, but in this case we will just move file from localhost to remotehost.

  - name: Copy nginx configuration.
    template:
      src: nginx.conf.j2
      dest: /etc/nginx/nginx.conf
  - name: Copy php file!
    template:
      src: index.php
      dest: /var/www/html/index.php

Now both our files are moved and jinja2 converted the {{ name }} variables into the values that we specified in the vars directive in the header.

After that, just add the tasks to reload nginx and the server should be up and running!

  - name: Start nginx
    service:
      name: nginx
      state: restarted

And when all is done, we can run the scripts:

ansible-playbook playbook.yml -i hosts

Visit the site at its ip or domain and we got ourself a server running nginx and php!

Full example playbook

---
- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true
  vars:
    servername: my.domain.tdl
    cert_path: /etc/ssl/certs/my.domain.tdl.crt
    key_path: /etc/ssl/certs/my.domain.tdl.key

  tasks:
    - name: Install UFW
      apt:
        name: ufw
        state: present
    - name: Setup rules
      command: "{{ item }}"
      with_items:
        - ufw default deny incoming
        - ufw allow ssh
        - ufw allow http
        - ufw allow https
        - ufw --force enable
    - name: Install php
      apt:
        name: php-fpm
        state: present
    - name: install nginx
      apt:
        name: nginx
        state: present
    - name: Copy nginx configuration.
      template:
        src: nginx.conf.j2
        dest: /etc/nginx/nginx.conf
    - name: Copy php file!
      template:
        src: index.php
        dest: /var/www/html/index.php
    - name: Start fpm
      service:
        name: php7.0-fpm
        state: started
    - name: Start nginx
      service:
        name: nginx
        state: started

Final words

Ansible is a great and powerful tool, in this part I just show the very basic of what ansible is to give a short introduction to it, when we later provision the servers that will run the K8S cluster, the files will be a whole lot more complex. I will in the next post go through how to let terraform run the scripts after creating the servers, so that terraform takes care of all the work through its apply command. One could choose to run terraform and then run the ansible scripts, but I personally prefer to have as few clicks as possible!

Also remember that the above script collection is NOT suitable to run on a production server right away, this is just an example!

As always, if you find any issues with the tutorial, have any input, any critique or just want to say hi, don’t hesitate to comment below!