Ansible!

This article is part 3 in a series: Terraform, Ansible and UpCloud

• 1 - Terraform and UpCloud
• 2 - Terraform and UpCloud Part 2
• 3 - Ansible!

This post is an updated version of this post. A lot have changed both for me and with ansible and terraform, so a renewal of the series was due!
The post is also part of my Ansible, Terraform and UpCloud series, although this one is more of a general ansible featured post.

In this post, I’ll introduce you to ansible and how to run ansible scripts on your development computer to install software on your remote servers, the post is a follow-up on the Terraform posts and we are still focusing on UpCloud.

What is ansible

Ansible is a tool which is used to automate tasks such as server provisioning, software installation and basically anything you dont want to do manually. Ansible is a lot like Terraform in some ways and one could use either or, personally though I prefer to use both, where I use Terraform for the actual server provisioning while I use Ansible to install and configure the servers.
I will in a later post go through how to mix the two and make Terraform start your ansible scripts for you, but for now, we focus on plain ansible.

Installing ansible

Ansible requires python, so make sure you have that installed! In this part I will focus on linux-like environments and use Ubuntu as the system to make it easy for all of us.

apt-get install ansible

The Ansible playbook

Ansible files are written in the YAML language. If you have no earlier experience with yaml I’d recommend that you go read up on it a bit before continuing reading this post. Yaml is a JSON superset and you can actually use JSON directly in a YAML file if you want. The syntax is quite easy to read and write though, so in my posts, I will use yaml instead of json.

We will use something called playbooks in this tutorial, and we will stick to one single playbook and one single server. Due to the fact that ansible requires python running on the remote machine we start with a server that we deploy manually. Later on, we will install python in the terraform provision script, but this post is all about ansible for now!

Deploy a new ubuntu server and ssh into it.

ssh root@server-ip
apt-get update 
apt-get install python -y

Now we have a server with all the dependencies that is needed for the playbook to be able to run.

When working with playbooks, we need a file that includes all the servers that we want to work with, this is called an inventory file and to make it easy, we call it hosts for now.
The hosts file should be in the same directory as the playbook files will reside in and should contain the following:

[machines]
<server-public-ip>

When the hosts file is created, we can start looking at the playbook.

The playbook is a yaml file containing all the tasks that should be run on the server, it always start with a --- at the top of the file and then the base is defined:

---
- name: My playbook
  hosts: machines
  remote_user: root
  become: true

In the above snippet we set the name of the playbook to My playbook, it could be whatever you want, it’s just a name.
After the name, we define which of the machines in the hosts file that should be included in the script, the [machines] bit in the hosts file indicates on which servers that are included in that specific configuration. You can add multiple groups like that, and you will when we later create more playbooks and stuff that should be installed on multiple servers that behaves differently.
We then define the user that will be doing the work on the remote machine, in this case we use the root user, just to make it simple, we also tell the script that we want to become the root user during the tasks.

The first task

The best directory structure to use with ansible is to keep all the playbooks in the same directory. Name them after the server type. We will in later posts take a look at groups and other things that will be used by all the playbooks, they will be placed in subdirectories.

As we use no groups in this post, we will put everything in the same file, later on we split it up and make the playbooks share all common tasks.

A lot of code editors such as most IDEs and text editors as sublime, atom or vs-code have support for yaml files, which makes it a lot more easy to see what the files contains and allows for a better syntax highlighting than a standard text editor as notepad, some of them also supplies plugins to allow for autocompletion and other type of help in ansible files, might be worth checking out!

In this example, the script we write will install applications to run a webserver with PHP and Nginx. We will set up UFW and we will go through some of the features in ansible.
The server already exists and it is an ubuntu server. We pretend that its IP is 257.0.0.1.

First thing first, we update the hosts file with the webserver IP.

[webservers]
257.0.0.1

Now, the host file have a single entry under the webservers group, to make ansible use the webservers group when we deploy the scripts, we have to update the playbook directives to use it.

---
- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true

The - name: ... part of the playbook “header” is used as a description and will be displayed when ansible runs. Only use one of them in a single playbook as you will want to keep your playbooks as clean and decoupled as possible.
Each task does also have a name property, which each and every task should have, just as with the playbook name, it will be printed during ansible run.

The first task we will create is the firewall task, installation of UFW, then we will run a few commands to set up the rules.

  tasks:
    - name: Install UFW
      apt:
        name: ufw
        state: present

The apt directive tells Ansible to install the package using the apt package manager, the package goes under name and the state indicates that the package have to be present for the task to be successful.
Some distros have UFW installed by default, but if it is already present nothing will be done, so all good!

There are some ways to make the installation use other package managers (such as yum or apk etc) but in this part we use ubuntu and keep it ubuntu specific.

  - name: Setup rules
    command: "{{ item }}"
    with_items:
      - ufw default deny incoming
      - ufw allow ssh
      - ufw allow http
      - ufw allow https
      - ufw --force enable

In the above task we use a special Ansible feature. The Command use a placeholder (the {{ item }} part). When using a placeholder like that, we can pass a set of values to the tasks, which it will then run one by one. To pass the values, we use the with_items property.

Now UFW is installed and enabled, the 80, 443 and 22 ports are open and we are ready to install php and nginx.

  - name: Install php
    apt:
      name: php-fpm
      state: present
  - name: install nginx
    apt:
      name: nginx
      state: present

The above commands install php and nginx, it uses the same command as when we installed UFW so further description is not really needed.

Php fpm (FastCGI Process Manager) is a php fast-cgi version which is often used on servers, it’s easy to set up with nginx hence fitting for the example.

The php.ini file that php comes with should probably be modified, but that is a tutorial in itself, so we skip that for now, nginx though could use a default configuration that fits our server, and why not do that with ansibles template system jinja2!

PHP FPM have in earlier version had a setting enabled cgi.fix_pathinfo which made it quite insecure, it might be worth checking if the version of fpm you run have this or not, or either way, you might want to set it to cgi.fix_pathinfo=0 to be sure.

Being able to use templates when provisioning a server is something wonderful. You always want to be able to reuse files, so templates is really the way to go. Our nginx config will be quite simple, but here goes:

user www-data; # The user that runs the webserver.
events {}      # Events directive is required, else server wont start.

http {

  server {
    # Redirect all port 80 to port 443 for TLS only!
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://$host$request_uri;
  }

  server {
    listen      443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    ssl_certificate     {{ cert_path }};
    ssl_certificate_key {{ key_path }};

    server_name {{ servername }};

    root /var/www/html;
    index index.html index.php;

    # This is the php fast-cgi settings.
    location ~\.php$ {
        try_files $uri =404; # This directive makes sure that the file exist, else it will return a 404 - not found.
        fastcgi_pass  unix:/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include       fastcgi_params;
    }
  }

}

We can also create an example php file that can be added to the server to show that it all works as intended:

<?php
echo 'Hi! You succeeded in provisioning a server with nginx and php with the use of ansible!';

In this example, we put the template in the same directory as the playbook just to keep it simple. There are other places where it is preferred to be placed, but we take that in a later post.

When saving a template file, use the j2 (jinja2) file type. I usually name my files as they would be named when used and append .j2.

Our directory should look something like this now:

ansible /
  playbook.yml
  nginx.conf.j2
  index.php

Copying configuration is quite simple but before we copy the file, we update the header of the file to include some variables that we want injected into the nginx configuration!

- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true
  vars:
    servername: my.domain.tdl
    cert_path: /etc/ssl/certs/my.domain.tdl.crt
    key_path: /etc/ssl/certs/my.domain.tdl.key

The variables we defined will - through the use of jinja2 be injected into the scripts when we move them to the server.

The cert_path and key_path is to the TLS certificate and key that the server uses, and due to this being an ansible specific tutorial I will skip the part with creation of tls certificates.

When copying a template from the local computer running ansible to a remote host is done using the template task type. When doing this, all the variables defined in ansible will be available for the template and any placeholder will be replaced with the real value.
Moving a file that is not a template can use the file task type instead.

  - name: Copy nginx configuration.
    template:
      src: nginx.conf.j2
      dest: /etc/nginx/nginx.conf
  - name: Copy php file!
    file:
      src: index.php
      dest: /var/www/html/index.php

Now both our files are moved and jinja2 converted the {{ name }} variables into the values that we specified in the vars directive in the header.
After that, just add the tasks to reload nginx and the server should be up and running!

  - name: Start nginx
    service:
      name: nginx
      state: restarted

And when all is done, we can run the scripts:

ansible-playbook playbook.yml -i hosts

Visit the site at its ip or domain and we got ourself a server running nginx and php!

Full example playbook

---
- name: Webserver installation
  hosts: webservers
  remote_user: root
  become: true
  vars:
    servername: my.domain.tdl
    cert_path: /etc/ssl/certs/my.domain.tdl.crt
    key_path: /etc/ssl/certs/my.domain.tdl.key

  tasks:
    - name: Install UFW
      apt:
        name: ufw
        state: present
    - name: Setup rules
      command: "{{ item }}"
      with_items:
        - ufw default deny incoming
        - ufw allow ssh
        - ufw allow http
        - ufw allow https
        - ufw --force enable
    - name: Install php
      apt:
        name: php-fpm
        state: present
    - name: install nginx
      apt:
        name: nginx
        state: present
    - name: Copy nginx configuration.
      template:
        src: nginx.conf.j2
        dest: /etc/nginx/nginx.conf
    - name: Copy php file!
      template:
        src: index.php
        dest: /var/www/html/index.php
    - name: Start fpm
      service:
        name: php7.0-fpm
        state: started
    - name: Start nginx
      service:
        name: nginx
        state: started

Final words

Ansible (just as with Terraform) is very powerful, it got a whole lot of features that would be impossible to cover in just one post, but the above should give you a hint on how it works and why it might be a nice addition to your toolbox!
In the next post I will try to walk through how to start ansible playbooks from terraform so that the two work together to provision and install servers.

Also remember that the above script collection is NOT suitable to run on a production server right away, this is just an example!

As always, if you find any issues with the tutorial, have any input, any critique or just want to say hi, don’t hesitate to comment below!