Combine Ansible & Terraform

This article is part 4 in a series: Server setup on UpCloud

• 1 - Terraform and UpCloud
• 2 - Terraform and UpCloud Part 2
• 3 - Ansible and UpCloud
• 4 - Combine Ansible & Terraform
• 5 - Etcd and SkyDNS on UpCloud.
• 6 - Etcd cluster for Kubernetes
• 7 - Firewall with ansible

Having two systems to create new servers with software feels a bit much, especially when both ansible and terraform have the capability to actually do all the work by them selves. I like to use terraform for the initial server setup, but ansible is a lot easier to configure for software setup, but one command should be enough, and that’s why we add a few lines of code to make terraform run the ansible scripts during the setup phase.

local-exec and remote-exec

Terraform have two built in provisioners that are good for doing other stuff than calling APIs. The local-exec provisioner executes commands on the local machine and the remote-exec provisioner executes commands on a remote machine. As you might remember, ansible requires Python to be installed on the remote machine as well as the local machine. Terraform does not, not only that, terraform even have its own SSH functionality! If we already had python installed on the remote host, we could use local-exec and run the ansible scripts right away, but we don’t (at least not without using specific python templates or doing manual stuff). Further on, if we did that, we would have to wait for the server to be fully set up and running before executing the ansible scripts. As it seems, local-exec will not wait for the server as remote-exec do (as it tries to connect via ssh till the server is available). So instead of running the playbook directly, we use the remote-exec provisioner to connect to the server and install python, then, when python is installed, we run the ansible scripts via the local-exec provisioner.

Remote

The definition of the remote-exec provisioner is quite simple, we defined it inside the resource that the ansible scripts belong to, at the end, after the resource details have been set. The commands that we wish to run is defined in a variable called inline, it could be either a single string or an array of strings, which, if an array, will be executed after each other, all commands added there will run on the remote host. Inside the remote-exec scope we also need to define a connection object.

The minimum information we need to provide is the host address, the connection type (ssh), the remote user to run the commands as and a ssh key which have a public counterpart added to the users authorized keys on the server. In this example, we use the id_rsa private key (the default name for the private key) and it should be located in the users .ssh directory. The file function that terraform supplies is used to load the key as a string value from the key file. Also, remember to set the permissions of the private key to 0600!

provisioner "remote-exec" {
    inline = [ # I prefer to define the inline commands as an array even if it's just one command.
        "apt-get -qq install python -y",
    ]

    connection {
        host        = "${self.ipv4_address}" # The `self` variable is like `this` in many programming languages
        type        = "ssh"                  # in this case, `self` is the resource (the server).
        user        = "root"
        private_key = "${file('~/.ssh/id_rsa')}"
    }
}

When the terraform apply command is invoked, terraform will now connect to the server and install python to it at the end of the provision script via remote-exec.

Local

The remote-exec provisioner could be used for all software installation, and that way we could just get rid of ansible all together, but as I said earlier, ansible have a nicer way of defining dependencies, and it have quite good control over what have been done on the server through its configuration settings (if, for example the apt object is set in ansible and the state is set to present, it will register that the program is already installed and then not bother installing it again). So, instead of using the remote-exec provisioner, we invoke ansible via the local-exec provisioner.

In this example, I have put all the ansible files inside a directory which resides in the same directory as the terraform scripts, the directory is called Ansible and the playbook we use is called playbook.yml. To allow ansible to connect to the remote server, we need to give ansible the private key to use and we also need to define an inventory. As the server is a newly created server, the ip-address to connect to is unknown, so we cant add the ip-address to a hosts file (we could write it to the file, but that is not a great way of doing it), fear not though, ansible allow for a list of hosts to be passed as a inventory when invoking the ansible-playbook command!

Defining a local-exec provisioner is as simple as the remote-exec, but it takes a few other arguments. As the playbooks are placed inside the Ansible directory, we use a working_dir variable to tell ansible where it should do its work. By the environment object, we can set environment variables to be used while the script is running, variables that ansible can use instead of using ansibles --extra-vars to set variables (which of course is a legal way of doing it too!).

To run the local command, we use a command variable, in which we just tell ansble to run the playbook, in this case, we also pass the private key and as I said earlier, the list of ip-addresses that the host or hosts have. To make it clear to ansible that it is a list of addresses that it is handed (even if it is just one), add a , after the ip.

provisioner "local-exec" {
    environment {
        PUBLIC_IP  = "${self.ipv4_address}"
        PRIVATE_IP = "${self.ipv4_address_private}"
    }

    working_dir = "../Ansible/"
    command     = "ansible-playbook -u root --private-key ${var.ssh_key_private} playbook.yml -i ${self.ipv4_address},"
}

If we run the terraform apply command at this point, we will first provision a new server, install python via the remote-exec provisioner and then, via the local-exec provisioner tell ansible to do its magic!

Resulting code

Directory structure:

Src /
  Provision.tf
  Ansible /
    playbook.yml
    nginx.conf.j2
    index.php

Final words

I would recommend that you in either the ansible script or a local-exec provisioner clause remove and then add the server host keys, that way you will not have to manually tell the script that you accept the host keys, and you can make sure that the ansible tasks don’t fail because the host is wrong (if you had to re-provision the server and it uses the same ip).

Ansible and terraform are two extremely powerful tools, what I have shown until now is just scraping on the surface of the magic it can do, and to be honest, even if I call it magic, it really isn’t. Play around with the tools, provision servers and install software, after a while you will notice how easy it becomes to work with remote machines without even having to access the machines in question!

As always, if you find any issues in the tutorial, have any questions, comments or just want to say hi! Do not hesitate to comment below!