How To Solve The Problem Of Centralized Let’s Encrypt Certificate by using Ansible
Hello everyone!
In this article, we are going to describe how we solved the problem of centralized certificate updating Let’s encrypt and we will manage the infrastructure using ansible.
In our decision we will use:
- Ansible
- Rsync, rsyncd
- Inotify, incron
- Certbot
- Nginx
We will propose two versions of the architecture, in which our solution can be useful. In turn, you can offer your options in the comments.
Option 1: You have several front-end servers with public ip (for example, 3), serving multiple domains. These domains can be added/deleted. In order not to monitor each of the frontend servers – it’s more convenient to do this on one letsencrypt server:
Option 2: You only have one server with public ip, and you need certificates on the servers inside the network:
Role description ansible
Repository with roles is available by reference.
There are 4 roles in the repository:
- nginx-simple
It installs nginx on all hosts and copies the basic configs. The role itself does not start from the playbook. It is triggered by meta dependencies from other roles. - lets encrypt-server
Configures rsyncd on the letsencrypt server host. In meta dependencies has the role of nginx-simple. Accordingly, first install nginx, and then the role of the letsencrypt server will be lost. - incron
Sets the necessary packages for incron and copies the basic config. The role also does not run as straight as nginx-simple. - front
In meta dependencies, it has the roles incron and nginx-simple. After them, the front role copies the nginx configs necessary for example.com adds the task in cron to pick up new certificates with a letsencrypt server and the task in incron to check the file changes and execute nginx -s reload
Let’s turn to practice:
Initial Configuration
Initially, we have:
- One server for the centralized issuance of SSL certificate (letsencrypt server)
- And, one or more publicly accessible servers with nginx (front)
On all servers installed Ubuntu 16.04.
Installing and Configuring nginx
First, install all nginx hosts from the general nginx-simple role, and distribute nginx configurations that are common for all hosts (nginx.conf, SSL parameters, certificate paths, etc).
For let’s encrypt server in the template … /sites-available/default.conf the .well_known folder will be located in /var/www/:
{% if letsencrypt_server %}
location /.well-known {
root /var/www/;
}
For the server/servers of the front group, since the .well_known folder is used not only for certificates but for other software, we import example.conf letsencrypt-proxy.conf into the config.exe config and nginx will look for the folder locally on the front server using the try_file :
{% if nginx_proxy_well_known %}
try_files $uri $uri/ @letsencrypt;
{% endif %}
Configurations for the domain will be populated depending on the variables from the inventory. In the repository, this is an example.com domain
Also, depending on the variable letsencrypt server, the role of nginx-simple will install on the let’s encrypt server certbot and add the cron task to update the certificates.
Getting a certificate
Since we solved this problem before the appearance of the wildcard certificate from Let’s Encrypt, we will consider both options for obtaining a certificate.
On the server lets encrypt server we perform:
certbot certonly --agree-tos -d example.ru --webroot -w /var/www/
If there is more than one domain, we add the subsequent ones using the -d switch.
To obtain a wildcard certificate, we will need to add records to the DNS TXT. At the moment, this is the only way to obtain such a certificate:
certbot certonly --agree-tos -d example.ru -d *.example.ru --preferred-challenges dns --manual --server https://acme-v02.api.letsencrypt.org/directory
Certbot will write what TXT records you will need to add.
Updating Certificates
We got the certificates, it remains to configure them to copy to the front server/server.
To do this, we will configure rsyncd on the let’s encrypt server with reading permissions for a restricted list of ip addresses:
hosts allow = {{ hosts_allow }}
hosts deny = *
list = true
use chroot = no
[cert]
path = /etc/letsencrypt/live/
uid = root
gid = root
read only = true
Every 5 minutes cron task from front servers will check if certificates are updated and pick them up. Because certificates are rotated, there are symlinks in the /etc/letsencrypt/live/{{domain}} folder.
Add the -L switch to pull out the original files:
/usr/bin/rsync -zavL --chmod=D0750,F640 --delete rsync://{{ hostvars['letsencrypt-server'].ansible_eth0.ipv4.address }}/cert /etc/letsencrypt/live/
Hook for nginx
We set up nginx, got certificates, took them to the front server. It remains to determine that the files in the /etc/lets encrypt/live/ {{domain}} folder have changed and execute nginx -s reload
- The subsystem of the kernel Linux inotify and the incron daemon help us in this.
The role of incron will install the necessary packages, and from the front role template, the task for monitoring the certificates and the desired hook will be added:
/etc/letsencrypt/live/{{ domain }}/ IN_CREATE,IN_DELETE,IN_MODIFY,IN_MOVED_TO nginx -s reload
Conclusion
We tried to describe in detail the entire installation and configuration process, all described in the playboy ansible – the article was very compact. As it often says, “a little more than 100 lines of code.” We will answer questions, criticisms, and comments with pleasure in the comments.
