Certificates: Let's Encrypt and auto-renewal

Why this matters

In the previous lesson, the server proved its identity with a certificate signed by a certificate authority (CA). Certificates like that used to be bought and renewed by hand once a year. Now there's Let's Encrypt: a CA that issues certificates for free and automatically.

Almost every service you'll maintain has a program that issues and renews its certificate. When that program breaks, the site goes down for every visitor at once, on the same day. You need to understand this chain before it breaks.

Lifetimes keep getting shorter

Let's Encrypt certificates live for 90 days by default. Nobody manages to renew them by hand every three months, so the whole process gets automated.

A short lifetime isn't a quirk of one CA anymore. Industry requirements (the CA/Browser Forum, ballot SC-081) cut the maximum lifetime of a public TLS certificate on a schedule:

Starting from	Maximum lifetime
March 15, 2026	200 days
March 15, 2027	100 days
March 15, 2029	47 days

Let's Encrypt moves even faster. From February 10, 2027, certificates will default to 64 days, and from February 16, 2028, to 45 days. You can already request a 45-day certificate (the tlsserver profile) or a 6-day one (the shortlived profile).

Nobody keeps up with reissuing a 45-day certificate by hand: miss one window and the site goes down. Let's Encrypt also stopped sending expiration-warning emails on June 4, 2025. Automating issuance is now mandatory, and keeping it running is your job.

ACME: proving you own the domain

Before issuing a certificate for example.com, a CA has to confirm you actually control that domain. The protocol a client and a CA use to agree on this is called ACME. The most commonly installed ACME client is called certbot.

The most common challenge type is HTTP-01:

1. The client (certbot) requests a certificate for example.com.
2. Let's Encrypt issues a random token.
3. Certbot drops a file named after the token at the path
   http://example.com/.well-known/acme-challenge/<token>
   The file holds the token and the ACME account key's fingerprint.
4. Let's Encrypt itself visits that address and checks the content.
5. It matches: the CA issues the certificate.

Only whoever controls the server the domain points to can put a file there. That's the whole basis of the check.

HTTP-01 has limits. The check only goes to port 80; you can't point it at another port. Let's Encrypt follows redirects (up to 10, only to http or https, and only to ports 80 and 443), so a blanket HTTP-to-HTTPS redirect doesn't block the check.

The second method, DNS-01, needs a TXT record, _acme-challenge.<domain>, in DNS instead of a file. It's needed for wildcard certificates (*.example.com) and for servers that aren't reachable on port 80 from outside. The cost is that the client needs access to your DNS API. There's also a third method, TLS-ALPN-01 on port 443, but it's mostly used by reverse proxy authors and large hosting providers.

certbot in practice

You can get a certificate for nginx with one command:

certbot --nginx -d example.com -d www.example.com

The --nginx plugin runs the ACME check itself, gets the certificate, wires the paths into nginx's config, and reloads it.

Another common option is webroot. Certbot only drops the token file into a folder that a running web server is already serving, and doesn't touch the config:

certbot certonly --webroot -w /var/www/certbot -d example.com

The file lands at /var/www/certbot/.well-known/acme-challenge/<token>. So the web server has to serve /.well-known/acme-challenge/... from that exact folder, or the check gets a 404.

The finished files land in /etc/letsencrypt/live/example.com/:

cert.pem        the server's certificate
chain.pem       the CA's intermediate certificate
fullchain.pem   cert.pem and chain.pem in one file
privkey.pem     the private key, keep it secret

nginx's ssl_certificate directive loads fullchain.pem. Browsers trust the root CA, but might not have the intermediate one. The server must send it itself, or the chain of trust breaks.

Auto-renewal

The certbot package installs a timer (a systemd timer or cron) that runs certbot renew twice a day. Starting with certbot 4.0, the certificate renews once less than a third of its lifetime is left. For a 90-day one, that's roughly 30 days before expiry.

Check that renewal will work without touching the production certificate:

certbot renew --dry-run

This command runs the whole process against Let's Encrypt's test server (staging) and saves nothing to disk. If it passes, certificate issuance is configured correctly.

But new files on disk don't yet mean the site serves the new certificate. A web server reads the certificate at startup and when its config reloads, then keeps it in memory. The --nginx and --apache plugins reload the server themselves. With webroot, certbot doesn't do that, and you add the reload with a deploy hook: a command certbot runs after every successful certificate issuance.

certbot renew --deploy-hook "systemctl reload nginx"

Or an executable script in /etc/letsencrypt/renewal-hooks/deploy/, which certbot runs on every renewal by itself. nginx's reload rereads the config and certificates without dropping current connections, while restart drops them.

Common mistakes

ssl_certificate loads cert.pem instead of fullchain.pem. The site often still opens in a browser: browsers can fetch the intermediate certificate themselves from a link in the server certificate, or already know it. But curl, clients in code, and other services don't do that and fail with an error like unable to get local issuer certificate.

Certbot renewed the files, but the site serves the old certificate. The server wasn't reloaded after the renewal. A couple of months later the old certificate expires, even though certbot renew reports success every time.

Webroot doesn't match what the web server actually serves. Certbot writes the token to one folder, and nginx looks for the file in another. Let's Encrypt gets a 404, and certbot's response shows Invalid response along with the token's address.

Port 80 is blocked by a firewall "because we only use HTTPS." HTTP-01 only goes to port 80, so the check fails. Either open port 80 or switch to DNS-01.

What you'll need for the tasks

The labs use a lab CA instead of Let's Encrypt: its own root certificate, root.pem, and an intermediate one. The domain site.lab resolves to 127.0.0.1 through the curl --resolve flag, so you don't need to touch /etc/hosts. In the first lab, instead of certbot there's its lab stand-in, setup/certbot.sh, with the same certonly and -d.

• curl --cacert <root CA> --resolve site.lab:<port>:127.0.0.1 https://site.lab:<port>/
• openssl s_client -connect 127.0.0.1:<port> -servername site.lab -showcerts (shows which certificates the server sends in the handshake)
• openssl x509 -in <file> -noout -subject -issuer -dates
• openssl x509 -in <file> -noout -text: the Subject Alternative Name block lists the names the certificate was issued for. A client checks the site's address only against this list
• docker compose -f setup/docker-compose.yml exec web nginx -t and ... nginx -s reload
• docker compose -f setup/docker-compose.yml logs web

In the next lesson we'll cover the reverse proxy: it's what actually serves this certificate, terminating TLS in front of the application.