DNS

Why DNS

Machines talk to each other over IP addresses. People are bad at remembering those. DNS (Domain Name System) is a distributed directory that turns a name into an IP address.

Without DNS you'd have to hardcode IPs into every link and config, and update them everywhere whenever a server changes. DNS gives you a stable layer of names on top of addresses that keep changing. For DevOps it's also a frequent source of incidents: "the site is down" often just means "the name resolves to the wrong place."

The addresses in this lesson's examples are made up, from the 192.0.2.0/24 range that RFC 5737 reserves for documentation. The real example.com currently resolves to different addresses, and they change from time to time.

How resolution works

The browser wants to open www.example.com. Here's what happens:

1. /etc/hosts: local overrides. The order of sources is set by /etc/nsswitch.conf, and files (that is, /etc/hosts) usually comes before dns. So an entry in /etc/hosts overrides any DNS answer.
2. The local resolver or OS cache: "seen this name recently?" If yes, and the TTL hasn't expired yet, the answer comes straight from the cache.
3. The DNS resolver from /etc/resolv.conf. Usually that's a router, a corporate DNS, or a public one (8.8.8.8, 1.1.1.1).
4. The recursive resolver does the legwork itself:
   • asks the root servers: "who knows about .com?",
   • asks the .com TLD server: "who knows about example.com?",
   • asks the example.com authoritative server: "what's the IP of www.example.com?",
   • returns the answer along with its TTL and caches it.
5. The browser gets the IP and opens a TCP connection.

In reality the cache kicks in almost every time: someone on your resolver already asked for this name a minute ago.

Record types

The main ones you'll run into at work:

Type	What it holds	Example
A	IPv4 address	example.com → 192.0.2.10
AAAA	IPv6 address	example.com → 2001:db8::1
CNAME	alias for another name	www.example.com → example.com
MX	mail server and its priority	10 mx1.example.com: where to send mail for @example.com
TXT	arbitrary text	SPF, DKIM, DMARC, domain-ownership checks
NS	name server	which DNS servers are authoritative for the zone
PTR	reverse resolution (IP → name)	logs, filters

CNAME works differently from the rest. Once the resolver gets one, it follows the chain and queries records for the alias's target instead. That's why dig on a name with a CNAME shows two lines: the CNAME itself and the target's A record.

Per RFC 1034, CNAME can't coexist with other records on the same name. And the zone apex (example.com, with no prefix) always has mandatory SOA and NS records, so a CNAME at the zone apex is forbidden. DNS providers get around this with a virtual ALIAS/ANAME record, which looks like an ordinary A record from the outside.

An MX record has a number before the server name, the preference. Mail is first tried against the server with the lower number, the rest serve as backup (RFC 5321, section 5.1).

A single domain often carries several TXT records at once: SPF (v=spf1 ...), DKIM (the public key for mail signing), DMARC (v=DMARC1; p=reject; ...), plus verification records for Google, Microsoft, and other services.

TTL and caching

Every record comes with a TTL (time-to-live) in seconds: how long the resolver is allowed to cache it. Typical values:

• 60-300 s: for records that change often (load balancers, CDNs);
• 3600 (an hour): a common default;
• 86400 (a day): for static records.

Before a migration (an IP change, a mail move) the TTL gets lowered to 60-300 s a day or two ahead, so old answers expire faster. After the migration it gets raised back.

dig

dig example.com                 # A record + the full answer
dig example.com +short          # just the address
dig example.com MX              # MX records
dig example.com TXT             # TXT records
dig @8.8.8.8 example.com        # ask a specific resolver
dig +trace example.com          # walk the whole chain from root

Read the full dig output starting from the ANSWER SECTION. Every line there has the same shape: name, TTL, class (IN), record type, value.

;; ANSWER SECTION:
www.example.com.    300    IN    CNAME    example.com.
example.com.        300    IN    A        192.0.2.10

At the end of the output, the SERVER line shows which resolver answered. The aa flag in the header means the answer came from the authoritative server, not from someone else's cache.

dig +trace is useful when you suspect one of the resolvers is serving a stale cache, or something is broken at the delegation step.

The older nslookup tool still shows up, but dig gives you more to work with.

dig isn't part of a base install: on Debian and Ubuntu it comes with the bind9-dnsutils package, on RHEL and Rocky with bind-utils, on Alpine with bind-tools. On someone else's server, where you can't install anything, getent hosts does a simple "does this name resolve" check.

$ getent hosts example.com
192.0.2.10      example.com  example.com

There's an important difference between them. dig always goes straight to the DNS server and knows nothing about /etc/hosts. getent hosts takes the same path as applications: through /etc/nsswitch.conf, meaning /etc/hosts first, then DNS. If dig and getent answer differently for the same name, check /etc/hosts first.

/etc/resolv.conf and /etc/hosts

/etc/resolv.conf tells the system which DNS to use. A minimal version:

nameserver 8.8.8.8
nameserver 1.1.1.1
search example.com

search appends a suffix to short names: db becomes db.example.com.

On Ubuntu (and anywhere systemd-resolved is enabled), editing this file by hand is pointless. There, /etc/resolv.conf is a symlink to a systemd-resolved file, with a single nameserver 127.0.0.53 inside: a local stub that knows the real servers itself. Your edits get wiped out on the next network settings update.

$ ls -l /etc/resolv.conf
/etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf
$ resolvectl status         # which servers are actually used

resolvectl status shows the real DNS addresses, and you need to change them where systemd-resolved gets them from: netplan, NetworkManager, or /etc/systemd/resolved.conf. On Debian 12 systemd-resolved isn't installed by default, and there /etc/resolv.conf is usually a real file.

Docker containers have a similar stub. On a user-defined network (created with docker network create or Compose), a container's /etc/resolv.conf contains nameserver 127.0.0.11. That's Docker's built-in DNS: it resolves other containers' names itself, and forwards the rest of the queries onward, to the host's DNS or to the servers from the dns option.

/etc/hosts holds local overrides that bypass DNS entirely. The most common cases:

127.0.0.1   localhost
::1         localhost
192.168.1.50  legacy-server

Useful for a dev environment, for a check before a DNS migration ("let's see how it behaves once the client already has the new IP"), and for debugging. The risk is that it's easy to forget a record like that.

Common mistakes

Changed the IP in DNS, but someone still gets the old server. That's TTL and caching somewhere along the chain: the OS, the browser, the ISP's resolver, its upstream, a corporate cache. What to do:

• flush the local OS cache (sudo resolvectl flush-caches on modern Linux, sudo systemd-resolve --flush-caches on older ones, ipconfig /flushdns on Windows, sudo dscacheutil -flushcache on macOS);
• check that the resolver isn't lying: dig @1.1.1.1 example.com asks around the local one;
• wait out the TTL. If a record had a TTL of 3600, up to an hour of some clients seeing the old address is normal.

Prevention: lower the TTL a day ahead of the migration, not on the day you do it.

DNS returns the new address, but the application still goes to the old one. dig shows the right IP, while curl and the application keep going to the wrong place. Common cause: a forgotten /etc/hosts entry added "just for the check." dig doesn't see it, applications do. Compare dig +short <name> and getent hosts <name>.

Looked up the name before creating the record. The resolver also caches a negative answer (NXDOMAIN, "no such name"), and the duration is set by the zone's SOA record (RFC 2308). If you checked the name a minute before creating the record, "name doesn't exist" can stick around for a while.

Mixed up the MX priorities. The primary mail server is the one with the lower number, not the higher one.

What you'll need for the tasks

• Docker 20.10 or newer with Compose v2, bash.
• docker compose -f setup/docker-compose.yml exec client <command>: run a command in the lab environment's container.
• dig <name>, dig <name> <type>, dig @<server> <name>, +short: read A, CNAME, MX records and their TTL (the "Record types" and "dig" sections).
• How to read a line from the ANSWER SECTION: name, TTL, class, type, value.
• getent hosts <name> and how it differs from dig (the "dig" section).
• The /etc/hosts format and the "hosts first, then DNS" order (the "How resolution works" and "/etc/resolv.conf and /etc/hosts" sections).

The next lesson covers HTTP and HTTPS, which carry most of your network traffic.