SSH: config, agent, bastion

Why this matters

Once you have more than a couple of servers, commands get bloated:

ssh -i ~/.ssh/deploy_key -p 2222 deploy@10.0.5.14

Remembering the IP, port, user, and key path for every host is impossible. The file ~/.ssh/config moves all of that into named blocks, and then you log in by a short alias. This lesson covers aliases, logging in through a bastion, and why it's better not to turn on agent forwarding.

~/.ssh/config: host aliases

# ~/.ssh/config
Host web
    HostName 10.0.5.14
    User deploy
    Port 2222
    IdentityFile ~/.ssh/deploy_key

Host db
    HostName 10.0.5.20
    User postgres
    IdentityFile ~/.ssh/deploy_key

Now ssh web expands into the full command from the block. The parameters match ssh flags: HostName sets the real address, User says who to log in as, Port sets the port, IdentityFile points to the key to use. Other programs built on top of SSH understand the alias too: scp web:/etc/hosts . and git over SSH read the same file.

The file must belong to you, and nobody else should be able to write to it. If a group can write to it, ssh refuses to work and prints Bad owner or permissions on ~/.ssh/config. Normal permissions: 600.

You can set values by a pattern. The Host * block applies to all hosts:

Host *
    ServerAliveInterval 60
    AddKeysToAgent yes

ServerAliveInterval 60 means: if nothing has arrived from the server for 60 seconds, the client sends it a keepalive message over the encrypted channel on its own. The connection doesn't sit idle, and a NAT or firewall along the way doesn't drop it. The default is 0, meaning the check is off. AddKeysToAgent yes adds the key to a running ssh-agent on first use, so the passphrase isn't asked again.

Block order: the first value wins

ssh reads the config top to bottom and for each parameter takes the first value it finds. Later blocks no longer change that parameter. Hence the rule: write specific hosts above, put the shared Host * at the very end.

An example mistake:

Host *
    Port 22

Host web
    HostName 10.0.5.14
    Port 2222

Here ssh web goes to port 22: the value from Host * is found first, and Port 2222 from the web block is ignored.

No need to guess. The -G flag prints the resulting parameters for a host without connecting anywhere:

ssh -G web

hostname 10.0.5.14
port 22
...

If the output doesn't match what you wrote in the block, some block above set that parameter first.

Bastion and ProxyJump

On a proper network, production servers aren't exposed to the internet directly. Only one gatekeeper host is visible from outside, called a bastion (or jump host). You reach the real server through it.

This used to be done with long ProxyCommand constructs. Now there's ProxyJump:

Host bastion
    HostName 203.0.113.10
    User jump

Host app
    HostName 10.0.5.30
    User deploy
    ProxyJump bastion

The command ssh app first opens a connection to bastion, and through it reaches the internal 10.0.5.30. ProxyJump can hold an alias from the same file, and then bastion's own User, Port, and IdentityFile apply.

The -J flag does the same thing from the command line:

ssh -J jump@203.0.113.10 deploy@10.0.5.30

-J has a trap. Flags like -i and -p apply only to the final server, not to the bastion. The command ssh -i ~/.ssh/lab_key -J jump@bastion ... presents bastion with the default keys, not lab_key, and gets Permission denied from jump@bastion. Settings for the bastion are safer to keep in ~/.ssh/config.

Traffic to the final server stays encrypted all the way from you to it: bastion forwards the bytes and sees where the connection is headed, but not its contents.

One more detail on the server side. ProxyJump works through TCP forwarding on bastion, which is allowed by sshd's AllowTcpForwarding setting. In OpenSSH it's yes by default, but some distributions change this in their own sshd_config. Alpine, for example, sets AllowTcpForwarding no, and a jump through such a bastion won't go through.

Why agent forwarding is dangerous

It's tempting to turn on agent forwarding (ForwardAgent yes or the -A flag) so you can keep using your own key from an intermediate server. The problem is that forwarding opens your agent's socket on that server. Root on the intermediate host (or whoever broke into it) can use that socket to sign requests with your key while you're connected. They can't copy the key itself, but they can log in as you.

Here's what that looks like on bastion, while user dev is logged in there with -A. Root finds the agent's socket and logs into the internal server with dev's key:

ls /tmp/ssh-*/

srwxr-xr-x    1 jump     jump             0 Oct  9 16:43 agent.35

SSH_AUTH_SOCK=/tmp/ssh-XXXXeCNIFK/agent.35 ssh -p 2222 deploy@app01.lab.internal whoami

deploy

ProxyJump is safer: it doesn't expose the agent on bastion, it just forwards an encrypted tunnel straight through. With ProxyJump, there's no agent socket on bastion at all. If you do need forwarding, turn it on in a specific host's block, not in Host *.

Common mistakes

• Host * at the top of the file. Its values win over everything written below. Check with ssh -G <alias>.
• ForwardAgent yes in Host *. The agent travels to every server you log into, including ones that aren't yours.
• ssh -i key -J bastion ... expecting the key to also work for bastion. -i applies only to the final server.
• A config writable by a group: ssh prints Bad owner or permissions and refuses to connect. A config created as root with 600 permissions can't even be read by ssh: aliases silently stop working (Could not resolve hostname app).
• ProxyJump is set up correctly, but bastion has AllowTcpForwarding no. The client sees channel 0: open failed: administratively prohibited and stdio forwarding failed.

What to remember

~/.ssh/config stores connection parameters in named Host blocks, and then you log in by a short alias. For each parameter the first value wins, so Host * goes at the end.

ProxyJump (or -J) lets you reach an internal server through a bastion without exposing your agent on the intermediate host.

Agent forwarding (-A) exposes your agent on someone else's machine. Turn it on for one host when you actually need it, not for every host at once.

SSH keeps coming up in the tracks ahead: git, Ansible, deployment. All of them rest on these keys and aliases.

What you'll need for the tasks

• Logging into a container as a given user: docker exec -it -u <name> <container> bash, leave with exit.
• Editor in the lab containers: vi. Without an editor you can write a file with cat > file <<'EOF' ... EOF.
• Host blocks, the parameters HostName, User, Port, IdentityFile, ProxyJump, ServerAliveInterval, ForwardAgent.
• Resulting host parameters: ssh -G <alias>.
• Logging in without a password prompt: ssh -o BatchMode=yes <alias> hostname.