How SSH Works: Three Layers, and the Settings That Actually Matter

Most people meet SSH as a password prompt and a scary warning once a year. Underneath is a clean three-layer design, specified in four RFCs, and every common SSH problem maps neatly onto one layer.

The three layers

 +-----------------------------------------------+
 | CONNECTION    channels: shell, exec, sftp,    |   RFC 4254
 |               port forwarding, agent          |
 +-----------------------------------------------+
 | AUTHENTICATION   publickey, password, ...     |   RFC 4252
 +-----------------------------------------------+
 | TRANSPORT     key exchange, host auth,        |   RFC 4253
 |               encryption, integrity           |
 +-----------------------------------------------+
 |               TCP (port 22)                   |
 +-----------------------------------------------+

SSH-2 is the version that was standardised by the IETF, and the only one in use.

Layer 1: transport, building a trustworthy pipe

When the TCP connection opens, both sides announce their supported algorithms and agree on one set: key exchange, host key type, cipher, message authentication. Then:

1. Key exchange

   Client and server run a Diffie-Hellman style exchange so both derive the same session keys, without those keys ever crossing the wire. An eavesdropper who records the whole conversation cannot reconstruct them.

2. Server authentication

   The server proves its identity by signing the exchange with its host key. The client compares that key against known_hosts. On first contact it asks you to trust the fingerprint, and remembers it.

3. Encryption and integrity on

   From here on, every packet is encrypted and carries an integrity check, so tampering is detected and content is private.

This is why the host-key warning matters. The whole scheme prevents man-in-the-middle attacks by pinning the server’s key on first use. If it changes unexpectedly, the protocol is telling you the thing you are talking to is not the thing you trusted before.

Layer 2: authentication, proving the client

Only after the encrypted pipe exists does the client identify itself, so credentials are never sent in the clear. Methods include:

Public-key authentication in practice:

ssh-keygen -t ed25519 -C "laptop"        # creates the key pair, set a passphrase
ssh-copy-id user@server                   # installs the public half
ssh user@server                           # signs in with the private half

Layer 3: connection, many things over one pipe

Once authenticated, the connection layer multiplexes channels over the single encrypted session. Each channel is independent, with its own flow control.

Diagnosing by layer

Run ssh -v (more v’s for more detail) and read which stage it stops at.

The server settings worth changing

# /etc/ssh/sshd_config
PasswordAuthentication no        # keys only, once you have confirmed key login works
PermitRootLogin no               # log in as a user, escalate with sudo
PubkeyAuthentication yes
AllowUsers deploy alice          # explicit allowlist

Reload the daemon after editing, and keep one session open while you test a second, so a typo does not lock you out.

Reaching internal hosts safely

Jump through a hardened bastion with ProxyJump rather than forwarding your agent into a shared machine.

ssh -J bastion.example.com app-server.internal

The jump host only relays encrypted traffic, and your key never lands on it. Agent forwarding, by contrast, lets anyone with root on the remote machine use your agent for as long as you are connected.

Takeaway

SSH works because of its layering: the transport layer creates a private, tamper-evident pipe and pins the server’s identity, the authentication layer proves the client, and the connection layer carries many independent channels over that one pipe. Learn the layers and the errors stop being mysterious, and the hardening list stays short.