Hardening SSH on a fresh server: most guides optimise for the wrong threat

The threat you were handed, not the one in the guide

A new machine gets a public address and, within minutes, it is being knocked on. The traffic is automated, untargeted and permanent: dictionaries of usernames, passwords lifted from old breaches, a sweep of every port in the hope that something answers. That is the weather on the public internet, not an attack on you.

Most hardening guides are written as though the reader's adversary had chosen them. That adversary exists — and almost nothing on the usual checklist inconveniences them for more than a few seconds. The realistic first-week threat model on a fresh server has three entries, and they are not equally likely:

1. Untargeted credential guessing. Removed entirely by deleting passwords, not by hiding the service.
2. A door left open by whoever touched the machine before you. A stale entry in an authorized_keys file, a default cloud-init account, a package that opened a port the day it was installed.
3. You. Locking yourself out, or writing a configuration that works until the next reboot.

Ranked by probability of ruining your week, the third comes first. Hence the thesis of this piece: hardening SSH is mostly an exercise in removing one entire class of credential without breaking your own access, and the parts of the checklist everyone argues about — the port number, the fail2ban tuning — are the parts that matter least. We will still do them, because they buy something real. Just not what people think they buy.

One boundary before we start: this is about the administrative door. It says nothing about the application you are going to run behind it.

Before the first edit: reserve a way back in

Every step below is reversible except the one that cuts your session. So set that up first.

• Keep the current session open for the whole operation. A reload does not drop established connections, and neither does a restart — but a configuration that fails to parse leaves nothing listening for the next one.
• Test your out-of-band access now, while nothing is broken. The provider console or KVM is the thing that saves you, and it very often wants a local password you have never set, or a rescue mode you have never booted. Discovering that at the moment you need it is the entire failure.
• sshd -t validates syntax. sshd -T prints the effective configuration, after every Include and Match block has been resolved. The second is the one that catches the mistake, and this article keeps coming back to it.

The unit is called ssh on Debian and Ubuntu, sshd on the RHEL family. Get that wrong in a runbook and the runbook fails on half your fleet.

1. Delete the password, not the port

This is the one measure that changes the shape of the problem rather than its volume. A password can be guessed from anywhere, by anyone, forever, at no cost. A key cannot. Once passwords are gone, the first category of the threat model is gone with them — not reduced, gone.

The order matters:

1. generate an ed25519 key (ssh-keygen -t ed25519) and give it a passphrase — a key with no passphrase is a password file with better marketing;
2. copy it over (ssh-copy-id) and open a second session with it before changing anything;
3. then, and only then, set PasswordAuthentication no and KbdInteractiveAuthentication no.

The second directive is not redundant. Keyboard-interactive is a separate authentication path that PAM can still satisfy with a password on some images, which is why so many servers that “disabled password login” still accept one. Do not trust the file, ask the daemon:

sshd -T | grep -Ei 'passwordauth|kbdinteractive|pubkeyauth|permitrootlogin'

That single line is worth more than the entire configuration file, because it reports what sshd resolved rather than what you believe you wrote.

While you are in there: MaxAuthTries does not count what it looks like it counts. Every public key your client offers counts as an attempt. A laptop with six keys in its agent can be disconnected before it ever offers the right one, which from the outside is indistinguishable from a lockout. Either keep the value generous, or pin the client side with IdentitiesOnly yes and an explicit IdentityFile in your ~/.ssh/config.

2. Root: the argument is blast radius, not strength

PermitRootLogin prohibit-password keeps key-based root access and refuses passwords. PermitRootLogin no forces you to arrive as a named account and escalate. The gain of the second over the first is not cryptographic — a root key is exactly as strong as any other key. The gain is attribution: every privileged action passes through a named account and leaves a sudo line in the journal, which is the difference between knowing that root did something and knowing who did.

The corollary nobody writes down: on a machine with no second administrative account and no console you have actually tested, PermitRootLogin no is not hardening. It is a lockout waiting for one bad key.

And underneath it sits the decision people make by accident — NOPASSWD in sudoers. With password authentication disabled, your private key is the only factor between the internet and the machine. If sudo asks for nothing, a stolen key is root immediately. If sudo asks for a password, that password is a second factor which lives in your head and never travels. The cost is that unattended automation breaks, which is precisely why people set NOPASSWD and then forget they did. Choose deliberately, per machine, and write the choice down next to the machine.

Finally, restrict who may even attempt to authenticate. AllowGroups ssh-users beats AllowUsers because it survives the next person joining the team, and it survives the day a package quietly creates a service account with a login shell.

3. Moving the port is theatre — and the payoff is real anyway

Bluntly: changing the port removes nothing. A full scan finds the service in seconds, and anyone who selected you as a target is not slowed down by it. If your posture depends on 2222 being unusual, you do not have a posture.

What it actually buys is a readable journal. On port 22 a modest VPS collects thousands of Invalid user admin lines a day, and your real events — a successful login at three in the morning, a ban that caught your own office address — are buried in them. Move the port and the log becomes a document a human can read, which is the precondition for noticing an anomaly at all. That is an operational gain. It is worth having, as long as you do not count it as a control.

Three mechanical traps, in the order they tend to bite:

• Socket activation. On recent Ubuntu releases sshd is started by ssh.socket, and the Port directive in sshd_config is then simply ignored — the listening socket belongs to the unit. You change the port, nothing happens, you reboot, and now you find out. The fix is systemctl edit ssh.socket: an empty ListenStream= to clear the inherited value, then the one you want.
• SELinux. On the RHEL family the daemon refuses to bind an unlabelled port until you run semanage port -a -t ssh_port_t -p tcp 2222.
• The privileged range. Below 1024, only root can bind. Above it, if sshd is ever stopped, any local process can take the port. Marginal on a single-admin box, less so on a machine with untrusted local users.

4. fail2ban, and what it is actually doing

The mechanism is worth stating, because it explains the limits. fail2ban reads the journal, matches failure patterns with a regex, and inserts a firewall rule for the duration of bantime. It is a log reader with a firewall attached. Nothing more.

Now put that next to the previous section. With password authentication removed there is almost nothing left to fail: the bot never reaches a prompt, and the journal fills with Connection closed by authenticating user root … [preauth]. fail2ban will happily ban those, and you should let it — but be honest about the purchase. You bought less log volume and fewer TCP handshakes. You did not buy a credential defence; you already had one.

The limits, plainly:

• a distributed botnet making one attempt per address never reaches maxretry. fail2ban is built against repetition, and repetition is the cheapest thing an attacker can give up;
• addresses are cheap, and in IPv6 effectively free: banning a single /128 while the client holds an entire prefix is a rounding error. Banning the prefix works, and takes collateral with it;
• it will ban you. Put your own ranges in ignoreip, and remember it will not save you the day you connect from a hotel;
• it cannot see through an intermediary. Behind a proxy or a NAT every attempt arrives from the same address, so the ban either does nothing or hits everyone at once;
• bantime.incrementis the one setting that meaningfully changes an attacker's economics: repeat offenders get progressively longer bans instead of a flat ten minutes.

The services where fail2ban genuinely earns its keep are the ones where passwords still have to exist: a login form, a mail relay, an endpoint you have not managed to retire yet. On a key-only SSH port it is hygiene, not defence.

Meanwhile sshd has its own throttle, which costs nothing and applies before a single log line is written: MaxStartups (the 10:30:60 form) caps unauthenticated connections in flight, and LoginGraceTime caps how long each one may sit there. Underused, and they keep working when your log reader is down.

5. The firewall: default deny, and the order that locks you out

The rule that matters is the default policy, not the exceptions: deny inbound by default, then allow the handful of ports you can name and justify. Everything a package opened at install time disappears, which handles the second entry of the threat model.

The order is not optional. Allow SSH first, enable the firewall second. Reversing those two commands is the most common self-inflicted outage on this list, and the one where the console you tested earlier stops being theoretical.

Better than moving the port, when your situation allows it: restrict the SSH rule to a source range. An allowlist is an actual control, where an unusual port number is only quieter logs.

What a firewall does not do, said out loud:

• it does not protect what you deliberately expose. A packet filter in front of a vulnerable web application changes nothing about the web application;
• egress stays open unless you decide otherwise, and filtering outbound traffic is real hardening with a real operational cost — every agent, backup job and package mirror then needs an explicit rule;
• on a cloud instance there is a second firewall at the provider, and it is the one you will forget. Check both, or you will spend an afternoon debugging a rule that was never the problem.

6. Unattended upgrades: the best ratio on the list

If you do only one more thing after removing passwords, do this one. Small servers are rarely lost to clever attacks. They are lost to a known vulnerability in something exposed, published weeks earlier and patched upstream long before anyone came near the machine. Automating security updates has no equal on effort against benefit.

Enable unattended-upgrades, keep the allowed origins limited to the security pocket rather than every update, set an automatic reboot inside an explicit window, and run it once with --dry-run --debug so you can read what it intends to do before it does it at four in the morning.

Then accept the three costs, because they are real:

• a service restarted mid-afternoon by needrestart is a small outage. Decide the window; do not discover it;
• automatic reboot means your application has to come back on its own. The risk you are taking on is not the update, it is untested boot ordering — check that everything you need is enabled, not merely active right now;
• it covers what the package manager installed, and nothing else. A binary you dropped into /usr/local/bin, a pinned container image, a runtime installed by a version manager: those stay yours, and they are exactly where the forgotten vulnerability will be.

What none of this protects against

A hardening article that ends on a victory lap is lying by omission. What you have done narrows one door. It does nothing about a vulnerability in the application you expose, nothing about a laptop stolen with an unencrypted private key on it, nothing about a compromised dependency in what you deploy, nothing about someone with legitimate access making a legitimate-looking mistake, and nothing about data loss — hardening is not backup, and the two get confused routinely.

No configuration makes a server invulnerable. What this one does is remove the cheap attacks entirely and make the expensive ones leave traces. That is the whole of the available win, and it is worth having.

The check that proves you did not lock yourself out

From a second terminal, with the first still open, in this order:

1. connect with ssh -v on the new port as the intended user, and read the authentication method in the output rather than assuming it;
2. run sudo -l, and if you configured a sudo password, actually use it once;
3. sshd -T, filtered on the directives you set: port, root policy, the two authentication switches, allowgroups and the maxauthtries value;
4. ss -tulpn — what is listening, on which address, owned by which process. This is where you find the thing you did not know was exposed;
5. the firewall rule list, on the machine and at the provider;
6. systemctl is-enabled on the ssh unit or socket, and on fail2ban and unattended-upgrades — is-active only tells you about right now;
7. reboot. Everything above proves the running state; only a reboot proves the persistent one, and a configuration that survives until the next kernel update is not hardening. Reconnect, then read journalctl -u ssh -b to see what actually started.

Then arrange to hear about a broken door from a machine rather than from your own next attempt. An SSH check against the port, at a slow interval, with enough hysteresis that one dropped packet wakes nobody, is the difference between learning about it tomorrow morning and learning about it in three weeks, at the exact moment you need to get in. Setting that without creating a new source of noise is the subject of a separate piece on alert noise.

Where Servor fits

Servor ships an SSH-hardening runbook among its 27 official ones — fail2ban, UFW and automatic updates are in there too. A runbook is the right shape for this work: a reviewed sequence with an explicit verification at the end, replayable on the next machine instead of retyped from memory. Execution goes through the agent, which opens an outboundconnection to the control plane, so tightening or restricting SSH does not cut your management path — SSH is used only long enough to bootstrap the agent in the first place. Every command lands in a single history with its output, exit code and duration, and the hash-chained audit log still answers “who changed sshd_config, when, and what did it return” six months later.

Two things worth knowing before you point it at a machine. The shared forbidden-command list refuses direct writes to /etc/passwd, /etc/shadow and /etc/sudoers, on the control plane and again on the agent — that is deliberate, and it means the sudoers decision stays a manual one. And if you would rather a machine only be watched, strict mode installs it supervision-only, with the control plane refusing every execution.

More on the operational half of this in our server operations guide, and on how a command gets approved, signed and verified in the Plan-Execute-Verify piece.