Set up and harden a fresh Linux server in one afternoon

A freshly installed server is reachable from the network but configured for convenience, not safety: you may be logging in as root with a password, nothing filters incoming traffic, and nobody installs updates. This tutorial walks through the first hour of work that every new server deserves, in an order that never locks you out.

Commands are shown for Debian/Ubuntu and for the RHEL family (RHEL, Rocky Linux, AlmaLinux 9) wherever they differ. The server is called web1.example.com with address 192.0.2.20; the backup target is backup.example.com.

What you will build

• A personal admin account (admin) that uses sudo, instead of working as root.
• SSH logins with keys only: password logins and direct root logins switched off.
• A host firewall that allows SSH and blocks other incoming connections (ufw on Debian/Ubuntu, firewalld on RHEL family).
• Security updates installed automatically every day.
• A correctly synchronized clock.
• A small script that warns about full disks and low memory every 15 minutes.
• A nightly rsync backup of /etc, /home and /var/www to another machine, with a tested restore.

Before you start

• A fresh server (VM, cloud instance or physical) running Ubuntu 22.04 or newer, Debian 12 or newer, or RHEL/Rocky/AlmaLinux 9, with root access or an initial user with sudo.
• Console access in case SSH breaks: a hypervisor console, cloud provider web console or a physical keyboard and screen. Do not skip this; hardening SSH over SSH is exactly how people lock themselves out.
• Your own computer with an SSH client (Linux, macOS and Windows 10/11 all include OpenSSH).
• A second Linux machine for backups (backup.example.com) with rsync installed and a user called backup. Any small VM or NAS that offers SSH works.
• Time: about 2 hours.
• Knowledge: basic shell use, editing files with nano or vi, and the idea of public and private SSH keys.

Step 1: Log in and install all updates

Why: images and installers are often weeks or months old. Patch first so that everything you configure runs on current packages.

ssh [email protected]

Debian/Ubuntu:

apt update
apt full-upgrade
[ -f /var/run/reboot-required ] && echo "reboot needed"

RHEL family:

dnf upgrade

Check: no errors in the package manager output. If a reboot is reported as needed (Debian/Ubuntu) or a new kernel package was installed (RHEL family), run reboot and log in again. If your provider gave you a non-root user instead of root, put sudo in front of the commands in this step.

Step 2: Create an admin user with sudo

Why: working as root means every typo runs with full power, and every attacker knows the user name root. A personal account with sudo leaves an audit trail and lets you turn off root logins later. Debian and Ubuntu give sudo rights to members of the sudo group; RHEL-family systems use the wheel group.

Debian/Ubuntu (adduser asks for a password and some optional details):

adduser admin
adduser admin sudo

RHEL family:

useradd -m admin
passwd admin
usermod --append -G wheel admin

On a minimal Debian install sudo may be missing; install it with apt install sudo. Choose a strong password even though you will log in with keys: sudo still asks for it.

Check: switch to the new user and ask sudo who you are. The answer must be root.

su - admin
sudo whoami
exit

Step 3: Set up SSH key login for the admin user

Why: an SSH key cannot be guessed the way a password can, and it never travels to the server. Run these commands on your own computer, not on the server. Skip ssh-keygen if you already have ~/.ssh/id_ed25519.pub.

ssh-keygen -t ed25519 -C "admin@laptop"
ssh-copy-id [email protected]

ssh-keygen asks for a passphrase; set one, it protects the key if your laptop is stolen. ssh-copy-id logs in once with the password and appends your public key to /home/admin/.ssh/authorized_keys with correct permissions. On Windows without ssh-copy-id, paste the contents of the .pub file into that file on the server by hand.

Check: open a new terminal and log in. You should get a shell without being asked for the server password (only for the key passphrase, if you set one).

ssh [email protected]
sudo whoami

Step 4: Turn off password and root logins

Why: with keys working, passwords are only an attack surface: bots try common passwords against every public SSH server all day. Put your settings in a drop-in file instead of editing the main sshd_config, so package updates do not overwrite them.

First confirm that the main config includes the drop-in directory (current Debian, Ubuntu and RHEL 9 releases do):

grep -i '^include' /etc/ssh/sshd_config

For each setting, sshd uses the first value it reads, and drop-in files are read in alphabetical order. Some images ship a file such as 50-cloud-init.conf that turns passwords back on, so name yours with a low number to make it win:

sudo tee /etc/ssh/sshd_config.d/00-hardening.conf > /dev/null <<'EOF'
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
EOF

sudo sshd -t
sudo sshd -T | grep -Ei '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin) '

sshd -t prints nothing when the syntax is valid. sshd -T prints the effective configuration; all three lines should show no. Keep your current SSH session open and restart the service:

# Debian/Ubuntu
sudo systemctl restart ssh

# RHEL family
sudo systemctl restart sshd

Check from your computer, in a new terminal: key login still works, and a forced password attempt is refused without even asking for a password.

ssh [email protected] 'echo key login ok'
ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password [email protected]
ssh [email protected]

The last two commands must fail with a Permission denied message. If the key login also fails, use your still-open session or the console to remove 00-hardening.conf and restart SSH.

Step 5: Turn on the firewall

Why: a host firewall makes sure that only services you deliberately open are reachable, even if a package starts listening on a new port. The golden rule: allow SSH before you enable the firewall.

Debian/Ubuntu with ufw. When first installed, ufw is disabled with a default policy of denying incoming and allowing outgoing traffic. Debian does not install it by default:

sudo apt install ufw
sudo ufw allow 22/tcp
sudo ufw enable
sudo ufw status verbose

ufw enable asks for confirmation because enabling a firewall can cut existing SSH connections; because port 22 is already allowed, answer y. Optionally replace the plain rule with sudo ufw limit 22/tcp, which denies an address that opens 6 or more connections within 30 seconds.

RHEL family with firewalld, which is normally installed and running already:

sudo systemctl enable --now firewalld
sudo firewall-cmd --get-default-zone
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

The default zone is usually public, which on RHEL also allows cockpit and dhcpv6-client. If you do not use the Cockpit web console, close it with sudo firewall-cmd --permanent --remove-service=cockpit followed by sudo firewall-cmd --reload. Changes without --permanent are lost at the next reload or reboot.

Check: ufw status verbose shows Status: active with 22/tcp ALLOW IN, or firewall-cmd --list-all shows ssh under services. A new SSH login from your computer still works. When you later run a web server, open 80 and 443 the same way (sudo ufw allow 80/tcp, or --add-service=http and --add-service=https).

Step 6: Install security updates automatically

Why: most break-ins use holes that were fixed long ago. Automatic security updates close them within a day without you logging in.

Debian/Ubuntu use unattended-upgrades. Ubuntu server usually has it already; on Debian install it. The two lines in 20auto-upgrades switch on the daily package list refresh and the upgrade run:

sudo apt install unattended-upgrades
sudo tee /etc/apt/apt.conf.d/20auto-upgrades > /dev/null <<'EOF'
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF
sudo unattended-upgrade -v --dry-run

By default it installs updates from the security archive. Which origins are allowed and whether the server may reboot by itself (Unattended-Upgrade::Automatic-Reboot) is set in /etc/apt/apt.conf.d/50unattended-upgrades. Logs go to /var/log/unattended-upgrades/.

RHEL family (9) uses dnf-automatic. Set it to security updates only, then enable the timer that downloads and installs them:

sudo dnf install dnf-automatic
sudo sed -i 's/^upgrade_type *=.*/upgrade_type = security/' /etc/dnf/automatic.conf
grep '^upgrade_type' /etc/dnf/automatic.conf
sudo systemctl enable --now dnf-automatic-install.timer
systemctl list-timers 'dnf-automatic*'

Check: the Debian/Ubuntu dry run ends without errors and lists what it would install; on RHEL, list-timers shows when dnf-automatic-install.timer runs next. Neither tool reboots by default, so check now and then whether a kernel update is waiting for a reboot.

Step 7: Make sure the clock is synchronized

Why: wrong time breaks TLS certificates, makes logs from different machines impossible to compare, and confuses scheduled jobs. Every major distribution ships an NTP client; you only need to confirm it is running.

timedatectl

Look for System clock synchronized: yes and NTP service: active. RHEL-family systems use chronyd, as does Ubuntu from 25.10 on; older Ubuntu releases use systemd-timesyncd. If the NTP service is inactive, install and start chrony:

# Debian/Ubuntu
sudo apt install chrony

# RHEL family
sudo dnf install chrony
sudo systemctl enable --now chronyd

chronyc tracking

Servers are easier to manage in UTC, which avoids daylight-saving jumps in logs: sudo timedatectl set-timezone UTC.

Step 8: Watch disk and memory

Why: a full disk is the most common reason a healthy server suddenly stops working. Start with the quick manual checks, then automate them.

df -h
free -h
uptime
sudo du -xh /var --max-depth=1 | sort -h | tail

The script below warns when any real filesystem is 85% full or less than 10% of memory is available. It writes a warning to the system log with logger, so you find it with journalctl -t check-resources. Save it as /usr/local/sbin/check-resources.sh:

#!/usr/bin/env bash
set -euo pipefail

DISK_LIMIT=85
MEM_MIN_PERCENT=10
MEMINFO="${MEMINFO:-/proc/meminfo}"
problems=0

while read -r use mount; do
  pct="${use%\%}"
  if [ "$pct" -ge "$DISK_LIMIT" ]; then
    logger -t check-resources "disk ${mount} is ${pct}% full"
    echo "WARNING: disk ${mount} is ${pct}% full"
    problems=1
  fi
done < <(df -P -x tmpfs -x devtmpfs | awk 'NR>1 {print $5, $6}')

total=$(awk '/^MemTotal:/ {print $2}' "$MEMINFO")
avail=$(awk '/^MemAvailable:/ {print $2}' "$MEMINFO")
avail_pct=$(( avail * 100 / total ))
if [ "$avail_pct" -lt "$MEM_MIN_PERCENT" ]; then
  logger -t check-resources "only ${avail_pct}% memory available"
  echo "WARNING: only ${avail_pct}% memory available"
  problems=1
fi

[ "$problems" -eq 0 ] && echo "OK: disks below ${DISK_LIMIT}%, ${avail_pct}% memory available"
exit "$problems"

sudo chmod 755 /usr/local/sbin/check-resources.sh
sudo /usr/local/sbin/check-resources.sh
echo '*/15 * * * * root /usr/local/sbin/check-resources.sh > /dev/null' | sudo tee /etc/cron.d/check-resources

Check: the manual run prints a line starting with OK: on a healthy server. To see a warning on purpose, temporarily set DISK_LIMIT=1 in the script, run it, and look at journalctl -t check-resources; then set it back. On a minimal RHEL install, cron may be missing: install it with sudo dnf install cronie and sudo systemctl enable --now crond.

Step 9: Back up with rsync

Why: hardening reduces risk; backups are what save you when something goes wrong anyway. rsync copies only what changed, so nightly runs are fast. The backup lives on another machine, so losing this server does not lose the copy.

Install rsync on both machines (sudo apt install rsync or sudo dnf install rsync). The backup runs as root on web1 so it can read every file, so give root its own key without a passphrase and allow it on the backup host:

sudo ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_ed25519
sudo ssh-copy-id -i /root/.ssh/id_ed25519.pub [email protected]
sudo ssh [email protected] 'mkdir -p /srv/backups/web1'

If the backup host also has password login disabled, append the contents of /root/.ssh/id_ed25519.pub to /home/backup/.ssh/authorized_keys on the backup host by hand instead of using ssh-copy-id. Then save the backup script as /usr/local/sbin/backup.sh:

#!/usr/bin/env bash
set -euo pipefail

DEST="${BACKUP_DEST:[email protected]:/srv/backups/$(hostname -s)}"
SOURCES=(/etc /home /var/www)

existing=()
for path in "${SOURCES[@]}"; do
  [ -e "$path" ] && existing+=("$path")
done

rsync -a --delete --relative "${existing[@]}" "$DEST/"
echo "$(date "+%F %T") backup finished: ${existing[*]} -> $DEST"

• -a (archive) copies recursively and keeps permissions, owners, times and symlinks.
• --relative keeps the full path, so /etc ends up in /srv/backups/web1/etc.
• --delete removes files from the backup that were deleted on the server. That keeps the copy exact, but it also means an accidental deletion reaches the backup at the next run. For history, keep several copies or snapshots on the backup host.
• Sources that do not exist (for example /var/www on a server without a website) are skipped.

Make it executable, test it once by hand, then schedule it nightly at 02:30:

sudo chmod 755 /usr/local/sbin/backup.sh
sudo /usr/local/sbin/backup.sh
echo '30 2 * * * root /usr/local/sbin/backup.sh >> /var/log/backup.log 2>&1' | sudo tee /etc/cron.d/backup

Check: the manual run ends with a backup finished line, and the files are on the backup host:

sudo ssh [email protected] 'ls /srv/backups/web1/etc | head'

Check that it works

1. From your computer, ssh [email protected] works with your key, while ssh [email protected] is refused.
2. sudo ufw status verbose or sudo firewall-cmd --list-all shows only the services you meant to open.
3. timedatectl reports a synchronized clock.
4. sudo /usr/local/sbin/check-resources.sh prints OK.
5. Restore test: copy one file back from the backup into a temporary folder and compare it with the original.

mkdir -p /tmp/restore-test
sudo rsync -a [email protected]:/srv/backups/web1/etc/hostname /tmp/restore-test/
diff /etc/hostname /tmp/restore-test/hostname && echo "restore ok"

Reboot once (sudo reboot) and repeat checks 1 to 3: all settings must survive a restart.

If something goes wrong

• Locked out of SSH after Step 4 or 5: log in on the console, then remove /etc/ssh/sshd_config.d/00-hardening.conf and restart SSH, or allow SSH again with sudo ufw allow 22/tcp / sudo firewall-cmd --permanent --add-service=ssh && sudo firewall-cmd --reload.
• Password login still works after Step 4: another file sets PasswordAuthentication yes before yours. Run sudo grep -ri passwordauthentication /etc/ssh/ and check that your drop-in sorts first and that sshd -T shows no.
• sudo says the user is not in the sudoers file: the group change is missing or the user has not logged in again since. Check with id admin as root and look for sudo or wheel.
• SSH key is ignored and the server asks for a password: permissions are too open. On the server: chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys; on RHEL also run restorecon -Rv ~/.ssh if you created the files by hand.
• The backup cron job does nothing: look at /var/log/backup.log. The first connection from root to the backup host must accept its host key; run sudo ssh [email protected] true once by hand.

Clean up

Everything here is meant to stay. If you used a throwaway server to practice, the cleanest undo is deleting the VM. To roll back single pieces:

sudo rm /etc/cron.d/backup /etc/cron.d/check-resources
sudo rm /usr/local/sbin/backup.sh /usr/local/sbin/check-resources.sh
sudo rm /etc/ssh/sshd_config.d/00-hardening.conf
sudo systemctl restart ssh    # sshd on RHEL family

Warning: removing the hardening file re-enables password logins. Do it only on a test machine. To delete the admin account, first check that you can still log in as another user with sudo rights, because this command also removes its home directory and keys:

id admin
sudo userdel -r admin

The backup copy on backup.example.com stays until you remove /srv/backups/web1 there; check its contents with ls before deleting it.

Next steps

• Install fail2ban to ban addresses that repeatedly fail to log in.
• Send logs and metrics to a central system (for example Prometheus node exporter and Grafana) instead of local scripts.
• Describe this whole setup in an Ansible playbook so the next server takes minutes, not hours.
• Keep versioned backups with a tool such as restic or BorgBackup, which add encryption and history.
• Review open ports regularly with sudo ss -tlnp and remove services you do not need.

Official documentation

• Ubuntu Server: OpenSSH server
• OpenSSH: sshd_config manual
• firewalld: firewall-cmd manual