Restore drill: prove your backup works in 15 minutes

A restore drill answers one question: if the original disappeared right now, could I get it back, and how long would it take? You do not restore everything. You restore a small, random sample into a place that cannot hurt production, verify it byte for byte, and write the result down. Fifteen minutes once a month catches almost every silent backup failure.

Before you start

• Know where the newest backup is and how to authenticate to it (repository password, S3 keys, PBS fingerprint). If finding this takes more than two minutes, that is already a finding.
• Have a scratch directory with enough free space, for example /tmp/restore-drill, and a scratch database you can drop afterwards.
• Start a timer. The drill measures reality, including the time you spend looking for passwords.

Step 1: restore a single file from a file-level backup

Pick a file that changes often, such as a config file edited last week. With restic:

export RESTIC_REPOSITORY=/mnt/nas/restic
export RESTIC_PASSWORD_FILE=/etc/restic/password
restic snapshots --latest 1
restic restore latest --target /tmp/restore-drill --include /etc/nginx/nginx.conf
diff /etc/nginx/nginx.conf /tmp/restore-drill/etc/nginx/nginx.conf && echo IDENTICAL

--target keeps the restore away from the live file, and --include restores only that path, so the whole thing takes seconds. With Borg the equivalent is borg extract, run from inside the scratch directory because it extracts relative to the current directory:

cd /tmp/restore-drill
borg list /mnt/nas/borg --last 1
borg extract /mnt/nas/borg::$(borg list /mnt/nas/borg --last 1 --format '{archive}') etc/nginx/nginx.conf
diff /etc/nginx/nginx.conf etc/nginx/nginx.conf && echo IDENTICAL

If the file differs, check whether it was edited after the snapshot before you call it a failure. If the file is missing from the backup, the include or exclude list is wrong, and that is the most valuable thing a drill can find.

Step 2: restore a database dump into a scratch database

A dump that exists is not a dump that imports. For PostgreSQL, create an empty database and restore the newest custom-format dump into it:

latest=$(ls -t /var/backups/postgresql/*.dump | head -n 1)
echo "$latest"
sudo -u postgres createdb restore_drill
sudo -u postgres pg_restore --dbname restore_drill --no-owner --exit-on-error "$latest"
sudo -u postgres psql -d restore_drill -c "SELECT count(*) FROM orders;"
sudo -u postgres psql -d app -c "SELECT count(*) FROM orders;"

--exit-on-error makes pg_restore stop at the first problem instead of printing warnings you might miss. Compare a row count or a max(updated_at) between the scratch database and production; the difference should match the age of the dump. For MySQL or MariaDB:

latest=$(ls -t /var/backups/mysql/*.sql.gz | head -n 1)
mysql -e "CREATE DATABASE restore_drill;"
gunzip -c "$latest" | mysql restore_drill
mysql -e "SELECT count(*) FROM restore_drill.orders; SELECT count(*) FROM app.orders;"

If the dump was created with --all-databases or contains CREATE DATABASE and USE statements, it will not land in restore_drill; restore it on a throwaway VM or container instead, never on the production server.

Step 3: verify from a different machine once a quarter

The monthly drill proves the data is intact. The quarterly drill proves you can reach it without the original server: on a laptop or a fresh VM, install the backup tool, fetch the credentials from wherever you keep them (password manager, sealed envelope, vault), and repeat Step 1. This catches the most common total failure: the only copy of the repository password lived on the server that died.

Step 4: record the result and clean up

Stop the timer and write one line into a log or wiki page: date, which backup, what was restored, identical or not, minutes taken, and anything that was harder than it should have been. Then remove the scratch copies.

rm -rf /tmp/restore-drill
sudo -u postgres dropdb restore_drill
mysql -e "DROP DATABASE restore_drill;"

Careful: dropdb and DROP DATABASE are irreversible; double-check that the name is the scratch database and not production before pressing Enter.

What a good drill log looks like

2026-10-14  restic nas  /etc/nginx/nginx.conf  identical  4 min  ok
2026-10-14  pg_dump app  orders=184233 vs 184250  ok  6 min  pg_restore needed --no-owner
2026-09-16  restic nas  /etc/ssh/sshd_config  MISSING  12 min  /etc excluded by typo in excludes.txt, fixed

Over time this log is also your evidence for audits and the honest basis for the recovery time you promise others.

Official documentation

• restic: Restoring from backup
• Borg: borg extract
• PostgreSQL: pg_restore