Cron-safe wrapper: lock, log and report failures for any scheduled command

A scheduled job has three classic failure modes: it runs again before the previous run finished and both runs fight over the same files, it fails silently for weeks because nobody reads cron mail, and when it finally does break there is no log of what happened. This wrapper fixes all three. Put it in front of any command in a crontab or a systemd service.

The script

#!/usr/bin/env bash
set -Eeuo pipefail

SCRIPT_NAME=$(basename "$0")
LOG_DIR=${CRONWRAP_LOG_DIR:-/var/log/cronwrap}
LOCK_DIR=${CRONWRAP_LOCK_DIR:-${XDG_RUNTIME_DIR:-/run/lock}}
MAIL_TO=""
WEBHOOK=""
TIMEOUT=""
NAME=""

usage() {
  cat <<USAGE >&2
Usage: $SCRIPT_NAME -n NAME [-l LOG_DIR] [-m EMAIL] [-w URL] [-t SECONDS] -- COMMAND [ARGS...]

  -n NAME      short job name, used for the lock and log file names
  -l LOG_DIR   where to write NAME.log (default: $LOG_DIR)
  -m EMAIL     send a failure report with the local mail command
  -w URL       POST a failure report as JSON to this webhook
  -t SECONDS   kill the command if it runs longer than this
USAGE
  exit 2
}

while getopts ":n:l:m:w:t:h" opt; do
  case "$opt" in
    n) NAME=$OPTARG ;;
    l) LOG_DIR=$OPTARG ;;
    m) MAIL_TO=$OPTARG ;;
    w) WEBHOOK=$OPTARG ;;
    t) TIMEOUT=$OPTARG ;;
    h) usage ;;
    :) echo "missing value for -$OPTARG" >&2; usage ;;
    \?) echo "unknown option -$OPTARG" >&2; usage ;;
  esac
done
shift $((OPTIND - 1))

[[ -n "$NAME" && $# -ge 1 ]] || usage
[[ "$NAME" =~ ^[A-Za-z0-9._-]+$ ]] || { echo "NAME may only contain letters, digits, . _ -" >&2; exit 2; }

command -v flock >/dev/null 2>&1 || { echo "flock not found: install util-linux (Linux) or flock from Homebrew (macOS)" >&2; exit 1; }
if [[ -n "$TIMEOUT" ]] && ! command -v timeout >/dev/null 2>&1; then
  echo "timeout not found: install GNU coreutils or drop -t" >&2
  exit 1
fi

mkdir -p -- "$LOG_DIR"
LOG_FILE="$LOG_DIR/$NAME.log"
LOCK_FILE="$LOCK_DIR/cronwrap-$NAME.lock"

log() { printf '%s [%s] %s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$NAME" "$*" >> "$LOG_FILE"; }

exec 9> "$LOCK_FILE"
if ! flock -n 9; then
  log "SKIP: previous run still holds $LOCK_FILE"
  exit 0
fi

report_failure() {
  local status=$1
  local tail_text
  tail_text=$(tail -n 20 -- "$LOG_FILE")
  local subject="[cronwrap] $NAME failed on $(hostname -s) with exit code $status"
  if [[ -n "$MAIL_TO" ]] && command -v mail >/dev/null 2>&1; then
    printf '%s\n\nLast log lines:\n%s\n' "$subject" "$tail_text" | mail -s "$subject" "$MAIL_TO" || true
  fi
  if [[ -n "$WEBHOOK" ]] && command -v curl >/dev/null 2>&1; then
    local payload
    payload=$(printf '%s\n%s' "$subject" "$tail_text" | python3 -c 'import json,sys; print(json.dumps({"text": sys.stdin.read()}))' 2>/dev/null || printf '{"text": "%s"}' "$subject")
    curl -sS -m 15 -H 'Content-Type: application/json' -d "$payload" -- "$WEBHOOK" >/dev/null || true
  fi
}

start=$(date +%s)
log "START: $*"

set +e
if [[ -n "$TIMEOUT" ]]; then
  timeout --kill-after=30 "$TIMEOUT" "$@" >> "$LOG_FILE" 2>&1
else
  "$@" >> "$LOG_FILE" 2>&1
fi
status=$?
set -e

elapsed=$(( $(date +%s) - start ))
if (( status == 0 )); then
  log "OK: finished in ${elapsed}s"
else
  if (( status == 124 )) && [[ -n "$TIMEOUT" ]]; then
    log "FAIL: timed out after ${TIMEOUT}s (exit $status)"
  else
    log "FAIL: exit code $status after ${elapsed}s"
  fi
  report_failure "$status"
fi

exit "$status"

How to use it

sudo install -m 755 cronwrap.sh /usr/local/bin/cronwrap

# Try it by hand first
cronwrap -n hello -l /tmp/cronwrap -- echo "it works"
cronwrap -n hello -l /tmp/cronwrap -- false; echo "exit was $?"
cat /tmp/cronwrap/hello.log

# In a crontab
30 2 * * * /usr/local/bin/cronwrap -n nightly-report -m [email protected] -t 3600 -- /usr/local/bin/nightly-report.sh

# In a systemd service
ExecStart=/usr/local/bin/cronwrap -n nightly-report -w https://hooks.example.com/abc -- /usr/local/bin/nightly-report.sh

What each part does

• exec 9> lockfile opens file descriptor 9 and flock -n 9 tries to lock it without waiting. If another run holds the lock the wrapper logs SKIP and exits 0, so cron does not mail you about an expected overlap. The lock is released automatically when the process exits, even if it is killed.
• All output of the command, stdout and stderr, is appended to NAME.log between START and OK or FAIL lines with timestamps, so grep FAIL tells you the history of the job.
• set +e around the command lets the wrapper capture the exit code instead of dying with it; the original code is then passed back with exit "$status", so systemd and cron still see the real result.
• timeout (from GNU coreutils) stops a hung job; exit code 124 means it timed out and --kill-after sends SIGKILL if the job ignores SIGTERM.
• The failure report contains the last 20 log lines. Mail uses the local mail command (install mailutils or s-nail and a working MTA); the webhook posts a {"text": ...} JSON body that Slack-style incoming webhooks and most chat tools accept.
• -- separates the wrapper’s options from the command, so the wrapped command can take its own -x style flags.

Notes and limits

• The lock is per machine. If the same job runs on several hosts, add a shared lock (a database row, Redis SET NX, or a scheduler) or let only one host run it.
• Log files grow forever. Add a /etc/logrotate.d/cronwrap entry, or point -l at a directory you already rotate.
• flock is part of util-linux and is on every mainstream Linux. macOS does not ship a flock command, so the wrapper refuses to start there until you install it with brew install flock.
• Careful: the wrapper does not add any safety to the command itself. A job that deletes files will still delete files; the wrapper only makes sure it runs once at a time and that you hear about failures.

Official documentation

• flock(1) manual page
• GNU coreutils: timeout
• Bash manual: Redirections