All use cases

Dev & Ops

The backup that didn’t run can’t tell you it didn’t.

Scheduled jobs fail quietly: the cron line gets commented out, the disk fills, the box never comes back from a reboot. Add one line to the end of the job and silence becomes the alert.

Backup doesn’t check inyour webhook fires.

Monitor a job

Alerts go out by email, by text, or as a webhook to your own systems.

3 min read

Cinematic vertical photo of an attractive, confident woman in her thirties in a sharp dark suit with a security badge on a lanyard, walking with purpose down…

nightly-db-backup due 03:05

01/29

01 The silence

Failures that never log an error

Most monitoring watches for something that happens: an error, a spike, a status code. A job that never starts produces none of those. There’s no stack trace for a cron entry that didn’t fire.

So the failure surfaces late, usually on the day you need the thing the job was supposed to make, when the newest restore you can find is weeks old.

A dead man’s switch inverts it. The job reports in when it succeeds. If the report doesn’t arrive on time, that absence is what fires.

  1. nightly-db-backup: ok, 412 MB

  2. nightly-db-backup: ok, 413 MB

  3. Host rebooted for a kernel update. Cron never comes back.

  4. No runs. No errors. No alerts.

  5. Restore requested. The newest backup is from Oct 02.

Twenty-one silent nights and not one error, because nothing ran. A check-in turns the first missed night into an alert.

02 The switch

How it works

Press check in before the dial runs out, or let it run and see what your person gets.

nightly-db-backup1d 1hleft on the clock

Simulation: 12 seconds stand in for 1d 1h. Nothing is sent.

  1. Create a countdown

    Set the interval a little longer than the job’s schedule: 25 hours for a nightly job, 70 minutes for an hourly one. Choose who hears about a miss.

  2. Check in at the end of the job

    Copy the countdown’s check-in link and take the token at the end of it. Once the job succeeds, POST to the check-in endpoint with that token. Each check-in restarts the interval.

  3. A missed run fires

    When the interval passes without a check-in, the countdown sends your email, a text, or a webhook with your own method, headers and body.

backup.sh: the check-in runs only if every line above it succeeded
set -euo pipefail
pg_dump "$DATABASE_URL" | gzip > /backups/db.sql.gz
curl -fsS -X POST \
  "https://countdowntrigger.com/api/public/check-in/$CT_TOKEN"

03 The message

What fires when a run is missed

You write the alert once. An email reaches you by default; it can also be a text, or a webhook to a URL you choose, by GET, POST or PUT.

  • Your words, unedited, under “Their message”
  • Who set it up, and the check-in that was missed
  • What to do next, and a link that explains the alert

Point the webhook at a chat channel, a pager or your own endpoint, with the headers and body that service expects.

Countdown Triggernotifications@countdowntrigger.com

ops did not check in — “nightly-db-backup”

ops did not check in

ops set up a countdown called “nightly-db-backup” and did not check in by Thu, Oct 15, 3:05 AM PDT. This message was sent automatically on their behalf.

Their message

nightly-db-backup has not checked in. The last successful run is more than 25 hours old. Check the backup host, the cron entry and the disk.

What to do: try to reach ops directly. If you cannot, and you believe they may be in danger, contact your local emergency services.

04 The fine print

What it is, and what it isn’t

  • Not an APM

    It watches one thing: whether a job reported in on time. Pair it with whatever tells you why.

  • No API keys

    The check-in is one unauthenticated POST carrying the countdown’s token. Treat the token like a secret: keep it in the job’s environment, not in the repo.

  • One-minute precision

    Intervals start at one minute. Give jobs with variable run times some slack.

  • Reminders are account-wide

    Every countdown emails its owner before each deadline, which is noise for a job that checks in by itself. Reminder emails are an account setting today: switching them off in Settings applies to every countdown on the account.

05 Questions

Before you set it

Is there an API?

Not a keyed one. Each countdown has a check-in token, and a POST to the check-in endpoint with it is the whole interface. The token is the credential, so keep it out of version control.

What does the webhook send?

Whatever you configure: the URL, the method (GET, POST or PUT), the headers and the body.

Can one account watch many jobs?

Yes. You can run up to 100 countdowns, one per job.

What if a check-in is sent twice?

Each check-in just restarts the interval, so a retry is harmless. One token accepts up to 20 check-ins a minute.