Overnight pipeline status line for on-call data engineers
Problem
When a pipeline fails overnight, the alert reaches the on-call data engineer as a page or a Slack ping with a job name and nothing else. To find out whether it matters, they have to open a laptop, VPN in, and load the orchestration dashboard, then scroll through run history and logs to work out when the job last succeeded and which scheduled report depends on it. Most of this happens from a phone in bed at 2am, tethered to a laptop because the orchestration tool has no mobile view worth using. By the time they have an answer, twenty minutes have passed, and half the time the honest conclusion is that it could have waited until morning, which they could have known from the first message.
Product idea
A chat and voice line the on-call engineer already has in their phone. They text or call the number, say or type the pipeline name, and get back the last run's status, the last known good run, and which scheduled reports or downstream jobs sit behind it, pulled from the orchestration tool's run history. If the answer is 'still broken, first report affected is at 7am', they know to act now. If it is 'failed once, retried, succeeded', they go back to sleep. Nothing is paged or escalated by the tool itself. It answers a question and gets out of the way. It does not diagnose the cause and does not open or close incidents; that stays in whatever incident tool the team already uses.
Who it is for
Data engineers and analytics engineers on an on-call rotation, and the head of data who is currently the fallback everyone calls when the rota holder cannot get a laptop open.
Possible first version
A phone number wired to SMS and a short voice menu, backed by a read-only connection to the pipeline orchestration tool's run history. It answers status queries for pipelines already registered in a lookup table mapping job name to the reports and downstream jobs it feeds. Voice replies are text-to-speech readbacks of the same data the SMS reply gives. Out of scope for version one: paging or escalation, root-cause diagnosis, and any write access to the orchestration tool. It only answers what is already true in the run history.
- Build classification
- Micro-tool
- Rough effort
- 8-10 day prototype
- Roles involved
- Data engineer, Analytics engineer, Head of data
- Relevant to
- Professional club, League office, Federation / governing body, Esports property
- Systems in play
- Pipeline orchestration and scheduling, Telephony and contact centre tools, Messaging apps
- Product framing
- Voice or chat interface
Questions we get asked
What do we need to give it before it can answer anything?
A read-only connection to your orchestration tool's run history, and a short lookup table someone writes once mapping each pipeline to the reports or jobs it feeds. That second part is manual to start: nobody has that mapping written down anywhere, so building it is most of the setup work. Without it the tool can say a job failed but not why that matters, which is the whole point. No warehouse access, no write permissions, and no connection to the incident or paging tool are needed on day one.
Does this replace our on-call paging tool?
No. The page is still what wakes someone up, and stays exactly as it is. This answers the question that follows the page: is this bad enough to act on now, or can it wait until the morning stand-up. If your paging tool already surfaces enough context to answer that from the alert itself, you do not need this. Most do not; they surface a job name and a status code, not what depends on it.
We already get pinged in Slack when a job fails. Why do we need a phone line as well?
Because opening Slack on a phone at 2am, finding the thread, and reading a stack trace is roughly the same friction as opening the dashboard. It is not actually faster, it just feels more modern. A text or a short call answers one question in one exchange without opening an app at all. If your team is genuinely comfortable digging through Slack threads half asleep, this adds little, and that is worth being honest about before building it.
What happens if the orchestration tool itself is down when someone calls?
It says so rather than guessing. If the run history API does not respond, the reply is that status is unavailable and to check the dashboard or logs directly, not a stale answer dressed up as current. This matters more here than in most tools, because a wrong 'all clear' at 2am is worse than no answer at all. The tool is deliberately conservative about what it claims to know.
Is this your workflow?
Tell us one sports workflow that still runs on paper, spreadsheets, WhatsApp or an outdated system. We will map it and show you what a simpler product looks like.
Tell us about itMore in Data platform & engineering
- Access permission drift monitor for warehouse and reporting toolsA monitor that compares entity access rules across the identity provider, warehouse and reporting tool, and alerts a named owner when they drift out of sync.
- Attendance reconciliation report across ticketing and till systemsA reconciliation report that turns a ticketing export and a till export for the same fixtures into one defensible attendance and spend figure, with the variance between them shown and explained.
- Chat intake assistant that triages ad hoc data requestsA chat assistant that answers a data request instantly when the metric already exists, and turns anything new into a structured, tracked ticket instead of a message lost in a chat thread.