Cycle-time dashboard for data requests from ask to delivery
Problem
A request for a new field or feed starts as a message in the data team's Slack channel or a ticket in the project tracker, and from there nobody can say with any confidence how long it actually takes to land in the warehouse. The team's own service-level page promises a turnaround measured in days. The lived experience, when anyone bothers to check, is weeks, and the gap between the two is never explained because no one has ever lined up the ticket's created date against the pull request's merge date against the date the table actually appeared in the warehouse. Requests that stall do so quietly: sitting in a reviewer's queue, waiting on a legal sign-off for consent-linked data, waiting on a stakeholder to confirm a definition. The head of data hears about the stall only when the requester complains.
Product idea
A dashboard that reconstructs the real journey of a data request without asking anyone to fill in a status field. It reads the ticket's created and closed timestamps from the tracker, the pull request's opened and merged timestamps from git, and the timestamp the resulting table or column first appeared in the warehouse, then lines the three up per request. The output is a cycle-time distribution per stage: intake to build start, build to review, review to warehouse. It flags the stage carrying the longest median dwell time and lists the specific requests currently stuck there, aged by how long they have sat. It does not try to manage the request or notify anyone; it only shows where the queue is actually stalling.
Who it is for
Head of data and analytics engineering leads who need evidence for where the request queue stalls, and the data engineers whose day-to-day backlog it reconstructs.
Possible first version
A single dashboard reading three manually exported CSVs: ticket timestamps from the tracker, pull request timestamps from git, and a list of warehouse table creation dates. It computes stage-by-stage cycle time, plots the distribution, and lists the ten oldest requests currently stuck in the slowest stage. Refresh is a manual re-upload, not a live connection. Out of scope for version one: any live API integration with the ticketing tool, git provider or warehouse; any alerting or notification; and any write-back that changes a ticket's status. It answers where the stall is, nothing more.
- Build classification
- Micro-tool
- Rough effort
- 10-day prototype
- Roles involved
- Head of data, Data engineer, Analytics engineer
- Relevant to
- Professional club, League office, Federation / governing body, Collegiate athletics
- Systems in play
- Service desk and ticketing tool, Version control and pull request history, Cloud data warehouse and lakehouse, Transformation and modelling framework
- Product framing
- Find the bottleneck
Questions we get asked
What do we need to have in place before this is even worth building?
It needs the three timestamp sources to actually line up: a ticket ID that appears somewhere in the pull request title or description, and a naming convention that lets the resulting warehouse table be traced back to the request that asked for it. If those links do not already exist as a habit, matching requests across systems is manual and the dashboard will be built on guesswork. Fixing that convention is worth doing before the build starts, not during it.
We already get a burndown report out of the ticketing tool. Isn't this the same thing?
A ticketing tool's own report counts tickets opened and closed, which is not the same thing. The actual stall in most teams sits between a ticket marked as accepted and a pull request opened weeks later, a gap invisible to the tracker because nothing happened inside it. This dashboard exists specifically to see that gap by joining the tracker's timestamps against git and the warehouse rather than relying on any one system's own status field.
Isn't this just going to be used to put pressure on individual engineers?
That risk is real, and it is a fair thing to raise before agreeing to build this. The dashboard reports dwell time by stage, not performance by named engineer, and the stuck-requests list shows what is stuck rather than who is slow. Whether it stays that way is a management decision the tool cannot enforce; it can be misused exactly like a burndown chart can.
What happens if the three exports don't line up cleanly for a given request?
Version one has no live connection, so a request that cannot be matched across all three exports simply drops out of the distribution rather than appearing with a wrong stage time. That is a real limitation of working from manual CSVs rather than live APIs, and it is the reason a live integration would be a sensible second phase once the matching logic has been proven on exported data.
Is this your workflow?
Tell us one sports workflow that still runs on paper, spreadsheets, WhatsApp or an outdated system. We will map it and show you what a simpler product looks like.
Tell us about itMore in Data platform & engineering
- Access permission drift monitor for warehouse and reporting toolsA monitor that compares entity access rules across the identity provider, warehouse and reporting tool, and alerts a named owner when they drift out of sync.
- Attendance reconciliation report across ticketing and till systemsA reconciliation report that turns a ticketing export and a till export for the same fixtures into one defensible attendance and spend figure, with the variance between them shown and explained.
- Chat intake assistant that triages ad hoc data requestsA chat assistant that answers a data request instantly when the metric already exists, and turns anything new into a structured, tracked ticket instead of a message lost in a chat thread.