Analytics++

At 14:35 UTC on July 15, the AI credit balance behind Daylight ran dry. A user who had signed up two minutes earlier finished her first journal entry, 98 characters, watched the confetti fire, and waited for the automatic report the product had just promised her. The job producing it died two seconds after her entry landed. No error ever reached her screen. She simply never got the thing she came for.

The next morning's scheduled analyst run picked up the trail. In a single investigation, it:

• Noticed the product had gone quiet overnight: zero automatic reports since mid-afternoon, and four job failures filed under the wrong error type.
• Pulled the failed jobs and found the same provider message on all four. The AI account had run out of credit.
• Checked the commit history to clear our own code, then read the failing worker and pinned the exact line where the pipeline dies.
• Kept reading and found a second casualty: her $15 founding-member discount gets granted a few steps past the failure point, so it never armed. She lost the report and the deal.
• Triaged the four affected users by what each one actually experienced. Three returning users lost a minor enrichment each. One brand-new user lost her entire first impression. It flagged her by name and emailed me.

Then it kept the case open for five days. It recommended a credit top-up plus a backfill of her failed entries, escalated when both recommendations sat unacted for 48 hours, caught that the top-up happened but the backfill never ran, and graded its own recommendation "failed" in its tracker. It closed the case on July 21, when she finally received her first report and her discount armed, five days late.

I did none of this. The investigator is an agent that wakes up every morning in a sandbox holding Daylight's production event stream, its error log, its codebase, and its git history. I wrote about the architecture in an earlier post. This post is about what to call the discipline the thing practices, because "analytics" undersells it and I have no better word. So I have been calling it analytics++.

The walls were about people

Watch what that one investigation did. It read a time series and spotted an anomaly, which is data science. It parsed provider errors and traced worker code, which is software engineering. It reasoned about a founding-member offer and a broken first-session promise, which is product. It triaged individual users by what each of them went through, which is support. It tracked remediation and graded the outcome, which is program management.

In most companies that chain crosses four or five job descriptions, and every handoff is a place the question can die. The data scientist sees reports flatline and files a ticket. The engineer sees a billing error and tops up the account. Nobody connects the two to one specific person's ruined first five minutes, because no single role owns the full chain.

Those job boundaries came from hiring reality. Expecting one human to be strong at funnel analysis, stack-trace forensics, attribution mechanics, product judgment, and per-user detective work was never reasonable, so we cut the work along the seams of what a person can be good at. The problems never agreed to the cut. Real incidents run through the whole stack at once, and the interesting ones live in the seams, exactly where everyone's dashboard ends and the next team's begins.

Machine reasoning changed the constraint. An agent can hold the event stream, the error log, the source code, the git history, and the ad platform inside a single investigation without getting worse at any one of them. Analytics++ is what analytics becomes once the walls come down.

Two more cases, both from a single day, make the argument better than any definition.

The buyer we had to un-believe

On July 10, Daylight got its first subscriber, and the daily report celebrated a milestone: the first buyer with no ad click attached. Our first organic customer. Eighteen hours later a deeper investigation took the milestone apart:

• It pulled her raw attribution data and spotted a gbraid parameter, the click ID Apple substitutes on iPhones for privacy.
• It matched her campaign ID against every previous buyer's landing page. Same ad campaign as all of them.
• It found the bug: our attribution kept only the standard click ID, so every iOS ad click had been counted as organic since launch.
• It sized the blast radius: roughly 5 percent of paid signups, all invisible to our paid-versus-organic split.
• It retracted its own published claim in writing, "That was wrong; corrected in this report," and the fix shipped the same day.

The flattering version of that story would have redirected marketing effort toward an organic channel that did not exist. A system that can celebrate a milestone on Tuesday and dismantle it Wednesday with receipts is doing something dashboards were never built to do: it holds beliefs, and it checks them.

Nineteen logins in twenty-four hours

Same day, same subscriber, different thread. She re-authenticated 19 times in her first day, and the system went looking for why:

• It measured the gaps between her logins. Only two of nineteen exceeded the hour-long session token's lifetime, and one gap was 24 seconds. Expired tokens were ruled out by arithmetic.
• A re-login after 24 seconds means the browser itself is discarding her session, so it split the user base to test that theory: every Android user held a single session for days, and all four chronic re-loggers were on iPhone.
• Verdict on her device: Private Browsing or an in-app browser wipes her cookies on every visit. Nothing we can fix server-side.
• Then it turned on our own code and found what we could fix: an eight-hour session cap in our config against the auth library's 400-day default, plus two code paths that turn a network hiccup into a login screen.
• Its summary line stuck with me: she demonstrated that motivated users tolerate this, and the other 96 percent of mobile users should never be asked to. The fix shipped that afternoon.

A support rep would have shrugged at one user's login trouble. A data scientist would never have read the auth config. An engineer would never have run the Android-versus-iPhone split proving the problem was real but not universal. The finding needed all three at once.

Severity in units of people

The common thread across these cases is the accounting. Error counts and metric deltas describe pain the way a machine experiences it. Blast radius, expressed as what specific humans went through, tells you what to do about it. The credit outage was five failed jobs and roughly zero dollars, and the report escalated it anyway, because one of those five jobs was somebody's first impression and, in its words, at higher volume this same failure mode is a mass first-impression outage. When packet sales went quiet for a stretch of mid-July, the same system investigated, concluded the quiet was baseline and the late-June hot streak had been a roughly 2 percent fluke, and declined to page me.

A dashboard can make neither call. Both require walking from the metric to the mechanism to the individual people involved, plus a memory of what normal looks like. That memory lives in a table: named beliefs, each pinned to the SQL that would falsify it, re-verified on a rotating schedule, retired in writing when refuted. The July 10 retraction is a row in it.

What it runs on

The stack underneath is small and boring on purpose: an append-only event stream, an error log with one fingerprinted row per distinct bug, sanitized views so the agent can reason about behavior without ever reading a word a parent wrote, a ledger of beliefs with their evidence attached, and the product's own repo and git history mounted next to it all, because metric moves have commits behind them. Wide latitude to read on the inside, hard walls on the outside. The details are in the architecture post; none of it took a platform team, and all of it accreted one useful table at a time inside a single Postgres schema.

The job req will still say "analyst"

Part of what always made a senior analyst expensive was the instinct to ignore the org chart: to walk from a funnel chart into a stack trace and out the other side into a product decision. That instinct now has a runtime. What stays human is the work I actually spend my time on now, choosing what the system can see, curating the beliefs it keeps, grading its calls, and acting on the ones that matter. Analytics got a superset, and my job moved up one level inside it.

If you run analytics, or you are deciding what your data team should look like two years from now, I would like to compare notes: kyle@monumentlabs.io.