Screener vs Reality: Recording What Actually Happened

A recruiter at a mid-size IT services firm asked us a question in August that we could not answer: “Of the candidates your AI said Go on, how many of them are actually still here?” We had a verdict for every one of the 1,000-plus pilot interviews we had run. We had an outcome for none of them. The screen ended at the recording and the transcript, and everything after that, offer, joining, attrition, lived somewhere else, if it lived anywhere at all.

That gap is what AI screening outcome tracking is supposed to close, and it is the reason HireQwik shipped an outcome capture layer to production on August 26. A screening record can now carry the real ending, hired, offered, not selected or withdrew, per candidate, and a JD-level card compares that ending against what the AI actually decided.

Why “the AI said Go” was never the finish line

Every screening vendor, us included, will happily show you a verdict distribution: how many Strong Go, how many Go, how many On Hold, how many No Go. That number describes the screen’s opinion. It says nothing about whether the opinion was right, because nobody ever went back and checked what happened to the person on the other end of it. Metaview’s screening research frames this as a signal-versus-noise problem: a screen that only reports its own confidence, without a downstream check, cannot tell the difference between being right and just being consistent.

The reason nobody checks is not laziness. It is that the ending lives in a different system, or in a recruiter’s head, or in an offer letter nobody logged anywhere the screening tool can see. A verdict you never audit against reality is not validated, it is just unexamined, the same problem we described when we wrote about why we hand-audit a sample of every No-Go batch. Outcome tracking is the same instinct applied to the whole funnel instead of a rejected sample: put the ending back next to the decision and look at both at once.

The Outcome control: giving a screening record its ending

The mechanic is a control on the /inbox Screened expanded row and on Interview Details. A recruiter sets one of four values, hired, offered, not selected or withdrew, against a candidate’s record. It is org-scoped, so one company’s outcomes never bleed into another’s, it is audit-trailed, and it is clearable if someone records the wrong thing and needs to fix it.

That sounds like a small addition. It is the addition that turns a screening tool into a system that can be checked. Before it existed, the only honest answer to “did the AI get this one right” was a recruiter’s memory of a specific candidate, which does not scale past a handful of names and does not survive the recruiter changing roles.

The Screener vs reality card: reading the verdict-by-outcome matrix

Per JD, HireQwik now builds a “Screener vs reality” card from a verdict-by-outcome matrix: for every combination of what the AI decided and what actually happened, how many candidates landed there. Strong Go candidates who got hired. Go candidates who were offered but withdrew. No Go candidates who somehow got hired anyway.

Reading that matrix is a different exercise than reading a verdict distribution. A verdict distribution tells a recruiter how confident the screen was. The matrix tells them how often that confidence matched what happened next, JD by JD, not averaged across every role the company has ever run through HireQwik. A rubric that looks fine in aggregate can be quietly wrong on one specific role, and the aggregate number is exactly what hides that.

The one cell that matters: hired_no_go

Every cell in that matrix is informative, but one gets its own name in the product because it is the miss that costs the most: hired_no_go, a candidate the screen rejected whom the team hired anyway. A manager override, a parallel process outside HireQwik, a second look nobody logged, whatever the specific route, the hire happened despite the No Go, not because of it.

Most screening reports do not surface this cell at all. They average it into an overall accuracy number, if they compute one, and a single-digit hired-anyway count disappears into a denominator of hundreds. HireQwik reports it as its own line instead of averaging it away, because the whole point of tracking outcomes is to see the misses that cost something, not just the ones that are statistically convenient to report.

A candidate an AI screen wrongly rejects is not a rounding error. It is a person your own hiring process later decided was worth hiring, on evidence the screen had access to and scored the wrong way. That is exactly the population our No-Go audit habit is designed to catch by sampling; outcome tracking catches the subset of it that a human somewhere in the org already proved was a real miss, because they hired the person anyway.

The bucket most reports would rather hide: never screened

The matrix has a second bucket that is easy to leave out and hard to justify leaving out: hires who were never screened through HireQwik at all. A referral who skipped the funnel, a walk-in from a campus drive who was hired off a paper form, a lateral pulled in through a personal contact. HireQwik surfaces this bucket visibly rather than excluding it from the report.

The instinct to hide it is understandable. It is not the screening tool’s failure, and reporting it looks like admitting the funnel has holes a vendor did not cause. But leaving it out flatters the funnel by construction, showing a cleaner picture of coverage than the one that’s actually true. If a third of a company’s actual hires for a role never touched the screen, that is the single most important fact about how well the screen is actually covering the hiring process, and it belongs on the same card as everything else, not in a footnote nobody reads.

A worked example, with the numbers rounded and disguised

Picture a fictional but realistic JD: a customer-support role at a mid-size BFSI campus drive, forty candidates screened over one intake. Say the AI called eighteen Strong Go, fourteen Go, five On Hold and three No Go. Before outcome tracking, that distribution is the whole story anyone gets. After it, the card adds a second layer: of the eighteen Strong Go candidates, sixteen were hired and two withdrew before joining. Of the fourteen Go candidates, nine were hired, three were not selected after a further round, and two are still pending. Of the five On Hold candidates, two were eventually hired after review and three were not selected. Of the three No Go candidates, two were not selected, matching the call, and one was hired anyway through a manager override, the JD’s one hired_no_go cell.

Nothing in that picture says the screen is broken. Sixteen of eighteen Strong Go hires joining is a strong match. But the single hired_no_go candidate is worth a specific, five-minute look, not because one data point proves a pattern, but because it is exactly the kind of case that used to be invisible entirely. Multiply this JD by every role a company runs through HireQwik in a season, and the value of the card is not any single cell, it is that every JD gets this same second layer instead of only the confident, high-volume ones a TA lead happens to remember to check by hand.

Who actually fills this in

A measurement layer is only as good as the discipline behind entering the data, and outcome tracking has an obvious failure mode: nobody bothers to record the ending, and the card stays empty forever. We thought about this before shipping, because a feature that requires a new habit and gives nothing back for the first few weeks tends to die quietly.

The design choice we made was to put the control exactly where a recruiter already is when the ending becomes knowable, the /inbox Screened row and Interview Details, rather than a separate reporting screen nobody opens on its own. A TA lead closing out a requisition is already touching those records to archive the JD; recording hired, offered, not selected or withdrew at that moment costs seconds, not a new workflow. It will still lag reality by however long it takes an offer letter to turn into a confirmed joining, and a recruiter who genuinely forgets will leave a gap. We are not claiming the discipline problem is solved, only that we tried to put the control at the point of least resistance instead of adding a new task nobody asked for.

There is a second reason this matters operationally: outcome entry is not a one-time action per candidate. A candidate marked “offered” can later become “not selected” if they decline, or the field can be corrected if a recruiter mis-clicked. That is why the control is clearable rather than a one-way commit. A hiring process is messier than a single state transition, and a measurement tool that assumes otherwise produces confidently wrong numbers instead of honestly incomplete ones.

Why counts, not rates, below ten

One more deliberate choice in how this reports: below ten outcomes for a JD, the card shows raw counts instead of a computed rate. Three hired-no-go candidates out of nine screened is not “33%,” it is three candidates, and presenting it as a percentage borrows a precision the sample size does not earn. A JD that has only produced eight outcomes yet is not ready for a rate-based read, and the card says so by refusing to compute one.

This matters more at Indian campus and bulk-hiring scale, where a single opening can pull thousands of applicants in a week but a specific role at a specific college might only produce a handful of actual hires. Small-sample honesty is not a nice-to-have here, it is the difference between a genuinely useful signal and a number that looks precise and is not.

What v1 deliberately does not do

Outcome data does not feed back into auto-decide thresholds automatically, and that is by design, not a missing feature waiting to ship. Auto-decide bands are a recruiter’s own configuration, untouched by anything this card shows them. We built the measurement layer first and are treating the “should this data retune the thresholds” question as a separate, harder decision that deserves its own scrutiny rather than a quiet default. We wrote about why that specific question deserves more caution than a build ticket, because the failure mode of getting it wrong compounds silently.

There is an honest wrinkle worth naming too: during the build, a hire-only week briefly got suppressed as “empty” in the weekly recap, the one week the recap exists for. It was caught and fixed before ship, not after a customer noticed. We would rather tell you that than pretend the feature arrived fully formed. It also lines up with a broader pattern SHRM’s 2026 research found: only about a quarter of organisations with an AI-use policy believe that policy is actually future-proof, which is another way of saying most teams shipping AI into a hiring process know they are not done governing it.

Where this connects to the evidence a recruiter actually hands someone

Outcome data on its own is a JD-level pattern. The record behind a single hire, offer or miss is still the interview itself, the transcript, the per-question scoring, the speech read. When a “Screener vs reality” card flags a hired_no_go cell worth a closer look, the next step is opening that specific interview, and handing the actual evidence to whoever needs to weigh in is the natural next action, not a separate project.

Outcome tracking will not tell you your AI screen is accurate, because “accurate” is not a number we are willing to publish without a real held-out benchmark behind it. What it will tell you, JD by JD, is whether your screen’s confident calls are matching what your own hiring process ends up deciding, and where they are not. For a screening layer that has run more than a thousand interviews across pilot campaigns, that is the first honest report card it has ever had. Talk to us if you want to see what the card looks like on your own JD funnel.