1,099 Interviews, 14 Campaigns: An Honest Post About Where AI Screening Breaks

Across 14 pilot campaigns we’ve run 1,099 AI voice interviews, and the number we get asked about most is the wrong one. Everyone wants to know the time saved — fair, it’s a real 89% reduction in HR time per candidate versus a manual phone screen. Almost nobody asks the harder question: across those 1,099 conversations, where did the screen get it wrong, and what did we do about it? This post is that answer, not the highlight reel.

The Numbers That Actually Matter

Start with what worked. One evening, one campaign, 3,000 candidates screened in about two hours — a task that would have taken a recruiting team roughly 18 hours of manual phone screening, compressed to 1–2. Auto-rejection ran above 60% on that campaign, which sounds aggressive until you remember what it replaces: a human screener working through the same queue at 10–15 minutes per call, running out of both time and attention long before the queue does. At our current price point of ₹39 per interview, running that volume through a manual phone-screen process instead — at ₹85–150 per candidate for a 10–15 minute call — would have cost roughly 2–4x more per candidate and taken a week instead of an evening.

None of that is in dispute. The part worth being honest about is what the 60% auto-rejection rate doesn’t tell you: it doesn’t tell you how many of those rejections were correct.

Where the Screen Actually Struggles

Three honest limitations, in the order our pilot partners have raised them:

Accent and dialect edge cases. A structured 15–20 minute voice conversation scores communication — pace, fluency, coherence of the answer — not just keyword content. That’s the whole point. But it also means a candidate with a strong regional accent speaking clear, correct English can occasionally score lower than a neutral-accent candidate saying less of substance. We catch most of this in calibration, but “most” isn’t “all,” and any vendor who tells you otherwise hasn’t run enough campaigns to have found the exceptions yet.

The middle band is genuinely hard. Strong Go and clear No Go calls are where the AI screen earns its keep — those decisions are usually unambiguous even to a human listening to the same recording. The candidates sitting in the “on hold” band are where judgment calls happen, and that band is exactly where we tell HR teams to keep a human in the loop rather than let a threshold decide alone.

Nervous candidates read differently than unqualified ones. A 20-minute structured conversation is less forgiving of interview nerves than a resume is. We’ve had pilot partners flag candidates who froze early in the call and recovered strongly by the closing minutes — a pattern a rigid pass/fail cutoff on early responses would miss entirely, and one reason we weight the full conversation rather than just the opening exchange.

Why We’re Telling You This

SHRM’s own research found that 88% of HR leaders now see AI screening as a compliance risk — not a technology risk, a compliance one. That number exists because too many vendors sell a screening tool as a black box that never explains a rejection. We built our pilot process around the opposite instinct: every auto-rejected candidate gets a reason tagged to the call, every “on hold” candidate gets surfaced to a human, and every campaign gets a calibration review before its next run. HyperVerge, one of our pilot customers, has run enough of these campaigns with us to catch calibration drift early — that’s a working relationship, not a vendor demo.

The Contrarian Take

The uncomfortable truth about AI screening isn’t that it makes mistakes — every screening method does, including the manual phone screen it replaces. The uncomfortable truth is that most vendors won’t tell you where theirs breaks, because admitting a screening layer has a middle band that needs human judgment sounds like admitting the product doesn’t fully work. It works. It also has edges. A vendor that can’t name its own edges hasn’t looked hard enough for them, or isn’t going to tell you what they found.

If you’re evaluating an AI screening tool for your next campus drive, ask the vendor this one question: “Walk me through a candidate your system got wrong, and what you changed afterward.” If they don’t have an answer, they haven’t run enough real campaigns to have one — 1,099 interviews is roughly how many it takes before the failure modes stop being hypothetical.

Want to see how we handle the middle band on your own applicant pool? Talk to us.