Reef TRHThe assistant answered in about four seconds. A support lead had pasted in a genuinely nasty ticket — a double charge, a return that had gone missing, a customer three emails deep into losing patience — and the model came back with a reply that was calm, accurate, correctly apologetic, and on brand. It would have taken a human agent fifteen minutes and a supervisor's glance. Someone in the room actually clapped. The pilot was greenlit before lunch.
Eleven weeks later it still wasn't live. Nobody had cancelled it — you don't cancel something that looked that good in the room. It was simply stuck, marooned in the endless "almost ready" where most AI pilots quietly go to die.
The story that follows is a composite: the details changed, the pattern exact. We've been called into this room more than once.
What the demo was built to hide
A demo is a performance, and like any performance it is staged. This one ran on twenty hand-picked tickets the team already knew the model handled well. It touched one system, read-only, on a laptop. It was watched by people who wanted it to succeed. None of that is dishonest. It's just what a demo is. The trouble is that a business doesn't run under any of those conditions.
Pointed at the real inbox, the assistant met the other ninety per cent. Three languages in a single thread. A screenshot where a sentence should be. An order number that lived in a system the demo had never been wired to. The model didn't collapse in an obvious way — it did something worse for trust. It stayed confident while quietly getting things wrong, inventing a refund policy that had never existed and citing a delivery guarantee the company didn't offer.
A proof of concept proves an idea can work once, in conditions you control. It says nothing about whether it works on the Tuesday-afternoon inbox, wired to the systems that hold your data, when no one is watching. Those are different problems — and the pilot had only ever solved the first.
Then there was the plumbing. Connecting the assistant to the order system — the thing that would let it stop guessing and actually look a charge up — turned out to be most of the real work: rate limits, authentication, a data-retention rule legal surfaced late. And beneath it all, a quieter gap — nobody owned the thing. It had an enthusiastic sponsor and no operator, no one whose job was to answer for it at 2am when it told a customer something untrue.
What actually got it across the line
Getting it live took roughly nine weeks, and almost none of that was model work. It began with a blunt triage: what was salvageable, what had to be rebuilt, and which failures were fatal rather than cosmetic. The idea was sound and the prompt work was fine. Everything around it was missing.
So we rebuilt the data path first, against the actual inbox instead of the curated sample, so the assistant could look an order up rather than imagine one. We added the guardrails that mattered: hard limits on anything touching money, a confidence threshold below which the system drafts a reply but does not send it, and a human on every refund above a set amount. We put in monitoring, so a dip in quality showed up on a dashboard instead of in a furious complaint. And we named an owner — one person accountable for the assistant the way they'd be accountable for any other system in production.
None of that is what the execution gap looks like in a keynote. It's what it looks like on a Wednesday. The distance between a prototype that proves an idea and a system a business can run on is rarely a modelling problem; it's data, guardrails, and ownership — the unglamorous engineering a demo is allowed to skip. A pilot stranded in its eleventh week is precisely the kind of thing a Production Rescue exists to close.
When it finally launched, it did not handle everything. It handled the sixty per cent of tickets it could handle safely, escalated the rest, and was honest about which was which. That's what shippable looks like — and it is far less photogenic than the demo.
Common questions
Why do most AI pilots stall after a successful demo? Because the demo and the production system solve different problems. The demo runs on curated inputs and an audience that wants it to win; production faces the unchosen inputs, the real integrations, and the failure someone has to own. The model is rarely the issue.
How long does it take to get a stalled pilot into production? This one took about nine weeks, nearly all of it in the data path, guardrails, monitoring, and ownership rather than the model. The honest answer depends on how much of that engineering the original pilot skipped — which is what a short triage is there to find out.
Do you rebuild it or fix what's there? Fix it, wherever we can. The idea and the prompt work are usually sound; the failure modes sit around them. Targeting those is faster and cheaper than starting from a blank page.
No one clapped when it went live. There was no gasp in a meeting room, no round of applause — just a quiet Tuesday, a queue that moved a little faster, and a number on a dashboard that stayed where it should. That's the tell, in the end. The pilots that get applause are the ones built to be watched. The ones that survive are built to be trusted when no one is.