Reef TRHA proof of concept answers one question: is this possible? A production system answers a harder one — is this still right on a Tuesday, run by someone who wasn't in the room when it was built, on the morning it half-breaks? Almost every AI project that later stalled had cleared the first question with room to spare. The second is where they died.
So the useful work before a greenlight isn't admiring the prototype. It's interrogating it. Twelve questions do most of that work, and they sort into four things a pilot has to survive: its data, its bad days, its 2am, and its own economics. Notice, as you go, how few of them are really about the model.
Treat each question as a gate, not a scorecard. You are not hunting for twelve reassuring answers; you are hunting for the one where the honest reply is "we don't know yet." That answer is the finding. It tells you where the project will stall — while stalling is still a line in a planning doc rather than a month-six emergency.
Does the data survive contact with the real thing?
More pilots die on data than on any modelling decision, and none of it shows in a prototype that was handed a tidy sample the night before. Start with the plainest question there is: where does the data this use case needs actually live, and who is the one person who can explain each source's quirks? If you can't name the systems and the humans, you can't cost the work — you're guessing at it.
Then ask what the demo is really running on. Curated examples prove the idea and hide the mess — the half-empty fields, the record a 2019 rename left ambiguous, the field three departments each fill in differently. That mess isn't an error waiting to be scrubbed; it's the business, honestly recorded, and it's exactly what the model meets the moment it leaves the stage. And ask what the thing has to plug into: a fifteen-year-old order system with its own rate limits, authentication, and data rules will often cost more engineering than the model itself. Map that before you commit, not after.
Can you trust the answer on a bad day?
A demo is a scripted best case. Production is every case, including the ones nobody rehearsed. So — do you have an evaluation set and a quality bar you would defend in a meeting, or just a good feeling? You cannot ship what you cannot measure, and "it works" stays a vibe until there is a labelled set behind it.
Next, the edge the demo never saw: the malformed paste, the question no one planned for. Decide the fallbacks, the hard limits, and the safe way to fail, and treat them as the product rather than a patch bolted on later. And decide, before launch, how you will catch a confident wrong answer before a customer does — guardrails, human review, monitoring, something named and owned. A probabilistic system will eventually invent a policy that doesn't exist; the only real question is whether you or your customer finds it first.
Who runs it at 2am?
Shipping the model is where the work starts, not where it ends. So name the owner: who is responsible for this in production at 2am when it falls over? If the answer is "the people who built the demo, in whatever time they can spare," the system has no real owner, and unowned systems rot. Decide, too, what belongs on the dashboard — drift, latency, cost, quality — and who actually watches it, because models degrade quietly as the world shifts under them. And insist on a way back: a kill switch, and a route to the last known-good state. If there is no safe way to turn it off, it isn't ready to turn on.
Is it worth building at all?
AI has unit economics, and sometimes the honest finding is that it's the wrong tool. Work out the fully loaded cost per call — tokens are the visible bill; evaluation, infrastructure, maintenance, and human oversight are the rest — and weigh value-per-call times volume against that total, not against the API price alone. Ask whether a rule, a script, or a database lookup would do the same job cheaper and more predictably; for plenty of tasks it will, and the model should be reserved for the part that genuinely needs one. Finally, define the smallest slice worth shipping and the specific number that clears it. A greenlight ought to be a threshold you agreed in advance, not a feeling in a review room.
That is twelve. Answer them honestly and you will not have removed the risk — you will have moved it. The expensive surprises shift out of month six and into the room where they still cost nothing worse than a hard conversation.
Questions buyers actually ask
Isn't a working demo already proof it will work in production? No. A demo proves the idea is possible on chosen inputs. Production tests it on the unchosen ones, under the integration, ownership, and cost constraints the demo never touched.
How long should running this checklist take? An afternoon, with the right three or four people in the room — whoever owns the data, whoever will operate it, and whoever holds the budget. If you can't get them around a table, that is your first finding.
What if we can't answer several of the questions? Good — you found the gaps while they were cheap. Unanswered questions are a map of where to look first, not a verdict that the project is doomed. The failure is meeting them for the first time in month six.
Run any pilot through these before you commit a roadmap to it. The ones that survive the questions are worth an implementation sprint rather than another prototype; the ones that don't were going to fail anyway — just later, and more expensively, in the POC graveyard.