Interactive Essay
Before You Recommend It
Seven questions about one specific AI tool, for the person who will have to defend the recommendation. The demo answers none of them.
A logistics client once asked me to evaluate an AI tool that promised to cut their dispatch planning time in half. The demo was flawless. The sales engineer ran a sample route, the model reorganised it in eleven seconds, and everyone in the room — including me, for about ninety seconds — was ready to sign.
Then I asked to see it run on last Tuesday's actual data: the one with three trucks down for maintenance, a driver who had called in sick, and a customer who had moved their delivery window twice. The tool choked. Not dramatically — it just quietly produced a plan that assumed all three trucks were available, because nobody had told it otherwise, and nobody had built a way to tell it otherwise.
The gap was not in the model. It was in the twenty minutes of unglamorous setup nobody had budgeted for.
Every vendor pitch is optimised to answer one question: can the model do the thing? That is rarely the question that decides whether the tool helps.
The question that matters is whether the tool can do the thing inside this organisation — with its data quality, its handoffs between departments, and its tolerance for being wrong in public. After twenty-six years of technology consulting I have watched three or four hype cycles come and go, and the pattern repeats: a genuinely useful capability arrives, the market gets breathless, and a meaningful fraction of early adopters buy for reasons that have nothing to do with fit. AI is no different from ERP or cloud migration in this respect. It is just faster, and the mistakes are cheaper to make and more expensive to unwind.
So before I tell a client to move forward with anything, I run the same five checks. None of them are exotic. All of them get skipped constantly, usually because the sales cycle rewards speed and nobody in the room is paid to slow it down.
Below they are seven questions, because two of the five have a second half that is easy to answer confidently and wrongly. Answer them about one tool you are evaluating right now — not about AI in general, which nobody can evaluate. The verdict at the end is not a grade. It is a list of what you would still have to justify if someone senior asked you tomorrow why you signed.
The checklist
If the verdict came out lower than you expected, that is the useful outcome. Almost nobody fails this checklist through carelessness. They fail it because each of these questions has an owner in theory and no owner in practice, and because the demo worked well enough that nobody went looking for the Tuesday-afternoon version.
The most common objection I get is some version of “we do not have time for a seven-point review, the market is moving fast.” I understand the instinct and I do not dismiss it — speed is a real advantage, and I have told plenty of clients to move quickly on the right opportunity. But I have never seen a company regret spending two extra weeks confirming exit cost and data ownership before signing. I have seen several regret not spending it.
The checklist is not a brake pedal. It is closer to landing gear: something you want confirmed and locked before you commit, not something you improvise once you are already coming down. Run well, it takes days, not months.
Ask these questions out loud, of the vendor, before the contract. The ones worth working with will have crisp, specific answers. The ones who get vague or defensive are telling you something as important as the pitch itself.
Which is the truer reason the last tool went in without a review like this: (A) nobody had the standing to slow the sales cycle, or (B) nobody wanted to be the person asking? A or B — and what did it cost?
More of this work
I write about the places where technology decisions meet ethics, consciousness, and the stories we tell about machines — across essays and a shelf of books on AI governance, the mind, and fiction.
Explore the catalogue