Six questions to disarm an AI proposal in fifteen minutes¶
A short conversation, not a toolkit card.
You are in a fifteen-minute meeting with someone proposing an AI solution. The vendor deck is polished; the pilot is half-sold. These six questions, walked in order, will tell you whether the proposal holds up. Each is answerable in a sentence or two by someone who knows the work — but each forces a pause first. The reflection is forced by the question; you read the answer, or the blank, for the signal.
Nothing here is a toolkit entry. The book's toolkit is for the people who run the full method. These questions are for the meeting before the method runs — when you need to tell, quickly, whether there is a real case to put through the method, or whether the proposal will dissolve on contact with it.
1. If AI were off the table tomorrow, what problem would you still be left with — and what would you actually do about it?¶
What it probes. Whether the problem exists independent of the tool, or whether the tool is looking for a hook.
A strong answer. A specific problem remains, and the person can describe a non-AI path they would take. "We'd still have a backlog, and we'd hire two more people and rewrite the intake form."
A weak answer. A shrug, a restatement that assumes AI, or silence. "Well, then we wouldn't be modern." The problem only exists because the tool exists.
2. Even if it worked perfectly, would you still not want a machine making this call?¶
What it probes. Reasons to keep AI out of a decision regardless of how capable the model becomes.
Supporting examples, if the expert draws a blank:
- once it is done, it cannot be undone
- the person has no way to push back on it
- they have no real choice but to accept
- they deserve a human making the decision
A strong answer. The person pauses and names at least one of those reasons in their own words. "Yes — if we get this wrong, we cannot walk it back, and the person affected has no way to argue before it's too late."
A weak answer. "If it worked, why would it be wrong?" The question is treated as nonsensical. The decision has never been tested on anything other than capability.
3. Three short beats, in order¶
(a) When did you last watch this work actually happen?
(b) Walk me through it, step by step.
(c) Where in that flow are the actual decisions made — and which of them is this system supposed to take?
What it probes. Observation of the real work, decomposition into steps, and the specific decisions AI is being pointed at. "Steps" and "decisions" are not the same thing; routing depends on finding the decisions inside the steps.
A strong answer. Recent, concrete observation; a clear step list; a pinpointed decision. "I watched it last week. It goes intake, capture, score, assign, re-check, admit. The system is supposed to take the score, not the rest."
A weak answer. "My team handles it." A vague flow. A monolithic "it will do the whole thing." The proposal is still one thing, not several pieces; the decision has not been located.
4. Where would it matter if the same case — same inputs, same situation — got a different answer on two different days?¶
What it probes. Which parts need the answer to be reproducible, and which can tolerate the variance that AI introduces by design.
A strong answer. A clear map across the steps. "The score has to be the same — if two runs disagree on the same case, people get hurt. The summary can vary. The assignment depends on what's available, so it will differ regardless."
A weak answer. "Shouldn't it always give the same answer?" The tool's probabilistic nature has not been faced. The expert assumes AI behaves like a rule.
5. When it gets a case wrong, who's on the receiving end — and how much of the harm can actually be undone?¶
What it probes. Blast radius (who is affected) and reversibility of the consequences — not whether the system's output can be retracted, but whether the harm to the person can be.
A strong answer. A named affected party, a specific worst-case, and a gradation of what can be repaired. "Wrongly flagged application — applicant waits two weeks, we can reissue in a day. Wrongly discharged patient — that one we cannot take back."
A weak answer. "The system will be accurate." Abstractions. No named party, no worst-case, no gradation. Harm has not been sized; reversibility has not been thought about.
6. Who is responsible when it's running, and what would have to happen for you to stop using it?¶
What it probes. Named ownership, and whether the commitment has any exit at all — through rollback, replacement, or retirement.
A strong answer. A specific person (not a team, not a vendor) and at least one concrete condition that would end the system's use. "Priya owns it. We stop using it if the error rate crosses the threshold we've written down, or if the underlying policy changes, or if something better comes along."
A weak answer. "The data team handles it. Why would we stop?" Ownership dissolves on contact with failure. The commitment is open-ended, which means it cannot be kept.
Reading the six answers¶
Not a scorecard. A pattern.
Most answers substantive and specific. The proposal has a real problem, a tested categorical clearance, observed work, a found decomposition, a reproducibility map, named harm, and named ownership with an exit. Run the book's full method — the ground under it is stable.
One or two weak. The proposal is not disarmed; it is underspecified. Send the sponsor back to those one or two points with a week to answer them, and re-meet.
Three or more weak. The proposal dissolves under its own questions. The honest next step is to say so, calmly, in writing — not as a refusal of the sponsor, but as a refusal of the proposal in its current form. If the underlying problem is real (question 1 survived), they can come back with a repaired brief, a non-AI route, or a smaller pilot. If question 1 did not survive either, there is no problem to solve with AI, and the fifteen minutes has done its job.
These six questions are not a substitute for the book's method. They are the fifteen-minute test for whether running the method is worth anyone's time.