Questions to ask an AI consultant

A list of questions is only half useful, because the difficulty is not asking them, it is recognizing a weak answer delivered confidently. So each of these has both.

What have you told a client not to automate?

Good: a specific example, with the reason. Low frequency, high exception rate, the human contact was the value, the process was broken and needed fixing on paper first.

Bad: hesitation, or a generic answer about how it depends. Somebody who has never talked a client out of something has never been diagnosing, only selling.

Walk me through a project that went badly.

Good: told without much prompting, with what they changed afterward. People who have done this work have scars and are usually relieved to discuss them.

Bad: a challenge that turned out fine. That is a sales story wearing a failure costume.

Who specifically will do the work?

Good: names, and a willingness to put them in the agreement.

Bad: "our team". The commonest substitution in consulting is a senior person selling and a junior person delivering.

What does this cost to run each month after it is built?

Good: an immediate answer covering model usage, platform fees, and their own ongoing involvement, plus the observation that usage-based costs scale with volume.

Bad: treating it as a detail for later. A build price with no run cost attached is an incomplete number and they know it.

What breaks?

Good: a list. Wrong outputs on unusual phrasing, failures when a source system changes a field, the case nobody anticipated, degradation as your business drifts from what it was told.

Bad: reassurance. Anybody who has run something in production can answer this in detail, and anybody who cannot has not.

What happens when it does not know something?

Good: it stops and escalates to a named person with the context attached, and here is how that route was tested.

Bad: an explanation of how capable the model is. The answer to this question always involves a human, and evasion here is the single most predictive bad sign in the whole list.

Whose accounts will this run under?

Good: yours, registered and billed to you, with administrator access your side.

Bad: theirs, for convenience. Convenient at setup and it means you cannot leave. Why this matters more than any exit clause.

Do I get the prompts and configuration in readable form?

Good: yes, exported, kept current, and it is in the agreement.

Bad: "that is our proprietary methodology". In AI work the configuration is the product. Understand what you are being told.

Could another firm execute your roadmap?

Good: yes, and here is why we write it that way.

Bad: an explanation of why their involvement is essential. If the advice only works when they build it, the diagnosis and the sales pitch were the same document.

What do you need from us, and how much of our time?

Good: specific. Access to these systems, an hour each with these roles, a decision-maker reachable within a day, and a baseline number.

Bad: "very little of your time". Untrue for any real engagement, and it means overruns caused by your unavailability will be framed as your fault later. Availability is the top cause of slippage.

Where does our data go, and does anyone train on it?

Good: which systems they touch, where processing happens, which third-party providers are involved, retention and deletion terms, and a flat no on training.

Bad: surprise that you asked. This should have come from them first.

What would you recommend if you did not build anything?

The one I would keep if I could only ask one, and it is uncomfortable for us too.

Good: a real answer that includes buying something off the shelf, fixing a process instead, or doing nothing yet.

Bad: an answer where the recommendation happens to be exactly what they sell. Every firm in this field has a hammer. What you are testing is whether they know it.

The pattern

Notice how many of the good answers are unflattering to the person giving them. Admitting failure, naming run costs, conceding you can leave, saying you need real time from their client.

Specificity that costs the speaker something is the signal. Confidence is not, and in this category confidence is abundant.

If you would rather test judgment than collect answers, run the meeting as a working session and hand them a process that should not be automated.

The mirror image of this page is worth reading too: what they should be asking you is a faster read on competence than anything you can ask them. And how to run the meeting itself.

If you want to point these at us, do: that is what the first call is for, and how we structure the engagement answers about half of them in advance.