How to judge an AI consultant's record

There is a genuine problem here. Most of this work happens inside other people's operations under confidentiality, so the detailed case study you want frequently cannot exist. Which conveniently also means anybody can claim anything.

Four kinds of evidence hold up regardless. None of them are case studies.

Demonstrated behavior under stress

The strongest available and the least fakeable. Ask to see something working and then try to break it yourself.

Feed it the awkward case from your own business. Ask something outside its scope. Say something ambiguous. What matters is not whether it handles everything, because nothing does. It is whether it stops cleanly and hands over, or improvises confidently.

A firm that has shipped this work knows their system's limits and will show you them without embarrassment. That confidence is itself the evidence.

Specific accounts of failure

Ask about something that did not work, and listen to the texture of the answer.

Real answers contain details nobody would invent: the exception rate that turned out to be triple the estimate, the vendor who renamed a field and broke everything silently for two weeks, the team who quietly kept using the spreadsheet, the data where the same customer existed three times with different identifiers.

Fabricated answers are structurally different. They describe a challenge, then a resolution, then a lesson, and they are suspiciously tidy. Nobody's real failure has a clean arc.

Technical writing that could only come from doing it

Look for published explanations of how something was built and what the tradeoffs were. Not predictions about how AI will change your sector.

The test is whether the writing contains decisions. Why this approach over that one, what was tried and abandoned, where the design is a compromise. That kind of detail is very expensive to fake and nearly always genuine.

References who will take a phone call

Written testimonials are marketing copy. A person on the phone is evidence.

When you get one, ask the questions that produce information: what took longer than expected, what went wrong, what would you do differently, would you use them again for the next thing. The last question is the one that matters, and hesitation in the answer is the answer.

If confidentiality prevents a reference, ask for one who will speak without naming their company's specifics. Most clients will.

What to discount

Certifications from bodies that exist to sell certifications, and partner badges that require a fee and a form rather than demonstrated skill.

Client logos with no description of what was done. Logos frequently represent a small piece of work, or work done by someone who has left, or a relationship that ended badly.

Awards from publications that sell awards.

Years in business, in this specific field. The technology that makes current work possible is recent, so a long history is evidence of surviving as a company rather than of expertise in this. Relevant experience beats total experience.

Team size.

The portfolio question worth asking

Who specifically did this work, are they still at the firm, and will they be on my project?

Borrowed portfolios are common: work done at a previous employer, by a partner, by a contractor, or by somebody who left last year. None of that is necessarily dishonest and all of it is irrelevant to you unless the person is still there and assigned to you.

Then put the names in the agreement.

Judging a firm with no track record

Sometimes the honest situation, and it does not have to disqualify anybody.

What replaces a record: make them demonstrate rather than describe, weight the failure stories heavily, and structure the engagement so being wrong about them is cheap. Buy the diagnosis rather than the build. Keep the term short. Require the accounts in your name and the configuration delivered in readable form, so leaving is straightforward. The anti-lock-in list.

Do that and a new supplier becomes a bounded risk rather than a gamble.

Applying it to us, plainly

Rapled is new. Our track record is short and we would rather say that than dress it up with logos and language.

So the two checks that should carry the weight are the first and second on this page: make us show you a working agent and let you try to break it, and ask us what went wrong on previous builds. Both are available in one conversation.

We also structure the engagement so that being wrong about us is cheap: fixed fee for the roadmap, a real stopping point, the roadmap executable by anyone, accounts in your name. How that works, and the red flags including ours.