How to Choose Who Builds It

You have three proposals. One is half the price of the others, all three demonstrated something impressive, and you have no reliable way to tell which of them will still be working in eighteen months.

The technical comparison will not settle it. The questions that actually predict the outcome are about process and ownership, and you can ask all of them without understanding any of the technology.

Why this decision carries more weight than it looks

MIT’s Project NANDA published The GenAI Divide in July 2025, reviewing over 300 disclosed AI initiatives. Against an estimated $30 to $40 billion of enterprise investment, roughly 95% of generative AI projects produced no measurable return.

Their study covers large organisations rather than businesses your size, and no measurable return frequently means nobody set up a way to measure. What it does establish is that most of these projects fail, and that they fail on process rather than capability. Which means the supplier’s method matters more than their technical claims, because the method is what determines whether anything ends up in production.

The eight questions

What will you tell me not to build. The most revealing question in the list. A supplier who has never talked a client out of something either has not been doing this long or is selling whatever is asked for. Expect a specific example.

What happens in the first three weeks. You want to hear that they will sit with the person who does the work today, and that something narrow goes into real use quickly. If the first three weeks are discovery workshops producing a document, the project is already shaped wrong.

What number will move, and how will we know. A supplier who does not ask what you measure today will deliver something nobody can defend at the next budget review.

What happens when it is unsure. Ask directly whether the system can decline to answer. Anything that always produces an output cannot be trusted with any of it, because you have no way of knowing which answers were guesses.

Whose name are the accounts in. Hosting, the AI provider, the database. If the answer is theirs, your production system runs on somebody else’s credentials.

If we left in three years, what would we take. The code, the data, the configuration and the prompts, in your repository, in your name. A good supplier answers immediately. A poor one talks about how nobody leaves.

Who owns it after handover, on my side. A supplier who has not raised this has not thought about month nine, which is where most of these die.

Show me something you built that stopped being used. Everybody has one. The answer tells you whether they understand why, and whether they will be honest with you when something is not working.

What the cheap quote usually means

Not always a bad sign, but check three things before treating it as a saving.

Whether it includes the failure paths, the alerting and the documentation, or only the part that works when everything goes right. Whether it includes the integration into your systems, which is most of the real cost. And whether there is any provision for the weeks after launch, when everything you actually learn arrives.

A quote at half the price is frequently a quote for the demonstration rather than the system. The difference surfaces in month two.

What the expensive quote usually means

Sometimes seniority and a proper process. Sometimes a discovery phase producing a document, a platform licence you did not need, and a team large enough that nobody is accountable.

The test is what you get in the first month. If the answer is a requirements document, you are paying for a process designed for organisations with a hundred stakeholders. You have three, and you can put them in a room.

Judge the questions they ask you

Probably the strongest signal available, and it costs nothing to observe.

A supplier who asks what the process looks like when it goes wrong, who does this job today, what happens if the output is wrong, and what you measure now, is thinking about production. One who asks mainly about your budget and your timeline is thinking about the sale.

Notice also whether they push back on anything. A supplier who agrees with everything in a first meeting will agree with everything later, including the things they should be telling you are a bad idea.

References, asked properly

Every supplier supplies references and every reference says it went well, so the standard call is theatre. Two questions make it useful.

Is it still running. Not was it delivered. A year later, is the thing in daily use, and by whom. This single question separates suppliers who build things that last from suppliers who build things that get signed off.

What went wrong and how did they handle it. Something always did. A reference who cannot recall any difficulty either had a trivial project or is being polite, and neither tells you anything. The answer you want is a specific problem and a supplier who raised it rather than concealing it.

Ask for a reference whose project did not go smoothly. How a supplier reacts to that request is itself the answer.

The in-house option

Worth considering honestly, because sometimes it is right and we would rather say so.

If you have somebody technical internally, the work is genuinely central to how you compete, and it will need changing every month, building it inside makes sense. Ongoing change is what external arrangements handle worst.

It goes wrong when the person is doing it alongside another full time job. That produces something undocumented that works until they are busy, and the business is then dependent on one person’s spare capacity. If it matters, it needs somebody’s actual time, whoever they work for.

Start small enough to be wrong

Whatever the answers, the safest structure is the same: a first piece of work small enough that choosing badly costs you a quarter rather than a year.

One narrow process, in production, with a number attached. You will learn more about a supplier in three weeks of real work than in any amount of reference checking, and if it goes well the second project is easy to scope and easy to fund.

Be wary of anybody who resists that structure and wants to begin with a platform, a programme or a twelve month engagement.

Sources

• Aditya Challapally, Chris Pease, Ramesh Raskar and Pradyumna Chari, The GenAI Divide: State of AI in Business 2025, MIT Project NANDA, July 2025.

AI Optimize answers all eight of those questions the same way in the first conversation, including the one about what we will tell you not to build. That work sits under Custom Software.