BUILD
Building with AI vs building AI into the product
5 min read
Using AI to build software faster changes a development team's throughput, while shipping AI inside the product changes what the software can guarantee — and only the second requires new thinking about testing, cost and privacy.
"We use AI" is doing a lot of work in supplier conversations at the moment, because it describes two unrelated things.
One is a development team using AI tooling to write and review code. The other is a product that calls a model at runtime as part of what it does. The first is a question about how your software gets made. The second is a question about how it behaves.
Building with AI: a question about the team
AI-assisted development is now ordinary. It speeds up the parts of the work that are well-understood and repetitive, and it needs the same review as any other contribution.
What it does not change is responsibility. Code is the supplier's regardless of what typed it, and the questions worth asking are unchanged: is it reviewed, is it tested, can someone explain it in six months, and who owns it.
- Is generated code reviewed to the same standard as written code?
- Who owns the output, and is that stated in the contract?
- Does anything proprietary leave your environment through the tooling?
- Can the team explain and maintain what was produced?
Building AI in: a question about the product
A model in the runtime path changes the product's properties. Identical input can produce different output. Response time depends on a third party. Cost scales with usage rather than being fixed by the hosting. And behaviour can change when a provider updates a model you do not control.
None of those is a reason not to do it. All of them are reasons to design for it rather than discovering them in production.
What has to change in how you build
Testing moves from exact assertions to evaluation: a set of representative inputs with acceptable and unacceptable outputs, run against each change so a regression in quality is visible. This is the discipline teams most often skip, and the one that determines whether the feature stays good.
Cost becomes a design parameter. A feature that calls a model on every page load has a bill proportional to traffic, which is a different financial shape from hosting and needs a limit set deliberately.
And failure needs a defined behaviour. The model will be slow, rate-limited or unavailable at some point, and the product should degrade to something useful rather than to an error.
The questions a buyer should ask
When a supplier says they build AI into products, the useful follow-ups are specific. Which model, hosted where, and what happens to the data sent to it? What is the fallback when it is unavailable? How is output quality measured as the product changes? What does it cost per thousand uses, and who watches that number?
Clear answers suggest a team that has run this in production. Vague ones usually mean a prototype that has not met real traffic.
Where it is actually worth it
AI earns its place where the task is genuinely open-ended — summarising unstructured text, routing by intent, drafting something a person edits, extracting structure from documents that arrive in inconsistent formats.
Where the task has a correct answer that a rule could produce, a rule is better: cheaper, faster, testable and explainable. A good supplier will say so rather than finding a way to use the model you asked for.
Questions we get asked
Does it matter which model a product uses?
Less than the architecture around it. What matters more is whether the system can change model without a rewrite, how data sent to the provider is handled, and whether output quality is measured. Teams tied to one provider's specific behaviour have more work ahead than they usually expect.
How do you test a feature that gives different answers each time?
With evaluation sets rather than exact-match assertions: representative inputs, criteria for acceptable output, scored on every change so quality regressions show up before users find them. It is closer to testing a search ranking than testing a calculation.
What should we check about data privacy before shipping an AI feature?
Where the provider processes data, whether it is retained or used for training, what your privacy notice says about it, and whether any of it crosses a border that matters to your obligations. These need deciding before launch, because retrofitting them usually means changing the architecture.