Who Is Responsible When AI Gets It Wrong

A system sends a client the wrong figure. A quote goes out with a price nobody approved. An enquiry is dismissed that should have been escalated.

The question that follows is not technical. It is who is answerable, and most businesses have never decided.

The regulatory picture, briefly

The European Union’s AI Act entered into force on 1 August 2024 and applies in phases. From 2 February 2025 the prohibitions on certain practices took effect, alongside obligations around AI literacy. Later phases cover general purpose models in August 2025 and high risk systems from August 2026. Penalties for prohibited practices reach thirty five million euro or seven percent of global turnover.

Here is the honest part: for a business operating in Quebec or the United States, none of that is directly binding unless you place a system on the EU market or your output reaches people there.

It matters anyway, for two reasons. It signals where regulation generally is heading, and Canada has been working through its own approach. And more immediately, your clients may be subject to it, which makes your systems part of their compliance picture whether or not they are part of yours.

Accountability does not transfer

Strip out the regulation and one principle survives every framework written so far.

The business that sends something is responsible for it. Not the tool, not the supplier who built the tool, not the model underneath. If a quote goes to your client with your name on it, it is your quote.

That sounds obvious and it is worth stating plainly, because a surprising amount of decision making inside businesses proceeds as though buying a system also transfers the responsibility for its output. It does not, and no contract with a supplier changes what your client believes when something goes wrong.

Decide the boundary before you need it

The practical work is deciding, in advance, what a system may do alone and what requires a person.

Draw the line by consequence rather than by volume. Answering a question about opening hours has a trivial cost if wrong. Committing to a price, a date or a capability does not. Anything that creates an obligation should route to a person before it leaves the building.

Write it down in plain language. Which outputs go direct, which are reviewed, and who reviews them. One page, and it takes an afternoon.

The record is what protects you

When something does go wrong, the difference between an awkward conversation and a serious one is usually whether you can show what happened.

What was asked. What material the system drew on. What it produced. Who approved it, if anybody. When.

That log costs nothing to build at the start and cannot be reconstructed afterwards at any price. Businesses that skipped it end up reconstructing events from individual inboxes, which is both slow and unconvincing.

Guardrails are a design decision

Most published failures are not the model being unpredictable. They are a system given more scope than anybody intended.

An assistant with access to your entire shared drive will answer questions about everything on that drive, to whoever asks. If salary information, client contracts and internal notes live in the same place, you have created a way to query them.

Permissions have to carry through to the AI layer, and the system should answer only from material it was deliberately given. That is the most commonly missed control and the one most likely to produce a genuinely bad outcome.

Say what it is

Some businesses give an assistant a human name and no disclosure, on the theory that people engage more readily. They do, until they realise.

Beyond the trust cost, disclosure is increasingly an expectation in regulation rather than a courtesy, and retrofitting honesty after somebody has felt deceived is considerably harder than starting there.

People are entirely comfortable dealing with a system that is useful. What they object to is discovering they were not told.

What to settle this month

• Which outputs can leave without a person. Written down, by consequence.

• Who reviews the rest, by name rather than by department.

• What is logged, and whether you could produce it if a client asked next year.

• What each system can see, and whether that matches what the person asking is entitled to.

• Whether clients are told when they are dealing with an automated system.

None of that requires a lawyer or a framework. It requires a decision, and having it recorded is what separates a business that can answer questions from one that cannot.

What to ask a supplier about liability

Three questions, asked before signing rather than after an incident.

What does your agreement actually cover if the output is wrong? Most supplier terms limit liability to fees paid, which for a modest monthly subscription is close to nothing against a real commercial loss. That is normal and you should know it rather than assume otherwise.

What controls exist to stop the failure in the first place? A supplier who talks only about accuracy has thought about the model. One who talks about thresholds, review points and logging has thought about your business.

What can you show me if a client challenges an output? If the answer involves reconstructing from logs nobody retains, you are the one who will be explaining it.

Insurance is worth a conversation

Most professional indemnity and general liability policies were written before any of this and say nothing specific about automated output.

That does not necessarily mean you are uncovered, and it does mean nobody has confirmed you are. A short conversation with your broker, describing what your systems actually do and where they touch client work, is cheap and occasionally produces a surprise worth knowing about in advance.

Regulated professions should do this first rather than last, because the answer may shape where the review boundary sits.

Sources

• European Commission, Regulatory framework for AI. The AI Act entered into force 1 August 2024, with prohibitions and AI literacy obligations applicable from 2 February 2025 and further phases through 2026 and 2027.

AI Optimize builds the review boundaries, the permissions and the audit trail as part of the design rather than as a policy written afterwards. That work sits under Custom AI Integrations.