What Happens to a System After Two Years

The system you built in 2024 is still running. Nothing has broken, nobody has complained, and the report goes out every Monday.

It is also handling about seventy percent of what it used to, correctly, and the other thirty percent is being quietly worked around by three people who have stopped mentioning it.

Systems do not break, they drift

A clean failure is the good outcome. Something stops, somebody notices, it gets fixed the same week.

Drift is the expensive one. The business changed and the system did not, so it carries on applying rules that were correct two years ago. Nothing errors. The output is simply less right each quarter, and there is no moment at which anybody is told.

The five things that actually change

Your service mix. You added a line of work the system knows nothing about, so those jobs get categorised as whatever is closest and every downstream number is slightly wrong.

Your client mix. Rules tuned for jobs under $20,000 behave oddly on the $200,000 work you now do, and nobody revisited the thresholds.

The people. The person who understood why a rule existed has left. The new person assumes it was deliberate and works around it rather than questioning it.

The systems it touches. A vendor changed a field, renamed a status, deprecated something. The integration still runs and now writes into the wrong place, or silently drops a value.

The exceptions. Handling that started at two percent of volume is now fifteen percent, because the business shifted, and a rule written for a rare case is now being applied constantly.

Why nobody catches it

Because the people closest to it adapt rather than report.

Somebody notices the output is wrong for a certain type of job, so they fix those by hand. That takes ten minutes and raising it takes a conversation and a project. Ten minutes wins, every time, and within a year the workaround is simply how the job is done and nobody remembers it was a workaround.

By the time it surfaces, the fix is larger than it would have been, and the business has quietly concluded that automation degrades and needs replacing periodically. What actually happened is that nobody looked.

The review that prevents it

Twenty minutes, twice a year, per system. Not a project.

• Sample ten recent cases and check the output against what a person would have done. Not a report of what it did. The actual work.

• Ask the people using it what they fix by hand. Phrase it as wanting to remove annoyances, not as an audit. You will get a list.

• Check the exception rate against when it was built. If escalations have tripled, the rules no longer match the business.

• Look at anything it connects to. Has a field, a status or a form changed since it was built.

• Re-check the number you recorded before it existed. This is the one that tells you whether it is still earning its place.

Where AI systems differ

Two differences worth knowing, because they change what the review has to look at.

The first is favourable. An AI step handles variation it was never explicitly told about, so a new job type or an unfamiliar document layout frequently just works. Rules based automation breaks on the first thing it did not anticipate; an AI step usually copes. That is genuinely why these systems last longer.

The second needs watching. Because it copes, it will also produce a confident answer on a case it should have refused, and confident wrong output is invisible in a way that an error is not. So the review has to sample real cases rather than check that the thing ran.

The practical control is the same as at build time: the system must be allowed to say it is not sure. Then the escalation rate becomes your early warning, because a system quietly handling more edge cases without asking is the signal that the world moved and nobody adjusted the threshold.

The handover that never happened

The single most common cause of undetected drift is a departure.

Somebody leaves, their accounts get closed, and an alert that used to reach a person now goes to a mailbox nobody opens. The system carries on running and its only warning mechanism has been silently disconnected. This is worth checking specifically, because it is invisible until the day something goes wrong and no one hears about it.

Test it rather than assuming. Deliberately trigger a failure once a year and confirm a human being actually receives the message. Most businesses that try this discover at least one alert going nowhere.

Write down why, not just what

The reason drift is hard to correct is that nobody recorded the intent.

A rule says jobs over $50,000 route to a director. In two years somebody will ask whether that threshold is still right, and the answer depends entirely on why it was set: because that was roughly the top decile of jobs, or because of an insurance condition, or because one director wanted visibility on large work. Those three imply completely different reviews.

One sentence per rule, recorded at build time, and the biannual review takes twenty minutes instead of becoming an investigation.

What to do when you find drift

Resist rebuilding. That is the instinct and it is usually wrong.

Most drift is a rule that needs changing, a threshold that needs moving or a new category that needs adding. Hours of work, not a project. The rebuild instinct comes from not having looked for two years, which makes the accumulated gap feel like a fundamental problem rather than five small ones.

Fix the five. Then put the review in the diary so it never accumulates again.

The version that should be switched off

Occasionally the honest finding is that the process no longer matters. The service line was discontinued, the client left, the regulation changed.

Turn it off, deliberately, and tell people. A system nobody needs still consumes attention every time it fails, still appears in every audit, and still has to be understood by whoever inherits it. Retiring things is part of owning them, and almost nobody does it.

AI Optimize builds systems that escalate rather than guess, and reviews them on a schedule so drift gets caught as five small fixes instead of one rebuild. That work sits under Workflow Automation.