We removed the wrong bottleneck
AI made software trivial to create and did nothing about the cost of owning it. In a regulated bank, ownership is the whole cost — and we're about to find out the hard way.
The pitch for AI coding is that it makes software cheap to build. It does. That is exactly the problem.
For thirty years the bottleneck in enterprise software was writing it — the typing, the boilerplate, the slog. AI removed that bottleneck almost overnight. But writing was never the expensive part. Owning the thing was: maintaining it, securing it, patching it, and — in a bank — being able to explain it to an auditor long after whoever wrote it has moved teams. AI collapsed the cost of creation and left the cost of ownership untouched. We optimised the cheap half.
The marginal cost of creation went to zero. Ownership didn't move.
When anyone can stand up an internal tool in an afternoon, you don't get ten times the leverage. You get ten times the inventory — a sprawl of near-identical, half-finished, unowned tools, each carrying the full lifetime cost of the software it resembles. The afternoon it took to build is a rounding error against the decade you'll spend owning it.
A handful of tools. Still roughly accounted for — for now.
We already have a name for this
This isn't a new risk. Banks have managed it for decades under a deeply unglamorous heading: End-User Computing. EUC is the governance category for the spreadsheets, Access databases and scripts that quietly run material processes outside the formal software lifecycle — built by someone in the business, owned by no one in particular, understood by fewer people each year.
The textbook EUC disaster is JPMorgan's 2012 "London Whale". According to the bank's own post-mortem, the risk model at the centre of a multi-billion-dollar loss ran through a chain of Excel spreadsheets updated by copying and pasting between them, with a formula that divided by a sum instead of an average — quietly halving the volatility it reported. The spreadsheet didn't cause the loss. It's why nobody saw it coming.
Vibe-coded internal tools are EUC v2. Same failure mode — material work, no owner, no provenance — except the artefact is now a web app with network access and a database, and the cost of producing one has fallen to nearly zero. The blast radius went up while the barrier to creating it went down. That is not a combination that resolves itself.
The regulator already closed the escape hatch
The comfortable assumption is that AI is unregulated, so there's runway before any of this needs governing. The opposite is true. In its 30 April 2026 letter to industry, APRA was blunt: "existing prudential standards apply to AI risk" — entities simply haven't "operationalised governance in practice". There is no separate, lenient AI regime coming. CPS 230 (operational risk) and CPS 234 (information security) already reach an AI-generated tool the moment it touches a material process or real data, which is to say immediately.
APRA also named the precise gap, and it named the right one: entities are "relying primarily on policy direction ... rather than enforceable technical restrictions". A policy that says don't vibe-code production tools is not a control. It's a hope.
The supply chain is the part that should worry you
There is a sharper edge than maintenance. Ask an LLM for code and it will, with complete confidence, recommend packages that don't exist. A 2025 USENIX Security study, "We Have a Package for You!", measured it: across 576,000 generated samples, commercial models invented non-existent package names about 5% of the time and open-source models more than 20% — and the same fabricated names recur. That repeatability is what turns a hallucination into an attack. Someone registers the package the models keep inventing, and the next person who accepts the suggestion installs it. At a 5–20% rate, an uncontrolled pip install channel isn't a risk you're running. It's a schedule.
Ownerless software breaks the org chart
Conway's Law says your software ends up shaped like the organisation that built it. AI quietly breaks the assumption underneath it — that there's an organisation behind the software at all. A tool an LLM generated in an afternoon, for one person, maps to no team, no cost centre, no accountable owner. In a bank with a named-accountability regime, that isn't only technical debt. It's an artefact nobody is answerable for, which is its own kind of finding.
None of this is an argument against AI coding. I run a platform that does it at scale. It's an argument that we removed the wrong bottleneck — and the bill for the one we left in place is coming due. The next post is about how to pay it down.
My own opinion, not the Group's.