The bottleneck has moved: why AI-generated code is a QA problem now

In our first post, we explained what we mean by AI-native testing and why the distinction matters. We talked about the spectrum from AI-assisted to AI-native, and we were honest about where AI shines and where it still needs humans.

This post is about why that distinction has become urgent. Something has changed in the last six months, and it affects every engineering team in South Africa whether they realise it or not.

Your developers got faster. Your testing did not.

In February 2026, ThoughtWorks gathered engineering leaders from across the industry for a Future of Software Development retreat in Deer Valley, Utah. The finding that stuck with us was simple: as AI takes over more of the coding, the care and attention that used to go into writing code does not just vanish. It needs to go somewhere else. Chad Fowler put it simply: "If we stop caring about the code, that discipline has to move somewhere else."

We think that "somewhere else" is testing. Here is why.

Your developers are already using AI coding tools. The question is not whether AI-generated code is in your systems. It is how much, and whether your testing can keep up.

We do not have to imagine what failure looks like

When we talk to engineering leaders about this, we sometimes hear "but our systems are stable." So let us look at what "stable" actually meant in South Africa over the last eighteen months.

In December 2024, Standard Bank's systems went down for five hours. Customers logged into the app and saw zero balances. Africa's largest bank, locked out of its own digital channels on month-end. Customers on social media said it was not a one-off: "Every month end it's the same story with Standard Bank."

On Black Friday 2025, FNB's app collapsed during the busiest shopping day of the year. Payments would not go through. Shoppers stood at tills unable to pay. FNB acknowledged "intermittent failures" after a 50% surge in transactions. Some customers were told to physically go to a branch or ATM to get their money.

Then in January 2026, Discovery Health revealed a system error that had been quietly paying out medicine claims at the wrong rate for an entire year. Instead of applying the correct percentages, the system paid everything at 100%. Nobody noticed for twelve months. It affected 16,500 members. Discovery first tried to claw the money back, sending letters demanding an average of R35,000 per member. The public outcry forced a reversal, and Discovery absorbed the full R125 million. The Council for Medical Schemes opened an investigation.

All of these failures happened with software that was mostly written and checked by people. Now consider what happens when the volume of code being shipped multiplies by five, which is what AI-assisted development is already doing across the industry. We keep asking the same question in client conversations: if Discovery's claims error had been introduced by an AI tool instead of a person, would your current process have caught it any faster?

The part that has not caught up yet

In our first post, we drew a line between AI-assisted and AI-native. We said AI-assisted is useful but not transformative. We stand by that. But what we are seeing now is that most organisations have started using AI to write code without updating how they check that code. It is an understandable gap. The tools moved fast. But it is a gap that needs closing.

Simon Willison has been one of the few people in the industry willing to say this out loud. Willison co-created Django, one of the most widely used web frameworks in the world, and coined the term "agentic engineering." In his recent conversation on Lenny's Podcast, he described a turning point in November 2025 when AI coding tools went from "mostly works" to "actually reliable." He now estimates that 95% of the code he produces is written by AI. That is roughly 10,000 lines a day. His prediction: by the end of 2026, half of all professional software engineers will be writing 95% of their code this way.

But here is what makes Willison worth listening to: he is honest about the limits. He says he can build software in an hour that looks like it took weeks. It has documentation. It has tests. It has clean structure. But he does not trust it. He has not used it long enough to believe in it.

This is what we see in conversations with clients. The software looks better than ever. But the confidence that it works as intended has not kept pace. In our first post, we said AI handles the repetitive 80% and humans handle the 20% that requires judgment. That is still true. But the 80% has grown so fast that the 20% requiring judgment is getting harder and harder to find.

The maths do not work for South African teams

When people overseas talk about this problem, they assume you have a big testing team. Ten, fifteen people whose only job is checking that the software works. That is not what it looks like here.

Most South African companies we talk to have twenty to forty developers and two or three people responsible for testing. Sometimes one. Sometimes it is a developer who also does testing when there is time. There is some automated testing in place, but it is patchy, it breaks often, and it covers maybe a third of what matters. Before every release, someone spends days manually checking things. Everyone knows it is not enough. The budget to change it has not been there.

Now your developers just got three to five times faster at producing code. QA capacity stayed exactly the same.

On top of that, the rules are getting stricter. The Information Regulator has already issued a R5 million POPIA fine. Between April and September 2025 alone, 1,607 data breaches were reported, up 60% from the year before. The maximum penalty is R10 million or ten years in prison. If software written by AI mishandles someone's personal data, "we did not know the AI did that" is not going to hold up.

A new category of risk

There is one more dimension we want to flag, because it connects directly to what we do.

In May 2025, a software bug at a single vendor took down NAPAfrica, South Africa's biggest internet exchange point. Most ISPs in the country were affected. The same day, a power fault at a Vodacom data centre in Midrand caused a second outage. Voice, data, and enterprise services went down. One bug. One vendor. National impact.

Now think about the fact that AI tools are writing code that touches your most sensitive systems, the ones that handle customer data, process payments, and connect to third parties. Willison has a name for the danger: the "lethal trifecta." When an AI tool can see private data, read content from outside sources, and make changes to live systems, you have a recipe for problems that nobody in the industry has fully solved yet.

In our first post, we said AI-native means the whole quality workflow is built around what AI is good at, with people providing judgment on the hard calls. This is exactly the kind of hard call we are talking about. Your quality process now needs to catch problems that simply did not exist two years ago. That is not just a testing question. It is a strategic one.

Where the attention needs to go

Chad Fowler asked the right question at that ThoughtWorks retreat: if we stop caring about the code, where does the discipline go?

We think the answer is quality checks that run at every stage of delivery, from the moment a requirement is written to the moment code runs in production. Not more people doing manual checks at the end. Not bolting a few AI features onto an existing process. A quality orchestration system where AI handles the checking and people handle the judgment calls.

This is what we built Kintova to do. Six capabilities that cover each stage: Requirement Quality, Code Quality, Test Quality, Data Quality, Release Quality, and Production Quality. Each one activates at a different point in your delivery process, so gaps get caught where they are cheapest to fix.

Three things worth doing in the next 30 days:

Find out how much AI-written code is already in your systems. Ask your engineering leads directly. Not whether they use AI tools, but how much of what is running in production was written by one. Most leaders we speak to are genuinely surprised by the answer.

Run the Discovery test. Take your last big release and ask: if something subtle had gone wrong, like claims being paid at the wrong rate for months, would our current process have caught it? If the honest answer is "probably not," that is exactly what our QA Discovery Sprint is designed to surface. One week, no preparation needed, and you get a clear picture of where the gaps are.

Compare how fast your team builds to how fast your team checks. If a feature can be delivered in a day but testing takes two weeks to catch up, that gap is where your risk lives. That is also where a different approach to testing starts to make real economic sense.

Further reading: Simon Willison's highlights from his Lenny's Podcast conversation covers the dark factory concept and agentic engineering patterns. The ThoughtWorks Future of Software Development retreat findings are worth reading in full. For the security dimension, Willison's series on prompt injection and the lethal trifecta are essential.