AI Supercharges Confidence, NOT Judgement: Why AI Is Making Technical Leadership Harder, Not Easier

AI has removed the barrier to producing code, architecture, and infrastructure, but not the barrier to judging whether that output is any good. This gap lets confident but inexperienced opinions outcompete calibrated expert scepticism in decision rooms, making technical leadership harder because organisations can no longer assume production capability implies sound judgement.

Put ten surgeons in a theatre. Each has twenty years behind them, thousands of procedures, and a working memory of every time an apparently simple decision went wrong at hour four. They examine the patient and arrive, independently, at the same conclusion: this operation is a bad idea.

Standing next to them is a man who has watched a great deal of Grey’s Anatomy. He has seen something that looks similar before, the procedure does not appear especially difficult, and he is quite enthusiastic about it. You would know instantly whose judgement counted, and you would not need a governance framework to work it out.

Something structurally identical is now happening in technology, except that the room is a steering committee, the patient is a production system that moves other people’s money, and nobody is entirely sure which of the participants has ever held a scalpel. This is the part of the AI transition that almost nobody is planning for, and it may prove more consequential than many of the model limitations we currently spend our time worrying about.

The uncomfortable part is that this is not a hunch. The research is now reasonably good, and it says something considerably more specific than “be careful with AI”: these tools systematically move people’s confidence faster than they move their accuracy, this happens to experts and novices alike in different ways, and training people not to do it mostly does not work.

1. AI Demolished the Production Barrier, Not the Judgement Barrier

There are two separate things going on here, and the industry keeps talking about them as though they were one.

1.1 The Barrier That Used to Do the Filtering

For the whole history of our industry there has been a crude but effective filter sitting between having an opinion about technology and producing technology. If you could not code, you could not produce much code; if you did not understand distributed systems, you could not produce a credible distributed architecture; and if you had never operated anything, your design would generally announce that fact within about two pages.

The filter was neither fair nor precise, but it did useful work, because it meant that the people generating technical artefacts had usually paid some price to learn how. Their scars were an implicit part of the artefact.

1.2 What Actually Got Removed

That filter is gone and it is not coming back. Someone with modest technical knowledge can now ask an agent to build an application, convert a framework, generate an architecture, stand up infrastructure, interrogate a database, produce a penetration testing plan, or write forty thousand lines of code before lunch, and the output frequently looks excellent. Sometimes it genuinely is.

But note precisely what was removed. AI demolished the barrier to production and did not touch the barrier to judgement. Those two things were bundled together for so long that we started to treat them as a single capability, and now that they have come apart, most organisations have no mechanism for telling them apart. Being able to produce a thing is not the same as being able to assess whether the thing is any good, which used to be a philosophical nicety and is now an operational risk.

2. “I Liked the Output” Is Not an Engineering Assessment

This shows up most clearly in a specific, recognisable scene, so it is worth walking through it in full.

2.1 The Twelve Hundred API Problem

Take a mature mobile banking application: ten-plus years of development, thousands of screens, somewhere around twelve hundred APIs, layers of accumulated architectural decisions, security controls added in response to specific incidents, dependencies nobody has audited since 2019, and a generous quantity of spaghetti that everyone has agreed not to discuss.

Now someone proposes using agents to migrate the whole thing from one stack to another, and a proof of concept gets built. It demos beautifully, as these things always do, because the demo path is the one thing an agent will absolutely nail. And then someone in the room says: I liked the output.

The relevant question is not whether they liked it. The relevant question, and I accept how confrontational this sounds, is whether they are qualified to know whether it is good.

2.2 The “Almost Right” Problem Is the Whole Problem

This is not an abstract worry. When Stack Overflow asked nearly fifty thousand developers what actually goes wrong with AI tooling, the top answer was not that the model cannot code, it was output that is almost right but not quite, reported by around two-thirds of respondents, with a further 45% saying that debugging AI-generated code eats more time than writing it themselves would have.

Read that finding through the lens of the demo, because “almost right” is precisely the failure mode that a demo cannot surface and an inexperienced reviewer cannot detect. Code that is obviously wrong is harmless, since it fails immediately and loudly, whereas code that is almost right passes review, ships, and waits.

METR found the same thing from the other direction when it compared automated scoring against human review of agent output on real tasks in large repositories: agents frequently produced functionally correct code that could not be used as-is because of test coverage, formatting and general quality problems, which means the benchmark scores we all quote at each other systematically overstate real-world usefulness. Passing the tests and being fit for the repository are two different bars, and only one of them was measured by the thing you saw in the demo.

2.3 Good Code Is a Claim About the Future

Good software is not software that looks elegant in a demo. Good software survives production traffic, partial failures, a determined attacker, a dependency upgrade, a regulator with a specific question about a specific transaction from fourteen months ago, and five years of other developers changing it under time pressure.

None of that is visible on the surface, because all of it is a claim about the future, and evaluating claims about the future of a system is exactly the skill that takes fifteen years to build and cannot be conferred by a good demo. The deeper the system, the wider the gap between how the output looks and what the output is. A proof of concept tells you the migration is possible and tells you nothing about whether it is survivable, and those are the only two questions that matter, in that order.

3. The Experts Are Not Anti-AI, and Reading Them That Way Is the Actual Failure

The reflex response to a cautious engineer is to assume they are behind the curve. It is worth checking that assumption before acting on it.

3.1 What They Are Actually Saying

Here is where these conversations reliably go wrong. Assume the same organisation has principal and staff engineers with fifteen or twenty years of hands-on experience: they know this application, they still write code, they use coding agents every single day and have strong opinions about which ones are better at what. And they say: do not use AI to blindly rewrite this application.

The lazy read is that engineering is being conservative again. Except that the same engineers are also telling you to use AI to generate tests and lift coverage into places that have never had it, to run penetration testing, to map the dependency graph with agents, to accelerate refactoring, to interrogate the codebase and surface things the organisation no longer knows about itself, to find patterns across twelve hundred APIs that no human is going to find by reading, and to automate the repetitive eighty percent of the migration work.

3.2 Resistance Versus Calibration

That is not resistance, it is calibration, and calibration is the single most valuable thing an expert produces. They are telling you where the tool is extraordinary and where the problem is still fundamentally an engineering problem requiring human judgement, and those are completely different positions. An organisation that cannot distinguish between them will systematically overrule the wrong people.

It is also worth noticing who in the industry is most cautious. In the same Stack Overflow data, experienced developers report the lowest rate of high trust in AI output and the highest rate of active distrust, at roughly 2.6% and 20% respectively, which is not a pattern consistent with technophobia so much as with the people who carry accountability having a better model of what the failure looks like. If your engineers were actually anti-AI they would not be handing you a list of eight places to deploy it aggressively.

4. Dunning Kruger Just Acquired a GPU

The popular version of this idea is a meme. The research version is more interesting, and more useful.

4.1 What the Original Research Actually Found

The Dunning Kruger effect has been flattened into a meme, so it is worth going back to the 1999 paper. Kruger and Dunning found that participants in the bottom quartile on tests of humour, grammar and logic sat at the twelfth percentile in reality and estimated themselves at the sixty-second, and their explanation is the part that matters here: the skills required to perform well in a domain are largely the same skills required to recognise good performance in that domain. Incompetence is partly self-concealing.

I should be fair to the critics, because the internet version of this effect is oversold. Gignac and Zajenkowski argue that the classic quartile graph is substantially an artefact of the better-than-average effect and regression to the mean, and there is a live methodological argument about how large the real asymmetry is. Fine. The leadership-relevant claim survives all of it, and it is narrower than the meme: evaluation is a separate skill from production, and you cannot bootstrap it from enthusiasm.

4.2 The Perception Gap Is Now Measured

What AI changes is the scale, and here the evidence is uncomfortably direct. In 2025, METR ran a randomised controlled trial on sixteen experienced open-source developers doing 246 real tasks in repositories they had worked in for an average of five years. Before starting, they forecast that AI would make them 24% faster; afterwards, having done the work, they estimated they had been 20% faster; the measured result was that they took 19% longer.

Be careful with that number, because I have watched it get weaponised on LinkedIn all year. It is one study, on early-2025 tooling, in mature repositories, with a small sample, and METR themselves have since redesigned the experiment after finding that a large share of developers now simply refuse to work without AI, which wrecks the control group; they believe current tools probably do produce a speedup they can no longer measure cleanly. Anyone citing “19% slower” as a settled fact about AI coding in 2026 is doing the same thing I am complaining about in this article.

The durable finding is not the slowdown but the roughly forty-percentage-point gap between what those developers experienced and what actually happened, and the fact that the gap did not close even after they had done the work. These were expert practitioners with strong incentives to be accurate about their own productivity, and their self-assessment pointed in the wrong direction entirely. That is the mechanism, and it has nothing to do with stupidity or bad faith: the feedback loop that used to keep confidence tethered to competence has been quietly cut, and nobody feels it being cut.

AI supercharges confidence. It does not supercharge judgement.

5. “Why Can’t It Be Vibed? I Only Want to Understand”

This is the sentence where the whole dynamic shows up in the room, and it is worth taking apart carefully.

5.1 The Most Reasonable-Sounding Sentence in the Room

The confidence problem rarely arrives as arrogance. It arrives as one of the most reasonable-sounding sentences anyone can say in a meeting: why can’t this just be vibed — I only want to understand why not.

Wanting to understand is the good instinct, and it is a perfectly fair question, provided the person asking has the substrate to hold the answer. That proviso is where the whole thing turns, and it is almost never examined.

Consider the same request in a domain where nobody is polite about it. Why can’t I score a goal in a World Cup final? I only want to understand. The answer is not a diagram, a tactics deck or a well-structured explanation of the run, the touch and the angle of the boot. The answer is that you have not spent your life training to be Ronaldo, and no explanation transfers the thing that makes it possible. We accept this without argument in sport, in music and in surgery, and then walk into a technology review and behave as though software were the one domain where twenty years of practice compresses neatly into a forty-minute briefing.

5.2 A Roof Needs Walls

Understanding is not a right if you have not invested in the knowledge the understanding has to sit on. I just want a roof on my house sounds entirely reasonable, and it is — provided you have built the walls for that roof to sit on. Nobody argues with the request. The request is fine. The question is whether there is anything underneath it to carry the weight.

Some explanations require prerequisites that take years to acquire, and the honest answer to “explain it to me so that I can decide” is sometimes: I can explain it, but the explanation will not put you in a position to decide.

Almost nobody will say that sentence out loud to a senior person, which is the actual organisational failure hiding in all of this. Telling an executive that they lack the background to evaluate an answer sounds like arrogance, reads as insubordination, and has ended careers, so instead people simplify. They produce the compressed version, the analogy, the diagram with four boxes. The compression is lossy by construction, and because it is simple it feels like understanding, so the executive leaves the room genuinely more confident and no better calibrated than when they walked in. The simplification was the harm. It was also, in the moment, the polite and professional thing to do, which is why it happens everywhere.

5.3 Ask for Evidence, Not for Understanding

The way out of this is not to demand better explanations, and it is certainly not for experts to hide behind complexity. An expert who answers “you wouldn’t understand” is failing at their job just as badly, and any organisation that tolerates that answer deserves what it gets.

The way out is to change what you ask for. Instead of “explain this until I can judge it myself,” ask what would demonstrate it to someone who cannot judge it directly: what experiment would settle this, what would we see if you are wrong, what does the failure look like in staging, what gate could we put in front of it that trips automatically. Evidence transfers across an expertise gap in a way that understanding does not. You do not need to hold the mechanism in your head if you can see the outcome and you have agreed in advance what outcome would change the decision.

That is delegation performed properly rather than abandoned, and it applies to me as much as to anyone reading this. I cannot personally evaluate every technical judgement made in my organisation, and pretending otherwise would be far more dangerous than admitting it.

6. The Asymmetry Nobody Priced In

The same tool, given to two different people, produces two very different outcomes, and only one of them is visible from outside.

6.1 A Multiplier for One Person, a Conviction Machine for the Other

Give an experienced engineer a coding agent and you get a multiplier, because they already carry the model of coupling, blast radius, failure modes, security boundaries, concurrency, data integrity, observability and what production does at month-end, and AI gives that model leverage. The results are genuinely startling. Hand precisely the same tool to someone with a shallow model of those things and you get output that looks similar and a person who is now significantly more certain.

The sharpest evidence for this comes from Stanford, where Perry, Srivastava, Kumar and Boneh ran a controlled user study on security-related programming tasks and found that participants with access to an AI assistant wrote significantly less secure code than those without, while being more likely to believe their code was secure. Less secure and more confident, in the same population, from the same tool. Their full paper notes that many participants simply accepted the assistant’s answer as final without further verification. That is not a productivity story, it is a confidence-competence inversion, and it is the single most important slide that never appears in an AI strategy deck.

6.2 This Predates the Model, and It Cannot Be Trained Away

There is a well-established name for the underlying behaviour and it long predates all of this. Human factors research calls it automation complacency and automation bias, and Parasuraman and Manzey’s review of the field contains two findings that every technology leader should internalise: it shows up in both novices and experts, and it cannot be trained or instructed away. We learned this from aviation, from radiology and from air traffic control long before transformers existed, and we are now relearning it in software, at speed, with a much shorter feedback loop between the error and the customer.

The first person in my example gained capability and the second gained conviction, and both experiences feel identical from the inside, which is what makes this so difficult to manage.

7. What the Organisational Data Says

Zoom out from the individual to the organisation and the picture gets clearer rather than murkier. Google’s 2025 DORA report, drawing on nearly five thousand technology professionals, lands on a single sentence: AI is an amplifier that magnifies the strengths of high-performing organisations and the dysfunctions of struggling ones. Ninety percent of respondents use AI daily and over 80% believe it has made them more productive.

The delivery data underneath that belief is split. AI adoption now correlates positively with throughput, so teams genuinely are shipping more, but it also continues to correlate with instability: more change failures, more rework and longer recovery. DORA specifically tested whether AI-driven speed would compensate by letting teams fail fast and fix fast, and found that it did not, because the bottleneck simply moved downstream into review, testing and operations, which is exactly where judgement lives and exactly where nobody added capacity.

So the honest summary of the evidence is not that AI does not work, because it plainly does. The summary is that AI raises the volume of everything, including the volume of decisions that require judgement, while simultaneously raising confidence, which is what keeps the judgement shortfall invisible until it arrives as an incident. If your engineering foundations are strong, the amplifier is a gift; if your organisation already struggles to tell good technical work from confident technical work, the amplifier is a problem you have just funded.

8. When Production Becomes Cheap, Judgement Becomes Expensive

There is a narrative building that AI makes deep technical expertise less valuable, and the opposite is happening. When anyone can generate code, the scarce skill becomes knowing which code should exist, and when anyone can point an agent at a system and ask it to rewrite the thing, the scarce skill becomes knowing which parts should be rewritten, which should be refactored, which should be left alone entirely, and in what sequence — because sequence is where migrations actually die.

Cheap production does not devalue judgement. It strips out everything that used to disguise the absence of it.

9. Leadership Still Means Applying Pressure

None of this licenses experts to hide behind their expertise, and the pressure has to be applied in the right place to actually work.

9.1 The Questions to Ask

I want to be unambiguous here, because the argument above is easily abused. The automation bias literature applies to your principal engineers too, sometimes twenty years of experience is twenty years of accumulated assumptions that nobody has re-tested since the assumptions were true, and deference is not the answer either, being merely a different way of failing to think.

So push, and push specifically:

• Why can’t AI do this? Name the mechanism, not the discomfort.
• What exactly goes wrong? Show me the failure mode.
• What would have to be true for this to work?
• Which portions can be automated even if the whole cannot?
• What does a safe experiment look like, and what does it cost us if it fails?
• Could agents take thirty percent of the human workload out of this? Fifty? Eighty?
• What would change your mind?

9.2 Controls, Not Hope

But notice what the research implies about how that pressure should be applied. If complacency and automation bias cannot be trained away, then telling people to review AI output carefully is not a control, it is a hope. The controls that actually work are structural: who is accountable for the review, what evidence a reviewer must produce beyond an approval click, what gets staged behind a flag, what has a tested rollback path, and which classes of change cannot be merged on a single pair of eyes regardless of whose eyes they are.

9.3 Production, Evaluation, Accountability

It also helps to separate three things that this whole debate keeps mashing together. Production has been radically democratised, anyone can generate the artefact now, and that is a genuine gain. Evaluation has not been democratised at all, remaining disproportionately concentrated in people who have operated systems at scale, and AI has made it more valuable rather than less. Accountability sits with leadership and always did; it cannot be delegated to the expert and it certainly cannot be delegated to the model.

That gives a cleaner division of labour than “trust the engineers.” The expert’s job is to supply calibrated judgement and to defend it under pressure, while the leader’s job is to decide how much weight that judgement deserves and then own the outcome. An expert who refuses to be challenged is failing at the first, and a leader who overrules calibrated judgement and then treats the resulting incident as an engineering failure is failing at the second.

Challenging expertise and overruling expertise are different acts, and only one of them requires you to possess equivalent knowledge. The leader’s job is not to become the most confident engineer in the room but to make sure the best engineering judgement in the room is properly tested and then properly heard, in that order, and that the testing does not quietly become a mechanism for arriving at the answer the loudest person already wanted. Otherwise you are not running a technology organisation, you are industrialising a leadership vacuum.

10. The Objections That Miss the Point

Four objections come up every time I make this argument in public, and each of them deserves a real answer rather than a dismissal.

10.1 “This Is Just Gatekeeping”

It would be, if the conclusion were that non-experts should not touch the tools, but that is not the conclusion. Product managers should build software, designers should write code, analysts should build agents, and executives should interrogate their own databases instead of waiting three weeks for a report. All of that is unambiguously good and I would like a great deal more of it.

Democratising the ability to do things is not the same as concluding that every opinion about how those things should be done now carries equal weight, and those two propositions get deliberately conflated, usually by whoever is losing the argument.

10.2 “The Model Writes Better Code Than Most of Your Engineers”

For a well-scoped function with a clear contract this is frequently true, and arguing otherwise is embarrassing. It is also irrelevant to the decision in question, because local code quality is not the constraint on the migration of a mature system. The constraint is system-level judgement: boundaries, sequencing, data integrity during cutover, what happens to in-flight transactions, which behaviours are load-bearing and which are vestigial.

The model is excellent at the unit; what it does not automatically possess is the operational and institutional context required to judge the whole. Note the word automatically, because you can supply a great deal of that context deliberately, and the teams getting real value out of agents are precisely the ones doing that work. It is also worth remembering that the METR trial found its largest negative effects exactly where context was deepest, with expert developers in repositories they knew intimately, which is the opposite of the environment where benchmark scores are generated.

10.3 “That’s a Prompting Problem”

Sometimes. But the interesting failures here are not the ones where the agent produced something wrong, they are the ones where it produced something plausible and the person reviewing it lacked the model required to notice. Better prompting does not fix that, it makes the output more plausible, which is the wrong direction entirely. The Stanford result is the cleanest illustration: participants who trusted the assistant less and engaged harder with their own prompts produced better security outcomes, which tells you the operative variable was the human’s scepticism rather than the tool’s quality.

10.4 “They Are Protecting Their Jobs”

Occasionally true and worth watching for, but the tell is easy, because someone protecting their position resists the tool everywhere. The engineers I am describing are the ones pushing agents into testing, dependency analysis, refactoring and security work faster than the organisation can absorb them, which is not the behaviour of someone building a moat. It is the behaviour of someone who has read the actual capability curve and is telling you where it currently bends.

11. Vibe Coding Works Right Up Until the Vibes Meet Production

There is a real place for this way of working, and there is a real limit to it, and the two need to be kept apart.

11.1 Where It Genuinely Belongs

I want to be clear that I love vibe coding. For prototypes, internal tools, scripts, experiments and greenfield ideas, the ability to describe intent and watch it get constructed is the most fun this industry has been in twenty years, and I do it constantly. The practitioners are also more discriminating about it than the discourse suggests, with around 72% of developers saying vibe coding is not part of their professional work while using AI heavily elsewhere in their workflow. They are not rejecting the tool, they are scoping it.

11.2 What a Sedimentary Codebase Actually Contains

That scoping matters, because there is an enormous difference between creating a new application and rewriting a mature one that contains years of accumulated business behaviour. A production banking application is not a set of screens attached to APIs, it is a sedimentary record in which every strange validation rule has a story and every awkward workflow probably exists because of a fraud event, a regulatory finding, an operational failure at 2am, or a customer edge case discovered the hard way in 2017. Some of it is genuinely dead weight, and some of it is the only thing standing between you and a repeat of an incident nobody currently remembers.

AI can read the code, and it can read the commit history, the tickets and the decision records too. What it cannot reliably do is reconstruct the institutional history that caused the code to become what it is, because the causal chain mostly was not written down: it was a phone call, a regulator’s raised eyebrow, a Friday night, someone who left in 2019. Frequently we cannot reconstruct it either, which is the actual problem. I spent twelve years writing algorithmic trading software before I ever went near cloud infrastructure, and the lesson that period beat into me was that a system which behaves perfectly in every test you thought to write can behave very differently against a live market. Correctness is not what your tests assert, it is what reality does not punish.

That is precisely why large migrations need careful engineering rather than more tokens. The tokens are cheap now; the consequences did not get any cheaper.

12. The Real Skill Is Knowing Whose Judgement to Trust

AI is going to blur every traditional boundary inside our organisations and I think that is broadly excellent, but blurred boundaries make the weighting problem harder rather than easier, and the weighting problem is now the job. There will still be moments when the person who has lived with a system for fifteen years sees something the rest of the room cannot articulate and cannot yet prove, and your entire value as a leader in that moment is whether you can tell the difference between that person and the most confident person.

So when ten experienced engineers tell you the operation is a bad idea, interrogate them properly: ask why, ask for evidence, ask whether the technology has changed the assumptions underneath their objection, and ask them to find a safer path, a smaller experiment or a staged version. Make them work for it. But if the only person in the room who thinks the operation looks straightforward is the one who has been watching Grey’s Anatomy, think very carefully before handing them the scalpel.

AI may well turn out to be the greatest productivity technology of our working lives, and it also creates a new organisational risk that most governance structures cannot currently see. The research is now clear enough that pleading ignorance is getting difficult: these tools reliably move confidence further and faster than they move accuracy, in experts as well as novices, and no amount of telling people to be careful appears to fix it.

That makes technical leadership harder, not easier. The scarce resource is no longer the ability to produce an answer, because AI will produce answers all day and night and they will be beautifully formatted. The scarce resource is knowing when the answer is right, when it is subtly wrong, when AI should lead, when a human must, and, above all, who in the room is actually qualified to tell the difference.

References and Further Reading

On the perception gap in AI-assisted development

• Becker, Rush, Barnes and Rein, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR, 2025 — blog write-up / paper
• METR, We Are Changing Our Developer Productivity Experiment Design, February 2026 — the necessary caveat on the above
• METR, Research Update: Algorithmic vs. Holistic Evaluation, August 2025 — why benchmark scores overstate real-world usefulness
• Stack Overflow, 2025 Developer Survey: AI and the accompanying analysis

On confidence exceeding competence

• Perry, Srivastava, Kumar and Boneh, Do Users Write More Insecure Code with AI Assistants?, ACM CCS 2023 (preprint)
• Kruger and Dunning, Unskilled and Unaware of It, Journal of Personality and Social Psychology, 1999
• Gignac and Zajenkowski, The Dunning-Kruger Effect Is (Mostly) a Statistical Artefact, Intelligence, 2020 — and the ongoing methodological debate
• Parasuraman and Manzey, Complacency and Bias in Human Use of Automation: An Attentional Integration, Human Factors, 2010

On organisational outcomes

• DORA, State of AI-assisted Software Development, Google Cloud, 2025 — announcement and analysis of the stability findings

Andrew Baker is Group CIO at Capitec Bank. He writes about enterprise architecture, engineering leadership and banking technology at andrewbaker.ninja.