Before We Teach AI
Before We Teach AI
This week I joined another data leadership networking event. Two presentations, one on deploying an AI solution and another on data governance in an AI driven world. Both were thoughtful, well delivered and clearly grounded in experience.
By the time it finished I wasnât thinking about either of them. I was replaying a conversation from the break.
Someone suggested that organisations should start thinking about AI agents the way they think about employees. Agents would need training, governance, continuous development and regular evaluation across their lifecycle. It was an attractive analogy and, judging by the nods around the room, it resonated with a lot of people.
It resonated with me too. Just not in the way I expected.
I didnât start thinking about AI. I started thinking about how much time and money organisations already put into helping people learn, and how often those investments fall short of producing lasting change. Mandatory compliance training. Cyber security awareness. Leadership programmes, technical certifications, whole libraries of online learning. It would be difficult to argue that any of it consistently produces better judgement, stronger capability or lasting change in behaviour.
The problem isnât that organisations fail to provide training. Itâs that training and learning are not the same thing.
Most of us have completed courses because we had to. Weâve acknowledged policies we didnât really understand and sat through presentations that made perfect sense at the time, then returned to exactly the same habits the following week. Knowledge doesnât automatically become capability, and behaviour rarely changes because somebody clicked Complete.
I spend most of my working life thinking about how people develop judgement in high stakes work, the kind where a decision made badly costs somebody something real. None of what builds that judgement looks like a training module. People improve because they try things, get them wrong, get feedback and adapt. Experience is mostly accumulated lessons from things that didnât go to plan.
Which makes the analogy less flattering than it was intended to be. If we really do treat AI agents the way we treat employees, we should be honest about what that involves. Weâd give them a badly written induction, ask them to confirm theyâd read it, and then evaluate them once a year against objectives nobody remembers setting.
So perhaps we were asking the wrong question. If organisations are still working out how people learn, why are we so confident weâve worked out how to teach machines?
The more I sat with that, the clearer it became that the conversation wasnât really about agents at all. It was about the organisations expected to build, govern and improve them.
An AI system doesnât learn in isolation. It inherits our data, our documentation, our processes, our governance and whatever objective we happened to choose. If those things are inconsistent, fragmented or poorly understood, introducing AI doesnât make them coherent. It gives the organisation another way of experiencing the consequences.
You see this most clearly when a company puts a model on top of its own documentation. The intention is reasonable enough. Staff spend too long hunting for answers, so let the system find them instead. What surfaces almost immediately is that three versions of the same policy exist, two of them contradict each other, and the one everybody actually follows was never written down. The work stalls, and it gets logged as an AI problem. It isnât. The organisation has been running on that contradiction for years. It worked because somebody in the middle quietly knew which version to apply.
Thereâs another part of learning we tend to skip over.
People get better through feedback. We notice a mistake, feel something about it, work out what weâd do differently and adjust. AI doesnât work like that. A deployed system doesnât learn from a poor decision. It repeats it, at the same speed and with the same confidence, until somebody notices, decides it matters, works out why it happened and improves the system around it. Every step in that chain is human. The learning still belongs to the organisation.
Which leads to the question I suspect weâll spend the next few years on.
Will we actually notice when AI gets something wrong?
The failure mode isnât obvious errors. A wrong answer that looks wrong gets caught. A wrong answer that looks right gets used. As these systems get more capable the balance shifts, and it shifts in an uncomfortable direction. More outputs are correct, which is the whole point, but the ones that arenât become harder to pick out, because they arrive with the same fluency and the same absence of hesitation as everything else.
At the same time the incentive to check falls away. Checking is expensive. If the system has been right for three months, the fourth month of scrutiny starts to look like waste. Confidence grows faster than scrutiny, and it grows quietly, because nothing has gone wrong yet.
The people best placed to notice are usually the ones the system was brought in to relieve. If youâve spent fifteen years doing the work, you can feel when an answer is off before you can explain why. That instinct is the last line of defence, and itâs also the first thing to appear in a business case as inefficiency.
So the better AI becomes, the more we need human curiosity, and the harder it becomes to justify keeping it.
Over the past year Iâve become less convinced that AI creates new organisational problems. I think it exposes the ones that were already there.
For decades weâve compensated for poor documentation because somebody knew the answer. Weâve worked around inconsistent processes because experienced colleagues filled the gaps. Weâve tolerated fragmented knowledge because people built informal networks that kept work moving. Humans are remarkably adaptable. AI is considerably less forgiving. It confronts organisations with the problems capable people have quietly been solving for years.
Perhaps the conversation shouldnât start with how we train AI.
Perhaps it should start with how organisations learn.
Because AI doesnât just inherit an organisationâs knowledge. It inherits its habits, its assumptions and its blind spots. The quality of an AI system is therefore unlikely to exceed the quality of the organisation around it.