Incorporating AI into your business is more than giving the employees access to a new tool

Like many companies, a customer of mine has finally "allowed" AI, and is organizing so employees can now use a bedrocked Claude Code in their work. The apparent plan is, again like with many companies, that the engineers will start experimenting with it, and that they will gradually learn how to incorporate AI more and more into the daily workings of the company.

This is not a great idea. It only exemplifies what I wrote earlier: that there is a lack of understanding around what companies need to do to embrace AI. Agents can do the work of hundreds or thousands of employees.

But the missing layer isn't how to replace a human worker's manual work with an autonomous agent, but how to get the agent to do the human worker's job at least as well as the human would.

Rote tasks are easy to automate. But when you're automating a customer service worker's tasks, you're automating a lot more than just rote work. How do you impose on such an agent all the cultural aspects of that person's work? What do you optimize for? So much of what a company IS lies in the culture and undocumented behavior of its employees. How do you encode that into an autonomous agent? How do you ensure that the agent makes the right call based on the contextual contents of a given situation? How can you make an agent detect the frustration in the voice of a yearslong customer which might be a hint of them being close to cancelling their relationship with you and moving to a competitor instead? In that situation,

is the ultimate goal of the agent still to reduce case resolution time, or to make sure the customer still feels valued and remains a customer when the case is closed?

This is probably most publicly exemplified by what cognitive scientist Gary Marcus dubbed the Klarna Effect. They were preening their feathers in the media, being early adopters of AI and firing their customer service staff, estimating to save $40M on AI agents doing the work of 700 human employees and reducing case resolution time from 11 minutes down to almost 2 minutes. 6 months later they were scrambling to hire back the people who were laid off, realizing their mistake.

But it's important to categorize this failure correctly. This was not proof that AI agents cannot replace human workers (although, at that time, the tech was probably still not mature enough).

This was a failure to understand how AI agents work, and what is needed to make them work the way you want them to.

And if you want them to be able to do the work of a human customer support agent, you need to encode not just what the human agent knows about the "rules" that govern the resolution of customer support cases, but also the aspects of dealing with humans on the other side of the table that we humans learn through, well, being human. If not, you get what Klarna got: an agent that optimizes for what it was made to do: reduce case resolution time to the detriment of customer satisfaction.

We have spent decades developing, incorporating and improving all kinds of methodologies and metrics to balance and align the people in an organization. To make sure they work towards the same goals, that they do their work in line with the stated values and goals of the organization. And, by being in an organization, talking to colleagues, watching senior employees handle their work, talking by the coffee machine, the company culture and values gradually seep into the fabric of the workplace by easing its way into the minds of the people who work there.

Whatever is not explicitly stated is inferred. Because that's how humans work.

Agents, however, do not work that way. Agents act based on what they have been told. All such inferred knowledge must be explicitly stated to an agent. The entirety of the company's culture and values must be encoded into the agents. But this is a type of work that we have NEVER had to do. We do not have the tools to do it, and we do not have the skill (yet). And, importantly, this is not a "tech skill". This has to be done by the whole company.

And it starts with management. It will require an enormous investment by the company that will likely dwarf any training investment the company has made previously. Some have estimated that it might cost the company more than their training costs for the last 25 years. Combined. So there is no way the tech department can drive this. Upper management needs to not only come on board, but to take the wheel and drive the company through this transition, because it will be fundamentally different from any transition the company has ever gone through.

If your company has opened up for the use of Claude Code or similar, and set an upper token limit per employee, you can safely assume that they have no idea what they are up against, and you'll have a steep wall to climb to make them realize what lies ahead. Token usage is going to be a small part of the incurred costs of this transition.

Time to get to work.