The silent CX crisis – who is the human in Human-Centered AI?
In the second of a three-part series, Natalie Calvert explains the problem with the term “human-centered” AI and why we must treat AI as a leadership test, not a technology race
Add bookmark
If you're a CX leader, the job you hold is the one this technology is aimed at. So, it is worth knowing how the last big bet turned out.
I am old enough to remember when this industry swapped people answering phones for IVR. More than 30 years ago. Customers still complain about it – the menu maze, the wrong hold music, the dead ends – at the biggest brands on the planet. Thirty years. One technology. Still not right.
Now the same industry is making a far bigger bet, with governments selling artificial intelligence (AI) as the engine of economic growth. The World Economic Forum expects 92 million jobs displaced by 2030, customer service among the most exposed. Thirty years to get one technology wrong. Now, we are about to repeat it at scale.
So, we have developed a response mantra for every conversation: Human-centered AI. The phrase is everywhere. The human is not.
Human-Centered AI has become the most repeated phrase in customer experience and the least defined. We mean it as a promise to protect the human. IT and finance hear a way to take out cost. Same organization, two languages – and the budget is sitting in their room, not ours. So, the phrase becomes a comforting label on a cost program: a human in the slogan.
Don't miss any news, updates or insider tips from CX Network by getting them delivered to your inbox. Sign up to our newsletter and join our community of experts.
A definition worth the name
Stanford's Institute for Human-Centered AI defines it as "technology built around human needs, values and wellbeing – widening human capability, not replacing it". At the same institute, Erik Brynjolfsson calls productivity by removing labor "a fallacy". Look at today's deployments against that. Removing the people who understand customers to pay for the technology is the opposite of augmentation.
That is not human-centered AI falling short. It is cost-centered AI.
So let me offer a clear definition.
Human-centered AI adds value for customers, for colleagues and for the organization. That is the test – not how many people the technology removes, but how much value it adds, for all three.
The two rooms – and the wrong language
Picture two rooms. In the first, customer leaders make the case for human-centred AI: protect the customer, protect the frontline. In the second, IT and finance make the case for cost.
Gartner surveyed 321 customer service leaders: 20 percent have already cut agent headcount because of AI; more than 80 percent expect to within eighteen months. Gartner's Emily Potosky named it – they are cutting agents to fund AI. The second room has the numbers. Morgan Stanley surveyed 935 executives across the five sectors most exposed to AI: productivity up 11.5 percent in a year, headcount down four percent.
But the choice is not fixed. The US posted a net jobs gain. The UK posted an eight percent net loss – the worst of any country surveyed. Same technology, different leadership decisions. By 2027, Gartner predicts, half the companies cutting service staff for AI will rehire for the same work under new titles. Cut, then rehire.
The first room is right. The second is funded.
We have perhaps five years of real transformation ahead, and we are at risk of wasting them. Keep making the human case in the language of empathy while the other room runs the numbers, and we lose – not because we are wrong, but because we are speaking the wrong one.
Human-centered AI is not a feeling to defend. It is an operating model to build – and a commercial case to prove in the board's own terms: loyalty, retention, lifetime value, risk and profit. Adoption is a purchase. Transformation is a redesign. The time to change the operating model is now, before the next five years automate the old one.
The human who remains
Three tiers of contact run through every operation. Respond is routine – AI will take it, and it should. Resolve is harder, and AI still handles much of it badly.
OnePoll quizzed 6,000 adults: 59 percent now find AI agents frustrating, up from 54 percent a year ago. Meanwhile, 82 percent have asked to speak to a real person – most more than once.
Then there is the third tier, Relationship – the shift most leaders have missed. When AI takes the routine, what reaches a human is no longer the easy nine in ten. It is complex, high-stakes and emotional – the complaint that becomes a moment of truth, the customer save. Relationship is not a tier at the top of the pyramid any more. It is the operation. Not in the future – but now.
The UK's Financial Conduct Authority found 49 percent of adults – 26.4 million people – with at least one characteristic of vulnerability. Only four in 10 ever tell their provider. Some firms reported finding none at all. This is not a minority to escalate. Stress, debt, illness, caring, loss – vulnerability comes and goes, and most of it is invisible. Every one of those customers, on a hard day, is a relationship call.
Handled badly, it escalates. Handled well, it becomes loyalty. Same call. The difference is who answers it – and how they are led.
Klarna learned this the hard way. It replaced the work of roughly 700 agents with AI. Its chief executive, Sebastian Siemiatkowski, later admitted cost had dominated and quality had dropped. Klarna is now rehiring humans.
The role is not disappearing. It is being promoted into the most consequential job in the operation. Which is why one word has to go. Agentic AI carries "agent" in its name. It is built to take the agent's place. The clue is in the title.
Put the human back, and the name changes with the work – advisor, executive, someone who handles relationship and judgement, not transactions. Three things still stand in the way: The operating model. The capability. The leadership to match.
What leaders do now: A road map
For customer leaders, the plan is three moves – none of it sentiment – and four decisions that belong in the boardroom.
Define human-centered AI in value terms: Adopt a single definition: Human-centered AI adds value for customers, colleagues and the organization – and measure every deployment against all three, not just cost.
Rebuild the operating model: Own the complex, high-stakes work end-to-end, with accountable leaders – not bolted onto a structure built for volume. Build for relationship as the operation, not the exception.
Build the capability: The work that remains is diagnostic, agile and relational. Re-skill people and team managers for judgement, not containment. The table below sets out the shift.

Lead on a different scorecard: Run every AI deployment through five tests – ease, speed, ownership, respect and value. Productivity without value is not progress. It is just a cheaper way to lose the customer.
Make loyalty and lifetime value the AI lens: For CEOs and CFOs, one question: what does AI deployment do to customer loyalty and lifetime value? Put that on the same slide as cost and productivity.
Name the person with veto power: For boards, a harder question: who has the authority to challenge any deployment that degrades the customer experience? Name them. At some point, inaction is a leadership decision.
Treat AI as a leadership test, not a technology race: PwC's 2026 AI Performance study found 74 percent of AI's economic value captured by just 20 percent of organizations – the ones pointing AI at growth and redesign, not cost alone. Within three years it will read as what it is: not a technology gap but a leadership gap – the record of who was in the room when the decisions were made.
Who is the human in human-centered AI?
Three humans, in fact. The customer wanting resolution, information or help. The employee, carrying the weight of delivering it. The leader, deciding what the gains are for.
Human-centered AI is not a technology decision. It is a CX decision about the humans that count – the customer, the employee and the leader – and how it is built into the new operating model.
This is the biggest opportunity the profession will ever have to change things for the better. If you're a CX leader, this is your decision. The cost of getting it wrong will not show on the dashboard – it will show as the customers who quietly never came back.
Within three years, the winners will be the organizations that used AI to make service better – not just cheaper. The rest will be rehiring the people they let go.
Be in it.
Quick links
- The silent CX crisis – who gives a damn?
- Is AI replacing CX jobs?
- Your contact center AI is succeeding, but are your customers still suffering?