As organizations experiment with giving AI agents greater responsibility for resolving customer issues, they need to find ways to effectively measure if those resolutions were successful.
We're learning as we go that traditional CX metrics fall short in an AI-first service environment. We still need customer contacts to be resolved, but capturing resolution immediately after contact ignores everything that could possibly happen after contact.
According to Nixalkumar Patel, a digital commerce leader focused on AI-driven customer experience who currently serves as a senior product manager at LG Electronics, CX leaders need to ask a second question after, "did AI resolve the case?"
Specifically, they need to ask if the resolution held.
Patel says: "That requires looking beyond the moment a conversation or case is closed and measuring what happened afterward."
Perhaps a refund was issued for the wrong amount. Maybe an address change never reached the fulfillment system. A cancellation appeared successful but had to be corrected when the order moved further through fulfillment. Or the customer never contacted support again, but an employee quietly repaired the transaction downstream. "The original interaction may still look like a successful AI resolution," Patel says. "The customer experienced something different: rework."
In this interview with CX Network, Patel proposes a new way to measure the real impact of AI in customer service and CX and sets out the calculations practitioners need to apply in their work.
How long should the resolution window be?
The first lesson is that AI resolution metrics need a longer observation window – and resolution itself needs a wider definition.
"AI is expanding what 'resolution' can mean," says Nixal Patel, senior product manager for LG Electronics North America. "An AI interaction may no longer end with an answer. It may initiate a refund, modify an account, change an order, reschedule an appointment or complete another action on the customer's behalf. That makes the quality of the resolution dependent on what happens after the conversation."
Existing CX metrics remain valuable, he says, but a problem persists: each observes a different part of the journey. "An issue can appear resolved at 10 a.m. and require corrective work at 4 p.m. The second event should change how CX leaders interpret the first," Patel says.
Which metrics are best for measuring AI?
Patel says the most appropriate metric to measure the success of AI in a service journey is Post-Resolution Rework Rate, or PRR.
"PRR is the percentage of AI-resolved customer interactions that later require corrective customer, employee or operational activity because the original resolution did not hold," he explains.
A simple calculation is:
PRR = (AI-resolved cases requiring corrective activity ÷ AI-resolved cases that have completed their observation window) × 100
"The observation-window requirement matters," Patel says. "A case resolved five minutes ago should not be compared with one that has had two weeks to generate downstream issues. PRR should only include cases old enough for the expected customer outcome to become observable."
PRR is not intended to replace existing CX measures, Patel explains. Instead, it adds another view: resolution durability.
What PRR adds to existing CX metrics
Automated resolution, first-contact resolution and repeat contact all answer useful questions. PRR asks a different one.
The distinction is especially important because not all failed resolutions create another customer contact.
- A customer may not know that an employee corrected the problem.
- An operations team may repair a transaction before the customer notices.
- A second refund may be issued under a different case.
- A system exception may be corrected downstream without reopening the original conversation.
"If the organization only watches for repeat contacts, some of that work disappears from the CX measurement model," Patel says.
Three types of post-resolution rework
Patel says that for PRR to become actionable, CX teams should identify who had to do the work after the original resolution.
Here are some key areas that often demand rework.
Customer rework
The customer has to return because the issue did not stay resolved.
Examples include:
- Contacting support again about the same issue;
- Repeating information already provided;
- Reopening a complaint;
- Using another channel to get the promised outcome.
"This is the most visible form of rework because the organization has transferred effort directly back to the customer," Patel says.
Employee rework
This is when an employee has to correct, reopen or finish work that the AI interaction appeared to complete.
Examples include:
- Manually adjusting an incorrect refund;
- Correcting customer or account information;
- Reopening a supposedly resolved service case
- Intervening in an action the AI should have completed correctly.
The customer may never see this activity, but the organization still incurs the cost.
Operational rework
A downstream process or operational team has to repair the outcome.
Examples include:
- Reversing an incorrect transaction;
- Correcting an order modification;
- Reprocessing a failed action;
- Repairing conflicting records across systems.
This category matters as AI agents become increasingly capable of taking actions rather than simply answering questions. Patel says the resolution may look complete in the conversational system while remaining incomplete elsewhere in the customer journey.
Matching the observation window to the customer outcome
Here's the thing about PRR: there is no single universal time window; different customer promises mature at different speeds.
Here's how to approach measurement windows:
Patel says the principle is simple: Measure the resolution when the customer outcome can reasonably be verified, not only when the conversation ends.
Five steps to put PRR into practice
To put this into practice, Patel suggests five steps practitioners can take:
1. Start with AI-resolved intents where outcomes can be verified
"Do not attempt to calculate PRR across every conversational intent immediately," he says.
Instead, practitioners should start with a small group of high-volume or high-consequence journeys such as refunds, billing corrections, account changes, order modifications or service requests.
"These provide clearer evidence of whether the promised outcome actually occurred," Patel clarifies.
2. Define what "staying resolved" means
As established, case closure cannot be the only definition of success.
"For each intent, establish the observable condition that represents durable resolution," Patel advises.
This means that for a refund, success might mean the correct amount reaches the appropriate payment state. For an account update, it might mean the change appears correctly in the relevant system of record. For a service request, it might mean the downstream activity was successfully scheduled or completed.
3. Connect the original resolution to later activity
This is likely to be the hardest part. "A new case number should not make the original issue disappear analytically," Patel states.
"CX teams need enough journey lineage to associate later events with the original interaction, whether those events occur in customer service, operations, payments, fulfillment or another system."
The objective is not to create perfect enterprise-wide data matching before starting. Patel says it is to create enough linkage for a defined set of journeys to identify whether corrective work followed an AI resolution.
4. Capture the reason for rework
A PRR percentage alone tells leaders that a problem exists. The reason codes tell them what to fix.
Useful categories might include:
- Incomplete resolution;
- Incorrect action;
- Misunderstood customer intent;
- Outdated or conflicting information;
- Downstream execution failure;
- Inappropriate automation;
- Missed escalation.
"The exact taxonomy will vary by organization, but it should point teams toward a corrective action," Patel says.
5. Combine PRR with existing CX and operational metrics
PRR alone is not a magic bullet. It is most useful when paired with other measures.
"A lower PRR is positive, but an organization should not achieve it simply by escalating every difficult interaction to an employee," Patel explains. "Likewise, a rising automated-resolution rate is not necessarily positive if the amount of downstream correction rises with it. The interaction between the two metrics tells leaders more."