8 Essential steps to take when preparing data for AI

When used to its full potential artificial intelligence (AI) is one of the most impressive technologies the modern organization has at its fingertips. For the last two years, CX Network's annual research into the state of CX has found that AI for operations is the number one CX trend influencing the work of practitioners. Conversational AI, generative AI and agentic AI also featured in the top 10 most selected responses. 

But things can easily go awry. In fact, research by McKinsey has found that 44 percent of organizations have experienced negative consequences from AI implementation. 

The fact of the matter is, AI is only a powerful efficiency driver when organizations put in the necessary work on its foundations. Failure to do so can quickly create a liability.

According to Sudarshan Reddy Gurampatti, senior machine learning engineer at JPMorgan Chase & Co., many teams spend weeks comparing models "but very little time looking at the data they are feeding into them". He adds: "In my experience, that's usually where problems start."

Featuring insights from Gurampatti, as well as AI and ML leaders at New York Life, LG Electronics and MCE Systems, this article outlines eight essential steps practitioners should take to ensure their AI will produce trustworthy outputs. 

Don't miss any news, updates or insider tips from CX Network by getting them delivered to your inbox. Sign up to our newsletter and join our community of experts. 

Learn More

1. Know what a correct answer looks like from the start

Shalini Sivasamy, senior AI/ML engineer for New York Life, says that to ensure an AI agent is producing trustworthy outputs, the starting point is defining what a correct answer looks like before building anything. 

Reflecting on her own experience, she explains: "On an insurance modernization project, we created a set of verified example answers, validated independently by two subject matter experts, before the system went live. This provided a fixed standard against which every future output could be measured, rather than relying on subjective judgment."

Once "correct" has been defined, the chief technology officer (CTO) for AI at MCE Systems, Alon Eirew, says practitioners must "rigorously iterate on the system's outputs". He explains: "This starts with benchmarking what a positive result looks like and even defining the criteria of what acceptable mistakes should be, then measuring and quantifying the validity or trustworthiness over several iterations."

2. Test, test and test again

Further to Eirew's point, testing is critical. Extensive testing should begin during the build phase using test data, with a focus on cases that do not follow the script. 

Hemang Upadhyay, senior manager of AI product management for LG Electronics North America, advises this includes unexpected scenarios, use cases that demand the model to hand off a query, or cases that should trigger guardrails. For conversational AI, test prompts should also reflect the unstructured way some customer talk; messy syntax, muddled intent and colloquialisms. "Testing only clean prompts creates false confidence," he warns. 

Once deployed, that testing should continue. Intent drift is real and largely invisible. It is only by continually testing models that organizations can ensure they are performing as originally programmed.

Sivasamy says: "Validation needs to be built in from the beginning rather than added later. A new AI system should go through a structured evaluation period with appropriate oversight before it is given access to customer or production data. 

"Identifying errors early in the lifecycle is far less costly than correcting them after they have reached end users," she adds.

According to Sivasamy, a willingness to invest in design stage testing "makes the difference between a system that holds up and one that needs constant correction".

3. Decide which data the model will work from

It may seem easier to allow a new AI tool access to all the data and knowledge available, but this sense of security is misplaced. Much like the laws of gravity mean what goes up eventually comes down, if an AI has ingested the information, there is every chance it can disclose it. This can happen accidentally in response to a well-intended prompt, or due to malicious prompt injection. 

In August 2025, Lena, the AI chatbot that assists Lenovo customers, leaked live session cookies data. Researchers at Cyber News discovered a 400-character prompt was enough for the bot to disclose the information, giving an attacker everything they need to bypass logins, hijack active chats, and access past conversations. This is just one example of the potential vulnerabilities effective guardrails can prevent.

Gurampatti says: "Not every document or system should be available to every user or every AI application.

"I've seen support systems return outdated pricing simply because an old document was never removed from the knowledge base. Customers usually don't know the document was outdated, they just know they received the wrong answer," he adds. 

4. Define what trusted data means 

Upadhyay advises that practitioners do not treat every data source equally. 

"For AI agents, especially in customer-facing or operational workflows, trusted data should be accurate, current, approved, attributable, and owned by a responsible business function," he explains. 

He says practitioners should start by identifying the source of truth for each type of answer the AI is expected to provide. 

"Product information, service policies, warranty rules, troubleshooting guidance, pricing, availability, customer history, and internal process documents may all live in different systems. If those systems conflict, the AI will inherit that conflict," he adds. 

5. Assign responsibility to a human

Accountability is key. Knowledge base management, data access, and testing must be orchestrated and owned and AI regulations, such as the EU AI Act and Health Insurance Portability and Accountability Act (HIPPA), explicitly outline this requirement. 

Upadhyay also recommends a content clean up. "This includes removing duplicate documents, outdated versions, conflicting instructions, incomplete FAQs, and unstructured content that lacks clear context. Many AI failures are not model failures; they are knowledge-management failures."

6. Add structure and metadata

Labelling data with more data may seem counterproductive, but it is key to the organization and structure AI models need to work accurately. 

Upadhyay says: "AI systems perform better when knowledge is organized with clear titles, categories, effective dates, ownership, product relationships, regional applicability, and usage context. This helps the AI retrieve the right information for the right situation."

Sivasamy adds: "Feeding the AI more documents does not improve results if those documents are poorly structured or inconsistently organized. On a banking compliance project, the way source documents were segmented and organized proved just as important as the choice of AI model. When done poorly, it produced answers that appeared accurate but were subtly misleading."

7. Ensure the system's reliability

Beyond data integrity, Eirew says there are two additional factors required to ensure trustworthy results from AI systems. 

The first factor is the reliability of the system itself. He explains: "Imagine an AI-based smart home theft detection system that constantly alerts you of false attacks, overloading the resident with notifications. Even if the system functions, it will erode trust if it's unreliable, because you will never be sure of when a real intruder has entered your home. 

The second factor is whether the output is grounded in reliable sources or facts. Eirew says: "Even if an AI system sounds confident, if it generates false, inconsistent or contradictory claims, it will easily lose the confidence of its users or, worse, impose a cost on businesses before they realize the issue."

8. And remember, rubbish in = rubbish out 

This one will never go out of fashion, but very much needs to be highlighted. 

Gurampatti says the quality of AI outputs "depends much more on the quality of the data than on the AI model itself". 

Basic questions to ask before introducing AI should include: Is the data accurate? Is it up to date? Is it complete? As outlined, teams should also remove data and knowledge that is duplicated, conflicting or outdated. 

Gurampatti adds: "AI performs much better when it knows which information is current and which information is no longer valid. For example, an older policy document may still exist in the system even though a newer version has replaced it. If the AI cannot distinguish between the two, there is a good chance it will return the wrong answer."

Quick links

• AI readiness gap: Why CX leaders hold the key to successful AI 
• The CX Recovery Layer: What happens when the agent is wrong?
• The EU AI Act compliance checklist for customer service teams