Many teams that use chat as a support channel have moved past decision-tree-based technology and onto AI, which promised to solve their chatbot-related gripes.
We recently surveyed more than 1,000 adults in the United States, both CX professionals and customers, to understand how they feel about that shift. We found that 90% of consumers are still repeating information to chatbots in the past year, which indicates that many solutions still leave a lot to be desired.
So, what gives? The devil is in the details—specifically, the type of AI used to power these lackluster chatbots. We’ll explain the nuances between different types of AI and how you can identify the solutions that will truly make a difference for your business and your customers this coming year.
Why chatbots are still so bad
Frustrating chatbots are usually based on retrieval augmented generation (RAG) models, the cheapest and most easily accessible type of AI. These RAG-based systems typically combine the ability to retrieve information from a knowledge base with additional generative AI technology (like ChatGPT) that speaks to customers. Technically, they don’t require generative AI to talk to a customer, but they wouldn’t be able to come close to producing conversational language without it.
RAG-based systems are used by companies that don’t have a mature AI model to handle sophisticated tickets. They’re cheaper since retrieving information requires less computational power than complex reasoning, which amounts to less infrastructure costs. Plus, their simplicity makes them easier to implement. These combined benefits helped some chatbot providers improve upon decision-tree-based chatbots of the past, but they still don’t eliminate huge challenges for their customers.
Unlike more advanced systems, RAG-based AI lacks language processing, reasoning, and decision-making capabilities. Its retrieval system is limited to information. It doesn’t understand it, analyze it, or take action based on it—it just regurgitates it.
For example, this popular stationary retailer uses a RAG-based chatbot to help customers find products on its website. It can answer questions about specific planners based on predefined categories. When you try to follow up on its response or ask multiple questions at once, the agent gets confused because it can’t directly find an answer based on the structure of its retrieval system.
These systems probably deflect some very specific issues—specifically, any that can be answered by a help center or knowledge base article. But they won’t deflect any more issues than a help center would and will probably frustrate customers who have issues that fall outside of this box.
70% of the CX leaders surveyed said they had already adopted AI, while only 17% of consumers reported that wait times have decreased over the past two years. Plus, 90% of customers have experienced the need to repeat information multiple times to get an issue resolved. We’d bet this is because RAG-based systems are the most common, even though they only achieve 10% to 20% of ticket resolution.
If you want a solution that will truly transform your CX in 2025, you need to test your AI agent to be sure it’s much more advanced than a RAG-based system.
How to tell whether AI Agent is truly transformative
If you think you’re stuck with an AI-powered chatbot using a RAG-based model, you want to hunt for a truly transformative AI agent instead. The AI agents that dramatically reduce your team’s workload and transform CX departments are the ones based on natural language processing (NLP), natural language understanding (NLU), agentic AI, or all three combined.
1. Look for a human-centric experience
An inherently human-centric experience is one with personalized messaging delivered with emotional intelligence.
Well, while customers are increasingly comfortable with AI solutions, they still value human interaction. Our survey found that the number one frustration in customer service interactions is when customers don’t feel they can speak to a human. A truly human eccentric AI-powered experience is one where customers don’t feel like they have to in order to be treated with humanity and empathy.
To do that, they must truly understand the context and intent behind what a customer is saying and respond with empathy. Natural language processing and understanding are the types of AI that are respectfully responsible for those capabilities.
Truly transformative AI agents will take this a step further and also consider it. You are Brand, policies, multilingual support, cultural sensitivity, and historical interactions.
To test this, ask an AI agent an unusual question or one with a tricky keyword and see if it responds accurately and empathetically.
This hypothetical example shows a frustrated customer who contacts Fetch, a rewards-based app, when her rewards haven’t been delivered. The Fetch system, powered by four thought solves, responds with empathy when she is clearly frustrated.
This AI agent also provides specific answers, not just relevant links or the regurgitation of a knowledge base. It understood the issue, decided the best response, and then framed it appropriately for this customer.
2. Test for advanced reasoning
A transformative AI agent will be able to think about the information it provides and the context provided by the consumer. It will be a reason to determine the best response, not just regurgitate your knowledge base.
This is done with a combination of natural language understanding (NLU) and natural language processing (NLP), which allows AI agents to respond with empathy—even when customers don’t give a lot of context or have multiple intense responses in a single message.
This hypothetical example shows how Forethought Solve accomplishes this on our website. A customer could hypothetically provide context that their team is having trouble digging through their knowledge base and ask if we have a product that could help them.
Solve understands the intent behind this request and retrieves the appropriate information and reasons for it before crafting a response directly relevant to the customer’s question.
If the customer follows up with an additional question, the AI agent understands it’s a follow-up question. In the below example, the customer makes a vague reference to the product they’re discussing with the AI agent, but the AI agent still understands and responds accurately.
To test an AI agent’s reasoning capabilities, ask about two different topics in the same question or express a single issue in several different ways. It should be able to detect multiple questions, answer them accurately, and understand follow-up questions in context with the entire conversation.
3. See if it can take action
Agentic AI will power the most advanced AI agents. Agentic AI allows an AI agent to reason and respond with empathy and take action based on its conclusions. Depending on its integration, it could process returns, call up order details, provide rewards, or reset account passwords.
Gather uses Autoflows for Solve, powered by Agentic AI, to answer customer questions. In this example conversation, Solve understands the context: a customer is looking for an online event platform within their budget. Solve starts by collecting the information it needs to make a good decision.
Once a customer provides that information, the company produces an empathetic response based on the information provided, data from the knowledge base, and its experience working with customers who have similar questions.
When asked why it decided to recommend a free plan, this AI agent responded by explaining its reasoning, which included information the customer provided.
Here, Autoflows reasoned Gather’s free plan was best for this customer based on the number of seats and price, then decided to recommend it autonomously, without a human agent intervening.
4. Look for autonomous improvement
A truly transformative AI agent will improve on its own. As it interacts with customers, it will decide which interactions go well and which need improvement and adjust its behavior accordingly. Forethought even makes recommendations on how to improve your knowledge base with these insights through Discover.
If you’re using Autoflows for Solve, you can automatically generate policies by allowing AI to identify gaps in your workflow based on help desk tickets. Below is an example of an automatically generated policy to guide AI in helping customers unlock a frozen account.
5. Customization
While having a system that improves independently is helpful, we believe AI should always be used in partnership with a human customer experience team. That team should be able to manually adjust its AI agent as business needs evolve.
For example, you might need custom workflows for the holiday season or specific messaging for different customer segments. In those cases, you need to be able to create custom intents and workflows that follow them. If you’re unfamiliar with intents, they are the backbone of machine learning models allowing computers to understand and interpret human language. They describe what a customer intends to communicate, while the workflow is designed to resolve that intent.
You should be able to customize both. Here’s an example of an intent within Forethought to change a forgotten password, as well as the training phrases that might be used to help AI identify it.
Once you’ve customized your intent, you can use Forethought’s business logic components to customize the steps you want AI to take to resolve it.
Forethought’s business components include conditions, forms, options, and more.
The future of chatbots is combining efficiency and empathy
AI agents are truly transformative when they combine efficiency with human-like empathy. Our study showed that 48% of customers find AI chatbots to be more helpful than human agents in some cases.
Unhelpful AI-powered chatbots are RAG-based and only work to retrieve information from a database. They can’t understand, reason, or make decisions on their own. But those built using NLP, NLU, and genetic AI technology have the power to become proactive team members—not just passive tools.
