When humans hallucinate, we see things that aren’t real. AI does something similar; it “sees” patterns and information that don’t actually exist. But it still confidently presents this misinformation to customers without realizing it’s wrong. To AI, the patterns seem real, but these errors are confusing and frustrating to your customers and can quickly erode trust.
While AI is incredibly powerful, studies show that chatbots can hallucinate up to 27% of the time. Even small missteps can feel bigger in customer service, where trust and clarity matter most. That’s why preventing these errors is so important—it helps ensure AI enhances the experience instead of causing friction.
We’ll explore AI hallucinations—what they are, why they happen, and how your team can take steps to avoid them.
What are AI hallucinations?
AI hallucinations happen when generative AI, like ChatGPT, produces false or misleading information but presents it as entirely true. These errors aren’t limited to generative models; conversational AI, vision AI, and recommendation systems can also make similar mistakes, like offering incorrect answers, misclassifying objects, or recommending items that don’t exist.
Hallucinations happen because AI doesn’t truly understand facts or reality. It predicts what words it should use based on patterns in its training data, and when it lacks enough context or information, it’ll say something that sounds right but isn’t. Take this exchange, for example:
Customer: “I saw on your website that jeans are supposed to be 50% off today for Black Friday. Is that true??”
AI agent: “Yes, our entire denim collection is currently on sale for 50% off. Add them to your cart, and the discount will apply automatically.”
In reality, a code might be required, or the discount could only apply after subscribing to a newsletter. These small but significant mistakes happen because generative models prioritize fluency. When they can’t confidently anchor their words to verified data, they fill the gaps with made-up details.
Why do AI hallucinations happen?
AI hallucinations happen because, at their core, conversational AI are brilliant guessers—not fact-checkers. They predict words based on patterns in their training data, not on true understanding.
Only AI built on natural language understanding (NLU) or agentic AI can truly understand the information it’s given and respond accordingly. When information is incomplete, outdated, or unclear, generative AI often tends to guess wrong.
1. Outdated or incomplete training data
Outdated or incomplete training data often leaves AI without the reliable information it needs to respond accurately. When faced with gaps, like questions about new policies or unique situations, it fills in the blanks with its best guess, creating answers that sound plausible but aren’t correct.
2. Ambiguous or poorly structured prompts
AI thrives on clear instructions, but it struggles to provide accurate answers when prompts are vague. Without specifics, like the exact issue or tool involved, the AI guesses with a response that sounds helpful but doesn’t actually solve the problem.
3. Architecture isn’t built for fact-checking or logical reasoning
Large language models (LLMs) are designed to predict words based on statistical patterns, not to verify facts or reason through inconsistencies.
Preventing hallucinations doesn’t require overhauling the AI model entirely. But, integrating accurate, verified sources through techniques like curated knowledge bases (KBs) or improved training prompts can keep AI grounded.
8 Types of AI hallucinations (with examples)
Not all AI hallucinations are the same. They change depending on how the system is built and what it’s trying to do. Knowing the different types can help you quickly find the source of the problem and take targeted steps to prevent it from happening again.
1. Factual hallucinations
Factual hallucinations happen when an AI system presents false information as fact. For example:
Customer: “I ordered a sofa yesterday. Can you tell me when it will arrive?”
AI Agent: “Your sofa has already shipped and will arrive by Friday.”
In reality, the order is still being processed and hasn’t shipped yet.
These hallucinations can happen when a response is based on general patterns—like typical shipping timelines—rather than pulling real-time, verified order status information. Preventing factual hallucinations requires grounding AI in accurate, up-to-date information.
2. Logical hallucinations
Logical hallucinations are when an AI contradicts or breaks logical reasoning within a single response. For example:
Customer: “Do I need a screwdriver to assemble my desk?”
AI Agent: “No, you don’t need a screwdriver. But some customers in past interactions reported using a screwdriver to attach the legs.”
Here, the AI combines conflicting information from the instruction manual and past customer interactions but fails to recognize the contradiction.
This happens because the AI has focused on combining sources smoothly without checking for logical consistency. To prevent these hallucinations, reconcile conflicting data before using it to train your AI.
3. Relevance hallucinations
Relevance hallucinations happen when an AI provides information that seems related to a customer’s question but doesn’t actually address it. For example:
Customer: “I upgraded to your Plus Fiber internet plan last month, which should provide 200 Mbps. Can you confirm my current speed?”
AI Agent: “Our Plus Fiber customers often praise the stability and reliability of their internet connections, and we’ve received positive feedback on how well the speeds accommodate streaming and gaming needs.”
When the customer asks about their internet speed, AI responds with general statements about customer satisfaction and internet quality. While this sounds helpful, it does not directly answer the customer’s question.
If the AI isn’t guided to focus on stored facts or is allowed to drift into generalities, it may produce an answer that feels relevant but ultimately misses the mark.
4. Hallucinated citations
Hallucinated citations happen when AI references a source that doesn’t exist or is incorrect. For example:
Customer: “Where can I find your return policy details?”
AI Agent: “Our return policy can be found in Section 4 of the Terms and Conditions updated in March 2023.”
In reality, the Terms and Conditions from March 2023 have not been updated. The AI fabricated both the section and the update date. Instead of referencing a verified document, it created a citation to sound authoritative.
If AI doesn’t have a built-in mechanism to confirm whether a source it cites is real, it could produce plausible-sounding but entirely fabricated references.
5. Procedural hallucinations
Procedural hallucinations happen when the AI gives an answer that seems convincing but describes features or processes that don’t exist. For example:
Customer: “I need to request a purchase order for 100 units of Part A from the finance team. How do I do that?”
AI Agent: “Open the Vendor Management tab in your ERP, click the ‘Speedy PO’ option, and enter the item details. This special feature automatically approves urgent requests and forwards them to finance within minutes.”
In reality, the ERP system has no “Speedy PO” option or automatic approval feature. The AI creates an answer that sounds logical and helpful but is misleading. These hallucinations occur because the AI predicts plausible patterns rather than verifying actual processes.
6. Context dropping
Context dropping happens when an AI loses track of earlier parts of a conversation and responds off-topic. For example:
Customer: “I’m calling about my late mother’s life insurance policy. She passed away last week, and I need to file a claim.”
AI Agent: “I’m sorry for your loss. Could you please provide your policy number?”
Customer: “Yes, it’s 12345. Also, can you tell me what the coverage includes?”
AI Agent: “Our life insurance plans offer a variety of coverage options, including term and whole life policies. Many customers choose riders to enhance their basic coverage.”
When the customer asks a follow-up about coverage, the AI shifts to a generic explanation of insurance options, ignoring the initial context. This happens because AI models, especially large language models, often struggle to retain full context over multiple exchanges.
7. Temporal inconsistencies
Temporal inconsistencies happen when an AI mixes up dates or timelines and gives incorrect event time frames. For example:
Customer: “I need to have these flowers delivered to my mom by Mother’s Day on May 10th. Can you confirm the delivery date?”
AI Agent: “Sure, your flowers will be delivered on May 15th, just five days after Mother’s Day.”
In this case, Mother’s Day is on May 12th, not May 10th, and the AI also suggests a delivery date after the requested deadline.
This is another example of AI generating a response based on patterns in its training data without accurately processing or verifying specific information like dates.
8. Overgeneralizations
Misattributions and overgeneralizations happen when an AI provides broad, vague statements instead of precise details. For example:
Customer: “I’m considering downgrading from a Platinum to a Gold membership. How much will the Gold membership cost?”
AI Agent: “Our Premium memberships offer comprehensive benefits, including unlimited access to all facilities and exclusive classes. Many members find the Premium tier enhances their fitness experience significantly.”
The customer wants specific pricing details for downgrading to a membership, but the AI responds with irrelevant information. The AI misattributes the query, pulling general information about higher-tier plans instead of answering the exact question.
Best practices to prevent hallucinations
When AI makes mistakes, it can frustrate your team and your customers. Here are some practical steps to help ensure your AI support stays accurate and trustworthy.
2. Implement grounding techniques to anchor responses
Grounding techniques connect AI’s responses to verified sources. This “anchors” its response to real-time, trusted data instead of relying solely on pre-trained patterns. To implement grounding techniques, you need to give AI access to high-quality data from sources like:
- Knowledge bases with organized FAQs, policy documents, and product guides
- CRM systems containing customer profiles, purchase histories, and preferences
- Support ticket records with detailed issue resolutions
- Feedback from surveys, reviews, and social media interactions
- Internal company documents, including policies, training materials, and procedures
It’s not just about access to data—it needs to be clean, up-to-date, and well-organized. AI systems perform better when data is categorized and searchable. You could do this using retrieval-augmented generation (RAG) AI, which pulls data dynamically from these sources. While RAG improves access to current information, it has limitations—such as an inability to fully understand or verify what it retrieves.
It’s better to work with a tool that can really understand the data it receives. Forethought processes and understands data using NLU and agentic AI, integrating with over 40+ platforms.
This example shows how a Forethought and Zendesk integration would allow Solve to summarize and serve a KB article within a chatbot.
1. Optimize training data and knowledge bases
AI is only as reliable as the data it’s using. Your training data and knowledge base (KB) need to be well-maintained and optimized. Here’s how to do it:
- Keep information current: Regularly update your KB with new features, pricing, or policy changes. For example, a messaging platform might launch a new feature allowing threaded replies. They should update their KB with an article that includes detailed instructions on how to use them.
- Clean your data: Remove outdated information, duplicates, or references to deprecated features. For instance, a technology company might need to regularly delete mentions of unsupported integrations so that the AI will not recommend tools that no longer exist.
- Use specific examples: Include step-by-step solutions for common customer issues. For example, an email platform might have a notifications feature. Instead of just providing AI with help tickets where agents inform customers this feature exists, create an article with step-by-step instructions on how to turn notifications on and off.
- Organize clearly: Divide your KB into sections. For a SaaS-based project management tool, for example, categories like “Account Setup,” “Feature Troubleshooting,” and “Integrations” might be helpful. Categories help AI create accurate answers quickly.
Forethought customers can automatically find gaps in their KB using Discover, which analyzes customer interactions over emails, chats, calls, and surveys to identify missing information. It then drafts articles that address those gaps for your review.
This example shows Discover making a recommendation for an article on how to track an order that hasn’t yet been delivered.
4. Apply effective prompt engineering
Prompt engineering is the practice of crafting clear and specific instructions for AI. These prompts align AI responses with company policies, tone, and goals, ensuring the chatbot delivers accurate, reliable, and contextually appropriate answers. By refining how your AI interprets and responds to queries, you can reduce errors and improve customer interactions.
Forethought makes this easier with two key tools: intents and instructional notes.
Intents help AI understand customer questions naturally and accurately. They act as a guide for interpreting user inputs and generating relevant responses.
Instructional notes let you fine-tune how AI responds by adding extra context or rules. You can guide it to match your company’s communication style, clarify technical terms, or avoid specific topics. For instance, you might add a note to always explain specific acronyms or escalate sensitive topics to a human agent.
To create the most effective prompts and avoid hallucinations, follow these best practices:
- Be specific and clear: Instead of training AI with vague instructional notes like “Returns 30 days,” specify, “Explain the return policy for electronics purchased within the last 30 days.”
- Provide context: When training AI to respond to account-related questions, add background details like “The customer is asking about their current subscription plan.”
- Set constraints: Limit the scope of possible AI answers by referencing only verified policies or specific KB articles, like “Pull details from the current Returns Policy document in the KB.”
- Use examples: Show AI how to structure response with samples like, “To troubleshoot login issues, direct the customer to reset their password through the Account Settings menu.”
- Add fallback instructions: Program AI to acknowledge when it lacks enough information by telling it to say, “I don’t have that information. Let me connect you to an agent who can help.”
6. Maintain continuous monitoring and feedback loops
AI isn’t a “set it and forget it” tool—it requires ongoing care to stay accurate. When you set out to detect hallucinations, you can prevent repeated mistakes with optimizations.
Forethought simplifies this process with advanced chat insights. These insights allow our customers to search and sort conversations based on CSAT scores, workflows, or keywords and dive into individual interactions to identify hallucinations or gaps in training.
You’ll want to establish a regular process for reviewing chat transcripts and interacting with the AI agent yourself. Conduct regular “stress tests” where team members ask hypothetical customer questions.
You might ask, “What are the return policies for a product purchased online during a promotional sale if the customer lives outside the U.S.?” Document inaccuracies, off-topic answers, or gaps in knowledge, and use these findings to refine training data, adjust prompts, and expand your KB.
The way forward
AI hallucinations show how important it is to combine accuracy with efficiency in customer service. Customers expect precise, reliable answers, which means building AI systems that grow and improve your business. By using clean data, monitoring performance, and refining tools, you can create AI that builds trust and improves the customer experience.
Forethought provides the tools to help you create accurate, dependable AI supporting your team and customers. Schedule a demo today.
