AI agents such as chatbots and voicebots are nothing new in customer service. With the help of large language models, they now achieve particularly natural and effective communication, but this also poses a serious challenge. Because AI agents tend to hallucinate. When chatbots or voicebots hallucinate, they invent information that sounds plausible but has no factual basis. And caution is advised, because these phenomena are far more than simple programming errors.

This is exactly what happened at Air Canada in 2024, when a chatbot promised a customer a non-existent refund policy for a flight. The airline had to pay for the mistake and refund the customer anyway. Such incidents illustrate that the risks of hallucinations go beyond a dissatisfied customer inquiry: They threaten your reputation, can result in costly legal consequences, and lead to operational inefficiencies as your employees have to correct the AI's mistakes.

Hallucinations arise when using Large Language Models (LLMs), which essentially function as probability machines. LLMs have been trained to analyze huge amounts of text and recognize patterns in order to then predict the statistically most likely next word that forms a coherent, human-like statement. It is this predictive ability that is at the root of the problem.

The model strives for a plausible-sounding answer, not necessarily a factually correct one.

If an LLM does not find clear information in its training data or in the direct context, it fills these knowledge gaps by generating stylistically and grammatically correct but factually invented information. One reason for this is that the models are more positively reinforced in training if they give any answer at all than if they admit to not knowing something.

Hallucinations must be distinguished from simple errors. A simple error occurs, for example, when a bot displays outdated opening hours because the underlying knowledge database has not been updated or selects the wrong one of two products stored in the catalog.

A hallucination, on the other hand, is more insidious, as the bot actively invents information. This type of misinformation is extremely difficult to detect and has far-reaching consequences for customer satisfaction and compliance.

The good news for customer service leaders is that hallucinations from chatbots or voicebots are not an uncontrollable fate, but can be controlled through a robust, multi-layered approach. This approach relies on four technological and procedural pillars that interlock seamlessly to significantly improve the factual fidelity and reliability of your AI agents.

The basis for reliable bot responses is a well-prepared knowledge store. This process is subdivided into chunking and vectorization.

Chunking involves breaking down large amounts of data into smaller units of information (chunks). The chunks used to generate the response should be coherent and complete. Errors often occur here, such as simple chunking according to a fixed number of characters, where a sentence can start in one chunk and end in another.

The ideal variant is therefore smart chunking or content-based chunking. Here, the document structure is used (e.g. headings or formatting) to generate logical and self-contained sections.

Example: In a mechanical engineering company with different machine models, each chunk containing technical details must specify the exact machine and model to which the information relates.

Following chunking, each text chunk is converted into a vector. A vector is a numerical representation that captures the semantic meaning of the text in a high-dimensional space. This step is essential, as the customer query, which is also vectorized, can be compared with the existing chunks for semantic similarity with the utmost precision during the subsequent search.

Implementing a RAG architecture is an effective way to prevent the invention of information and force the LLM to stick to reality. RAG equips the model with an external, verified knowledge repository (e.g. FAQs, internal manuals) as a single source of truth, so that it does not rely solely on its own, potentially erroneous training data

Each user request is processed in four steps:

• The request is interpreted first.
• In the retrieval phase, the system then searches your verified knowledge database for relevant facts.
• Only in the augmentation phase are these facts transferred to the LLM as a clear context together with the original question.
• Finally, in the generation phase, the model formulates a natural response based strictly on the verified facts.
• In a further step, advanced systems perform a final fact check, in which the generated answer is checked again together with the facts.

This approach drastically reduces the likelihood of the AI agent hallucinating. This leads directly to a higher first resolution rate (FCR) and a lower escalation rate, as customers receive reliable answers.

While RAG regulates what the bot says, Guardrails and Prompt Engineering determine how it says it and what it is allowed to talk about.

Guardrails are programmatic rules that set clear, predefined boundaries for the AI model. Examples include thematic restrictions, a ban on making unauthorized promises or guarantees, and the establishment of sentiment analyses that immediately trigger a proactive handover to a human employee in the event of highly disgruntled customers.

In addition, precise prompt engineering provides an unambiguous framework for action. An optimized system prompt can instruct the model to answer questions based solely on the context provided. If the information is missing, the model is instructed to admit uncertainty and hand over to a human colleague instead of guessing.

Despite all the technology, human supervision is the third, indispensable pillar. The human-in-the-loop approach specifically integrates human intelligence into the AI process to serve as the ultimate safety net and quality assurer.

Calls in which the bot was unsure or which were rated negatively by the customer should be viewed by employees and the corrections should flow directly back into the system as learning feedback.

Clear escalation paths also ensure a seamless handover from the bot to a human agent as soon as the conversation becomes too complex or emotional.

The success of your Conversational AI strategy stands and falls with the containment of your AI agent's hallucinations. BOTfriends is your strategic partner for implementing these measures.

Our platform is designed from the ground up to minimize the risks of generative AI.

The question today is no longer whether you use generative AI, but how you integrate it into your processes in a safe, responsible and controlled manner. Although LLMs are constantly being optimized to detect hallucinations, if you want to be on the safe side and stand out from the competition, you should leave nothing to chance with a platform like BOTfriends X.