Practical Approach to Mitigating AI Hallucinations in Customer-Facing LLMs (Python Implementation)

Google Cloud Build with AI AI Solutions
1

Hello everyone,

As we integrate Large Language Models (LLMs) into real-world business applications, one of the most critical challenges we face as Data Scientists is AI Hallucination. When dealing with customer service or enterprise data, a fluent but factually incorrect response can severely impact business integrity and digital sovereignty.

I’d like to share a practical approach we’ve been refining at Whitecyber Data Science Lab. We use a method called Contextual Anchoring (a fundamental part of RAG) combined with strict boundary prompting to force the LLM to rely only on the provided truth source, completely restricting its internal, pre-trained knowledge for specific queries.

Here is a simplified Python implementation demonstrating this concept for a hypothetical healthcare clinic chatbot. The goal is to prevent the model from assuming business hours or insurance policies.

import os
from openai import OpenAI # Or adapt to Vertex AI / Gemini API

# Initialize Client
client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

# =====================================================================
# STEP 1: Establish the "Trusted Knowledge Base" (Data Sovereignty)
# In production, this is retrieved dynamically via vector DBs.
# =====================================================================
clinic_policy_document = """
[OFFICIAL CLINIC POLICY]
1. Operating Hours: Monday to Friday, 08:00 AM - 04:00 PM.
2. The clinic is CLOSED on Saturdays, Sundays, and National Holidays.
3. Insurance: We only accept private insurance (Prudential, Allianz). We DO NOT accept state insurance.
4. General Practitioner Consultation Fee: $10.
"""

# =====================================================================
# STEP 2: The User Query (Potential trap for hallucination)
# =====================================================================
user_query = "Can I visit the clinic this Sunday using my state insurance? How much is the fee?"

# =====================================================================
# STEP 3: Hallucination Mitigation (System Prompting)
# Forcing the "Learning by Outcome" rule: prioritize truth over fluency.
# =====================================================================
system_instruction = f"""
You are the official virtual assistant for the Clinic.
Your primary directive is to answer user queries with STRICT DATA INTEGRITY.

ANTI-HALLUCINATION RULES:
1. You MUST ONLY answer based on the [OFFICIAL POLICY] text below.
2. DO NOT use any outside knowledge or assumptions.
3. DO NOT guess or fabricate answers.
4. If the user's question cannot be answered using the text below, you MUST reply exactly with: 
   "I apologize, but that information is not available in our current policy database. Please contact human support."

[OFFICIAL POLICY]
{clinic_policy_document}
"""

# =====================================================================
# STEP 4: Execution
# =====================================================================
def ask_assistant(question):
    try:
        response = client.chat.completions.create(
            model="gpt-3.5-turbo", 
            temperature=0.0, # CRITICAL: Temp 0.0 minimizes creative hallucinations
            messages=[
                {"role": "system", "content": system_instruction},
                {"role": "user", "content": question}
            ]
        )
        return response.choices[0].message.content
    except Exception as e:
        return f"System error: {e}"

# Output
print("User : ", user_query)
print("Bot  : ", ask_assistant(user_query))

Why this works effectively:

  1. temperature=0.0: This drastically reduces the model’s creative variance, making the output highly deterministic.

  2. Explicit Fallback Constraints: Instructing the model exactly what to say when the data is missing prevents it from trying to be “helpful” by making things up.

I believe that building robust AI isn’t just about training better models, but building stricter validation frameworks around them.

How does everyone else here handle boundary enforcement in your production LLMs? Do you rely more on pre-prompting, or do you use secondary models to verify the output before sending it to the user?

Would love to hear your thoughts!
:star_struck: