Maintaining a “living ground truth” is one of the biggest hurdles in moving from a cool agent demo to a reliable tool people that can actually use. In a fast-moving business, information is fluid. For a large organization, data isn’t just a static PDF; it’s a breathing ecosystem of pricing, policies, and updates.
We recently partnered with Alaska Airlines, a global airline that faced a profound challenge. Before this project, they had no automated help on their website at all. Every single customer question, from “Can I bring my cat?” to complex baggage claims, was handled via a manual email form.
This legacy process created a massive “knowledge lag.” During peak travel seasons, customers waited up to two weeks for a response. Since every query required a human, the cost was high and the customer frustration was higher. Alaska didn’t just need a chatbot; they needed a way to value their customers’ time.
Here are five lessons that we learned while building Alaska’s new AI-first concierge service:
1. The Sitemap: Building a Knowledge Pipeline
The first step in improving CX was ensuring the agent always had the right answers. Instead of feeding the agent individual, static HTML files of the airline’s site, which creates a maintenance nightmare, we linked the agent’s datastore directly to the organization’s sitemap.xml.
-
The Component: A sitemap is a file that provides a “map” of all important pages on a website, allowing the agent to know exactly where to find information.
-
The Mechanism: We enabled Sitemap Refresh, a feature that allows Vertex AI to reference this map to identify exactly which pages need to be recrawled. This ensures the agent is only as old as Alaska’s last website deployment.
-
The Technical Edge: By verifying the site domain through the Google Search Console, we granted the crawler authorized, advanced indexing access. This ensures every sub-page and pattern within the domain is indexed securely and comprehensively, without manual intervention.
-
The Result: If the marketing team updates a baggage fee on the website, the agent learns it automatically. For a customer who used to wait two weeks, getting a factual answer in two seconds is a game-changer.
2. The Vertex AI Reranker API: Ensuring Precision and Trust
Crawling a website is only half the battle; once all the data is indexed, the agent must find the exact fact among thousands of pages. Standard search might find five pages that mention “dogs,” but only one has the current weight limit for cabin travel.
-
The Component: The Vertex AI Reranker API is a specialized tool that acts as a “quality filter”. It takes a list of potential answers and re-orders them based on which one most accurately fits the user’s specific question.
-
The Solution: We integrated this API to act as a second-stage filter. It analyzes the context of the conversation to ensure the agent doesn’t just give a “related” answer, but the correct one.
-
The Refinement: Unlike basic keyword matching, the Reranker API understands nuances. It knows the difference between a user asking about a “service animal” versus a “pet,” which ensures that the agent selects the policy that actually applies to the scenario.
-
The Impact: This drastically reduces hallucinations (where the agent makes things up) and ensures that the agent provides high-accuracy answers. By providing the correct answer on the first try, the airline builds the customer trust that is often lost in automated systems.
3. Orchestration: The Steering Playbook
Managing an airline’s operations is complex. You wouldn’t ask a pilot to handle a refund or a mechanic to book a ticket. We applied this same logic to the agentic architecture.
-
The Component: In Vertex AI Agent Builder (now Gemini Enterprise Agent Platform), we used Generative Playbooks, which are specific sets of natural language instructions that define how an agent should handle a particular task.
-
The Steering Playbook: We deployed a “manager”, or Steering Playbook, that serves as the initial point of contact. Its sole job is to listen to the customer, identify their intent, and route the conversation to the correct “Specialist” agent, like a “Baggage Expert” or “Mileage Specialist”.
-
Business Value: This prevents the agent from becoming overwhelmed by a monolithic prompt (one giant, confusing set of instructions). Since each specialist has its own focused set of instructions, the agentic system becomes much easier to test, update, and scale as it grows.
Creating a Generative Playbook
Before we dive deep into the architecture of our customer solution, let’s look at the iterative loop of defining, testing, and revising a Generative Playbook.
The process starts by initializing a new playbook: defining the logical flow through Steps, integrating APIs via Tools, and establishing Parameters to pass context. Most crucially, we provide few-shot examples to train the model on how these should be executed. Next comes an iterative loop of testing and validation using the in-built agent simulator, analyzing stack traces to identify reasoning failures, and refining the components until the agent performs correctly and consistently.
The Architecture: Bringing it All Together
Now that we understand how Playbooks work, let’s look at the architecture that used Playbooks at its core to manage the agent’s end-to-end workflow.
When a visitor asks a question (“How can my children fly alone?”), the query is transmitted to the Agent Builder backend. A webhook then triggers a Cloud Function that calls the Vertex Search Engine API. The API crawls the airline’s website data store and retrieves a generative answer with linked citations. This information is processed by Gemini to ensure it is concise before being sent back to the user. Finally, all interactions are recorded in BigQuery, where a Cloud Function executes every 15 minutes to push data to Cloud Monitoring for ongoing performance analysis.
4. Warm hand-off: Seamless Escalation to Telephony Platforms and CRM Systems
Perhaps the biggest leap in Customer Experience was how the system handles the “tough stuff.” Not every question can or should be answered by a virtual agent. Success isn’t just about how many calls an agent “deflects”; rather, it’s about how well it sets up the human team for success.
-
The Webhook: A Webhook is a technical bridge that allows the agent to send information to or trigger an action in an external system in real-time.
-
The Cloud Function: A Cloud Function is a small piece of code that runs in the background to process that information and perform a specific task, like packaging data for a human agent.
-
Stateful Warm Handoffs: We replaced the frustrating legacy “email and wait” process with a stateful warm handoff. When the agent hits its logic limit, it triggers a Webhook and Cloud Function to bundle a “context packet”, including the user’s name, verified intent, and a summary of the conversation between it (the “Ask Alaska!” virtual agent) and human user till that point.
-
The Integration: This packet is pushed directly into the human agent’s console via a telephony provider such as Google Cloud CCaaS.
-
The ROI: Human experts skip the “discovery” phase. They no longer ask, “What is your account number and how can I help you?” Instead, they say, “I see you’re having trouble with your mileage plan, let’s fix that.” This leads to significantly lower Average Handle Time (AHT) and much higher Customer Satisfaction (CSAT) scores.
The Value Shift: Automating the Simple, Mastering the Complex
The real value of this project wasn’t just technical: it was human. Before this, the airline’s staff was buried under a mountain of repetitive emails, leading to burnout and slow service. By rethinking the architecture, we shifted the team’s role from managing backlogs to designing better customer experiences.
We prioritized building a loop of continuous improvement. Every interaction is recorded in BigQuery and visualized via Looker Studio, giving the business process team real-time visibility into customer intent and areas for agent refinement. By automating the 80% of inquiries that are repetitive and simple, we’ve empowered Alaska’s support staff to focus on the 20% of cases that require human empathy and complex problem-solving.
This isn’t just about efficiency; it’s about empowering people to do the work they’re best at. By eliminating knowledge lag and high-friction manual processes, the airline hasn’t just updated its tech stack; it has set a new standard for customer service.
Try “Ask Alaska!” out for yourself here!
Best,
Gayatri (gayatriap@) and Aakansha (maakansha@)