Dialogflow CX: Vertex AI Search (RAG)

Google Cloud Build with AI Conversational Agents (Dialogflow CX)
1

I am building a production voice support agent using Dialogflow CX integrated with Twilio (voice) and Vertex AI Search (RAG).

The architecture intentionally separates responsibilities:

Dialogflow CX flows → routing, confirmations, and high-risk / transactional actions

Vertex AI Search (RAG) → informational “How do I…?” explanations

Voice-first interaction (no chat UI)

Expected behaviour

After a RAG response is delivered (via Vertex AI Search):

The agent should continue the conversation

Either:

Ask a clarifying follow-up (e.g. “Do you want me to walk you through this now?”), or

Route the user to the correct Dialogflow CX flow/page based on intent

Actual behaviour

Vertex AI Search returns a correct, grounded answer

After the response:

The conversation ends, idles, or escalates

No follow-up intent capture occurs

CX does not transition to another page or flow

The RAG response itself is correct — the issue is conversation continuation and routing after RAG.

Current configuration (summary)

Dialogflow CX agent (voice)

Twilio for telephony

Vertex AI Search connected as a Data Store

Start Flow used for routing only

Default Welcome Intent disabled

RAG invoked via fulfilment for informational queries

Transactional processes handled in scripted CX flows

What I have already ruled out

Default Welcome intent interception

Obvious page or event handler misconfiguration

Webhook timeouts or fulfilment errors

Twilio audio cut-off issues

Question

Is there a recommended Dialogflow CX design pattern for:

Continuing a conversation after a Vertex AI Search (RAG) response, and

Cleanly transitioning back into CX intent/page routing

without prematurely ending or escalating the interaction?

Specifically:

Should follow-up confirmation be handled in CX rather than via RAG?

Is there a known best practice for chaining RAG → intent capture → flow transition in voice agents?

Any guidance, references, or examples would be appreciated.

2

You can do this things in many ways .you can design change and as per your business need works well go for that

1.Configure your Data Store prompt (or Generator) to explicitly append a follow-up question (e.g., “Would you like me to start that process?”) to the RAG response, ensuring the user knows it is their turn to speak.

image
image597×521 45.1 KB

2. Routing :- RAG to Intent Capture and Flow Transition:

  • Create a dedicated follow-up page (e.g., “post_rag_confirmation”) that activates after RAG via transition.
  • On this page:
    • Use entry fulfillment to prompt a clarifying question (e.g., “Do you want me to walk you through this now? Say yes or no.”).
    • Define routes for intent capture:
      • Intent: “confirmation.yes” → Transition to a transactional flow (e.g., “reset_account_flow”).
      • Intent: “confirmation.no” → Route back to Start Page or escalate.
      • No-match handler: Reprompt with shortened question (e.g., “Sorry, yes or no?”)
    1. Use flow to this way use webhook to call data store
3

To fix this “dead-air” issue in a voice-first environment, the core problem is that a Data Store fulfillment acts as a terminal response unless a state transition is explicitly forced.

The Recommended Design Pattern: “The Wrapper Page”

The best practice is to move the RAG invocation from a global route or the Start Page into a dedicated “Informational Handling” Page.

1. Create a “Post-RAG” State:

Instead of staying on the same page, set the Transition of your Data Store route to a new page (e.g., Info_Follow_Up). This ensures the state machine is “expecting” more input after the RAG answer.

2. Handle Follow-ups in CX, not RAG:

Vertex AI Search is great at grounding facts but poor at managing conversation state.

• In the Entry Fulfillment of your Info_Follow_Up page, add a static Text Response (e.g., “Does that help, or would you like to speak to an agent?”).

• This forces the Twilio stream to stay open and triggers the “listening” state.

3. Chaining via Conditional Routes:

To transition back to transactional flows, use Intent Routes on this “Post-RAG” page.

• If the user says “Yes” → Transition to Start Page / Main Menu.

• If the user triggers a transactional intent (e.g., “How do I pay?”) → Transition to the specific Payment page.