Vertex AI Conversational Agent

Google Cloud Build with AI Vertex AI - Custom ML & MLOps
1

Hi Everyone,

I’ve created an agent in Vertex AI Conversational Agent using the “Build Your Own” (not chat) option. I’ve also connected it to a BigQuery dataset via the Data Store tool. I’ve defined instructions in the Playbook and confirmed that the agent is linked to the data store correctly.

However, when I test the agent, it doesn’t return any responses based on the structured data (from BigQuery or a CSV in Cloud Storage). The agent keeps responding with “I don’t have that information” or similar fallback replies.

I’m unsure how to proceed—structured data appears to be set up correctly, but the agent isn’t retrieving or referencing it during conversation. I feel stuck at this point and would really appreciate any guidance, troubleshooting tips, or best practices to resolve this.

Thanks in advance!

2

Hi @Gargeya,

Welcome to the Google Cloud Community!

It looks like you are encountering an issue where your Vertex AI Conversational Agent, despite being correctly linked to structured data via BigQuery or CSV, isn’t retrieving or referencing that information during your interactions, leading to fallback responses like “I don’t have that information.”

Here are the potential ways that might help with your use case:

  • Structured Data Store Type: Make sure your Data Store is clearly set up as a ‘Structured data’ source when connecting to BigQuery or CSV files, as this helps your agent recognize and use the data effectively.
  • Playbook Instructions: You want to make sure your Playbook instructions provide clear, actionable guidance, telling your agent exactly when to use the data store and how to reference its schema so it can respond with relevant, structured information.
  • Confirm Agent-Data Store Linkage: You may want to double-check that your Data Store is correctly linked to your agent and ensure that the “Search and retrieve information from your data stores” Generative AI capability is enabled both are essential for helping your agent respond reliably with structured data during conversations.
  • Permissions: Make sure your service account is granted the BigQuery Data Viewer or Storage Object Viewer role, as these permissions are essential for retrieving data from your connected sources.

In addition, you may want to check this lab that walks through Vertex AI agent setup and data store integration, it’s a useful way to confirm your configuration and make sure no key steps were overlooked.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

3

Hi Marvin,

Thanks so much for the detailed guidance!

I’ve double-checked everything—permissions are set correctly, and the data store is accurately linked as a structured data type. I also enabled the “search and retrieve information from your data stores” capability.

After doing a lot of research, I’ve realized the root issue seems to be that Vertex AI agents don’t inherently understand or query structured/tabular data unless it’s presented in natural language form (like descriptive summaries). For example, tables like:

program_name      location     retention_summary  
Engineering        Illinois     75% stayed in-state 5 years after graduation  
Biology            Wisconsin    62% moved out of state within 10 years

work well because the model can semantically match queries like “How many Biology students stayed in-state?” with the retention_summary.

However, my data is purely numeric like:

label_cipcode	y1_instate	y1_out_of_state	y1_instate_perc	...

And that seems to be the issue—the model doesn’t know how to translate queries like “What is the in-state ratio for year 5 for Engineering?” into column lookups unless there’s a layer to interpret it.

I’ve seen suggestions to set up a Cloud Function to create dynamic SQL queries and route user questions through it—essentially enabling SQL-style structured querying from the conversational agent.

Would you recommend going that route? Or is there an alternative way to dynamically connect the structured tabular data to the agent’s reasoning without Cloud Functions?

Thanks again for your help—this has been really informative so far!

Best,
Gargeya