Vertex AI Search Fails to Ingest Data from Data Stores Sourced from either Google Drive or Google Cloud Storage

[j-fujimori]

Hello, I am trying to build a chatbot for internal company regulations using Google Cloud’s Vertex AI Search. However, I am facing a critical issue where the data store does not ingest any data from the data source, resulting in zero search results.

Project ID: (PII Removed by Staff)

Methods Attempted:

1. Using Google Drive as a Data Source:

• Symptoms:

  • Created a data store specifying a specific folder within a Google Drive Shared Drive as the data source. The status becomes “Active”, but the document count remains 0.

  • The Search Preview consistently returns “No results found.”

  • The Logs Explorer shows no logs related to the data connector.

  • The test also failed when targeting “All of Drive” instead of a specific folder.

• Troubleshooting Steps Performed:

  • Granted the “Workforce Pool Admin” (roles/iam.workforcePoolAdmin) IAM role to the user account.

  • Configured Google Identity as the ID Provider in AI Applications authentication settings.

  • Confirmed that “Smart features for other Google services” is enabled in the Google Workspace Admin Console.

  • Granted “Viewer” permissions for the target Google Drive folder to the designated service account (service-... **at** gcp-sa-enterpriseknowledge.iam.gserviceaccount.com).

  • Performed a full cleanup (delete and recreate) of both the app and the data store multiple times.

  • Created a simple test .txt file in the target folder, but it was not detected.

2. Using Google Cloud Storage (GCS) as a Data Source:

• Symptoms:

  • As an alternative to isolate the issue, I used GCS.

  • I uploaded files to a GCS bucket and created a data store using that bucket as the data source. The status becomes “Active”, but the document count remains 0.

  • The Search Preview still returns “No results found.”

• Troubleshooting Steps Performed:

  • GCS bucket permissions were left at the default (Not public).

  • Configured periodic synchronization for the data store.

  • Created the app and data store from a completely clean state.

Conclusion: Regardless of whether the data source is Google Drive or GCS, the Vertex AI Search data store is failing to ingest any data. I have verified all known user-configurable settings without success. I suspect there may be a special policy or restriction at the project or organization level that is not visible to me and is fundamentally blocking the data ingestion process.

Could anyone provide insights into similar issues, or suggest any other areas I should investigate?

[a_aleinikov]

Hi! @j-fujimori This usually happens because the Vertex AI Search service account (service-<project-number>@gcp-sa-enterpriseknowledge.iam.gserviceaccount.com) doesn’t have the right permissions. Give it Viewer on the Drive folder or Storage Object Viewer on the GCS bucket, and make sure both the Drive API and Vertex AI Search API are enabled. After that, ingestion should start working.

[j-fujimori]

Thank you for your reply. Unfortunately, we have already performed all the steps you suggested, but the issue persists.

• The Vertex AI service account has been granted the “Viewer” role on the target Drive folder and the “Storage Object Viewer” role on the GCS bucket.

• Both the Vertex AI API and the Drive API are enabled in our project.

Even with these configurations, no data is ingested, and no logs are generated. Do you have any other suggestions on what we could check?

[Sam_Wathen]

Hi @J ,
I am running into the same problem while trying to set up a Vertex AI Search sourced from Google Drive for my startup. Have you had any success figuring out the issue since your last update?

Thanks!

Sam

[j-fujimori]

Hello Sam.
I gave up on the problematic environment and recreated it in a different environment. I think the version of Vertex AI provided by the region was probably affecting it.

[Tomáš_Dvonč]

Hello, I am facing this exact issue.

I am the Owner of project (PII Removed by Staff) and my Google Drive data store is not indexing.

I have already:

1. Granted the service-<project-number>@gcp-sa-enterpriseknowledge.iam.gserviceaccount.com the Viewer access directly to the correct Drive/Folder ID.

2. Confirmed the folder is not empty and contains indexable PDFs.

3. Confirmed the Google Drive API is enabled.

4. Deleted and recreated the data store multiple times.

Just like everyone else here, my document count remains - and I see no error logs in Logs Explorer. The import is failing silently, even though all permissions seem correct.

[duckdepaepe]

Exact same problem here. When i look in the Drive logs in google admin, there nothing in the logs that indicates the files from the drive are being indexed.

[Salman_Khan1]

Any update on this, did anyone cracked it ? I am stuck on this for a couple of weeks now.