Automatic refresh for datastore indexation

Google Cloud Build with AI Custom ML & MLOps
1

Hi,

I’m using the Discovery Engine of Datastore to index my website content and create a conversational agent.

How can I schedule the advanced indexing automatically? The indexing process takes too much time. Is it possible to perform automatic refreshes?

2

Hi @yasmine ,

Discovery Engine doesn’t support automatic re-indexing out of the box, but you can set it up yourself:

How to Schedule Automatic Index Refresh:

  1. Use Cloud Scheduler (GCP’s cron job service)

  2. Trigger a Cloud Function or Cloud Run that:

    • Calls the Advanced Website Indexing API

    • Passes your Datastore/Website ID

  3. Frequency: Set it to run daily or weekly depending on how often your content changes

This setup gives you full control over index refresh timing.

3

Hi @yasmine,

Welcome to the Google Cloud Community!

Yes, automatic refreshes are possible. You may find this documentation useful in resolving your issue, as it outlines the recommended solution and details how to handle website re-crawling in Google Cloud’s Discovery Engine, including both automated and manual refresh methods.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.