Vertex AI batch inference - input/output from/to multi-region BigQuery dataset

Google Cloud Build with AI Custom ML & MLOps
, ,
1

Hi,

I have been doing batch inference in Vertex AI with custom model. The input data is from BigQuery table. Last week it worked, but for some reason now I can’t select the same table I did then, because of region vs. multi-region. Happens when trying to do this on the GCP console.

When writing the path to the input table I get the following error: “BigQuery resource must be in europe-west3. The selected resource is in EU.” Even though the BigQuery dataset was in multi-region (EU) last week when I was able to run the batch inference. Last week I was also able to browse other multi-region datasets in the input selection, but now I see nothing. For output selection I can browse and select multi-region (EU) datasets/tables.

Has something changed in Vertex AI during the weekend, or why can’t I select the input to be multi-region dataset. Or should it have been possible from the start. And why the output table can be selected from multi-region? If I were to use GCS buckets for the input/output, those could be selected from multi-region buckets.

From what I understand the models can’t be saved to multi-region. So would the best solution here be to create a dataset with the same region (europe-west3) as the model for the input data. Or is there way to get the input from multi-region dataset.

2

Hello @LeeviHHHH,

I don’t see any changes in the Vertex AI release notes, so I would say no changes have been applied to region management for batch inference. That said, Vertex AI is moving so fast that something may have changed without notification. Maybe a Googler could tell us more about that?

Anyway, I’ve been using batch inference for a while. I checked this morning, and if I choose a batch inference region that is different from the region of my BigQuery tables, the batch inference service cannot resolve any table.

Know that I have multiple tables that are stored in the “EU” region, but since batch inference cannot be started as multi-regional, I have to pick a specific region.

Also, picking “europe-west1” will show the tables that are in the “europe-west1” region only, not EU.