Search for Retail - problems exporting product catalog to BigQuery

Google Cloud Build with AI AI Solutions
1

After integrating real-time user event recording into GCP Search for Retail we wanted to do some basic analytics on the products and events.

We have found instructions on exporting user events to BigQuery – this looked like exactly what we need. We have completed the procedure for exporting user events without problems. However, we cannot seem to go through the instructions for procedure for exporting products without errors.

The steps we have followed:

We have created the target data set in BigQuery that would hold the exported data in the US multi-region:

andr_0-1721299737153.png
andr_0-1721299737153.png1172×1779 237 KB

Then:

gcloud config set project [project-id]
gcloud auth application-default login
cat > request.json << 'EOF'
{
  "outputConfig": {
    "bigqueryDestination": {
      "datasetId": "retail_data",
      "tableIdPrefix": "exported",
      "tableType": "view"
    }
  }
}
EOF

Then:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://retail.googleapis.com/v2alpha/projects/[PROJECT_ID]/locations/global/catalogs/default_catalog/branches/default_branch/products:export"

We get the following error:

{
  "error": {
    "code": 403,
    "message": "Your application is authenticating by using local Application Default Credentials. The retail.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds .",
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "SERVICE_DISABLED",
        "domain": "googleapis.com",
        "metadata": {
          "service": "retail.googleapis.com",
          "consumer": "projects/[project-number]"
        }
      }
    ]
  }
}

Looking up Troubleshoot your ADC setup we find “If you are calling the REST or RPC API directly, use the x-goog-user-project HTTP header to specify a quota project in each request.” Let’s try that:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    -H "x-goog-user-project: [PROJECT_ID]" \
    -d @request.json \
    "https://retail.googleapis.com/v2alpha/projects/[PROJECT_ID]/locations/global/catalogs/default_catalog/branches/default_branch/products:export"

Which returns a new error:

{
  "error": {
    "code": 404,
    "message": "Method not found.",
    "status": "NOT_FOUND"
  }
}

We tried this procedure locally and via Cloud Shell – same result.

We tried using “0” as the branch ID instead of “default_branch” – same result.

This is where we have gave up trying to export product catalog.

Has anyone else encountered this and found a working method for exporting product catalog to BigQuery?

2

I got exactly the same issue. Been trying lots of things, but ultimately I get the same 404 error as above. Following https://cloud.google.com/retail/docs/export-data-into-bq?hl=en says to use products:export, but reading through the REST API reference documentation I cannot see that endpoint https://cloud.google.com/retail/docs/reference/rest . Specifically, https://cloud.google.com/retail/docs/reference/rest/v2alpha/projects.locations.catalogs.branches only lists “get” and “list”.

3

Thanks for your message, glad to know someone else is experiencing the same problem.

Based on the paths mentioned in the Export to BQ documentation, I think the REST API documentation for the “export” calls would be in these places:

  1. Products: projects.locations.catalogs.branches.products
  2. User Events: projects.locations.catalogs.userEvents

There’s quite a bit of methods there, such as “import” or “purge”. There is no “export” in either. But since the “export” method for User Events is working, I think these methods might undocumented.

1 Like