Vertex AI Search (Discovery Engine): key property values (e.g. title) are not returned in a standard field for structured data stores

[aleksvujic]

Description

I’m using a Vertex AI Search application (engine) connected to 2 structured data stores. I’m calling it through the Discovery Engine API (servingConfigs/*:search, v1, eu location, .NET client library Google.Cloud.DiscoveryEngine.V1 version 1.14.0).

data-store-1 (field title is marked with key property = title):

data-store-2 (field summary is marked with key property = title):

Request:

POST https://eu-discoveryengine.googleapis.com/v1/projects/{project}/locations/eu/collections/default_collection/engines/{engine}/servingConfigs/default_serving_config:search
{
  "query": "compiler flag",
  "pageSize": 3,
  "contentSearchSpec": {
    "snippetSpec": { "returnSnippet": true },
    "summarySpec": { "summaryResultCount": 5, "includeCitations": true }
  }
}

Response (the first document is from data-store-1 and the second document is from data-store-2):

{
  "results": [
    {
      "id": "123",
      "document": {
        "name": "projects/.../dataStores/data-store-1/branches/0/documents/123",
        "structData": {
          "title": "How to initialize session",
          "url": "https://www.website.com/documentation/how-to/index.html",
          "content": "Welcome! This page contains frequenty...",
          "products": ["gui"],
          "keywords": ["select"]
        },
        "derivedStructData": {
          "snippets": [{ "snippet": "...", "snippet_status": "SUCCESS" }],
          "url": "https://www.website.com/documentation/how-to/index.html",
          "can_fetch_raw_content": "true",
          "is_exact_match_query": 0
        }
      },
      "retrievalSignals": {
        "retrievalSources": [
          "SEMANTIC_SEARCH"
        ],
        "semanticRelevanceScore": 0.44342685
      }
    },
	
	{
      "id": "456",
      "document": {
        "name": "projects/.../dataStores/data-store-2/branches/0/documents/456",
        "structData": {
          "summary": "Enable automatic updates",
          "url": "https://www.website.com/releases/update",
          "content": "To enable automatic updates...",
          "products": ["gui"],
          "keywords": ["update"]
        },
        "derivedStructData": {
          "snippets": [{ "snippet": "...", "snippet_status": "SUCCESS" }],
          "url": "https://www.website.com/releases/update",
          "can_fetch_raw_content": "true",
          "is_exact_match_query": 0
        }
      },
      "retrievalSignals": {
        "retrievalSources": [
          "SEMANTIC_SEARCH"
        ],
        "semanticRelevanceScore": 0.31827265
      }
    }
  ]
}

Problem

Both data stores map a field to the title key property, but each uses a different field name:

1. data-store-1: field title
2. data-store-2: field summary

The response doesn’t use those mappings to present the title in a consistent way:

1. The title value appears only in structData, under each store’s own field name (structData.title in one result and structData.summary in the other).
2. derivedStructData has no title field, and neither does document itself.
3. url does appear in derivedStructData for both results. A title equivalent doesn’t.

This means the client needs to be aware of the schema of every individual data store connected to the application just to extract a basic, common property such as the result title. That seems particularly surprising for an application (engine) that is intended to provide a unified search experience across multiple data stores: the application knows that title is the key property for each data store, but that information is not reflected in a common field in the search response.

In other words, the engine provides a single search endpoint over multiple data stores, but the client still needs to maintain knowledge of the individual data-store schemas in order to render a basic search result. Adding another data store, or changing which field is mapped to the title key property, would require a corresponding client-side change.

So a client that wants to show something as simple as a result title has to know the internal JSON layout of every data store connected to the app. For each result it has to:

1. parse the data store ID out of document.name (.../dataStores/data-store2/...),
2. keep its own per-store mapping (data-store-1 → title, data-store-2 → summary),
3. update that mapping whenever a data store is added to the app or its schema changes.

That seems to defeat, at least partially, the purpose of an application (engine) that combines several data stores behind one search endpoint. The schema already says which field holds the title, but the client still can’t get a standardized value back.

Question

Is there an existing option that returns key property values (title, uri, description, category, …) in standardized fields for structured data stores?