Introducing Observability for Conversational Analytics in Google Cloud

kgrinevskaja-16 · September 1, 2026, 7:04pm

Authors: Kate Grinevskaja and Kiko Aumond

Conversational Analytics can transform enterprise data exploration. By translating natural language questions into accurate SQL queries and rich data visualizations, Conversational Analytics agents across Google Cloud empower users of all skill levels to rapidly extract meaningful insights from their data.

Yet as organizations move conversational data agents from initial pilots into critical production workflows, leaders face a common challenge: AI agents have traditionally functioned as “black boxes.”

Without granular visibility into agent behavior, platform owners and FinOps teams may encounter several key roadblocks:

Trust & Transparency Gaps: Limited insight into the intermediate reasoning steps, retrieved schema metadata, and query generation logic behind every answer.
Cost Attribution Challenges: Difficulty breaking down operational costs between LLM token usage and downstream data warehouse execution.
Troubleshooting Complexity: Pinpointing whether a slow response was caused by schema retrieval, model inference latency, or expensive warehouse queries.

To solve these challenges, Google Cloud provides Observability for Conversational Analytics. Built natively into Google Cloud’s Observability platform, this launch provides out-of-the-box visibility into token consumption, execution latency, and end-to-end trace statuses—with no custom instrumentation or required code changes.

Architecture of Conversational Analytics Observability routing through Google Cloud Observability into BigQuery Agents Hub, Observability Analytics, and Looker.

How Conversational Analytics Observability Works

Observability operates across Google Cloud’s data analytics and operations ecosystem via a unified architecture:

Cloud Trace (Span-Level Transparency): Every interaction turn automatically generates a distributed trace containing hierarchical OpenTelemetry spans detailing metadata resolution, model reasoning, SQL generation and execution, and query execution.
Cloud Monitoring (Time-Series Telemetry): Aggregated metrics, including token volumes, active sessions, latency percentiles, and error counts, are continuously emitted under the BigQuery Conversational Analytics namespace for platform health tracking and alerting.
BigQuery In-Console Agents Dashboard: Accessible within BigQuery Agents Hub, agent owners gain a high-level view of active conversational sessions, user engagement trends, query execution durations, cost related metrics, and error rates where their agents live.

image1512×771 165 KB
Observability Analytics: Cloud Trace telemetry is automatically exposed as queryable BigQuery linked datasets (_Trace.Spans._AllSpans), allowing teams to run SQL queries across spans, join with billing tables, information schema and build custom Data Studio dashboards.

Looker In-IDE Agents Observability Dashboard - Looker’s native observability provides an out-of-the-box dashboard containing various useful metrics that help admins understand token usage, user engagement with agents, responses and feedback from the agents.

Admins can now predict and manage token costs while continuously optimizing agent accuracy, response quality, and user sentiment. For more details check out the blog here.

How to Use Observability Analytics for Custom FinOps & Org-Level Insights in BigQuery

While the Trace Explorer UI is fantastic for single-turn debugging, enterprise FinOps and platform teams often need to aggregate telemetry across the organization to calculate Total Cost of Ownership (TCO) and understand adoption patterns.

What is Observability Analytics?

In Google Cloud Observability, Observability Analytics automatically provisions a dataset directly connected to your Cloud Trace telemetry. This dataset can be queried via Observability Analytics. Alternatively, If you want to query traces from BigQuery Studio, join traces with existing enterprise warehouse tables, or build Looker dashboards, create an Observability Linked Dataset.

You can access Observability Analytics by:

Navigating to Cloud Trace > Observability Analytics in the Google Cloud Console.
Querying the linked dataset directly inside BigQuery Studio.
Connecting the linked dataset to Data Studio or Looker to create executive-level usage and cost dashboards.

Example 1: Multi-Project Token Aggregation & Adoption Tracking

To understand LLM token usage across multiple Google Cloud projects inside your organization, you can query your exported _AllSpans trace sink tables using a UNION ALL.

In Google Cloud Console, navigate to Cloud Trace > Trace Scopes (or Observability Scopes).
Click Create Trace Scope (e.g., name it org-ca-trace-scope).
Add all GCP projects in your organization that deploy or use CA agents.
Open Trace Explorer, click on the Scope dropdown in the top toolbar, and select your newly created trace scope (org-ca-trace-scope).

Here is a query with an example project and data set:

WITH raw_spans AS (
  -- Tier 1: Scan, filter, and extract JSON fields ONCE into typed columns
  -- Project 1: Finance Data (Example)
  SELECT
    'finance-data-prod' AS gcp_project_name,
    trace_id,
    JSON_VALUE(attributes, '$."gen_ai.conversation.id"') AS conversation_id,
    SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.input_tokens"') AS INT64) AS input_tokens,
    COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.cache_read.input_tokens"') AS INT64), 0) AS cached_input_tokens,
    SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.output_tokens"') AS INT64) AS output_tokens,
    COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.reasoning.output_tokens"') AS INT64), 0) AS reasoning_output_tokens
  FROM `finance-data-prod.us._Trace.Spans._AllSpans`
  WHERE
    start_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY)
    AND (name = 'generate_content' OR STARTS_WITH(name, 'generate_content'))

  UNION ALL

  -- Project 2: Marketing Analytics (Example)
  SELECT
    'marketing-analytics' AS gcp_project_name,
    trace_id,
    JSON_VALUE(attributes, '$."gen_ai.conversation.id"') AS conversation_id,
    SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.input_tokens"') AS INT64) AS input_tokens,
    COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.cache_read.input_tokens"') AS INT64), 0) AS cached_input_tokens,
    SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.output_tokens"') AS INT64) AS output_tokens,
    COALESCE(SAFE_CAST(JSON_VALUE(attributes, '$."gen_ai.usage.reasoning.output_tokens"') AS INT64), 0) AS reasoning_output_tokens
  FROM `marketing-analytics.us._Trace.Spans._AllSpans`
  WHERE
    start_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY)
    AND (name = 'generate_content' OR STARTS_WITH(name, 'generate_content'))
),
typed_spans AS (
  -- Tier 2: Compute uncached vs cached input tokens
  SELECT
    gcp_project_name,
    trace_id,
    conversation_id,
    COALESCE(input_tokens, 0) AS input_tokens,
    cached_input_tokens,
    GREATEST(0, COALESCE(input_tokens, 0) - cached_input_tokens) AS uncached_input_tokens,
    COALESCE(output_tokens, 0) AS output_tokens,
    reasoning_output_tokens
  FROM raw_spans
)
-- Tier 3: Aggregation across projects
SELECT
  gcp_project_name,
  COUNT(DISTINCT trace_id) AS total_requests,
  COUNT(DISTINCT conversation_id) AS total_conversations,
  SUM(input_tokens) AS total_input_tokens,
  SUM(cached_input_tokens) AS total_cached_input_tokens,
  SUM(uncached_input_tokens) AS total_uncached_input_tokens,
  SUM(output_tokens) AS total_output_tokens,
  SUM(reasoning_output_tokens) AS total_reasoning_output_tokens
FROM typed_spans
GROUP BY gcp_project_name
ORDER BY total_requests DESC;

Example 2: Correlating Data Agents to BigQuery Compute Spend (Slots & Bytes Scanned)

A common FinOps challenge is measuring the precise BigQuery execution cost (slots and bytes scanned) originating from specific data agents.

Because execute_tool spans are children of the root Chat span, we can write a recursive CTE to traverse the span tree. This accurately propagates the agent_id down to the execution telemetry, letting us group Data Warehouse metrics (like bq_slots_ms) by Agent and User:

WITH RECURSIVE
  FilteredSpans AS (
    SELECT
      trace_id,
      span_id,
      parent_span_id,
      name,
      JSON_VALUE(attributes, '$."gen_ai.agent.id"') AS agent_id,
      JSON_VALUE(attributes, '$."user.id"') AS user_id,
      SAFE_CAST(JSON_VALUE(attributes, '$."gcp.geminidataanalytics.bq_slots_ms"') AS INT64) AS bq_slots_ms,
      SAFE_CAST(JSON_VALUE(attributes, '$."gcp.geminidataanalytics.bq_bytes_processed"') AS INT64) AS bq_bytes_processed
    FROM `my-project.my_bq_logs._AllSpans` --this is an example
    WHERE start_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
  ),
  span_tree AS (
    -- 1. Base Case: Anchor on root Chat spans
    SELECT
      trace_id,
      span_id,
      span_id AS root_chat_span_id,
      agent_id,
      user_id,
      name,
      bq_slots_ms,
      bq_bytes_processed,
      1 AS depth
    FROM FilteredSpans
    WHERE
      name = 'google.cloud.geminidataanalytics.Chat'
      AND agent_id IS NOT NULL

    UNION ALL

    -- 2. Recursive Case: Traverse down child spans by matching parent_span_id
    SELECT
      child.trace_id,
      child.span_id,
      parent.root_chat_span_id,
      COALESCE(child.agent_id, parent.agent_id) AS agent_id,
      COALESCE(child.user_id, parent.user_id) AS user_id,
      child.name,
      child.bq_slots_ms,
      child.bq_bytes_processed,
      parent.depth + 1 AS depth
    FROM FilteredSpans AS child
    JOIN span_tree AS parent
      ON child.trace_id = parent.trace_id
      AND child.parent_span_id = parent.span_id
    WHERE
      child.name != 'google.cloud.geminidataanalytics.Chat'
      AND parent.depth < 20
  ),
  agent_tool_usage AS (
    SELECT
      agent_id,
      user_id,
      root_chat_span_id,
      bq_slots_ms,
      bq_bytes_processed
    FROM span_tree
    WHERE name LIKE 'execute_tool%'
  )
SELECT
  agent_id,
  user_id,
  COUNT(DISTINCT root_chat_span_id) AS total_questions,
  ROUND(COALESCE(SUM(bq_slots_ms), 0) / (1000.0 * 3600.0), 4) AS total_slot_hours,
  ROUND(COALESCE(SUM(bq_bytes_processed), 0) / POWER(1024, 4), 4) AS tb_processed
FROM agent_tool_usage
GROUP BY agent_id, user_id
ORDER BY total_slot_hours DESC, tb_processed DESC;

This recursive query unlocks clarity on how much BigQuery computing architecture is consumed by our Natural Language agents.

Enabling Managed Observability

Observability is an opt-in enterprise feature designed for governance and security. Platform administrators can activate observability across BigQuery and the Conversational Analytics API. For Looker, start here.

Topic		Replies	Views
Gain Full Visibility into AI Agents: BigQuery Agent Analytics Observability via Looker Technical Tips & Tricks lookml	0	141	June 30, 2026
Introducing Data Token Observability for Conversational Analytics in Looker Announcements gemini-in-looker , looker	2	400	August 19, 2026
Tracking Agent Telemetry and FinOps Metrics in BigQuery Data Analytics bigquery , looker , analytics-general	1	28	September 6, 2026