More in my A2A Deep Dive Series:
- The A2A 1.0 Milestone: Ensuring and Testing Backward Compatibility
- Running and Debugging A2A Agents in Gemini Enterprise
- Using A2A Extensions for Injecting Session State into ADK Agents exposed as A2A servers
When developers first started building agentic systems, the default motion was to build everything from scratch. If an application needed to interact with Salesforce, Jira, ServiceNow or Workday, the developer would simply build custom agents for those platforms themselves.
While it is technically feasible, doing so means you assume quite substantial maintenance tax. You are now responsible for tracking schema drift, managing API versioning, and rewriting and evaluating prompt logic every time a platform updates its capabilities. The maintenance tax of keeping those custom bridges from collapsing can many times consumes the entire project.
Therefore the future of AI architecture does not seem to be about writing every line of domain logic — it is rather about orchestration. Industry leaders are moving beyond providing APIs and are building official, domain-specific AI agents. As a result futre of AI Architecture is not about building isolated AI agents but rather reusing that virtual workforce of specialized AI agents, e.g by leveraging the Agent-to-Agent (A2A) protocol. In such an approach you offload the maintenance tax to the teams with the actual domain expertise, completely agnostic to how that specialist was built under the hood.
I am starting a series of articles where I will show examples of pioneers contributing to that global virtual workforce. We’ll start by looking at our first example: The BigQuery Data Engineering Agent, and show how your AI Agent can hire its expertise.
BigQuery
Before we look at how to orchestrate the BigQuery Data Engineering Agent, we have to understand the platform it operates on.
BigQuery is a serverless enterprise data warehouse that scales from zero to virtually any volume without the need manual infrastructure provisioning or sizing compute clusters. It is widely recognized for its ease of use and continually pushing the boundaries of highly scalable architecture through a wave of innovations like continuous queries which enable true real-time, event-driven analytics directly within the warehouse, global queries which empower teams to seamlessly join and analyze data across multiple geographic regions without the overhead of data movement, managed disaster recovery for cross-region storage and compute failover, alongside BigMetadata to efficiently manage metadata even when it itself becomes “Big Data” and BigLake to efficiently manage, secure, and integrate open-table formats like Apache Iceberg into your broader multicloud data lakehouse.
BigQuery didn’t just recently add AI as an afterthought — it was one of the very first platforms to democratize machine learning at enterprise scale.
By introducing BigQuery ML, it allowed data specialists to build, train, and deploy ML models directly using SQL, without ever moving the data outside the warehouse. These models could then be called as simple SQL functions within standard analysis. Their out-of-the-box forecasting models like ARIMA_PLUS, in particular, remain pure gold for enterprise analytics.
Today, BigQuery has evolved into a comprehensive AI engine capable of analyzing any modality — from structured tables to videos, audio, documents and images — at BigQuery scale. At to this native support for graphs and vector searches on billions of records with Google ScANN algorithm.
It features native integration with the Vertex AI Model Garden, giving developers direct access to Gemini, Anthropic models, and a host of open-source models offered via Model-as-a-Service (MaaS), alongside models hosted by customers on Vertex AI Endpoints. BigQuery even serves state-of-the-art models straight from Google DeepMind, such as Gemma embeddings, TimesFM and AI weather forecasting models like GraphCast and GenCast capable of predicting weather conditions and extreme events up to 15 days in advance.
BigQuery now hosts also an ecosystem of native agents out-of-the-box, including agents for conversational analytics, data preparation, data exploration and the focus of this blog post: data engineering.
Conversational Analytics
Dataprep
Its true differentiator here is the underlying Knowledge Catalog. Knowledge Catalog is a Gemini-powered data catalog that provides universal business context and governance for your entire data estate. By automatically extracting semantics from both structured and unstructured data, it builds a dynamic context graph. This graph is foundational to all generative AI use cases: it grounds AI agents in enterprise truth and drastically reduces hallucinations.
For those interested in building their own agents on BigQuery, it is worth mentioning that BigQuery is natively integrated with the Google Agent Development Kit (ADK) and offers a native Google-managed Model Context Protocol (MCP) server.
2. Data Engineering Agent
Data Engineering Agent makes it easy to build and maintain data pipelines in BigQuery. Users describe pipeline requirements in plain language, and data engineering agent generates the necessary SQLX code (an open-source extension of standard SQL injected with configuration and templating capabilities), adhering to data engineering best practices that you can customize through instructions. For example, a user might prompt: “Create a pipeline to load data from table A, standardize the date formats, remove duplicate entries, and load it into a BigQuery table named Y”.
When an existing pipeline needs updating, users simply tell the agent what to change. It analyzes the existing codebase, proposes modifications, and leaves the final review and approval to the user. The agent automatically retrieves resource metadata, business glossaries and data profiles from the Knowledge Catalog to ground generation in customer reality.
3. A2A interface
The Data Engineering Agent is exposed as an A2A Agent. Operating on A2A 0.3 spec, the agent connects to a specific BigQuery Dataform repository and workspace (analogous to a Git respository and branch for your codebase, respectively).
Let’s use first the A2A Python SDK and build an A2A client to test communication:
First, we need to understand exactly where our agent lives. The Data Engineering Agent is available on a specific REST endpoint tied to your Google Cloud environment:
AGENT_URL = f"https://geminidataanalytics.googleapis.com/v1/a2a/projects/{project}/locations/{location}/agents/dataengineeringagent"
Note: Replace {project} with your GCP Project ID and {location} with the GCP region of your Dataform repository.
Before we can talk to the agent, we need to authenticate. We do this by fetching a Google Cloud bearer token using Application Default Credentials (ADC):
def get_bearer_token() -> str | None:
"""Fetches a Google Cloud bearer token using Application Default Credentials."""
try:
credentials, project = default(
scopes=["https://www.googleapis.com/auth/cloud-platform"]
)
request = req()
credentials.refresh(request)
token = credentials.token
return token
except Exception as e:
print(f"Error getting credentials: {e}")
print(
"Please ensure you have authenticated with 'gcloud auth application-default login'."
)
return None
BEARER_TOKEN = get_bearer_token()
With our token ready, we can initialize the A2A connection. The A2A protocol relies on an “Agent Card” — a metadata document that describes the agent’s capabilities and supported communication transports. We use a A2A SDK resolver to fetch this card, configure our client to use JSON over HTTP, and establish the connection:
async with httpx.AsyncClient(
headers={
"Authorization": f"Bearer {BEARER_TOKEN}",
"Content-Type": "application/json",
},
timeout=60.0,
) as httpx_client:
resolver = A2ACardResolver(
httpx_client = httpx_client,
base_url = AGENT_URL,
agent_card_path = "v1/card"
)
card = await resolver.get_agent_card()
config = ClientConfig(
httpx_client=httpx_client,
supported_transports = [
TransportProtocol.http_json
],
streaming = False
)
client = await ClientFactory.connect(
agent=card,
client_config=config
)
We now have the A2A Client successfully declared.
Next, we need to configure our message payload. The Data Engineering Agent uses A2A extensions, mechanism which make A2A messages incredibly flexible. Extensions enable us (among other things) to append structured information alongside our main text prompts.
(If you want to understand more about A2A extensions, I highly recommend checking my other blog post focused strictly on this topic: “Using A2A Extensions for Injecting Session State into ADK Agents exposed as A2A servers”.)
Here is a list of A2A extensions we need to activate:
extensions = [
"https://geminidataanalytics.googleapis.com/a2a/extensions/messagelevel/v1",
"https://geminidataanalytics.googleapis.com/a2a/extensions/instruction/v1",
"https://geminidataanalytics.googleapis.com/a2a/extensions/gcpresource/v1",
"https://geminidataanalytics.googleapis.com/a2a/extensions/pipelinecontext/v1",
"https://geminidataanalytics.googleapis.com/a2a/extensions/conversationtoken/v1",
"https://geminidataanalytics.googleapis.com/a2a/extensions/finishreason/v1"
]
Next, we declare the actual metadata dictionary. This is where we pass the extra, structured information required by the activated A2A extensions. Because we activated these A2A extensions, the A2A Data Engineering Agent knows exactly how to parse this dictionary.
metadata = {}
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/conversationtoken/v1"] = ""
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/gcpresource/v1"] = {
"gcpResourceId": f"projects/{project}/locations/{location}/repositories/{repository}/workspaces/{workspace}"
}
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/instruction/v1"] = {
"agentInstructions": [{
"definition": "Always use uppercase for SQL keywords.",
"name": "style_guide.md"
}]
}
All we need to do now is assemble and send the final message. We construct a standard message payload containing our plain-text prompt, attach a unique ID to track the conversation context, and fire off the request along with our metadata and list of extensions we want to activate:
message_payload = {
'role': 'user',
'parts': [{'kind': 'text', 'text': USER_QUERY}],
'messageId': uuid4().hex,
'contextId': '959304c1-e814-4486-b9a7-efe885a6b066'
}
# Send raw dictionary
response_stream = client.send_message(
request = message_payload,
request_metadata = metadata,
extensions = extensions
)
Once the execution completes, if we go to BigQuery and open our Dataform workspace (the dev branch we targeted), we will see the newly generated SQLX objects automatically written to the repository.
4. Using A2A Data Engineering Agent from ADK
Now that we know the raw A2A communication works, we can elevate our architecture and orchestrate our expert directly from the Google Agent Development Kit (ADK)!
First, we need to define our Data Engineering Agent as a remote specialist within the ADK framework. We use the RemoteA2aAgent class, pointing it to our established agent endpoint.
data_engineering_specialist = RemoteA2aAgent(
name="data_engineer",
description = "This is data engineering agent that help with every data engineering task",
agent_card=(
f"{AGENT_URL}/v1/card"
),
use_legacy=False, ## If false, send request to the server including the extension indicating that the server should use the new implementation.
a2a_request_meta_provider = my_a2a_meta_provider, ## Callable[[InvocationContext, A2AMessage], dict[str, Any]] callable that takes InvocationContext and A2AMessage and returns a metadata object to attach to the A2A request
config = agent_config
)
# 2. Agent Composition
root_agent = LlmAgent(
model="gemini-2.5-flash",
name="unicorn_orchestrator",
instruction="Coordinate user requests and delegate data engineering tasks to the data engineering specialist.",
sub_agents=[data_engineering_specialist],
)
Notice the a2a_request_meta_provider argument—this is where we tell the ADK how to inject our required metadata into the outgoing A2A messages:
def my_a2a_meta_provider(context: InvocationContext, message: A2AMessage) -> Dict[str, str]:
"""
Extracts context and message details to build a metadata object
(typically a dictionary or list of tuples) for the A2A request.
Args:
context: The context of the current invocation (e.g., tracing, auth).
message: The actual app-to-app message being dispatched.
Returns:
A dictionary of metadata headers to attach to the outgoing request.
"""
# 1. Initialize your metadata object (usually a dict for headers)
metadata: Dict[str, str] = {}
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/conversationtoken/v1"] = "" ##"cmVxdWVzdGNvbnZlcnNhdGlvbnRva2Vu"
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/gcpresource/v1"] = {
"gcpResourceId": f"projects/{project}/locations/{location}/repositories/{repository}/workspaces/{workspace}"
}
metadata["https://geminidataanalytics.googleapis.com/a2a/extensions/instruction/v1"] = {
"agentInstructions": [{
"definition": "Always use uppercase for SQL keywords.",
"name": "style_guide.md"
}]
}
return metadata
This function receives the InvocationContext, giving us capabilities to inject dynamic values directly from the ADK session state. This is the perfect place to enforce enterprise coding standards via the agentInstructions extension.
With everything wired up, we can finally unleash our orchestrator. Let’s run our root agent and give it the following task:
“In project bigquery-public-data check dataset bitcoin_blockchain and for every object in that dataset generate a workflow named wf_{object_name} which counts the number of records in that table."
The root orchestrator passes the instructions and metadata to our A2A Data Engineering Agent. When the execution completes, navigating to BigQuery and checking the code in our targeted Dataform workspace reveals the new SQLX workflows generated automatically by our virtual specialist!
Summary
The era of monolithic AI agents — where developers had to custom-build, update, and maintain every single third-party integration — is giving way to a new paradigm. As we’ve explored with the BigQuery Data Engineering Agent, the future of AI architecture may be about orchestration of remote specialists exposed using the Agent-to-Agent (A2A) protocol.
By adopting the A2A protocol, we can stop writing custom agents for third-party platforms and start “hiring” official, domain-specific remote specialists. We showed how to connect one such specialist directly via the A2A Python SDK or compose multi-agent systems using the ADK. The example with the Data Engineering Agent demonstrates the use of A2A extensions, which can be used to enforce strict coding standards and guide these agents to perform complex pipeline generation grounded exactly in our environment metadata.
The BigQuery Data Engineering Agent is just the first example of the growing A2A ecosystem. As more platforms expose their native capabilities via A2A, your orchestrators will have access to highly specialized, global virtual workforce.
This article is authored by Lukasz Olejniczak — Customer Engineer at Google Cloud. The views expressed are those of the authors and don’t necessarily reflect those of Google.
Please clap for this article if you enjoyed reading it. For more about google cloud, data science, data engineering, and AI/ML follow me on LinkedIn