Supercharge Your AI Agents: Applying Enterprise Governance to GenAI

Google Cloud Apigee Knowledge Hub
, ,
1

Generative AI agents are moving from experiment to enterprise. As developers build powerful, autonomous agents using frameworks like the open-source Agent Development Kit (ADK), a new challenge emerges: How do you manage, secure, and scale them in production?

These agents make calls to Large Language Models (LLMs), but this traffic is often a “black box,” leaving you with questions:

  • How do you control runaway costs from token consumption?

  • What happens if your primary LLM provider has an outage?

  • How do you enforce security and compliance on the agent’s requests?

  • Who is consuming these services and how?

The answer is to apply the same proven patterns we use for all enterprise services: API management.

The New Pattern: Apigee as the AI Agent Gateway

We are excited to introduce a powerful new pattern that places Apigee in the critical path of an agent’s LLM calls.

image|733.0689075630253x413.0902527075812
image|733.0689075630253x413.0902527075812960×540 44.2 KB

Instead of an agent calling an LLM directly, it calls an Apigee proxy. Apigee then securely manages the “southbound” call to the LLM backend (like Gemini, Claude, or any other). Apigee is leveraged as the unified gateway to secure, manage, and scale the entire Generative AI workflow. This includes acting as the AI Gateway layer for all LLM interactions and providing the core API Gateway functionality for the tools and enterprise APIs utilized by the agents.

To make this simple, a new first-party Apigee wrapper for the ADK is released in v1.18.0. This will allow developers to easily configure any ADK-built agent to route its LLM calls through Apigee, instantly unlocking a suite of production-ready capabilities. (Note: The initial release will support Gemini and Vertex endpoints for generateContent calls only).

In your adk code, simply instantiate the ApigeeLlm wrapper and pass it to your Agent Development Kit’s (ADK) LlmAgent. For more info, check out the adk docs.

from google.adk.agents import LlmAgent
from google.adk.models.apigee_llm import ApigeeLlm

# Instantiate the ApigeeLlm wrapper
model = ApigeeLlm(
    # Specify the Apigee route to your model. For more info, check out the ApigeeLlm documentation.
    model="apigee/gemini-2.5-flash", 
    # The proxy URL of your deployed Apigee proxy including the base path
    proxy_url=f"https://{APIGEE_PROXY_URL}", 
    # Pass necessary authentication/authorization headers (like an API key)
    custom_headers={"foo": "bar"}
)

# Pass the configured model wrapper to your LlmAgent
agent = LlmAgent(
    model=model,
    name="my_governed_agent",
    instruction="You are a helpful assistant powered by Gemini and governed by Apigee.",
    # ... other agent parameters
)

What You Get

By routing agent traffic through Apigee, you immediately gain enterprise-grade governance without modifying your agent’s core logic:

  1. Dynamic Circuit Breaking & Reliability: This is the coolest part. Imagine your agent makes a call to Claude, but it fails or returns a token error. With an Apigee policy, you can instantly catch that specific error and automatically retry the request against a different model, like Gemini. The agent and end-user never see the failure; they just get a successful response.

  2. Cost Control & Quotas: Prevent runaway agent costs by applying rate-limiting and quota policies to token consumption—just like any other API.

  3. Performance & Caching: Use policies like semantic caching to store and serve common agent responses, dramatically reducing LLM calls, latency, and cost.

  4. Security & Data Masking: Integrate with solutions like Model Armor to sanitize the user prompts sent to and responses received from large language models (LLM) and Sensitive Data Protection (SDP) or Data Loss Prevention (DLP) to automatically inspect and mask sensitive data (like PII) in both the agent’s prompts and the LLM’s responses.

  5. AI Productization & Governance: This pattern turns your “agent” into a manageable “AI product.” You can now bundle its capabilities, apply advanced security (access control, service tiers, quotas, etc), track fine-grained analytics, and even publish it on a developer portal.

This governance story extends beyond the agent’s LLM calls to the tools it uses. A common challenge is that agents can’t use legacy APIs because their specifications lack richness or don’t exist at all. This is where SpecBoost, another new AI-powered capability, comes in. SpecBoost enhances your existing OpenAPI specs—or generates new ones from scratch—to make all your enterprise APIs fully discoverable and ready for agent consumption.

This pattern represents a massive step forward in bringing the reliability, security, and governance of Apigee to the new world of generative AI.

Keep an eye on our channels for the official release! For When it lands, you can find examples in the apigee-samples and adk-samples repositories.

Special mention to Geir Sjurseth and Seyed Ahmad for collaborating this article with me.

:thought_balloon: Tailor your solution with help from a Google Cloud Sales Specialist

4 Likes