Designing secure enterprise Agentic AI platform: where should the trust boundaries sit?

[aman_swarnkar]

As AI agents move from isolated experiments into enterprise workflows, the architecture around the agent becomes as important as the agent itself.

Once an agent can invoke tools, access business data, call APIs, or act on behalf of users, I think the trust model needs to answer a few questions explicitly:

• Who initiated the interaction?
• Which agent is executing the action?
• Whose authority is being used for the downstream operation?
• Where should policy be enforced before a tool or service is accessed?
• How do we keep these interactions observable and auditable?

My current design position is to keep user identity, Agent Identity, delegated authority, tool access, and runtime policy enforcement as separate concerns rather than allowing them to collapse into a single trust domain.

I put together a generic Google Cloud reference architecture exploring these boundaries using publicly documented capabilities such as Agent Identity, Agent Gateway, Agent Registry, private connectivity, Model Armor, and observability.

I’m interested in how others are approaching this: where do you draw the most important trust boundary in an enterprise agent platform?

Full architecture and write-up:

(URL Removed by Staff)

[alexanderleo]

I think the clearest boundary is when an agent’s proposed action turns into a real tool call. That’s the moment it moves from suggestion to execution. Keeping user identity separate from delegated authority is important. Just because someone asks an agent for help doesn’t mean they want the agent to do everything it’s capable of.

If permissions get revoked in the middle of a workflow the system should stop the agent from making any tool calls. It should not proceed with actions that require authority the user no longer grants. The agent should be designed to check permissions at each step—before making a call—. Halt immediately if access is pulled. This way the user stays in control even if their permission changes, during a task.

[aman_swarnkar]

I agree, the transition from reasoning to an actual tool call is a critical enforcement point.
I would also evaluate authorization as close to execution as possible, particularly for privileged operations because both user permissions and policy state can change during a longer-running workflow.

This is also why I prefer keeping Agent Identity and delegated user authority separate. At execution time, the system should be able to answer both: which agent is attempting the action and is the user’s delegated authority still valid for that action?

The mid-workflow revocation case you mentioned is an important one because it shows why authorization cannot simply be treated as something established once at the beginning of an agent session.

[devin_davis1]

Great framing—especially the distinction between user identity, agent identity, delegated authority, tool access, and runtime policy enforcement. In enterprise agent systems, collapsing those into one “trusted agent” boundary is where many of the hardest failures begin.

I would draw the most important trust boundary at the point where an agent’s reasoning becomes an external effect.

An LLM or agent should be able to interpret a request, retrieve context, plan, and propose an action. But it should not be the ultimate authority for whether that action is permitted. Tool invocation, data access, API calls, workflow changes, and user-impacting outputs should cross a separate policy-enforcement layer that evaluates the request using more than the model’s intent.

Conceptually:

allow=f(initiating user,agent identity,delegated scope,requested tool,target resource,data sensitivity,runtime context,risk level)allow=f(initiating user,agent identity,delegated scope,requested tool,target resource,data sensitivity,runtime context,risk level)

A few design principles I have found important:

• Treat agent output as untrusted intent. A tool call proposed by the model is not an authorization decision. It should be schema-validated, policy-checked, and constrained before execution.

• Use separate identities. The user should remain the initiating principal; the agent should have its own workload identity; and downstream access should use a short-lived, narrowly scoped delegation rather than broad, standing credentials.

• Put the gateway in front of effectful tools. The agent gateway should mediate tool/API access, validate parameters, enforce allowlists and resource constraints, apply rate/cost limits, and produce a durable audit event for every attempted action—not only successful ones.

• Make delegation explicit and attenuating. An agent should never gain more authority than the initiating user, and ideally receives less: a capability limited to a specific action, target resource, time window, and budget.

• Separate planning from execution. It is safer for an agent to produce a structured action proposal—such as a typed tool-call manifest—than to hold direct access to production APIs, databases, deployment systems, or external communications.

• Require human approval for irreversible or high-impact actions. Examples include production deployments, financial transactions, external publication, permission changes, deletion, bulk data exports, or actions that exceed a budget/risk threshold.

• Make observability part of the authorization design. The audit record should link user request, agent/version, retrieved context, policy evaluation, delegated credential, tool request, response, downstream resource, and final outcome. Otherwise post-incident reconstruction becomes guesswork.

One subtle risk is that retrieval context and tool outputs can become indirect instruction channels. So I would also treat documents, webpages, tickets, and API responses as untrusted data by default: they can inform the agent, but they should not be able to expand permissions, override policy, or trigger actions by themselves.

A useful mental model is:

text

User request
→ authenticated session
→ agent planning / retrieval
→ structured action proposal
→ policy and authorization gateway
→ short-lived scoped delegation
→ tool or service execution
→ immutable audit and observability record

For research and engineering workflows, this distinction is particularly valuable. An agent may propose code, an experiment, a cloud job, or a publication artifact—but separate gates should decide whether it can modify a baseline, spend compute, submit a hardware job, access restricted data, or publish externally.

The key architectural principle for me is:

The agent can recommend an action; an independently governed policy layer decides whether the action may occur.

That separation makes the system easier to reason about, easier to audit, and much safer to scale across teams and tool ecosystems.

[aman_swarnkar]

I agree with this. I like the way you are defining the boundary where the agent’s reasoning actually becomes an external action.

One thing I would add is that having permission to use a tool should not automatically mean the agent is allowed to perform every action through that tool.

We still need to validate the actual request - what action is being performed, on which resource, with what parameters, and under whose authority.

So for me the flow would be something like:

Plan → action proposal → validation → authorization/policy check → execution → audit

I also agree on treating retrieved content and tool responses as untrusted input. They can influence the agent’s reasoning, but they should not be able to increase the agent’s permissions or bypass the policy layer.

One interesting question here is for long-running workflows: should we re-check authorisation before every effectful tool call instead of validating it only once at the beginning of the workflow?