Multi-account AI agents: orchestrating cross-LOB tool calls with AgentCore Gateway and MCP

Build a multi-account AI agent that reasons across Line‑of‑Business accounts

Imagine a single agent that answers a query requiring a bank balance from account A, a credit score from account B, and policy text from account C, without copying raw datasets into a central store. This reference pattern does that: each Line‑of‑Business (LOB) exposes narrowly scoped tools and retrieval endpoints, and a central integration layer orchestrates them.

Why enterprises care

Large organizations keep LOB accounts separate for ownership, compliance, and billing. Copying data into a central warehouse creates duplication, a bigger audit surface, and governance headaches. The alternative is to leave data where it lives, expose only the specific tool outputs needed at query time, and orchestrate those tools from a platform control plane.

Architecture at a glance

• Three-layer model, a platform account for the control plane and model governance, LOB accounts that host data and MCP servers, and an integration layer (AgentCore Gateway) that federates tool discovery and invocation.
• Platform account, in the reference deployment the platform runs the agent control plane (AgentCore Runtime) and hosts LLM inference via Amazon Bedrock. That centralizes model selection, guardrails, and inference billing.
• LOB accounts, each LOB packages domain operations and retrieval as MCP servers (examples: get_balance, get_credit_score, search_lending_policies) and keeps source assets, DynamoDB, S3 PDFs, vector indexes, inside its own account.
• AgentCore Gateway, presented as a single MCP endpoint to the agent; it registers LOB MCP targets, provides unified tool discovery and semantic search, centralizes authentication/authorization, and surfaces observability and audit logs.

Where inference runs, the tradeoffs (important)

The sample deployment places LLM inference in the platform account. That simplifies model governance but creates a central trust boundary: any tool outputs that are included in prompts or context become visible to the model provider and to the platform account’s runtime. The authors summarize the data-minimization goal like this:

“Only the specific data a request needs flows out at query time, so the underlying datasets do not leave their owning account.”, Senthil Kamala Rathinam, Karthik Tharmarajan, and Shashi Makkapati

That statement describes the intended pattern, but it is not an absolute guarantee. What actually leaves an LOB account depends on tool design, serialization, prompt construction, and logging. Concrete mitigations you should deploy if inference is central:

• Return structured, minimal tool results (typed schemas, enumerated fields) rather than full documents. Enforce maximum result size.
• Redact or pseudonymize PII at the MCP server before returning results. Prefer deterministic schemas so downstream code can validate fields automatically.
• Filter prompts before sending them to the model: strip chain-of-thought, remove raw identifiers, and enforce strict template-based prompt building.
• Shorten token lifetimes for any exchanged tokens and prevent long-lived memory of retrieved content by avoiding persistent prompt histories or enabling ephemeral contexts where available.
• Consider colocating inference with the data (run Bedrock or an LLM runner in the LOB account or use an encrypted enclave) when regulatory or contractual constraints forbid central inference.
• Audit and redact model logs; keep access to model transcripts tightly controlled and retain only what’s necessary for compliance.

If your compliance posture requires that model providers never see certain classes of data, treat the platform-hosted inference option as a convenience for low-sensitivity flows and use LOB-local inference for sensitive transactions.

MCP and the Gateway: the integration surface

The reference uses the Model Context Protocol (MCP) as the tool interface: each LOB exposes tools and retrieval endpoints as MCP servers, and the AgentCore Gateway aggregates those targets so the agent can discover and call them through a single MCP API. The sample shows MCP over Streamable HTTP and lists an example supportedVersions array (for example: [“2025-03-26”]). Treat version strings in examples as author-provided and verify supportedVersions in your environment.

Key responsibilities attributed to the Gateway in the reference include:

• Registering and routing to LOB MCP servers and providing unified tool discovery (semantic search).
• Centralizing authentication and retrieving credentials for outbound calls via AgentCore Identity (the reference demonstrates OAuth 2.0 client credentials M2M flows).
• Evaluating fine-grained authorization policies (Cedar/Policy in AgentCore) before tool invocation and producing observability logs for audit.

Note: many of these product-specific capabilities are described in the reference material. Verify exact feature availability, configuration and CLI/SDK behaviors against official product documentation before production deployment.

Authentication and authorization patterns

The recommended flows in the reference map to standard web identity patterns:

• Inbound user auth, users authenticate with an OIDC provider (example: Okta) and receive a JWT containing claims (sub, groups, aud). A Gateway inbound authorizer validates issuer, audience (the sample shows an allowedAudience example of [“lobfederation”]), signature and expiry.
• Gateway outbound auth, the Gateway uses OAuth 2.0 client credentials (machine-to-machine) to obtain tokens for calling LOB MCP servers. The sample registers credential providers in AgentCore Identity (example credential provider name: “lobfederation-okta-m2m” and an example scope: “lobfederation.invoke”).
• On‑behalf‑of (OBO), an on‑behalf‑of token exchange preserves user identity downstream for per‑user auditing and enforcement. The reference notes AgentCore Identity can support OBO, but the sample uses M2M because the example Okta developer account did not. Confirm OBO support with your IdP and AgentCore documentation if you require downstream per‑user auditability.
• Policy enforcement, fine‑grained authorization policies (expressed in Cedar/Policy in AgentCore in the reference) should be evaluated at the Gateway in enforcement mode to block disallowed tool calls; examples in the reference include comments like // Permit all authenticated users to invoke read-only tools and // Block destructive operations regardless of user.

Choosing M2M vs OBO: a quick decision guide

• Choose OBO when downstream calls must carry the specific user identity for compliance, audit trails, or legal accountability (for example, a user-initiated financial transaction).
• Choose M2M when calls are read-only or generic service calls, when IdP constraints prevent OBO, or when you prioritize lower latency and simpler token handling.
• Operational tradeoffs, OBO adds token-exchange latency, complexity, and IdP requirements, and M2M scales better but centralizes auditability in the platform account unless you propagate user context in application-level logs.

Operational checklist and security hardening

Consolidate these actions into your PoC and pre-production checklist:

• Network: prefer VPC/ENI or interface VPC endpoints via AWS PrivateLink over public endpoints. Use allowedWorkloadConfiguration to limit which workloads can invoke runtimes.
• Observability: capture Gateway data-plane logs in Amazon CloudWatch Logs and enable AWS CloudTrail data events (advanced event selector) to record per-call tool invocations. The reference notes OpenTelemetry tracing support, verify and enable as needed.
• Auth basics: validate JWT signature, issuer, audience and expiry. Enforce minimum claims. Issue narrow scopes (e.g., lobfederation.invoke) and short token lifetimes for high-sensitivity flows.
• Secrets & rotation: rotate client secrets or certificates automatically (recommend weekly/30-day rotation cadence depending on sensitivity) and prefer short-lived tokens or rotating credentials where supported.
• Least privilege: limit which Gateway identities can call each LOB target and require explicit Cedar policies for any write/destructive tools.
• Resilience: implement retries with exponential backoff, circuit breakers, and graceful degradation (cached or simplified answers) when LOB MCP servers are unavailable.
• Logging & retention: redact PII from logs, keep an auditable hash of returned results, and align retention with regulatory needs, note that enabling CloudTrail data events increases logging volume and costs.

Threats to watch and mitigations

• Token replay or theft, require jti/nonce claims, short token lifetimes (e.g., 5-15 minutes for sensitive flows), and immediate revocation paths. Monitor abnormal token usage.
• Compromised Gateway, isolate management access, use hardware-backed keys where available, separate management and data plane, and ensure LOBs can revoke Gateway credentials quickly.
• MCP server impersonation, use mutual TLS (mTLS) or signed JWTs between Gateway and LOB MCP servers and validate issuer and audience on each side.
• Model-data leakage, prevent chain-of-thought logging, filter prompts, and enforce minimal, structured tool outputs. Consider LOB-local inference for highly sensitive data.
• Supply-chain/model poisoning, validate and monitor training/knowledge-base updates, and gate model updates with offline evaluations tied to business KPIs.

Proof of Concept: measurable goals and tests

Run a focused PoC to validate the pattern before rolling it into production. Concrete success criteria:

• Latency: for interactive scenarios target 95th-percentile round-trip latency under 500-800 ms for simple tool calls; instrument and capture p95 and p99.
• Correctness: sample 1, 000 agent sessions and keep incorrect tool routing under 1%; record end-to-end accuracy against ground truth.
• Security: confirm no raw source documents are persisted in platform logs for a representative sample of queries; confirm redaction and prompt-filtering rules work as expected.
• Cost: estimate cost per 1, 000 requests including Gateway, inference, and LOB compute; set a bound and iterate on caching/replication if cost is too high.
• A/B test plan: split live traffic (e.g., 90/10, then 75/25) to compare a baseline routing strategy with an optimized routing strategy (different retrieval sizes, different tool descriptions). Evaluate business KPIs such as resolution rate or time-to-task completion.

What to verify with vendors and internal teams

Before production, confirm these product and operational details:

• Official AgentCore and Amazon Bedrock documentation for Runtime, Gateway, Identity, Policy (Cedar), Evaluations and Optimization, feature parity and any known limits.
• MCP (Model Context Protocol) specification and supportedVersions for your runtime and Gateway implementation.
• IdP capabilities: whether your Okta (or other IdP) tenant supports On‑Behalf‑Of token exchange and the exact claim mappings you need.
• Service quotas and throughput limits for the Gateway, token issuance, and LLM inference; plan autoscaling and circuit breakers accordingly.
• Repository artifacts and deployment scripts used in your PoC (the reference includes a sample GitHub repo you can start from, verify its contents and clean-up scripts before running).

Resources

• Sample repo referenced in the deployment notes: https://github.com/aws-samples/sample-amazon-bedrock-agentcore-banking-mcp-multi-account
• Authors of the reference material: Senthil Kamala Rathinam, Karthik Tharmarajan, and Shashi Makkapati.

Key takeaways, quick Q&A

• How can an agent operate across multiple AWS LOB accounts without centralizing datasets?

  Expose narrowly scoped tools and retrieval endpoints as MCP servers in each LOB, and use a central AgentCore Gateway to discover, authorize, and forward calls. Only minimal tool results (not raw datasets) are returned at query time, subject to your tool and prompt design.

• Where should inference run: central platform or LOBs?

  Both are viable. The reference places inference in the platform account for centralized model control, but that creates a trust boundary where tool outputs are included in prompts. If you cannot tolerate that exposure, colocate inference in LOB accounts or use encrypted enclaves/local runners.

• When should you use M2M (client credentials) vs OBO (on‑behalf‑of) token flows?

  Use OBO when downstream calls must carry user identity for compliance and per‑user auditability. Use M2M for read-only or generic service calls, or when IdP limitations make OBO impractical; M2M is simpler and typically scales better.

• What are the essential security controls to enable before any pilot with production data?

  Enforce strict JWT validation, use short token lifetimes and rotating credentials, enable PrivateLink/VPC connectivity, require mTLS or signed requests between Gateway and MCP servers, enable CloudTrail data events for auditing, and apply prompt‑level redaction and result schemas to minimize leakage risk.

• What should a short PoC measure?

  Measure latency (p95/p99), correctness of tool routing, evidence of no raw data exfiltration in sampled logs, and cost per 1, 000 requests. Add an A/B experiment and validate your authorization model (M2M vs OBO) and revocation procedures.

The pattern connects distributed services with a single agent control plane while preserving LOB ownership, if you treat the platform inference boundary as a deliberate, audited trust zone. Verify product features and quotas with vendor docs, run a focused PoC with measurable SLOs, and enforce a minimum security baseline (mTLS, short-lived tokens, CloudTrail data events) before scaling to production.