Smartsheet’s Remote MCP Server on AWS: A Blueprint for Token‑Efficient, Governed AI Agents

How Smartsheet built a remote MCP server on AWS

Smartsheet says its Model Context Protocol (MCP) implementation saved more than 3 billion tokens and drove over 87 percent week‑over‑week user growth in the first four weeks after general availability, according to their internal telemetry. Those are blunt signals: a protocol gateway with focused optimizations can materially change how AI agents interact with enterprise work data.

Executive summary

Smartsheet built a remote MCP server on AWS to give first‑party and third‑party AI agents structured, governed access to product data and capabilities. The design centralizes policy and tracing, cuts token use with progressive disclosure and token‑efficient serialization, runs continuous LLM‑in‑the‑loop canaries, and adds security controls like WAF, mTLS, and OAuth. The outcome: fewer tokens, faster adoption of agent workflows, and a repeatable blueprint for SaaS platforms exposing tools to agents.

Why a remote MCP server?

Agents need more than open endpoints. They need typed tools, observable tracing, and controls that stop hallucinated calls and runaway costs. A remote MCP server acts as a single gateway. It translates model intent into safe, schema‑validated tool calls, applies governance, and serializes responses in a token‑efficient way so models can do useful work without exhausting context windows or leaking data.

Architecture at a glance

High level: AI clients talk to the MCP server, which proxies requests to Smartsheet’s domain services and queries an intelligence layer for cross‑project insights.

  • AI clients → stateless MCP server containers (AWS Fargate on Amazon ECS).
  • MCP server calls domain services and an intelligence layer (Smartsheet cites Amazon Neptune for graph queries and Databricks for analytics/feature engineering).
  • Change‑event ingestion uses Amazon Kinesis Data Streams and Amazon Managed Service for Apache Flink, landing in Amazon S3 under a medallion‑style ingestion pattern.
  • LLM inference and agent runtimes integrate with Amazon Bedrock (Smartsheet notes Bedrock AgentCore as an evolving capability).

Edge protections include AWS WAF and AWS Shield behind an Application Load Balancer, OAuth validation and an OAuth proxy to reject unauthenticated calls, and mTLS plus private VPC subnets to protect backend services.

How Kinesis, Flink and SQS fit together

Smartsheet uses Kinesis and Flink as the canonical pipeline for change‑event ingestion into the intelligence layer (bronze→silver→gold/medallion). Structured analytics events used for offline validation and analytics go onto Amazon SQS for downstream processing. In short, Kinesis and Flink handle the primary change‑data flow and real‑time transforms. SQS carries analytics and telemetry events for offline analysis and resilience.

Scaling, deployment, and resilience

  • Stateless MCP containers run on ECS with Fargate and ECS Auto Scaling. Compute‑aware target tracking lets bursts of agent traffic scale out automatically.
  • Images live in Amazon ECR and CI/CD runs on GitLab CI, with runners hosted on AWS. Deployments use ECS deployment circuit breakers to roll back failing containers and roll out to the smallest region first.
  • Canary tests run every 15 minutes and execute multi‑step MCP workflows end to end, including the model in the loop, to detect regressions that only appear under agent conditions.
  • Observability uses OpenTelemetry traces, Kinesis to Amazon OpenSearch Service for logs and analytics, CloudWatch for infra metrics, Datadog for per‑tool APM, and PagerDuty for incident routing.

Governance, tool contracts, and safety

Smartsheet enforces governance across the MCP layer so agents can’t invent operations or escalate privileges:

  • Tools are published with strict, typed schemas generated from Pydantic models (JSON Schema). That constrains parameters and prevents hallucinated or malformed calls.
  • Tools carry annotations like readOnlyHint and destructiveHint, and access is tiered by organization and tool capability.
  • Per‑user metering tracks agent requests by identity header rather than IP, enabling session‑level quotas and layered rate limiting via AWS WAF rules and downstream enforcement.
  • Defense in depth includes edge WAF and Shield, an ALB with OAuth validation, mTLS for service‑to‑service authentication, private subnets, and domain service permissioning.

Smartsheet reports parity of experience by running the same MCP layer for both internal Smart Assist and external assistants (examples cited include Amazon Quick and Claude Desktop), while applying organization and tool‑level access controls to limit what each agent can do.

Observability and testing with nondeterministic models

LLMs are probabilistic and break classic unit testing. Smartsheet responds with continuous, LLM‑in‑the‑loop testing: end‑to‑end workflows in CI/CD and frequent production canaries that exercise the full MCP flow. To make that practical, teams typically combine:

  • Schema validation and strict tool contracts so outputs are syntactically checkable.
  • Golden‑path assertions for critical workflows and fuzz tests for edge behavior.
  • Canary strategies that limit cost and noise, for example running cheaper models for health checks, keeping contexts small, and using thresholding to avoid false positives when an LLM produces unusual but acceptable output.

Smartsheet’s canaries run every 15 minutes and include multi‑step MCP workflows with the model in the loop. Their write‑up does not publish exact cost controls or the low‑cost model(s) used for canaries, so teams should plan budget guardrails and failure handling for similar setups.

Token efficiency: where the math adds up

Token costs and context limits are practical levers. Smartsheet focused on three optimizations:

  • Progressive disclosure: return metadata and sampled rows first (fields like is_sampled, rows_in_sheet, rows_actual, filters_applied) so agents request more only when needed.
  • Schema‑driven tool contracts to reduce back‑and‑forth and miscalls.
  • Token‑efficient serialization: a proprietary format that avoids repeated key names and uses more token‑friendly delimiters to reduce structural overhead.

According to Smartsheet’s internal tests, proprietary serialization reduced response token counts by roughly 35-47 percent on data‑heavy responses. A representative 33‑item filtered query produced about 3, 900 tokens with the optimized serialization versus just over 6, 000 tokens for equivalent JSON, about a 35 percent reduction. The team also reports that JSON structural overhead typically consumed 15-25 percent of a response’s token count.

Smartsheet’s public write‑up does not specify which tokenizer or model family those measurements used; the numbers reflect response token counts observed in their internal telemetry and representative tests. If you plan to replicate this work, validate token savings against the specific model and tokenizer your agents use, because per‑token accounting varies by model family.

Security, identity, and the items still worth asking

Smartsheet describes a layered security posture: WAF and Shield at the edge, an ALB with OAuth validation and an OAuth proxy, mTLS for service authentication, and private subnets. Per‑user metering is based on an identity header rather than IP so rate limits can follow a user identity.

Several important operational and privacy details are not published and are worth raising with any vendor or partner:

  • Tenant isolation and logical separation: ask how the MCP enforces strict multi‑tenant isolation in practice and where cross‑tenant queries are prevented.
  • Embeddings and vector storage: Smartsheet references Neptune and Databricks for the intelligence layer but does not publish where embeddings live (S3, OpenSearch, a managed vector DB, or Bedrock). Verify embedding storage, access controls, and encryption at rest.
  • Identity binding: confirm the authoritative identity source (OAuth token/JWT claims, mTLS client certs) and how header spoofing is prevented. Identity must be cryptographically bound to requests.
  • Retention, PII handling and field‑level controls: ask for retention windows, redaction or field masking processes, and whether analytics events sent to the intelligence layer are pseudonymized.
  • External agent onboarding: request details on how third‑party agents receive credentials, for example mTLS certs, short‑lived tokens, or a brokered gateway, and what tooling exists to revoke or scope access quickly.

Early outcomes Smartsheet reports

  • Over 3 billion tokens saved (Smartsheet internal telemetry).
  • Over 87 percent week‑over‑week user growth during the first four weeks after GA (Smartsheet figure).
  • Proprietary serialization reductions of roughly 35-47 percent in internal tests on data‑heavy responses; an illustrative 33‑item query fell to ~3, 900 tokens from just over 6, 000 for JSON.

Practical checklist for teams building agent gateways

  • Run a central protocol gateway rather than letting agents call domain services directly. This centralizes policy, tracing, and serialization.
  • Publish strict, typed tool schemas (JSON Schema from Pydantic or equivalent) so models call only valid operations.
  • Design for progressive disclosure: return metadata and samples first, then fetch more on demand.
  • Test with the model in the loop. Use frequent canaries and CI workflows that exercise multi‑step agent flows, but control costs with cheaper models and short contexts.
  • Meter by cryptographic identity (tokens or certificates), not IP headers, and combine WAF rules with downstream quota enforcement.
  • Document where embeddings and analytics land, who can access them, and the retention policies applied.

Questions to ask your CTO / cloud team

  • How do we bind agent identity cryptographically?
    Use OAuth tokens or JWT claims, or mTLS client certificates as the authoritative identity. Avoid acceptance of unauthenticated headers, identity must be cryptographically verifiable.
  • Where will embeddings and vector indexes live?
    Confirm storage location (S3, OpenSearch, a managed vector DB) and ensure strict access controls, encryption at rest, and audit trails for any embedding store.
  • What’s our token‑cost budget for canaries and production canaries?
    Define model choices and budget limits for health checks. Consider running canaries on lower‑cost models or with reduced contexts to limit expense and false positives.
  • How do we enforce tenant isolation and field‑level PII controls?
    Ask for architecture diagrams and proofs: logical isolation, access‑control lists at the tool level, field masking, and retention windows for sensitive fields.
  • Which tools and operations are safe for external agents?
    Map feature parity versus access parity. Run the same MCP layer for internal and external agents, but scope tools and capabilities per organization and integration.

People and provenance

The MCP implementation was led by Vasil Kosturski (Principal Engineer, Smartsheet). AWS partners who contributed architecture guidance include Pyone Thant Win (Partner Solutions Architect) and Rony Blum (Senior Solutions Architect). Smartsheet cites MCP protocol references with dates: Elicitations (2025‑06‑18), Tasks (2025‑11‑25), and an MCP Apps blog post dated 2026‑01‑26. Galina Jordanowa supported marketing and GTM communications.

Final thought

Smartsheet’s pattern is straightforward and repeatable: centralize the agent gateway, make tool contracts explicit, optimize token use where it matters, and treat models as part of your testing surface. Those engineering bets buy lower operational costs and safer agent behavior, two things every enterprise needs before it lets an AI act on sensitive work data.

Key takeaways, questions you might be asking

  • How does the MCP server connect to Smartsheet’s data and insights?
    Smartsheet’s remote MCP server proxies agent calls to domain services and queries an intelligence layer (Neptune for graph relationships and Databricks for analytics), with change events ingested via Amazon Kinesis and Apache Flink into S3 in a medallion pattern.
  • What AWS services power the architecture?
    Core services include AWS Fargate with Amazon ECS for containers, Amazon Kinesis Data Streams and Managed Apache Flink for ingestion, Amazon S3 for storage, Amazon Bedrock for inference, and Amazon Neptune for graph intelligence. Operations use Amazon ECR, GitLab CI, Amazon OpenSearch Service, CloudWatch, Datadog, and PagerDuty.
  • How much token savings did Smartsheet report?
    Smartsheet reports over 3 billion tokens saved in internal telemetry. Proprietary serialization reduced response token counts by about 35-47 percent in internal tests. A representative 33‑item query fell to ~3, 900 tokens from just over 6, 000 for JSON.
  • How do they test systems that include LLMs given nondeterminism?
    They run LLM‑in‑the‑loop end‑to‑end tests in CI/CD and as production canaries every 15 minutes, and use agent‑first tracing to correlate tool call chains. Teams should combine schema checks, golden‑path assertions, and cost‑aware canary strategies.
  • What security patterns protect against bad agent behavior?
    A layered approach: AWS WAF and Shield at the edge, ALB with OAuth validation, mTLS for service authentication, private subnets in a VPC, domain service permissioning, and per‑user metering that tracks identity headers bound to cryptographic credentials.
0