Amazon Bedrock AgentCore for appointment scheduling: Natera’s HIPAA‑safe, low‑latency voice agent

Natera’s intelligent appointment scheduling with Amazon Bedrock AgentCore

Shorter silence. Fewer hang-ups. Better outcomes for patients who already have enough on their plates.

“Booking a phlebotomy appointment shouldn’t be a hassle for oncology patients already managing treatment.”

Here’s what Natera built, how they reduced perceived wait times, and the measurable impact of their engineering choices.

The problem they solved

Natera needed a voice agent to replace manual scheduling and a third‑party voice provider for mobile phlebotomy appointments. The constraints were clear: telephony flexibility, strict privacy controls for protected health information (PHI), and a production-quality system that could hide backend latency without sounding robotic or misleading callers.

What they built, core architecture and patterns

Dual‑WebSocket bridge: one socket to the telephony provider (Twilio) and another to the real‑time model/voice stack. Separating streaming from inference lets each side be upgraded or swapped independently.
Event‑driven latency masking (filler loop): when a backend tool call takes time, a low‑latency model returns short contextual acknowledgements so callers don’t hear dead air. Fillers are generated in parallel with tool calls and tuned from measured tool latencies.
Progressive trust model: conversations begin as unauthenticated sessions keyed by an actor ID (the system uses a SHA‑256 hash of the caller’s phone number as the unauthenticated actor ID). After verification the session migrates to an authenticated actor ID and gains access to personal‑data‑processing tools and memory.
AgentCore runtime and memory: migrating from Amazon ECS to Amazon Bedrock AgentCore removed container management overhead. AgentCore memory persists session state (backed by DynamoDB) so conversations survive short interruptions and context is available to agents.
RAG (retrieval‑augmented generation): Bedrock Knowledge Bases + Amazon Titan Text Embeddings V2 over documents stored in S3 provide grounded answers to patient questions.
Event pipeline: cross‑channel events stream through Amazon MSK and are processed by AWS Lambda, which writes activity summaries into AgentCore memory for a live view of recent patient actions.
Guardrails and compliance: the deployment uses Amazon Bedrock Guardrails and runs on HIPAA‑eligible AWS services under a Business Associate Agreement (BAA), with additional controls that gate access to PHI until verification completes.

Latency masking: why small responses matter

What patients experience, the time from their last utterance to the first meaningful agent audio, is the most important latency metric. Natera reports this “perceived latency” as the primary UX measure and tuned the system around it.

Observability traces (AgentCore traces exported to Amazon CloudWatch) revealed that roughly 70% of perceived latency came from a single external vendor API (unnamed in public reporting). To avoid leaving callers in silence while that API answered, Natera runs a filler loop. A fast, low‑latency model creates brief, contextual lines that acknowledge the wait without promising outcomes.

Calibration was pragmatic and measurement driven. They measured P50 (median) latencies per tool and set the filler trigger to P50 minus one second. Example P50s cited include authentication APIs averaging 2.5 seconds and scheduling APIs averaging 4 seconds, producing typical filler triggers near 1.5 seconds into a tool call.

System prompt (filler generation template):

“You are a scheduling assistant. The system is currently executing [TOOL_NAME]. The patient last said: “[LAST_UTTERANCE]” Current workflow step: [STEP_NAME] Generate exactly one short sentence (under 15 words) that:, Acknowledges the brief wait naturally, Does NOT promise a specific outcome, Matches the context of the current step”

They used a fast, low‑latency model for fillers (the source cites models such as “Claude Haiku” as examples) while heavier Bedrock model inference and tool calls completed in the background.

Validation, early production and what the numbers mean

Validation: Natera ran 500 scripted end‑to‑end call simulations and reported 100% tool‑calling accuracy for those scenarios. This means the system correctly routed and invoked backend tools with the expected parameters in all 500 scripted tests. It does not imply exhaustive coverage of every possible edge case in real‑world traffic.
Perceived latency: median time to first meaningful response was 6.8 seconds. The Bedrock‑only portion, reported as the time spent in Bedrock model inference and related Bedrock processing, was 6.2 seconds.
Cost: Natera reported per‑completed‑call cost of under USD 0.01, measured as the total cost for a full end‑to‑end patient interaction in their validation and early production window. A detailed component‑level breakdown (model inference, telephony minutes, third‑party API charges, and fixed engineering/monitoring overhead) was not published.
Development timeline: four months of iterative development and testing preceded validation.
Early production (4‑week window): 4, 744 calls handled, a 5.5% increase in call volume versus the prior system. Short‑call rate (under 30 seconds) fell from 22% to 12%. Verification/completion rate rose to 66% from 64%. Average duration for resolved interactions increased from 79 seconds to 101 seconds, which indicates callers were staying to complete tasks rather than dropping off. NPS survey response rate rose from 1.09% to 1.60%, a 47% relative increase in responses.
RAG testing: an internal 200‑query test set achieved over 90% response accuracy for general patient inquiries in their evaluation framework. The source describes this as internal testing rather than an external benchmark.

How measurements were defined. Perceived latency is the time from a patient’s last utterance to the first meaningful agent audio. Validation used scripted scenarios. Production comparisons use a four‑week window baseline. Observability relied on AgentCore traces and CloudWatch to break down per‑component latencies (for example, per‑tool P50s).

Migration pain points and observability wins

Moving from Amazon ECS to AgentCore required refactoring container‑specific code, adapting to the AgentCore microVM invocation model, externalizing session state (actor‑keyed) into AgentCore memory, and implementing WebSocket connection pooling for telephony concurrency. These are non‑trivial engineering efforts. The team externalized state to enable session continuity and to let AgentCore microVMs remain stateless for scale.

Observability paid for itself. Tracing identified a single external vendor API responsible for most perceived latency. That finding directly informed filler timing, tool timeouts, and remediation conversations with vendor partners.

Safety, privacy and governance: what they did and what to watch

Infrastructure and contracts: the system runs on HIPAA‑eligible AWS services under a Business Associate Agreement (BAA).
Memory gating: AgentCore memory access is gated by the progressive trust model so unauthenticated sessions cannot invoke patient‑identifying tools.
Guardrails: Amazon Bedrock Guardrails are used to reduce unsafe disclosures and enforce runtime output constraints.

One important privacy nuance: the unauthenticated actor ID is reported as the SHA‑256 hash of the caller’s phone number. Hashing a low‑entropy value like a phone number without a secret salt or HMAC is not strong anonymization. Teams should treat that construct as an identifier rather than a privacy boundary and use keyed hashing or tokenization if they need resistance to brute‑force reversal.

Open questions and limits to the public reporting

Which specific vendor API caused the bulk of the perceived latency? The public report does not name the vendor. Teams should instrument end‑to‑end traces across all third‑party integrations to detect similar bottlenecks.
Which foundation model(s) drive core reasoning in production? The report references Titan embeddings for RAG and cites fast models for fillers, but exact production LLM names and versions for decisioning were not disclosed.
Detailed cost breakdowns (inference vs telephony vs third‑party API) and fixed OPEX/engineering amortization were not published. Expect per‑call costs to vary by model choice and scale.
Operational edge cases like long‑lived WebSocket reconnections, multi‑call concurrency pressure, and failover strategies are described at a high level. Teams should test resilience under realistic fault scenarios.
Hallucination detection and continuous RAG monitoring are acknowledged, but the precise algorithms, thresholds, and escalation rules were not detailed publicly.

Practical advice for product and engineering leaders

Measure perceived latency, not just model latency. Define time_to_first_meaningful_audio and instrument traces that map telephony events to model and tool calls so you can tune fillers against real P50s.
Separate streaming and inference. A dual‑WebSocket bridge or equivalent isolation keeps telephony resilient while you iterate on model providers or RAG pipelines.
Implement progressive trust. Start with a low‑trust session (actor ID) and only unlock PHI‑handling tools after verification to reduce risk and simplify compliance audits.
Calibrate fillers from real measurements. Use per‑tool P50 latencies and conservative offsets (the Natera approach was P50 minus one second) so fillers neither interrupt nor overpromise.
Use RAG with guardrails and human oversight. Ground answers with company docs and embeddings, validate on representative test sets, and set up human escalation thresholds for safety‑critical responses.
Instrument vendor SLAs. Observability lets you identify slow third parties early and implement fallbacks or remediation before callers experience issues.

Next practical steps to consider

Break down per‑call costs by component (model inference, telephony, third‑party APIs, storage, orchestration) to understand economics as you scale.
Enhance fillers with brief options for callers when latency exceeds a higher threshold (for example, “Would you like a callback instead?”) so long timeouts degrade gracefully.
Build developer self‑service with templates, test harnesses, and governance so product teams can safely iterate without ballooning risk.
Set up continuous RAG validation and hallucination monitoring tied to human review for new or changed knowledge base documents.

Key takeaways, quick questions you might ask

Did the system consistently call the right backend tools?

Yes for the 500 scripted end‑to‑end simulations Natera ran, the team reported 100% tool‑calling accuracy across those scenarios. That result reflects scripted coverage and should be complemented with ongoing production monitoring for edge cases.
How long did callers wait for a meaningful response?

Median perceived latency (time from caller speech to first meaningful agent audio) was 6.8 seconds; the Bedrock‑only portion (reported by the team as the time spent in Bedrock model inference and processing) was 6.2 seconds.
Was the solution cost‑effective?

Natera reported per‑completed‑call cost under USD 0.01 for their validation and early production windows. A public, component‑level cost breakdown was not provided, so expect variation as you change models, telephony, or scale.
Did caller engagement improve in production?

Yes. In a four‑week early production window the system handled 4, 744 calls (a 5.5% increase over the prior system), reduced short calls (under 30s) from 22% to 12%, and increased NPS survey response rate from 1.09% to 1.60% (a 47% relative increase in responses).
How did they prevent sensitive data leaks during calls?

They gate access with a progressive trust model (unauthenticated sessions keyed by a SHA‑256 hash of the caller’s phone number, then migrated to authenticated actor IDs post‑verification), run on HIPAA‑eligible AWS services under a BAA, and use Bedrock Guardrails. Note: hashing a phone number alone is not strong anonymization; keyed hashing or tokenization is recommended for stronger privacy guarantees.

Natera’s work shows pragmatic patterns for turning an ML proof‑of‑concept into a usable, patient‑facing system: instrument aggressively, separate media streaming from inference, use short contextual fillers to hide latency, and gate PHI with progressive trust. Those principles translate beyond healthcare to any business workflow where silence costs you customers.