Agentic LLMs: Conducting Video Intelligence with AWS Transcribe, Rekognition, and Bedrock

Teams drown in hours of footage. An LLM can act as the conductor.

Agentic systems are an LLM that reasons at runtime about which tools to call and in what order. Instead of many fixed ML pipelines, a single reasoning agent interprets a natural-language question, “Did Alice enter the warehouse after 4pm?”, decides whether to transcribe audio, run face-search, call a one-call video analysis, or chain tools together, then returns a timestamped answer.

AWS Professional Services published a deployable reference implementation that demonstrates this pattern. The companion GitHub repository is https://github.com/aws-samples/sample-media-analysis-agent and the sample ties together the Strands Agents SDK, Amazon Bedrock (example model shown in the repo), Amazon Transcribe, and Amazon Rekognition. It includes deployment scripts, security guidance, a living threat model, and example code for runtime tool orchestration.

What the system looks like, at a glance

  • The agent is built with the Strands Agents SDK. AWS capabilities (Transcribe, Rekognition, BDA, S3 helpers) are exposed as Python functions decorated with @tool. Each tool’s docstring serves as the agent’s tool contract.
  • Model used in example code (example-only tag): us.anthropic.claude-sonnet-4-5-20250929-v1:0. The sample sets model max_tokens = 4096.
  • Runtime platform in the reference deployment: Amazon ECS on AWS Fargate, internal Application Load Balancer (ALB) reachable via Amazon CloudFront, authentication through invitation-only Amazon Cognito with mandatory TOTP MFA.
  • Python requirement: Python 3.11 or later. Prereqs (per the sample README): pip install strands-agents strands-agents-tools.
  • Deployment orchestration and artifacts appear in the repo (examples cited include deploy/deploy-ecs.sh and docs/threat-model.md).

How the agent reasons

“You are a video intelligence assistant. For each user query:
1. Determine whether it requires spoken content analysis,
visual content analysis, or both
2. Check if prior analysis results are already cached
3. Invoke the appropriate tools
4. Synthesize results into a clear answer with timestamps”

The sample includes example agent traces that show multi-step tool chaining and caching. One short trace:

[Think] This is a face-matching query. I need to index the reference photo first, then search the video.
[Act] analyze_reference_image(image=”reference.jpg”)
[Observe] {“faces_detected”: 1, “face_id”: “face_001”, “confidence”: 99.8}
[Think] Face indexed. Now searching the video.
[Act] search_faces_in_video(video_s3_key=”security_cam_july5.mp4″, collection_id=”user_collection”)
[Observe] {“matched_faces”: [{“timestamp”: “00:14:32”, “confidence”: 97.2}, {“timestamp”: “00:47:15”, “confidence”: 94.8}]}
[Response] Yes, the person appeared twice: at 14:32 (97% confidence) and 47:15 (95% confidence).

Mechanics to note: the agent checks a cache before calling tools, re-uses analysis outputs for fast follow-ups, and uses tool docstrings and the system prompt to map intent to capabilities. The sample also includes fallback rules and multi-modal orchestration, correlating transcript timestamps with Rekognition detections.

Performance, cost, and practical defaults

The reference implementation reports these sample measurements and defaults (time-sensitive):

  • Initial analysis of a new video: “5 to 10 minutes depending on length and services required.”
  • Cached follow-ups: “return in under a second.”
  • Typical transcript-based query on a 60-minute video: approximately $1.50 for the initial transcription plus Bedrock reasoning. Subsequent questions cost primarily the Bedrock per-turn amount.
  • Sample cost table for a 60-minute video (pricing cited as of July 2025):
    • Amazon Transcribe: “60-minute audio transcription”, $1.44
    • Amazon Rekognition: “Face search (60-min video)”, $6.00*
    • Amazon Rekognition: “Label detection (60-min video)”, $6.00*
    • Amazon Bedrock: “Agent reasoning (per turn)”, $0.05 to $0.15
    • Amazon S3: “Storage (500 MB, 24 hours)”, “<$0.01”

    *The $6.00 Rekognition charges are presented in the sample as one-time per-video costs. Subsequent queries on cached outputs only incur the Bedrock reasoning cost.

  • Cache guidance in the sample: reuse cached results if “< 24 hours old". Users can bypass cache by saying "re-analyze" or "fresh analysis".

Important caveats: pricing, model tags, and measured latencies depend on time and environment. The sample figures come from the reference materials and are useful for early budgeting, but run a small pilot on representative content and verify prices on the Bedrock, Rekognition, and Transcribe pricing pages.

Trade-offs: BDA (one-call) versus Transcribe + Rekognition

The reference sample wires in Amazon Bedrock Data Automation (BDA) as an optional one-call analysis that can return SUMMARY, CHAPTERS, and TRANSCRIPT in a single API call. In the sample, BDA accepts video objects up to 10 GB. The Transcribe and Rekognition path provides finer control, with face collections, label detection, and custom confidence thresholds. That path also lets you separate costs.

  • Choose BDA when you need a fast one-call summary and chaptering experience.
  • Choose Transcribe and Rekognition when you need face collections, custom thresholds, or to combine diarization with fine-grained visual indexing.
  • Measure both on your content. The sample recommends A/B testing accuracy, latency, and cost between the two paths.

Security, governance, and operational guardrails

The sample deployment includes concrete controls and a living threat model. Notable defaults in the reference materials:

  • Invitation-only Amazon Cognito with mandatory MFA via TOTP.
  • Per-user S3 prefixes with ownership validation and a 24-hour lifecycle for temporary artifacts.
  • Internal ALB accessible through CloudFront. Images in Amazon ECR and KMS are used for encryption key management.
  • Explicit warnings that Rekognition face collections persist independently of S3 lifecycle and must be deleted explicitly.
  • Development tooling used in the sample workflow: Kiro and Automated Security Helper (ASH) for spec-driven development and automated IaC and static scanning.

For production readiness, add these concrete controls and checks:

  • Enable S3 server-side encryption with customer-managed AWS KMS keys. Do not schedule production KMS keys for deletion. Test teardown scripts only in non-production accounts.
  • Use VPC endpoints for Rekognition, Transcribe, and Bedrock where available, and restrict public egress. Configure CloudFront origins with signed headers or private origin access patterns.
  • Use short-lived pre-signed upload URLs and per-user quotas. Log uploads and enforce rate limits.
  • Apply strict IAM least-privilege roles per tool and per action. Maintain allowlists and deny-lists for tool parameters.
  • Implement formal schema validation for tool outputs (JSON contracts) and add a validator layer so the agent cannot act on malformed or hallucinated results.
  • Keep audit logs for model and tool calls, with redaction rules for PII and biometric data and retention policies aligned with legal requirements.
  • Configure Bedrock Guardrails or equivalent safety controls for sensitive outputs. Keep a living threat model and test for prompt-injection and tool-exploitation vectors.
  • Run a legal and privacy review before using face-matching or biometric processing. Implement documented consent workflows, DPIAs (Data Protection Impact Assessments), and vendor controls to address GDPR, BIPA, and other jurisdictional risks.

Limitations and operational notes

  • Initial analysis time: the sample reports “5 to 10 minutes” for many 60-minute videos. Longer videos, 2 to 3 or more hours, may take significantly longer to index.
  • Cached follow-ups: the sample reports “return in under a second” after analysis artifacts are stored.
  • Web UI upload limits in the sample: accepts MP4, MOV, AVI, and MKV files up to 2 GB. BDA in the sample accepts video objects up to 10 GB via API.
  • Default face-match tool confidence_threshold in the sample: 80.0. Rekognition low-confidence fallback guidance in the sample: <60%.
  • Concurrent capacity guidance in the reference materials: “roughly six concurrent users per task at the defaults”, validate with your load tests and scale tasks or batch indexing for heavy ingestion.
  • Rekognition face collections persist separately and require explicit deletion via Rekognition APIs. Plan cleanup scripts and lifecycle governance.
  • Customer-reported outcome: an unnamed “major media and entertainment company” reported “approximately 80 percent” reduction in manual review time across a backlog of more than 200 multi-hour recordings. This was reported by the customer during the AWS Professional Services engagement and is not independently verified in the sample materials.
  • Deployment time: the sample reports stack create typically completes in “15 to 20 minutes” with most of that time attributed to CloudFront propagation. Measure this in your target region and account.
  • Teardown stack deletion order in the sample README: (1) video-analytic-agent-ecs (application), (2) video-analytic-cognito (identity), (3) video-object-locator-infra (storage).

Production checklist with measurable acceptance criteria

  • Model & pricing validation: confirm Bedrock model availability and pricing for your region. Recompute per-turn cost using representative prompt and response token counts.
  • Pilot benchmarking: run 10 representative videos and capture median and 95th-percentile for (a) first-run indexing time, (b) cost per indexing, (c) mean cached-response latency, and (d) Bedrock token usage per interaction.
  • Accuracy metrics: for speech, report Word Error Rate (WER). For face matching, report precision and recall at your chosen threshold. Log false positives and negatives and tune thresholds.
  • Security checks: validate IAM roles, enable SSE-KMS, confirm VPC endpoints, verify short-lived presigned uploads, and ensure audit logs redact PII before long-term storage.
  • Cleanup automation: implement scripts for ListCollections and DeleteCollection for Rekognition, and verify they run on teardown without leaving collections behind.
  • Legal & privacy: complete a DPIA for biometric workloads, record lawful basis and consent, and document retention and erasure procedures.
  • Human-in-the-loop: route results below configured confidence thresholds to a review queue. Measure reviewer throughput and expected time savings before scaling.

When not to use this pattern

  • You need perfectly deterministic, auditable decisions that cannot rely on probabilistic model outputs.
  • Your data residency policy forbids sending media to the regions or models you plan to use, or you require fully offline processing.
  • Real-time millisecond latency is mandatory for every query without the opportunity to pre-index or cache.
  • You cannot tolerate any biometric processing risk or legal exposure for face-matching in your jurisdiction.

Quick, actionable next steps with ownership

  • Security / Infra: clone the repo (https://github.com/aws-samples/sample-media-analysis-agent), run the smoke deploy (expect about 15 to 20 minutes in the sample), and verify VPC endpoints, KMS, and IAM least-privilege.
  • Product / Data: execute an end-to-end pilot on representative 60-minute recordings and deliver the benchmarking metrics listed above.
  • Legal / Compliance: run a DPIA and confirm consent and retention policies before enabling face-search features.
  • Engineering: add JSON schema validation for tool outputs, implement cleanup scripts for Rekognition collections, and tune cache TTLs per workflow.

Key questions and quick answers

  • How fast are follow-up answers after the video is analyzed?

    According to the reference implementation, cached answers typically return in under a second once analysis artifacts (transcript, Rekognition indices, or BDA outputs) are stored.

  • How long does the first analysis take?

    The sample reports roughly 5 to 10 minutes for initial analysis of many 60-minute videos. Longer recordings or more complex tool chains will take longer. Benchmark on your content.

  • What will it cost to index a 60-minute video?

    Using the sample’s pricing snapshot (July 2025): Transcribe about $1.44. Rekognition face-search and label detection are each about $6.00 (one-time per video in the sample). Bedrock reasoning per turn is about $0.05 to $0.15. Treat these as estimates, re-check live pricing and run token-cost estimates for your model prompts.

  • Are face collections automatically deleted with S3 lifecycle?

    No. Rekognition face collections persist independently and must be explicitly deleted via the Rekognition APIs. The sample calls this out and includes guidance to implement deletion scripts.

  • Is the reported ~80% manual-review reduction a verified metric?

    That number was reported by a customer during an AWS Professional Services engagement and is noted in the sample materials as customer-reported and not independently verified.

Agentic orchestration moves design complexity into a single, conversational conductor that maps user intent to specialized tools. It is a useful pattern for conversational exploration of media, but it concentrates governance, security, and legal risk in the model and tools layer. Start conservative: pilot on representative content, measure costs and accuracy, add strict tool contracts and validators, and keep humans in the loop where lives, livelihoods, or privacy are at stake.

Authors of the reference implementation: Michael Li and Kara Yang. Companion repository: https://github.com/aws-samples/sample-media-analysis-agent. Pricing and model tags in the sample are time-sensitive, confirm current values before production deployment (pricing cited above is as of July 2025).

0