Enterprises still wrestling with massive codebases or multi-document research have three priorities: scale, cost, and compliance. AWS and Moonshot AI claim Kimi K3 on Amazon Bedrock addresses all three, but those are vendor claims you should verify on your workload. Here’s what’s new, what’s real, and how to test it.
What AWS and Moonshot say
According to an AWS blog post by Alex Thewsey, Saurabh Trikande, William Yap, Tanvi Girinath, and Sofian Hamiti, Moonshot AI’s Kimi K3 is now available on Amazon Bedrock. AWS relays Moonshot’s claim that Kimi K3 is “its most capable model and the first open model to reach 2.8 trillion parameters, ” with native vision and a 1‑million‑token context window. The post also cites an “approximate 2.5x improvement in scaling efficiency over Kimi K2.”
Platform-level capabilities on Bedrock that AWS highlights for Kimi K3 include tool calling, structured outputs, reasoning, response streaming, and support for OpenAI‑compatible Responses and Chat Completions APIs. Bedrock also supports the Amazon Invoke and Converse APIs via the bedrock‑runtime endpoint.
“Your data is processed within the AWS data boundary, is not shared with the model provider, and is not used to train the underlying model.”, AWS
“Zero data retention is always enabled for inference requests, while zero operator access prevents even AWS operators from accessing your prompts and completions during inference.”, AWS
What to treat as vendor claims (and why to verify)
- 2.8 trillion parameters: AWS and Moonshot report this number. Parameter count is a descriptive spec, not a standalone proof of superior performance. Ask for the Moonshot model card and architecture details (trainable vs. sparse parameters, effective parameterization) before using the figure to justify procurement.
- 1‑million‑token context window: AWS reports the capability. Tokenization behavior varies by tokenizer and input; test with your real inputs to see the effective context you get after tokenization and how latency and costs scale.
- “Approximate 2.5x improvement in scaling efficiency”: This is a vendor metric. Request the benchmark methodology (datasets, baselines, hardware) before treating it as an apples‑to‑apples performance claim.
- “Open‑weight” label: The term implies greater transparency, but the operational meaning matters. Confirm whether weights are downloadable, what license applies, and any export or usage restrictions.
- Security guarantees: AWS states inference data is processed inside the AWS data boundary, not shared with Moonshot, and not used to train the model, and that Bedrock enforces zero data retention and zero operator access. Treat these as platform statements, validate them against Bedrock security docs and any attestations you require for compliance.
Key platform mechanics you should know (exactly as AWS describes them)
- Model identifiers and geography: Use global.moonshotai.kimi-k3 for the global cross‑Region profile, or us.moonshotai.kimi-k3 to keep processing within the US geography. AWS notes the global profile costs approximately 10% less than a geographic profile.
- How to call it: Try Kimi K3 from the Amazon Bedrock console (Test → Playground) or programmatically via the bedrock‑runtime endpoint (supports OpenAI‑compatible Responses and Chat Completions APIs), or via Amazon Bedrock Invoke and Converse APIs.
- Prerequisites: An active AWS account with Amazon Bedrock access, Python 3.10+, and IAM permissions: bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, and bedrock:CreateInference.
- Explicit prompt caching: Kimi K3 is the first open‑weight model on Bedrock to support explicit prompt caching. You can mark a reusable prompt prefix with a prompt_cache_breakpoint (minimum 1, 024 tokens before the breakpoint). In explicit mode, tokens written to cache are billed at a higher rate but are kept in cache for at least 30 minutes. Subsequent matching requests that hit the cache are billed at a discounted rate and the input tokens will not count against input‑tokens‑per‑minute quotas. Bedrock returns metadata such as cached_tokens to indicate cache hits.
Where Kimi K3 could matter, real use cases
Given the vendor claims, Kimi K3 is positioned for long‑horizon and multimodal workflows where stitching together retrieval systems and chunked prompts is painful. Practical applications include:
- Code assistants that must reason across very large repositories and dependency graphs.
- Legal or regulatory workflows that require cross‑document analysis of hundreds of pages.
- Long‑running agents and multi‑turn research assistants that benefit from sustained session context.
- Multimodal tasks combining images and long textual context (e.g., technical documentation plus diagrams).
Where to be skeptical, quick checklist
- Parameter counts are a signal, not a guarantee. Ask for model cards, evaluation results (perplexity, instruction following, code benchmarks), and reproducible tests.
- Confirm what “open” or “open‑weight” means operationally: downloadable weights? permissive license? usage limits?
- Prompt caching has tradeoffs: measure write cost vs. read discount, cache retention/eviction policies, and behavior under load and across model updates.
- Validate the security language (zero retention, zero operator access) with AWS security docs and any compliance attestations your organization requires.
- Get clarity on per‑token rates for Kimi K3 (separately for regular vs. cached tokens) from the Amazon Bedrock pricing page, vendor posts often omit concrete per‑model rates.
A practical, three‑step pilot to validate claims on your workload
Don’t buy on specs alone. Run this quick pilot and capture the metrics that matter.
-
Sandbox test (Playground)
In the Bedrock console Test → Playground, run 10 representative prompts (short, medium, and very long near the 1M token claim). Record token counts, median/95th latency, and whether the response fits your task. Measure tokenization delta (chars → tokens) for your typical documents.
-
Integration test (programmatic)
Call the model programmatically via bedrock‑runtime using the profile you need (global.moonshotai.kimi-k3 or us.moonshotai.kimi-k3). Test explicit prompt caching: write a reusable prefix (≥ 1, 024 tokens) with prompt_cache_options set to explicit, then send a repeated request to confirm a cache hit and inspect metadata (e.g., cached_tokens). Measure:
- Median and 95th percentile latencies with and without cache hits
- Cache hit ratio and billing delta (write cost vs. read discount)
- Throughput under concurrency and any errors or eviction behavior
-
Compliance and operational test
Ask AWS for written clarifications tied to your use case: what “zero operator access” covers, how data is isolated in the AWS data boundary, and whether logs or telemetry could be exposed for debugging. Collect the Bedrock security documentation and any SOC/FedRAMP/ISO evidence you need.
Practical metrics to capture during the pilot
- Median/95th latency per prompt length bucket
- Cost per 1M tokens (separately for regular input, cached writes, and cache hits)
- Cache write cost, cache read discount, and cache retention/eviction observations
- Tokenization ratio for your input types
- Functional accuracy for your tasks (e.g., cross‑file code reasoning, contract clause extraction)
Quick operational checklist
- Confirm Bedrock access and required IAM permissions: bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, bedrock:CreateInference.
- Choose the right profile: global.moonshotai.kimi-k3 (cost‑efficient) or us.moonshotai.kimi-k3 (US residency).
- Prototype in the console, then test programmatically via bedrock‑runtime and the OpenAI‑compatible Responses/Chat Completions APIs or Bedrock Invoke/Converse APIs.
- Experiment with explicit prompt caching for large reusable prefixes; measure the billing tradeoffs and cache behavior.
- Validate security claims against AWS documentation and any compliance attestations you require.
Key questions, straight answers
- Is Kimi K3 actually 2.8 trillion parameters and the first open model at that scale?
AWS relays Moonshot AI’s claim that Kimi K3 has 2.8 trillion parameters and calls it “the first open model to reach 2.8 trillion parameters.” Treat that as the vendor’s stated specification and verify details (including the meaning of “open”) with Moonshot AI model documentation and Bedrock model cards.
- Can I use Kimi K3 for million‑token prompts today?
AWS reports Kimi K3 supports a 1‑million‑token context window and that it’s available on Bedrock in global and US profiles. Prototype on the Bedrock console or via the bedrock‑runtime API to measure latency, cost, and tokenization behavior for your specific inputs.
- Will explicit prompt caching save money and latency for repeated large prompts?
According to AWS, explicit prompt caching lets you write a reusable prefix (minimum 1, 024 tokens) at a higher write cost; cached tokens are kept at least 30 minutes and later cache hits are billed at a discounted rate and don’t count against input‑tokens‑per‑minute quotas. Measure the write/read cost tradeoffs and observe cache eviction policies on your workload.
- Are my prompts private and not used to train the model?
AWS states Bedrock processes inference data within the AWS data boundary, does not share it with the model provider, and does not use it to train the underlying model, and that zero data retention and zero operator access are enforced for inference requests. Treat these as platform claims and validate them against Bedrock security documentation and any compliance evidence you require.
- How do I call Kimi K3 programmatically?
Use the bedrock‑runtime endpoint (supports OpenAI‑compatible Responses and Chat Completions APIs) or the Amazon Bedrock Invoke and Converse APIs. The published identifiers are global.moonshotai.kimi-k3 and us.moonshotai.kimi-k3.
Where to look next
Review the AWS blog post authorship and model card details, check Moonshot AI’s Kimi K3 pages (e.g., kimi.ai/kimi-k3 and kimi.com/blog/kimi-k3), and examine the Bedrock prompt‑caching and security documentation before deploying sensitive workloads. The Moonshot AI on AWS samples repository contains code examples to get started with integrations such as OpenCode and Hermes Agent.
Three immediate steps for teams: (1) request the Moonshot model card and benchmark methodology; (2) run the three‑step pilot above and capture the metrics that matter to your use case; (3) obtain written security/compliance clarifications from AWS for your deployment. Big context windows and prompt caching can change how you build automation, but only real tests and documented security assurances should guide production decisions.