TL;DR
- Run a terminal-native agent (OpenCode) locally and route inference to open‑weight models hosted on Amazon Bedrock to keep controls, logs, and billing inside your AWS account while avoiding per-seat lock‑in.
- Pick different models for different agent roles: a long‑context reasoner for debugging, a balanced open‑weight for architecture/multi‑file work, and an MoE throughput model for bulk codegen.
- Expect token inflation from agentic workflows, plan quotas and observability, and build a cost model that includes planning, handoffs, and verification tokens.
- Proof it fast: install OpenCode, map two models (reasoner + builder) in opencode.json, run a test that generates code + tests, and measure tokens, latency, and failure modes.
How to run a quick experiment (what to do first)
- Confirm Bedrock access and model availability for the models you want to test.
- Install OpenCode (the verification example referenced v1.18.20) and set AWS credentials (IAM keys, IAM Identity Center, or Bedrock API key).
- Create an opencode.json that maps agent roles to Bedrock model identifiers (example below).
- Run a short workflow that asks the agent to generate unit tests, build code, and run your test harness. Measure token usage, p95/p99 latency, and how often the agent produces usable output.
- Iterate routing rules and add quotas/alerts for tokens and requests before scaling.
Why teams are moving toward open‑weight models for engineering
Many engineering teams prefer open‑weight models because they make customization, cost control, and transparency practical. McKinsey’s “Open‑source technology in the age of AI” report (2025) found 76 percent of organizations expect to increase open source AI usage, and leading AI adopters are 40 percent more likely to use open‑weight models. When a workload needs fine‑tuning, deeper inspection, or strict IP handling, open‑weight models often look more attractive than opaque hosted models.
Benchmarks from vendors and customers also shape decisions. CrowdStrike reported a fine‑tuned NVIDIA Nemotron variant hit 96% valid query accuracy in their test, compared with 61% for GPT‑4o and 94% for Claude Sonnet 4.5. Treat vendor reports as indicative, not definitive. They show potential, but you must validate performance against your own tasks.
Why Amazon Bedrock is a practical managed backend
Amazon Bedrock gives a managed inference layer that ties into AWS controls like IAM, CloudTrail, and VPC/PrivateLink, and it exposes a single API for multiple model providers. That makes it useful when you want enterprise logging, private connectivity, and one surface for several models. Verify model licensing, regional availability, and contract terms before you commit.
Security & compliance
- Bedrock integrates with IAM for least‑privilege access, CloudTrail for audit trails, and PrivateLink/VPC options for private connectivity. Bedrock is listed in the source material as having compliance coverage including HIPAA, SOC 2, ISO 27001, FedRAMP, and GDPR. Confirm which specific programs and regions apply to your workload.
- AWS states that Bedrock does not use customer inputs or outputs to train foundation models, but always confirm the current terms for the specific model you select and any partner policies that may differ.
Pricing tiers, quotas, and regional routing
- Bedrock offers tiered options for latency and cost: Priority (latency‑sensitive), Standard (on‑demand inference), and Flex (for variable‑latency workloads). The source describes Flex as roughly 50% lower cost than Standard for suitably latency‑tolerant workloads, so use AWS pricing and quotes for exact numbers.
- Default platform limits cited are 100M tokens per minute and 10K requests per minute. Treat these as planning figures and request quota increases when scaling a production system.
- Bedrock supports geographic and global inference profiles. Example profile names in the examples are global.moonshotai.kimi-k3 (routes across supported commercial Regions) and us.moonshotai.kimi-k3 (keeps processing within US geography). The source notes global cross‑Region inference can be ~10% less expensive than strict geographic routing, so verify current pricing and regional routing behavior for your account.
OpenCode + Bedrock: a terminal agent that routes to multiple models
OpenCode is, as the project describes it, “an open source, terminal-native AI coding agent built in Go.” It uses Language Server Protocol (LSP) diagnostics to understand project structure, can edit files and run shell commands, and connects to many LLM providers.
OpenCode supports mapping agent roles to different models so you can assign a high‑reasoning model to planning and a throughput model to code generation. Here’s a working example opencode.json mapping you can adapt (verify exact model IDs in your Bedrock console):
{
“$schema”: “https://opencode.ai/config.json”,
“model”: “amazon-bedrock/us.openai.gpt-oss-120b-1:0”,
“agent”: {
“plan”: {
“model”: “amazon-bedrock/global.moonshotai.kimi-k3”
},
“build”: {
“model”: “amazon-bedrock/us.nvidia.nemotron-super-3-120b”
}
}
}
In this example the default model is the open‑weight GPT‑OSS 120B hosted via Bedrock for balanced generation, the “plan” role routes to Moonshot AI Kimi K3 for deeper reasoning, and the “build” role routes to NVIDIA Nemotron 3 Super 120B for throughput‑optimized generation. Confirm the exact provider/model strings and version tags in your Bedrock model catalog before using them in production.
Practical prerequisites
- Active AWS account with Amazon Bedrock access and appropriate quotas.
- Model access or entitlements for Kimi K3, GPT‑OSS 120B, and NVIDIA Nemotron 3 Super 120B (as relevant to your tests).
- AWS credentials (IAM access keys, IAM Identity Center, or Bedrock API key) configured in your environment.
- Node.js 18+ (for some npm installs) or Homebrew on macOS/Linux to install command‑line tooling; the example OpenCode verification used v1.18.20.
Which model for which role, the pragmatic breakdown
- Reasoning and deep debugging: Moonshot AI Kimi K3 is positioned in the examples as a long‑context reasoner and is advertised with a 1M‑token context window. Treat that as a vendor‑claimed capability and verify it in the model documentation before relying on it for very long contexts.
- Multi‑file generation and architecture: GPT‑OSS 120B (the open‑weight 120B model) is a balanced choice for scaffolding modules and synthesizing coordinated multi‑file changes.
- High‑throughput code generation: NVIDIA Nemotron 3 Super 120B uses a Mixture‑of‑Experts (MoE) architecture that activates a subset of parameters per token to optimize throughput. The examples reference an MoE activation pattern (e.g., activating ~12B of 120B parameters per token). Treat such numbers as vendor‑reported performance characteristics to be validated for your workload.
Developer prompts and quick examples
OpenCode’s CLI lets you swap models inline and run prompts from your terminal. Example prompts used in the walkthroughs include:
“Generate comprehensive unit tests for @${file}. Use pytest with fixtures.”
“This Step Functions workflow hangs ~2% of the time under load.”
Those are production‑style prompts: one asks for a scaffolded test harness, the other is a diagnostic prompt to triage a flaky orchestration. Use your test harness to measure whether generated tests pass and whether suggested fixes for the workflow actually resolve the hang.
From local to production: orchestration and scaling
Local model routing is great for individual developers. In production you’ll want an orchestrator that routes subtasks to the best model and preserves continuity. The source example cites Ethara.AI using Oh‑My‑OpenAgent as an orchestration layer in front of Bedrock‑hosted models. Ethara.AI’s approach hands off plan/build/test/verify subtasks to the best model and reunifies results in a coordinator.
Two operational realities to plan for:
- Token inflation: Analyst reports (Gartner, 2026) cited in the source material estimate that agentic workflows can multiply token consumption by 5-30x. Use such estimates as directional guidance and run your own benchmarks, because agentic pipelines add planning, decomposition, and verification prompts that all consume tokens.
- Context continuity: If you switch models mid‑workflow, the new model has no implicit session memory. You must serialize and pass context explicitly, which increases token use and may eat into per‑token cost savings unless your orchestrator minimizes redundant context transfer.
Security, governance, and practical guardrails
Bedrock gives controls you’ll use in governance: IAM for least privilege, CloudTrail for auditing, VPC/PrivateLink for private connectivity, and Guardrails for content moderation and PII redaction. Validate each model vendor’s data handling and licensing. “Open‑weight” covers many license types, from permissive to restrictive to research‑only.
Practical governance checklist:
- Maintain a model inventory with license and commercial‑use flags.
- Lock down model invocation with least‑privilege IAM policies and network controls.
- Log prompts/responses for observability but redact secrets and PII; store artifacts (prompts, responses, verification) for audits.
- Run continuous A/B tests and a synthetic test harness to detect drift or regressions.
Example policy (illustrative, adapt to your IAM structure):
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“bedrock:InvokeModel”
],
“Resource”: [
“arn:aws:bedrock:us-east-1:123456789012:model/*”
]
}
]
}
Cost architecture, where money shows up
Cost levers for a Bedrock + OpenCode stack include per‑token charges, orchestration compute, storage for context and artifacts, and engineering time to maintain prompts and routing logic. Bedrock pricing tiers (Priority, Standard, Flex) let you match latency and cost profiles. The source describes Flex as roughly 50% lower cost than Standard for variable‑latency workloads and notes global cross‑Region routing can be about 10% cheaper than geographic profiles. Always confirm current pricing and per‑model rates in the AWS pricing portal and capture quotes for your target regions.
A quick cost model idea: total_work_cost = sum(token_costs_per_step) + orchestration_compute + storage + monitoring. Add a multiplier for agentic workflows to approximate planning and verification tokens, then iterate with measured numbers from test runs.
When not to use this pattern
- If you need capabilities that only closed commercial models provide (for example, proprietary reasoning features or certain safety guarantees), a pure open‑weight + Bedrock approach may not meet requirements.
- If you’re a tiny team with no AWS footprint and you need a zero‑ops, fully hosted experience, the integration and governance overhead here may not be worth it initially.
- If licensing or export controls prevent you from using particular open‑weight models in production, you’ll need to pick alternatives or negotiate vendor terms.
Quick technical and operational checklist to get started
- Confirm model availability and licensing for your target models (Kimi K3, GPT‑OSS 120B, Nemotron 3 Super 120B) in the target regions.
- Provision Bedrock access and set up least‑privilege IAM, CloudTrail, and PrivateLink where required.
- Install OpenCode and verify your opencode.json mappings (the example above is a working pattern to adapt).
- Run baseline benchmarks: latency, token usage, pass/fail rate for generated tests, and hallucination/failure rates for a representative set of engineering tasks.
- Set quotas, alerts, and a cost dashboard; request Bedrock quota increases before you hit limits.
Key takeaways, questions you’re probably asking
- Can I run an AI coding agent without sending proprietary code to a public API?
Yes. Running a local agent like OpenCode that invokes Bedrock-hosted models keeps inference and logs within your AWS account controls (IAM, CloudTrail, PrivateLink). Verify the specific model provider policies and your contract terms to confirm how inputs/outputs are handled.
- Which model should I use for debugging versus mass code generation?
Use a long‑context, high‑reasoning model (the walkthrough cites Moonshot AI Kimi K3 with a vendor‑claimed 1M‑token context) for debugging and root‑cause analysis; use an MoE throughput model (NVIDIA Nemotron 3 Super 120B) for bulk codegen; and use a balanced open‑weight model like GPT‑OSS 120B for multi‑file generation and architecture tasks. Validate model claims against vendor docs and your own benchmarks.
- Will multi‑model orchestration actually save money?
Potentially. Routing each subtask to the appropriately priced model can lower per‑task token cost, but agentic orchestration increases total token consumption (analyst estimates in the source suggest a several‑fold multiplier). Build a cost model that includes planning, context handoffs, verification tokens, orchestration compute, and storage.
- How do I manage compliance and auditing?
Use Bedrock’s IAM, CloudTrail, VPC/PrivateLink, and Guardrails; maintain a model inventory tied to licenses, redact PII in logs, and store artifacts for audits. Map Bedrock’s compliance programs (HIPAA, SOC 2, ISO 27001, FedRAMP, GDPR) to your control requirements and confirm region‑specific applicability.
- What operational practices prevent model‑drift and regressions?
Run continuous A/B testing and synthetic validation against a representative engineering test harness, track p95/p99 latencies and hallucination failures, set failure budgets, and retune or re‑fine‑tune models with guarded datasets when quality drops.
Final practical note
Try a low‑friction experiment: install OpenCode locally, map a reasoning model and a builder model in opencode.json, and run a workflow that generates tests, builds code, and runs the test harness. Measure tokens, latencies, and failure modes. Iterate on routing, minimize redundant context transfers, and add observability and quotas before you scale. The pattern, a terminal agent invoking Bedrock‑hosted open‑weight models, gives engineering teams control and flexibility, provided you design for governance and cost from day one.
“OpenCode is an open source, terminal-native AI coding agent built in Go.”, OpenCode