If you need a model that can hold entire contracts, multi-document pipelines, or long agent histories in one session, Tencent’s Hy3 is worth evaluating.
Tencent’s Hy research team has published Hy3, a 295-billion-parameter Mixture‑of‑Experts (MoE) language model. They report it can handle a 256K‑token context and activate up to 21 billion parameters per token (top‑8 routing). Tencent says the weights are published under Apache License 2.0, and the release includes deployment recipes, an OpenAI‑compatible API surface, and a lower‑memory FP8 checkpoint variant called Hy3‑FP8.
Quick technical orientation, what those headline numbers mean
- Total parameters: 295B (model capacity across all experts).
- Active parameters per token: up to 21B, because Hy3 is a sparse MoE that routes each token to a subset of experts (192 experts, top‑8 routing, eight experts may be selected per token, so per‑token activation can vary).
- Context window: 256K tokens, roughly on the order of 180k, 200k English words (≈ a few hundred to ~600 pages depending on formatting), large enough to hold multi‑document pipelines or full legal briefs in one conversational session.
- License: Tencent reports weights under Apache License 2.0 (check the repository’s LICENSE file before assuming commercial reuse).
MoE architectures keep a very large parameter budget while running only a fraction of it per token. Hy3’s reported active fraction (21B ÷ 295B ≈ 7.12%) lowers steady per‑token compute compared with a dense model of the same total size, but it brings operational issues, such as routing overhead, potential expert hotspotting, and more complex sharding logic.
Key engineering features (one line each)
- Mixture‑of‑Experts (MoE) with top‑8 routing: multiple expert modules exist; a router chooses up to eight experts per token so the model only executes those subnets for that token.
- Multi‑Token Prediction (MTP): a layer intended to predict multiple tokens at once to increase decoding throughput when used with speculative decoding.
- Speculative decoding support: vLLM and SGLang integration to use faster proposers and verification steps to accelerate generation under certain workloads.
- Hy3‑FP8 checkpoint: an FP8 checkpoint option to lower serving memory (FP8 can reduce footprint but requires careful validation to avoid numeric instability).
- OpenAI‑compatible API & reasoning flag: Tencent exposes an API with a reasoning_effort parameter (“no_think” default, “low”, “high”) to trade latency for deeper internal chain‑of‑thought-style computation.
What Tencent reports about performance and reliability
All scores and internal metrics below are vendor‑reported by Tencent. Validate them on your own benchmarks before making production decisions.
Coding benchmarks (vendor‑reported)
- SWE‑Bench Verified: 78.0
- SWE‑Bench Pro: 57.9
- SWE‑Bench Multilingual: 75.8
- Terminal‑Bench 2.1: 71.7
- DeepSWE: 28.0
STEM / reasoning benchmarks (vendor‑reported)
- GPQA Diamond: 90.4
- USAMO 2026: 72.0
- IMOAnswerBench: 90.0
- HLE (with tools): 53.2
Tencent also ran a blind comparison with 270 experts and 312 valid comparisons. They report Hy3 scored 2.67/4 versus GLM‑5.1 at 2.51/4, with the biggest advantages in frontend development, CI/CD, and data & storage tasks. See that as a promising signal, but ask for the blind‑test methodology, prompt set, and statistical analysis before using it as an independent comparative ranking.
Tencent reports post‑iteration improvements: hallucination rate down from 12.5% to 5.4%, commonsense error rate down from 25.4% to 12.7%, multi‑turn intent tracking internal issue rate down from 17.4% to 7.9%, and MRCR long‑dialogue benchmark up from 42.9% to 75.1%.
Deployment notes and friction points
- Hardware guidance: Tencent recommends 8 GPUs (they cite examples such as H20‑3e). Verify exact per‑GPU memory needs in their repo, the release notes should state required memory and recommended instance types.
- Serving stack: vLLM and SGLang are demonstrated for MTP/speculative decoding; example vLLM flags include tensor parallelism and speculative‑method mtp. These options change sharding and latency/throughput tradeoffs, test for your workload.
- FP8 caution: Hy3‑FP8 reduces memory but can introduce numerical edge cases. Run an FP8 validation suite on representative inputs and maintain FP16/FP32 fallbacks for sensitive workloads.
- Hosted access: OpenRouter lists a tencent/hy3:free route at $0 per token per the release notes, with the free tier scheduled to end on July 21, 2026. Confirm current pricing and hosted SLA before planning deployments.
- API controls: reasoning_effort is documented as an API parameter; Tencent suggests “no_think” for direct answers and “high” for math, coding, and multi‑step tasks. For deterministic outputs (code, math proofs) tune temperature and top_p downward from the published defaults (Tencent recommends temperature = 0.9, top_p = 1.0) to reduce nondeterminism.
Where Hy3 fits strategically
Tencent positions Hy3 as a capacity‑efficient MoE alternative to larger sparse models. It has fewer total parameters than some 700B+ MoEs, but it can still hit high active compute when routing fires many experts. That can give you very long context and strong peak capability without the same dense serving cost, if you can manage routing balance, sharding complexity, and validation of FP8 and speculative decoding in production.
Practical strengths to evaluate include long‑document summarization and retrieval‑augmented pipelines, multi‑turn agent scaffolds that need consistent tool calling and intent tracking across long histories, and math/STEM workflows that benefit from the “high” reasoning mode. Practical risks to test include expert hotspotting (inputs that repeatedly activate the same small set of experts), tail latency increases for long contexts, and FP8 numeric edge cases.
Concrete validation checklist and experiments to run before production
Don’t accept vendor claims at face value. Run these measurable tests and gather these metrics:
- Latency & throughput under long context: run a 50K and a 100K token end‑to‑end prompt and report P50/P95/P99 latency, tokens/sec, and memory per GPU. Acceptance example: P95 latency below your workflow threshold (specify your target seconds) and stable throughput at scale.
- FP8 vs FP16 numeric stability: validate Hy3‑FP8 on a representative math and code test set (compute pass/fail rates, numerical diffs, and examples of divergence). Acceptance example: less than X% functional regression vs FP16 on critical tasks.
- Hallucination and factuality suite: measure hallucination rate on a labeled dataset; define hallucination per example and report % per response. Acceptance example: hallucination rate below vendor baseline or below a product‑specific threshold.
- Routing hotspot test: craft inputs that tend to trigger the same expert subset and measure expert utilization skew, latency spikes, and recovery under concurrent requests.
- Tool calling and multi‑turn intent tracking: run scripted agent workflows that call external tools repeatedly across long sessions; measure success/failure rates and the internal issue rate over many runs.
- Blind comparative POC: run a head‑to‑head with your current production model on your task suites and have blinded raters score outputs; collect confidence intervals and statistical tests.
Short prescriptive POC plan (2-4 week baseline)
- POC target: a 50-100K token contract summarizer or a CI/CD multi‑step automation agent.
- Success criteria (examples): P95 latency under your SLA; summary factuality and ROUGE/L better than current baseline; hallucination rate reduced by at least Y percentage points; cost per 1K tokens within acceptable budget.
- Deliverables: benchmark scripts, raw logs (latency, memory), hallucination annotations, FP8 validation report, and routing utilization heatmaps.
Transparency and red‑flags to demand from Tencent (or any MoE vendor)
- Model card and paper with architecture/training details, tokenizer, and compute used.
- Exact LICENSE file and any dataset or third‑party restrictions.
- Benchmark scripts, prompt templates, and dataset versions for all reported scores.
- Blind‑test methodology, annotator selection criteria, and raw or aggregated annotation data with confidence intervals.
- FP8 validation notes showing any accuracy or stability deltas vs FP16/FP32.
- Deployment logs showing P95/P99 latency and throughput for recommended 8‑GPU configurations and per‑GPU memory requirements.
Risk checklist, what can go wrong fast
- Hotspotting: skewed input distributions can overload a few experts, spiking latency and throttling throughput.
- FP8 numeric regressions: reduced numerical range can break subtle math or physics computations unless validated.
- Operational complexity: MoE sharding, speculative decoding pipelines, and FP8 fallbacks add engineering debt compared to dense model deployments.
- Vendor metrics vs reality: vendor‑reported hallucination and benchmark gains are useful signals, but they must be reproduced on your test sets and prompt patterns.
Final read for product leaders
Hy3 is a serious engineering play: long context, sparse capacity, MTP/speculative decoding, and an FP8 checkpoint for lower memory. For companies building products that truly need a single conversational session to contain many documents or long agent chains, Hy3’s architecture is worth testing. For everyone else, weigh the engineering complexity and validation burden against the upside.
Before committing a production pipeline, ask for the model card, the FP8 validation report, benchmark scripts, blind‑test methodology, and exact hardware memory requirements. The headline numbers are compelling. The operational details determine whether Hy3 becomes an enabler or an expensive experiment.
Key takeaways, questions you might ask, with short answers
- What makes Hy3 different from a large dense model?
Hy3 is a Mixture‑of‑Experts model with 192 experts and top‑8 routing. It reports 295B total parameters but activates up to 21B parameters per token (≈7.12% active fraction), trading dense per‑token compute for sparser execution and higher nominal capacity.
- Can I run Hy3 in production without enormous cost?
Tencent recommends 8 large‑memory GPUs and provides FP8 checkpoints plus speculative decoding recipes to reduce footprint. That can lower costs, but you must measure P95/P99 latency, throughput, and FP8 accuracy on your workload, the devil is in routing balance and numeric stability.
- Is Hy3 better at reasoning and coding than comparable models?
Tencent reports strong STEM and coding results (for example, GPQA Diamond 90.4 and SWE‑Bench Verified 78.0) and a blind‑test edge vs GLM‑5.1 (2.67 vs 2.51). Treat these as vendor‑reported signals and reproduce them on your prompts and datasets before drawing conclusions.
- How do I control the model’s internal effort for hard tasks?
Hy3 exposes a reasoning_effort API flag (“no_think”, “low”, “high”); Tencent recommends “high” for math, coding, and multi‑step tasks. Lower temperature and top_p settings are advisable for deterministic outputs, tune per task.
- Are the weights really free to use commercially?
Tencent reports an Apache 2.0 license for the weights. Apache 2.0 is permissive, but confirm the repository’s LICENSE and any linked dataset or third‑party constraints before commercial use.