テンセント、2950億パラメータのオープン MoE モデル「Hy3」を公開（1トークンあたり 210 億パラメータ活性化、コンテキスト長 256K）

Tencent’s Hy team released Hy3. Hy3 is a 295B-parameter Mixture-of-Experts (MoE) model. It activates only 21B parameters per token. The weights ship under the Apache License 2.0. Hy3 is aimed at reasoning, agentic workflows, and long-context tasks.

What is Hy3?

Hy3’s architecture contains a sparse MoE with 192 experts and top-8 routing. Only 8 experts fire per token, so compute stays low.

The model also uses a Multi-Token Prediction (MTP) layer. MTP predicts several tokens at once for faster decoding. Both vLLM and SGLang enable it through speculative decoding.

PropertyValue

ArchitectureMixture-of-Experts (MoE)

Total parameters295B

Activated parameters21B

MTP layer parameters3.8B

Layers (excluding MTP)80

MTP layers1

Attention heads64 (GQA, 8 KV heads, head dim 128)

Hidden size4096

Intermediate size13312

Context length256K

Vocabulary size120832

Experts192 experts, top-8 activated

Supported precisionsBF16

A separate Hy3-FP8 checkpoint is also released. FP8 lowers the memory footprint for cheaper serving.

Benchmark and Performance

The research team published scores across coding, agents, and STEM. On coding, Hy3 reports 78.0 on SWE-Bench Verified. It also reports 57.9 on SWE-Bench Pro and 75.8 on SWE-Bench Multilingual. Terminal-Bench 2.1 lands at 71.7, and DeepSWE at 28.0.

On STEM and reasoning, the numbers climb higher. Hy3 reports 90.4 on GPQA Diamond and 72.0 on USAMO 2026. IMOAnswerBench reaches 90.0, and HLE (with tools) reaches 53.2.

The research team ran a blind test with 270 experts. That test collected 312 valid comparisons on real workflows. Hy3 scored 2.67 out of 4, ahead of GLM-5.1 at 2.51. The edge was clearest in frontend development, CI/CD, and data and storage.

imagehttps://hy.tencent.com/research/hy3

Reliability and Production Behavior

The research team focused much of this release on production reliability. Three failure modes got direct attention, backed by internal numbers.

Tool calling and output formatting: The team fixed baseline stability issues that broke agents. Invalid calls that trigger infinite loops dropped. Hy3 also generalizes across agent scaffoldings. On SWE-Bench Verified, accuracy variance across CodeBuddy, Cline, and KiloCode stays within 4%.

World knowledge and anti-hallucination: The target behavior is simple: answer when grounded, flag when evidence is missing. In internal evaluations, the hallucination rate fell from 12.5% to 5.4%. Commonsense error rates fell from 25.4% to 12.7%.

Multi-turn intent tracking: Joint SFT and RL improved coreference and constraint tracking. The internal issue rate dropped from 17.4% to 7.9%. On the MRCR long-dialogue benchmark, scores rose from 42.9% to 75.1%.

How to Call Hy3

Hy3 exposes an OpenAI-compatible API. You deploy it with vLLM or SGLang, then call the endpoint. One flag, reasoning_effort, controls how much the model thinks.

Copy CodeCopiedUse a different Browser

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(

model="hy3",

messages=[

{"role": "user", "content": "Refactor this function and explain the change."},

],

temperature=0.9,

top_p=1.0,

# reasoning_effort: "no_think" (default), "low", "high" (deep chain-of-thought)

extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},

)

print(response.choices[0].message.content)

Use no_think for direct answers, and high for math, coding, or multi-step tasks. Tencent research team recommends temperature=0.9 and top_p=1.0. You can also try Hy3 without local hardware. OpenRouter lists a tencent/hy3:free route at $0 per token. That free tier is scheduled to end on July 21, 2026.

Where Hy3 Fits: Use Cases

Hy3 is built around agent-style, long-context work. A few concrete examples:

Coding agents: Feed a full repository into the 256K window. Ask Hy3 to fix a failing test with reasoning_effort="high". Stable tool calls help it run edits across many files.

Document processing: Pass a long contract or filing as context. The anti-hallucination training reduces fabricated clauses and misquotes.

Financial analysis: Combine tables and prose in one prompt. Ask for a grounded summary that flags missing data rather than guessing.

Frontend and game development: Generate a React component or a small game loop. The blind test showed a frontend advantage over GLM-5.1.

Hy3 vs GLM-5.2

Tencent’s research team benchmarked Hy3 against GLM-5.2 in its own appendix. GLM-5.2 is roughly a 744B MoE with about 40B active parameters. Hy3 is less than half that total size, with 21B active. On coding, GLM-5.2 leads across the suite.

BenchmarkHy3 (21B active)GLM-5.2 (~40B active)

SWE-Bench Verified78.084.2

SWE-Bench Multilingual75.883.0

Terminal-Bench 2.171.781.0

DeepSWE28.046.2

Total / active params295B / 21B~744B / ~40B

LicenseApache 2.0Open weights

The focus here is about size, not just score. Hy3 trades some coding accuracy for a far smaller active footprint. That footprint matters when you self-host and pay for GPUs.

Deployment Notes

Hy3 has 295B total parameters, so serving needs real memory. Tencent’s research team recommends 8 GPUs, such as the H20-3e or cards with larger memory. vLLM and SGLang both ship recipes with MTP enabled. A minimal vLLM launch looks like this:

Copy CodeCopiedUse a different Browser

vllm serve tencent/Hy3 \

--tensor-parallel-size 8 \

--speculative-config.method mtp \

--speculative-config.num_speculative_tokens 2 \

--tool-call-parser hy_v3 \

--reasoning-parser hy_v3 \

--enable-auto-tool-choice \

--port 8000 \

--served-model-name hy3

For compression, The research team points to its AngelSlim toolkit. AngelSlim covers quantization, low-bit methods, and speculative sampling. Tencent also provides a complete finetuning pipeline for Hy3.

Try It: Interactive Explorer

The demo below is an interactive explorer for Hy3. It visualizes MoE routing, reasoning modes, benchmarks, and sparse efficiency.

(function(){

window.addEventListener('message',function(e){

if(e.data && e.data.type==='hy3-resize'){

var f=document.getElementById('hy3-frame');

if(f && e.data.height){ f.style.height=e.data.height+'px'; }

}

});

})();

Check out the Technical details, Model weight on HF and GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post Tencent Releases Hy3: An Open 295B Mixture-of-Experts (MoE) Model with 21B Active Parameters and 256K Context appeared first on MarkTechPost.