--- title: "Kimi K3: the 2.8T open model that finally beats Opus 4.8" canonical: "https://agenticup.dev/posts/kimi-k3-benchmarks-opus-48/" pubDate: "2026-07-20T00:00:00.000Z" description: "Kimi K3 is the first Chinese open-weight model to beat Opus 4.8 on frontier benchmarks. At 2.8T parameters with 16 of 896 experts active, it leads Program Bench but hallucinates at 51%." tags: [kimi k3, moonshot ai, open-source, model comparison, coding agents, benchmarks] --- **TL;DR:** Moonshot released Kimi K3 on July 16, the largest open-weight model ever at 2.8 trillion parameters. It is the first Chinese open-weight model to beat Opus 4.8 on frontier benchmarks, leading Program Bench at 77.8 and scoring 1668 Elo on agentic tasks. But the hallucination rate climbed to 51%, making K3 a coding specialist rather than a generalist. Full weights drop July 27 under Modified MIT. > **Key takeaways:** > - Kimi K3 is the first Chinese open-weight model to beat Opus 4.8 on frontier benchmarks, ranking #3 on the Artificial Analysis Intelligence Index > - Leads coding benchmarks (Program Bench 77.8, Terminal Bench 2.1 at 88.3) and agentic tasks (1668 Elo on GDPval) > - Hallucination rate rose to 51% on AA-Omniscience (up from K2.6's 39%), compared to Claude Fable 5 at 16% > - API pricing at ₹250/$3.00 input / ₹1,250/$15.00 output per MTok. 3x the price of K2.6, comparable to Claude Sonnet 5 > - Architecture built for coding: MXFP4 quantization baked into post-training, 1.78% expert activation, KDA hybrid attention for 6.3x faster decoding at 1M context --- I spent the morning running Kimi K3 through its paces. The model led Program Bench at 77.8. It scored 1668 Elo on agentic tasks. It edged GPT-5.6 Sol on Terminal Bench. Then I looked at the hallucination number. 51%. Up from 39% on K2.6. Claude Fable 5 sits at 16%. The same architecture that makes K3 the strongest coding model in open weights also makes it the most unreliable knowledge model in its class. That tension is the story. Moonshot released Kimi K3 on July 16 with no press conference. Just a model ID, a pricing page, and a quiet blog post. At 2.8 trillion parameters with 896 experts activating 16 per token, it is the largest open-weight model ever released. [Artificial Analysis](https://artificialanalysis.ai/articles/kimi-k3-achieves-3-in-the-artificial-analysis-intelligence-index-comparable-to-opus-4-8-and-gpt-5-5) ranks K3 fourth out of 189 models on its Intelligence Index, comparable to Opus 4.8 and GPT-5.5, behind only Claude Fable 5 and GPT-5.6 Sol. First Chinese open-weight model to reach that tier. Here is what the numbers mean for agent builders. ## What does Kimi K3 score on key benchmarks? Kimi K3's benchmark table is more honest than most model release tables. Moonshot publishes scores where their model leads and scores where it trails, side by side. | Benchmark | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol | Opus 4.8 | GLM-5.2 | |----------:|--------:|---------------:|-------------:|---------:|--------:| | Program Bench | 77.8 | 76.8 | 77.6 | 74.2 | 72.1 | | Terminal Bench 2.1 | 88.3 | 84.6 | 88.8 | 85.0 | 81.0 | | FrontierSWE | 81.2 | 86.6 | 71.3 | 75.1 | 74.4 | | DeepSWE | 67.5 | 70.0 | 73.0 | 66.8 | 62.3 | | SWE Marathon | 72.4 | 68.1 | 63.0 | 60.2 | 13.0 | | AA Intelligence Index | 57 | 60 | 58 (Sol) | 56 | 51 | K3 leads Program Bench outright. It ties GPT-5.6 Sol on Terminal Bench. On FrontierSWE, it beats GPT-5.6 Sol by nearly 10 points. 81.2 versus 71.3, though it trails Fable 5 at 86.6 by a similar margin. The catch comes on DeepSWE. GPT-5.6 Sol leads at 73.0, Fable 5 follows at 70.0, and K3 sits at 67.5. On the hardest repo-level bug fixing tasks, the closed frontier still leads. On agentic tasks, the picture is clearer. Artificial Analysis runs a dedicated agentic evaluation on GDPval. K3 scores 1668 Elo. Ahead of Opus 4.8 at 1600, GLM-5.2 at 1514, and GPT-5.5 at 1494. Only Fable 5 at 1760 stays ahead. K3 also takes the #1 position on AutomationBench-AA, a Zapier-style SaaS workflow evaluation scoring 53%. ## How does the architecture make K3 a coding specialist? Kimi K3 runs three architectural innovations that were designed for one thing: efficient long-context coding work. **Kimi Delta Attention (KDA)** is a hybrid linear attention mechanism. Standard full attention scales quadratically with context length. At 1M tokens, the KV cache alone becomes prohibitive. KDA alternates linear-attention layers and full-attention layers in a 3:1 ratio. Three linear layers handle local sequence structure cheaply. One full-attention layer preserves global information flow. The result is 6.3x faster decoding at 1M-token contexts. Here is how it works. In a standard transformer, every token attends to every other token. At 1M tokens, that is a trillion operations per layer. KDA replaces three out of four layers with a linear approximation that scales subquadratically. The fourth layer still runs full attention to prevent information bottlenecks. The model gets the speed of linear attention with the quality of full attention. Why it works: most tokens in a long context only need local information. The function signature at line 50 does not need to attend to the import at line 3. By routing those local interactions through cheap linear layers, KDA frees compute for the global patterns that benefit from full attention. [Moonshot's technical blog](https://platform.kimi.ai/docs/guide/kimi-k3-quickstart) reports KDA matching or beating full-attention baselines on both short and long-context tasks. **Attention Residuals (AttnRes)** solves a different problem. Standard residual connections accumulate representations uniformly across depth. Early-layer signals get diluted in deeper layers. AttnRes replaces this with selective retrieval across depth, letting the model pull high-value representations from earlier layers instead of inheriting whatever was accumulated. Moonshot reports approximately 25% higher training efficiency at under 2% additional compute cost. **Stable LatentMoE** is where the scale becomes practical. K3 activates 16 of 896 experts per token. A 1.78% activation ratio. For reference, DeepSeek V3 activates roughly 8 of 256 experts at 3.1%. K3 is more aggressively sparse. The routing mechanism, called Quantile Balancing, derives expert selection from router-score quantiles rather than heuristic load-balancing hyperparameters. This is cleaner than most MoE implementations and more stable at 2.8T scale where hyperparameter sensitivity compounds. The catch: the 1.78% activation means that on any given token, 98.22% of K3's parameters are dormant. The 16 activated experts reflect a narrow slice of the training data. Given that the architecture and cache optimization skew heavily toward code, those 16 experts are usually the right ones for a coding problem. When the question is about geography, history, or biology, they are probably the wrong 16. **MXFP4 quantization** is the decision that explains most of K3's tradeoffs. The model uses 4-bit microscaling for weights with 8-bit for activations, applied from the supervised fine-tuning stage onward. Most quantization is post-training: train at full precision, compress afterward. Moonshot baked the quantization into the post-training pipeline itself. The model was explicitly optimized to behave correctly under 4-bit weight precision during alignment. The payoff is operational. Moonshot reports greater than 90% cache hit rates in coding workloads, which is what the ₹25/$0.30 per MTok cache-hit input price reflects. The architecture is built around coding as a workload, not as an add-on use case. ## Why is the hallucination rate 51%? The AA-Omniscience Index measures factual knowledge reliability. It scores models from -100 to 100, giving credit for correct answers and penalizing confidently wrong ones. Kimi K2.6 scored a 6 on this index with 33% accuracy and a 39% hallucination rate. K3 improves the score to 18 and pushes accuracy to 46%. The hallucination rate rises to 51%. Claude Fable 5 scores 40 on the same index with 61% accuracy and a 16% hallucination rate. K3 is correct more often than K2.6 but wrong with confidence far more often than any frontier model. The architecture explains the direction of this tradeoff. MXFP4 at 4-bit precision compresses the weight space significantly. Factual recall stored in lower-resolution representations degrades when queries land outside the training distribution that was most heavily represented. The 1.78% expert activation means the 16 experts handling a factual question were trained primarily on code and coding-adjacent data. When the prompt asks about API syntax, those experts fire accurately. When it asks about historical events, they guess. [TemperatureZero's analysis](https://temperaturezero.com/2026/07/17/kimi-k3-closed-coding-gap-hallucination-rate-51/) makes this concrete: K3 consumed 13,241 reasoning tokens to generate a 3,417-token SVG response for Simon Willison's pelican-on-a-bicycle test. A nearly 4:1 reasoning-to-response ratio. Cost: ₹21/$0.25 for a task that runs for pennies on every other frontier model. The excessive reasoning is a feature when you are debugging a 10,000-line codebase. It is a tax when you are answering a factual question. K3 does not distinguish between the two at inference time. Willison also identified an apparent 85-token hidden system prompt that inflates per-call costs in ways not visible at the API level. Combined with K3's high verbosity. 130 million output tokens versus a 63 million median for comparable models. The effective cost per task can exceed what the list price suggests. The hallucination direction and the verbosity direction both follow from optimizing for long-horizon coding. The model reasons hard because the hardest coding tasks require sustained reasoning. It hallucitates more on factual questions because its precision budget was allocated to code, not general knowledge. ## How much does K3 cost and who should use it? Kimi K3 costs ₹250/$3.00 per million cache-miss input tokens and ₹1,250/$15.00 per million output tokens. Cache-hit input drops to ₹25/$0.30 per million tokens. This is a 3x price increase from K2.6 at ₹79/$0.95 and ₹333/$4.00. The pricing positions K3 against Claude Sonnet 5 (₹250/$3.00 input, ₹1,250/$15.00 output) rather than the budget tier that Kimi models traditionally occupied. [Artificial Analysis](https://artificialanalysis.ai/articles/kimi-k3-achieves-3-in-the-artificial-analysis-intelligence-index-comparable-to-opus-4-8-and-gpt-5-5) calculates the cost per Intelligence Index task at ₹78/$0.94. Similar to GPT-5.6 Sol at ₹87/$1.04 and roughly half the price of Opus 4.8 at ₹150/$1.80. Higher than open-weight peers like GLM-5.2 at ₹27/$0.32 and DeepSeek V4 Pro at ₹3/$0.04. The weights arrive July 27 under Modified MIT. K3 at 2.8T parameters requires multi-GPU clusters for inference. This is not a model that runs on a MacBook. It is an inference-provider model that competes with closed APIs on capability rather than undercutting them on price. **Use K3 for:** long-horizon coding tasks, agentic workflows with heavy tool calling, automated software engineering evaluations, and any workload where coding accuracy matters more than factual recall. **Do not use K3 for:** factual question answering, knowledge work requiring reliability, customer-facing chatbots where hallucination is visible, or cost-sensitive pipelines where the $15/MTok output rate compounds. The decision framework is simple. If your task involves code, K3 is the strongest open-weight option available. If your task involves facts, the hallucination rate disqualifies it for production use without a verification layer. ## FAQ > **Can I use Kimi K3 through existing tools?** > Yes. The Moonshot API uses an OpenAI-compatible Chat Completions interface. Set the base URL to https://api.moonshot.ai/v1 and use model ID kimi-k3. It works with Claude Code, OpenCode, Codex, Hermes Agent, and any tool that supports the OpenAI API format. > > **How does K3 compare to DeepSeek V4 Pro?** > K3 scores significantly higher on every benchmark (AA Index 57 vs 44, FrontierSWE 81.2 vs 29.0) but costs more per token (₹1,250/$15.00 output vs ₹30/$0.27). V4 Pro is the budget option. K3 competes on capability. > > **Will the hallucination rate improve when weights are released?** > Likely not. The MXFP4 quantization was baked into the post-training pipeline. The alignment phase happened under 4-bit constraints. Running at full BF16 precision may shift the numbers slightly but the training distribution, code-heavy, is the deeper constraint. > > **What hardware do I need to run K3 locally?** > 2.8T parameters at MXFP4 is approximately 1.4TB of memory. You need a multi-GPU cluster with at least 16 H100s for inference. This is an API-first model for most teams. > > **Does K3 support vision?** > Yes. K3 accepts text, image, and video inputs with native multimodal understanding. Output is text-only. The vision capabilities combine with coding for game development, frontend engineering, and CAD workflows. ## Related Posts - [GLM-5.2: MIT open-source model that rivals Opus 4.8](/posts/glm-52-mit-open-source-model-rivals-opus-48/). The previous open-weight benchmark that trailed Opus 4.8 by 1%. K3 now leads it. - [Kimi K2.7 Code: the first open-source model that competes with Claude Code](/posts/kimi-k2-7-code-review/). Moonshot's coding-focused model that set the stage for K3's architecture. - [The 2026 LLM architecture landscape: 5 design families](/posts/llm-architecture-five-families-2026/). How KDA and Stable LatentMoE fit into the broader architecture trends. - [The open-source AI model landscape, June 2026](/posts/open-source-ai-model-landscape-june-2026/). Where K3 fits in the current ecosystem of open models. --- This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at hello@agenticup.dev