Kimi K3 Pushes Open AI Models to 2.8 Trillion Parameters

Kimi has introduced Kimi K3, a native multimodal model with 2.8 trillion parameters, a context window of up to one million tokens, and built-in vision capabilities. The company describes it as the first open model in the roughly three-trillion-parameter class and its most capable system so far. It is intended for demanding work that unfolds over long periods, including software engineering, research, knowledge tasks, and complex reasoning. Kimi acknowledges that K3 still falls behind the strongest proprietary systems overall, but says it reached frontier-level results across its evaluation suite and surpassed the other tested models consistently.

The model is available through Kimi.com, Kimi Work, Kimi Code, and the Kimi API. At launch, it operates with maximum reasoning effort by default. Lower- and higher-effort options are planned for later updates, which should give users more control over the trade-off between speed, computational cost, and answer quality. Kimi is coordinating with inference providers and open-source maintainers to align implementation details before the wider ecosystem rollout. The company says the complete model weights will be published by July 27, 2026, while a forthcoming technical report will provide fuller information about the architecture, training process, and evaluations.

K3 continues Kimi's effort to push the size limit of openly released models. According to the company, Kimi systems defined the upper boundary of open-model scale during nine of the previous twelve months. Size alone, however, is not presented as the main advance. K3 incorporates Kimi Delta Attention and Attention Residuals, two architectural changes intended to move information more effectively across both long sequences and many network layers. It also expands the sparsity of its Mixture-of-Experts design: only 16 of 896 specialized expert components are activated for a given computation, alongside a Stable LatentMoE framework. This selective activation is meant to provide enormous total capacity without using every parameter for every token.

Combined with revised data and training methods, these changes reportedly deliver about 2.5 times the overall scaling efficiency of Kimi K2. In practical terms, Kimi claims K3 turns a given amount of computing power into useful capability more efficiently than its predecessor. Its million-token context and architectural changes are particularly relevant to long-running tasks, where the model must retain and connect information from large codebases, lengthy documents, tool outputs, and earlier decisions.

Kimi emphasizes K3's ability to work through extended engineering assignments with limited human supervision. The model can reportedly maintain long coding sessions, explore very large repositories, and coordinate terminal tools. Its native vision system also lets it combine software development with visual feedback. K3 can examine screenshots or other visual material while refining games, front-end interfaces, computer-aided design projects, and interactive 3D experiences. The claimed “vision in the loop” workflow means the model can write code, inspect the rendered result, identify visual or functional problems, and iterate rather than relying only on textual descriptions.

One evaluation gave models as long as 24 hours in identical sandboxes to optimize four GPU-kernel tasks. These covered workloads related to Attention Residuals, Kimi Delta Attention, and a multi-head latent-attention kernel with a head dimension of 512, running on NVIDIA H200 hardware and a general-purpose GPU from another vendor. Kimi reports that K3 was competitive with Claude Fable 5 under an evaluation that may have involved fallback behavior, while substantially outperforming Opus 4.8, GPT 5.6 Sol, and GPT 5.5. The company notes that Fable 5's figures came from a third party and that some model trajectories used small reductions in numerical precision that remained inside the accepted tolerance. An early K3 version also performed most of the Kimi team's kernel-optimization work during the final stages of development.

The company then tested whether the model could construct an entire GPU programming system rather than merely improve isolated kernels. K3 created MiniTriton, a compact compiler modeled on Triton's general approach. It included a tile-level intermediate representation built over MLIR, compiler optimization passes, a PTX generation pipeline, a domain-specific language frontend, and a runtime. On the supported roofline benchmarks, Kimi says MiniTriton matched or exceeded Triton and torch.compile, including victories over Triton on some workloads.

MiniTriton was also tested on an end-to-end nanoGPT training workload. The system reportedly maintained stable convergence, with its loss curve staying close to the reference despite minor divergence. This matters because realistic training validates the interaction of the compiler's complete pipeline, whereas a fast result on a narrow microbenchmark may not prove that the overall system works reliably. Kimi argues that the experiment demonstrates K3's capacity to design a coherent compiler from its programming interface through intermediate transformations, machine-code generation, and execution. Its newly built Tensor Core path reportedly already competes with Triton's extensively optimized implementation.

A separate proof of concept extended the model's work from software into hardware design. During one autonomous run lasting 48 hours, an early K3 designed, optimized, and verified a chip intended to run a small model based on its own architecture. It used open-source electronic-design-automation tools and the Nangate 45-nanometer library. In simulation, the resulting design fit within four square millimeters, met timing at 100 MHz, and exceeded 8,700 decoded tokens per second. It contained 1.46 million standard cells, 0.277 MB of SRAM, and an INT4 multiply-accumulate array with integrated dequantization. The experiment is presented as evidence that a single model can connect reasoning, coding, visual inspection, compiler construction, and chip design across unusually long autonomous workflows. These results remain company-reported demonstrations, and the technical report and open-weight release will be important for independent verification.