On the same day, DeepSeek officially announced that, after releasing a series of large models, the TileLang compilation tool, and an inference operator library, it is now fully open-sourcing its own internal inference framework, DeepInfer. What the company wants to fill in is a link that has long stayed out of the spotlight—the inference framework.
In DeepSeek's view, the upper limit of a model's capability is determined by the model itself, but "how to supply ultra-large-scale models at a reasonable cost" relies on this intermediate architecture. Training lets a model learn; inference is what puts it to use. When massive numbers of users make calls simultaneously, the inference framework is responsible for orchestrating and scheduling vast, complex computing clusters, and its efficiency directly moves the compute costs and power consumption of AI services—it doesn't grab attention like model weights do, yet it is the critical gate for bringing laboratory capabilities into the real world.
Open source means more than giving away code—it must also drive down the price of using AI
DeepSeek's intent is clear: lay out its internal deployment experience to help developers and enterprise organizations improve the throughput and latency of large-scale inference, so they can build private or public cloud deployments more efficiently, cutting compute costs and burning less electricity. In a word: make intelligence reach more people at a cheaper price, delivering "truly accessible large models for all".
This framework is not just talk. Based on DeepInfer, DeepSeek has officially run a high-throughput, low-latency large-scale inference service for DeepSeek V4.1 Flash; going forward, inference code for new models and subsequent tuning will also be continuously updated here. Judging from the publicly listed capabilities, DeepInfer integrates high-performance operators and a deeply optimized inference engine, supporting features such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and it is simultaneously compatible with both NVIDIA and Huawei Ascend platforms—at a time when export controls are pushing domestic substitution, this "dual-stack" compatibility is especially practical for domestic developers.
Open-sourced alongside the framework is deepseek-recipe, a production-grade token encoding/decoding component whose processing logic is fully aligned with DeepSeek's official API, ensuring the community gets an experience identical to the official deployment. The two codebases are hosted on GitHub at deepseek-ai/DeepInfer and deepseek-ai/deepseek-recipe respectively.
From models to operator libraries to inference frameworks, DeepSeek is moving its full toolkit for "building models" into the open-source community. When compute cost has become the biggest barrier to AI adoption, handing over the efficiency core may move the ecosystem more than simply releasing a few weights—after all, whoever can help the industry squeeze more output from every kilowatt-hour and every card holds the real leverage in the story of accessible AI for all.