Home › News › DeepSeek Open-Sources DeepInfer

DeepSeek Fully Open-Sources Its Internal Inference Framework DeepInfer, Aiming to Make Large Models Accessible to All

Published: 2026-10-20  |  Category: Open Source  |  Read: about 5 min

On October 20, 2026, DeepSeek officially announced that it is fully open-sourcing its internal inference framework DeepInfer, together with the production-grade token codec component deepseek-recipe. This is another major move from DeepSeek after open-sourcing its models, aimed squarely at "broad access to large models"—making global AI services fairer, more transparent and more trustworthy, and delivering more powerful intelligence at a cost everyone can afford.

Repositories

DeepInfer: Core Capabilities of a High-Performance Inference Framework

DeepInfer is a high-performance inference framework that DeepSeek has long used internally, and it already powers the high-throughput, low-latency large-scale inference service for DeepSeek V4.1 Flash. Fully open-sourcing it now means developers around the world can obtain the same inference optimization capabilities that DeepSeek uses in production.

Core Features

  • High-performance operator integration: a deeply optimized inference engine with targeted optimizations for large-model inference workloads
  • Distributed inference support: native support for multi-node, multi-GPU deployment, easily handling ultra-large models
  • Speculative Decoding: significantly faster generation with lower inference latency
  • SWA Bounded Replay: optimized memory usage and compute efficiency in long-context scenarios
  • Structured output: supports JSON Schema and other structured output formats for enterprise-grade applications
  • Multi-platform compatibility: supports both NVIDIA and Huawei Ascend platforms
Technical highlight: DeepInfer's speculative decoding can speed up generation by 2-3x, while SWA Bounded Replay optimization in long-context scenarios cuts memory usage by more than 40%. These optimizations are critical to lowering inference costs.

deepseek-recipe: A Production-Grade Codec Component

The companion open-source deepseek-recipe is the production-grade token codec component DeepSeek uses internally, fully aligned with the processing logic of the official DeepSeek API. That means community developers get an API experience identical to the official deployment.

Core Value

  • Full alignment with official logic: ensures community versions behave consistently with the official API
  • Production-grade stability: validated in DeepSeek's internal large-scale production environment
  • Easy integration: clear API interfaces and documentation for fast integration into existing systems

Multi-Platform Support: NVIDIA + Huawei Ascend

One of DeepInfer's highlights is support for both NVIDIA and Huawei Ascend. Under the current international circumstances, this strategy matters a great deal:

  • NVIDIA platform: supports mainstream GPUs such as A100, H100, H200 and Blackwell
  • Huawei Ascend platform: supports domestic AI chips such as Ascend 910B and Ascend 950DT

This dual-platform strategy not only reduces developers' dependence on a single hardware vendor, but also gives strong support to the development of the domestic compute ecosystem. Combined with DeepSeek's previously open-sourced Ascend adaptation components (TileLang for Ascend, DeepGEMM-Ascend, DeepEP-Ascend and others), DeepSeek is building a complete domestic compute inference ecosystem.

Industry significance: The full open-sourcing of DeepInfer marks an important step for domestic large models from "open-sourcing models" to "open-sourcing inference infrastructure." It will substantially lower the barrier for SMEs and developers to deploy high-performance large models and accelerate the spread of AI technology.

Future Plans: Continued Open-Sourcing of New Model Inference Code

DeepSeek made it clear in its announcement that inference code for future new models and subsequent performance optimizations will also be continuously open-sourced in DeepInfer. That means DeepInfer will be a long-term, continuously evolving open-source project rather than a one-off release.

For developers, this is an important signal:

  • You can build long-term stable inference services on DeepInfer
  • You can get inference optimization support for new models as soon as they arrive
  • You have the opportunity to contribute to the community and help move the framework forward

Why Some Components Are Not Yet Open-Sourced

The DeepSeek team also candidly explained that the KVCache management and load balancing components in the inference system are deeply coupled with internal infrastructure, so the relevant code is not yet fully open-sourced.

Note: Although core inference capabilities are fully open-sourced, some components deeply coupled with DeepSeek's internal infrastructure are not yet available. Developers may need to adapt and optimize them for their own scenarios.

Industry Reaction: The True OpenAI Spirit

DeepSeek's open-source move has drawn a positive response from the industry. Several insiders described it as "the true OpenAI spirit"—open-sourcing not just the models, but the core infrastructure that keeps them running.

Against the backdrop of white-hot competition in large models, DeepSeek's choice to open-source its core internal inference framework reflects a firm commitment to broad technology access. As its announcement puts it:

"With fully open-sourced models, operator libraries and inference frameworks, we look forward to a not-too-distant future in which the reach and quality of AI services worldwide expand and improve as never before—becoming fairer, more transparent and more trustworthy. That will accelerate technology iteration across society, reduce energy waste, and let limited resources flow more efficiently between infrastructure and end-user devices—delivering more powerful intelligence at a cost everyone can afford, along with the goodwill and boundless possibility that come with intelligence."

Practical Impact for Developers

For developers at large, the practical value of open-sourcing DeepInfer includes:

1. Lower Deployment Costs

Through speculative decoding, memory optimization and other techniques, inference costs on the same hardware can drop by 30%-50%.

2. Better Service Performance

Distributed inference and high-performance operator support make large-scale concurrency possible and significantly improve service response times.

3. Greater Self-Reliance

Support for the domestic Ascend platform reduces dependence on NVIDIA, which carries important strategic significance under current international circumstances.

4. Faster Technology Iteration

The power of the open-source community will drive rapid framework evolution, and developers can obtain the latest optimization techniques immediately.

Conclusion

DeepSeek's full open-sourcing of the DeepInfer inference framework is an important milestone in the history of domestic large models. From open-sourcing models to open-sourcing inference infrastructure, DeepSeek is building a complete open-source ecosystem for large models. This not only lowers the barrier to using AI technology, but also injects strong momentum into the domestic compute ecosystem.

For developers, now is the best time to embrace DeepInfer. Whether you are building enterprise-grade AI services or exploring innovations in large-model applications, DeepInfer will be a trustworthy choice.

— Curated and published by AICat —

Related Links

← Back to news list