DeepSeek Open-Sources DeepInfer Framework for Efficient AI

DeepSeek has open-sourced DeepInfer, the in-house framework for running its own models. The move reveals some of the technical secrets behind how the Chinese AI developer achieves high efficiency and low compute costs when running its models at scale. #🇨🇳 ChinA.I. 🤖🧠🦾🤖 Access to DeepInfer is likely to help third-party AI inference platforms that host DeepSeek models, such as Fireworks and Baseten, run DeepSeek’s latest models more efficiently and earn higher margins. But DeepInfer could also have a broader impact. Other AI labs in China and the U.S. could study DeepSeek’s open-source framework and learn how to increase the inference efficiency of their own models, developers say. DeepInfer supports Nvidia’s Blackwell chips as well as Huawei’s Ascend 950 chips, DeepSeek said. DeepInfer, however, doesn’t reveal everything. DeepSeek, for example, hasn’t open-sourced some of its proprietary techniques, such as those for managing memory and controling how the model distributes traffic across servers. While DeepSeek’s revenue is minuscule compared to OpenAI and Anthropic, the Chinese company is achieving healthy margins. Its gross margin for the business of selling access to its models in the first seven months of this year was 82.9%, higher than both Anthropic and OpenAI during similar periods https://github.com/deepse..