Chinese startup DeepSeek made headlines earlier this year when it released a reasoning model called R1 that managed to compete with AI models from major American tech companies, despite operating on a modest budget. Now, in collaboration with researchers from Tsinghua University, DeepSeek has published a paper detailing a new method in reinforcement learning, according to a report by SCMP.
The new approach focuses on aligning AI responses more closely with human preferences. It does so by rewarding models for producing more accurate and understandable answers. While reinforcement learning has been effective in specific domains, it has often struggled when applied to broader, more general problems. DeepSeek’s solution aims to overcome this by combining generative reward modeling (GRM) with principle-based self-criticism tuning.
Improved Reasoning with Fewer Resources
The newly proposed method reportedly enhances the reasoning capabilities of large language models (LLMs). According to the paper, this hybrid reinforcement learning approach outperformed existing methods in various benchmarks. It achieved the highest performance scores for general queries while consuming fewer computing resources, making it not only more effective but also more efficient, notes NIX Solutions.
The models developed under this framework are being called DeepSeek-GRM, short for Generalist Reward Modeling. DeepSeek has stated that these models will be open source, though no specific release date has been provided so far. We’ll keep you updated as more details become available.
What’s Next for DeepSeek and the Industry
In addition to this advancement, Reuters recently reported that DeepSeek plans to release DeepSeek-R2, the successor to its R1 reasoning model, sometime in April. This follows a trend among major AI developers—including China’s Alibaba Group Holding and San Francisco-based OpenAI—who are also working to enhance the reasoning and self-improvement capacities of their models, as noted by Bloomberg.
With ongoing innovation and increasing competition, the field of AI reasoning continues to evolve rapidly. Yet we’ll keep you updated as more integrations and developments emerge from DeepSeek and others in the industry.