News / All News / Article

Penguin Geeks | A Brief Introduction to the Principles Behind DeepSeek-R1

Article images may contain Chinese text.

1. Deep Thinking: The Gateway to AI Reasoning



In recent years, large language models (LLMs) have demonstrated extraordinary capabilities through pretraining on vast amounts of data. Yet as models continue to grow, the human-generated corpora used to train them are gradually approaching their limits. Remarks made six months ago by former OpenAI Chief Scientist Ilya Sutskever proclaimed the end of the pretraining era, suggesting that the traditional approach of improving model capabilities simply by increasing parameter counts is gradually losing ground.

Image

In Thinking, Fast and Slow, psychologist Daniel Kahneman divides human thinking into two modes: [System 1], which represents unconscious, fast thinking, and [System 2], which represents rational, slow thinking. LLMs operate more like fast thinking, rapidly retrieving information from vast stores of knowledge, but complex problems call for the deeper thinking of [System 2]. From reports of OpenAI's mysterious Q-star project to the subsequent launch of o1, the world's first large model for deep reasoning, these developments marked AI's entry into an era of 'slow thinking.' Rather than relying on immediate responses, models solve problems through multistep reasoning, much as humans do. Amid this transformation, DeepSeek-R1 burst onto the scene, not only challenging technological monopolies through its open-source approach but also demonstrating to the world another possibility for reasoning capabilities through 'pure reinforcement learning.'

Image

2. Pure Reinforcement Learning: AI's 'Wilderness Survival' Adventure



When tackling the challenge of deep reasoning, traditional technical approaches still rely on supervised fine-tuning (SFT), which requires large amounts of manually annotated chain-of-thought data, also known as reasoning-process data. This is expensive and offers limited generalization. The DeepSeek team, however, chose a counterintuitive path: using reinforcement learning (RL) alone to elicit the model's autonomous reasoning capabilities. Without process supervision, training relies solely on feedback from the outcomes of policy evaluation. It is like a baby learning to walk through countless attempts and falls, without demonstrations or guidance from adults, relying only on its own feedback and adjustments. This is the training logic behind DeepSeek-R1-Zero.

Through rule-based reward signals such as answer accuracy and adherence to formatting requirements, the model continually optimizes its policy during RL and independently generates long sequences of reasoning steps. Interestingly, although the reward rules do not explicitly encourage 'self-verification,' reflective behavior emerges spontaneously in the model, including human-like 'aha' moments. These remarkable results have inspired curiosity and further exploration. In replication tests, some research institutions found that self-reflective patterns were already present at the initial stage of RL—that is, in the base model, DeepSeek-V3-Base—and called this superficial self-reflection (SSR). Although self-reflection in this setting does not necessarily lead to a correct final answer, the long-form reasoning capability that the model gradually learns stems from carefully designed RL optimization. By maximizing the overall reward, the model naturally develops efficient and robust behavioral policies.


Image
Benchmark Performance of the DeepSeek-R1 Model


This ability to 'learn without a teacher' also comes at a cost: the model can become trapped in repetitive loops, generate lengthy, unintelligible content, and even mix Chinese, English, and other languages in its output. To address these issues, DeepSeek-R1 introduces multistage training. It first uses a small amount of cold-start data for SFT to guide the model's basic reasoning capabilities, then optimizes deep reasoning through reinforcement learning. During the convergence stage, rejection sampling is used to select high-quality data, after which multitask data is used to integrate general capabilities. Finally, RL across all scenarios aligns the model with human preferences, ultimately producing exceptional performance comparable to o1 on tasks such as mathematics and coding.


Image

Chain of Thought in DeepSeek's Large Reasoning Model


Traditionally, chain of thought has been regarded as synonymous with rigorous AI logic, designed specifically for analytical tasks such as mathematics and coding. Yet in the final stage of RL, DeepSeek-R1 uses not only high-quality reasoning-process data—approximately 600,000 examples—but also carefully selected general-task data—approximately 200,000 examples—to strengthen the model's generalization on general tasks. After a period of use and practical experimentation, people have been pleasantly surprised to find that deep reasoning also performs impressively on general tasks involving language and writing. Through its 'slow thinking' process, the model simulates key elements of human creativity when generating text: planning and iteration. For example, when imitating classical Chinese writing styles, the model must balance rhyme, allusions, and the integration of modern meaning. This does not depend solely on inspiration and intuition; it requires breaking down the target style, designing an expressive framework, and repeatedly refining details. DeepSeek-R1's chain-of-thought mechanism gives it a 'drafting' capability for such complex tasks, using multistep reasoning to anticipate effects and adjust wording, ultimately producing writing with greater coherence and stylistic consistency.
Image

3. Open Source and Transparency: Breaking AI Hegemony



DeepSeek-R1's dramatic arrival is significant not only for its disruptive technological innovation but also for its open attitude toward sharing through open source. From the model architecture and training code to the full weights of its 685 billion parameters, as well as six distilled models based on open-source foundation models such as LLaMA3 and Qwen2.5, the DeepSeek team has made every detail public. This candor and openness have earned widespread praise and an enthusiastic response from research institutions and developers on small and midsize teams around the world. DeepSeek-R1's improved mixture-of-experts (MoE) model increases the efficiency of load balancing across experts through a lossless auxiliary strategy. Its multi-token prediction (MTP) technology significantly increases decoding speed during inference, accelerating the breakdown of complex problems. Together with its low-level operator optimization capabilities, these advances demonstrate its deep technical expertise and outstanding capacity for innovation.

Image

No wonder people online call DeepSeek the truly 'open AI'!


Inspired by DeepSeek-R1's open-source release, the community quickly came together, producing a series of outstanding projects that reproduce R1's training process, including SimpleRL, OpenR1, LogitRL, and TinyZero. Even prominent research institutions, including teams at Stanford led by Fei-Fei Li and at Washington, successfully distilled Google's Gemini-2.0-Flash-Thinking model at an extremely low cost of approximately US$50, achieving performance close to that of leading reasoning models. These results fully demonstrate the strong scalability and broad application prospects of DeepSeek-R1's technology, offering a new paradigm and pathway for the democratization of AI: technological breakthroughs are no longer monopolized by giants but become the jointly created achievements and shared resources of collective intelligence.
Image

4. Lessons and Outlook: The Boundless Possibilities of AI's Future



DeepSeek-R1's practical results reveal an important conclusion: reinforcement learning can not only optimize answers but also elicit a model's 'metacognition.' When reward mechanisms are designed appropriately, AI spontaneously explores more efficient reasoning strategies, and this emergent behavior extends far beyond the boundaries humans have predefined. However, the pure RL approach places extremely high demands on computing power and data quality, while the unpredictability of model behavior remains a potential risk.

In the future, as more teams join this exploration, the convergence of 'reinforcement learning + reasoning' may give rise to a new generation of artificial general intelligence. Perhaps DeepSeek-R1's significance is much like the image evoked by its name: a whale-like spirit finding its way through the deep sea, lighting a beacon that guides the industry toward the unknown.























Image