https://beaiera.top/tags/scaling-factor/Recent content in Scaling Factor on Richelieu's BlogHugozh-cnMon, 01 Jun 2026 11:40:00 +0800https://beaiera.top/posts/deepseek-routed-scaling/Mon, 01 Jun 2026 11:40:00 +0800https://beaiera.top/posts/deepseek-routed-scaling/<h2 id="背景">背景</h2> <p>DeepSeek-V2/V3 系列模型采用了 MoE（Mixture of Experts）架构，其中 <code>routed_scaling_factor</code> 是一个重要的超参数，用于缩放 routed expert 的输出。该系数来自模型 config，在 <code>DeepseekV2MoE.__init__</code> 中初始化：</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="bp">self</span><span class="o">.</span><span class="n">routed_scaling_factor</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">routed_scaling_factor</span> </span></span></code></pre></div><p>默认值通常为 <code>1.0</code>，但 DeepSeek-V2 系列（如 deepseek-v2、deepseek-coder-v2）设置的典型值是 <code>2.5</code> 或 <code>1.0</code>，取决于具体子模型。</p> <h2 id="控制开关">控制开关</h2> <p>在 vLLM 的 <code>deepseek_v2.py</code> 中，关键代码如下：</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">apply_routed_scale_to_output</span> <span class="o">=</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">is_rocm_aiter_moe_enabled</span> </span></span><span class="line"><span class="cl"> </span></span><span class="line"><span class="cl"><span class="n">routed_scaling_factor</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">routed_scaling_factor</span><span class="p">,</span> </span></span><span class="line"><span class="cl"><span class="n">apply_routed_scale_to_output</span><span class="o">=</span><span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">is_rocm_aiter_moe_enabled</span><span class="p">,</span> </span></span></code></pre></div><p>这个 bool 值决定了 <code>routed_scaling_factor</code> 由谁处理——是 kernel 内部还是 runner 外部。</p>