共 60 个 commit，涉及 217 个文件，+7106/-4113 行变动。

概要

统计项	数值
Commit 数	60
变更文件	217
新增行数	+7106
删除行数	-4113

Commit 列表

📦 Other

• abc7154 #46831 — [CI/Build][CPU] Add test image cache clean-up (#46831)
  • 作者: Li, Jiang | +38/-0 | 1 个文件

  • Add image build cache clean-up to reduce issue. ## Test Result —

• 4e07ca2 #44800 — [Core] Add VLLM_GPU_SYNC_CHECK env var (#44800)
  • 作者: Nick Hill | +279/-0 | 5 个文件

  vLLM now uses asynchronous scheduling by default and in the majority of cases. Performance relies on the absence of any gpu<->cpu synchronizations on the main cuda stream, but such syncs can be opaque and it is easy for them to creep in accidentally. This change adds a VLLM_GPU_SYNC_CHECK env var which enables torch.cuda.set_sync_debug_mode for the model forward pass and sampler, so that we can ea…

• e71bc6d #46799 — [Rust Frontend] Use oss-harmony for Harmony output processing (#46799)
  • 作者: Bugen Zhao | +51/-427 | 2 个文件

  Switch the library used for Harmony output processing from Inferact/openai-harmony (introduced in #46696) to oss-harmony, a more actively maintained fork of the original library. This gives the Rust frontend two practical benefits: - picks up upstream Harmony bug fixes and improvements - embeds the tokenizer vocabularies in the library itself, removing the runtime download path and the related req…

• 8921c4b #46122 — [ROCm] [Performance] Optimize aiter moe for DeepSeekV4 (#46122)
  • 作者: TJian | +28/-15 | 1 个文件

  NOTE: This feature is validated with aiter v0.1.15.post1, we require the AITER on upstream to be updated. Mxfp4MoeBackend.AITER_MXFP4_BF16 is introduced only for DeepSeek V4 original weights. It follows the speed of light reference (ATOM repo) https://github.com/ROCm/ATOM/blob/d7964d50be17a3910dec1d22cf1d4f6205764cb4/recipes/mesh/multi-node-atom.md?plain=1#L74 in preparing the weights. 1. lmeval b…

• 8e39424 #46419 — [ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4 (#46419)
  • 作者: qli88 | +27/-8 | 4 个文件

  [ROCm][feature] Enable MiniMax-M3-MXFP4 with AITER MoE This feature requires AITER version bump (latest version). Accuracy test: 1. vLLM server start: VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 VLLM_USE_BREAKABLE_CUDAGRAPH=0 vllm serve /data/amd-MiniMax-M3-MXFP4/ –block-size 128 -tp 8 –attention-backend TRITON_ATTN –tool-call-parser minimax_m3 –enable-auto-tool-choice –reasoning-pars…

• 950ee4c #44226 — [API] Add token offsets to render endpoints (/v1/…/render) (#44226)
  • 作者: Hyunkyun Moon | +516/-20 | 13 个文件

  vLLM’s render endpoints (/v1/completions/render and /v1/chat/completions/render) are GPU-less routes that external serving systems layered on top of vLLM use to fetch preprocessing results (rendered token IDs, chat-template-applied prompts, etc.). Typical consumers include inference orchestration / scheduling layers like llm-d that need to make prefix-cache-aware routing or load-balancing decision…

• d980a3c #46780 — [ROCm] Fix AITER_UNIFIED_ATTN Dispatching After AITER Bump (#46780)
  • 作者: Micah Williamson | +30/-3 | 6 个文件

  After bumping AITER to v0.1.16.post2, we are seeing failures such as the following: Here we update the AITER_UNIFIED_ATTN backend to specify its compatible dtypes.

• bf292b5 #46071 — [Docs] Remove BambaForCausalLM from supported hybrid models list (#46071)
  • 作者: AgenticSpark | +1/-1 | 1 个文件

  #45990 removed BambaForCausalLM: vllm/model_executor/models/bamba.py was deleted and the model moved into _PREVIOUSLY_SUPPORTED_MODELS in registry.py. docs/usage/v1_guide.md still lists BambaForCausalLM among the supported hybrid models, so this drops it from that list. This PR originally also repointed a stale BambaForCausalLM reference link in docs/contributing/model/basic.md, but that line was …

• c7645bc #46706 — Remove grok model arch from vllm (#46706)
  • 作者: Tiezhen WANG | +3/-1401 | 13 个文件

  Background We are running a sprint to reduce the technical debt of the vLLM code base starting by removing obsolete model architectures based on vLLM internal usage tracking. The usage of both Grok models is very low, less than 10 hours in the past 3 months. ## Test Result —

• 5b33041 #46773 — [ModelRunner V2] Fix whisper test (#46773)
  • 作者: Nick Hill | +6/-3 | 2 个文件

  Add len method to work with existing test: https://github.com/vllm-project/vllm/blob/a2e8ec3d52ab4e163501c8c7bee8c03ca8359a7a/tests/models/multimodal/generation/test_whisper.py#L363 — —

• e312c5c #46507 — [Rust Frontend] Make Granite4 string argument scanning incremental (#46507)
  • 作者: Reid | +256/-33 | 4 个文件

  Granite4 tool calls can encode arguments either directly as an object or as a JSON string containing the serialized arguments: The object form already carries scan state across streaming chunks. The string form was still parsed with the one-shot json_str helper, which cannot remember how much of an incomplete string it has already scanned. When a long string-valued argument arrived over many chunk…

• 1502cf6 #45263 — Fix relative allowed local media paths (#45263)
  • 作者: Matti4 | +18/-1 | 2 个文件

  Fixes handling of relative allowed_local_media_path values in MediaConnector. Previously, the allowed path could remain relative while file URLs were resolved to absolute paths, causing valid local media files to be rejected. This resolves the allowed local media path during connector initialization and adds a regression test covering sync and async local image loading.

• 552a9db #44667 — [NVFP4][Emulation] Fuse NVFP4 weight dequantization with compute in triton kernel for w13/w2 MOE MLP linears (#44667)
  • 作者: fxmarty-amd | +906/-68 | 3 个文件

  On platforms without native NVFP4 hardware support (e.g. AMD MI350), NVFP4 MoE weights are currently dequantized for MOE emulation in BF16. The previous implementation from https://github.com/vllm-project/vllm/pull/35737 https://github.com/vllm-project/vllm/pull/40033 dequantized the full weight tensors w13 and w2, including ALL experts, before calling the standard Triton MoE GEMM kernel through T…

• 02a1f23 #46761 — [DFlash] Fuse precompute kv per-layer rmsnorms (#46761)
  • 作者: Giancarlo Delfin | +108/-18 | 3 个文件

  Context Currently in DFlashQwen3Model during precompute_and_store_context_kv we invoke rms_norm for each layer in a loop, resulting in multiple kernel launches. This is done because rms_norm only supports broadcasting the single [hidden_size] weights to all rows of the input. # This PR Adds support to rms_norm for passing in a weight of shape [num_rows, hidden_size], which enables applying a dif…

• 652d962 #46448 — [Model Runner V2][Spec Decode] Reduce TP communication for draft token generation (#46448)
  • 作者: yiheng | +34/-3 | 1 个文件

  use_local_argmax_reduction is already supported in the V1 speculative decoding path (SpecDecodeBaseProposer), but Model Runner V2 draft sampling in DraftModelSpeculator still always calls compute_logits() and all-gathers full vocabulary logits before argmax. This PR wires the existing config flag into the MR V2 greedy draft path via get_top_tokens(), reducing TP communication from O(vocab_size) to…

• 5314665 #46770 — [Model Runner V2][DFlash] Enable dflash attention backend selection (#46770)
  • 作者: Giancarlo Delfin | +3/-1 | 1 个文件

  Context Currently, the speculative config’s attention backend is not applied to the DFlash drfat model in Model Runner V2. The consequence is that if the attention backend defaults to FlashInfer, I get the error: Currently, DFlash only supports flash attention, so we need a way to override it via the speculative config. # This PR Simply passes the speculative config attention backend to the DFla…

• 32bb319 #46746 — [ModelRunner V2] Bound memory for large logprobs requests (#46746)
  • 作者: Nick Hill | +18/-10 | 1 个文件

  We support passing num logprobs as -1 which means entire vocab. In this case, the temp MRV2 kernel was exploding due to PADDED_TOPK=triton.next_power_of_2(num_logprobs) .

• ae7c8ec #46696 — [Rust Frontend] Switch rustls to native-tls/OpenSSL (#46696)
  • 作者: Bugen Zhao | +89/-441 | 8 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Related discussions: - #46052 @wseaton - https://github.com/vllm-project/vllm/pull/45890#issuecomment-4783509287 @tahsintunan Eliminate rustls from the Rust frontend dependency tree and keep the HTTP/TLS stack on native-tls / OpenSSL, which is friendlier for compliance-sensitive deployments. Also adds a cargo-deny bans check for the crypto provider crate…

• 1d3f4cb #46719 — [Rust Frontend] Extract renderer fixture test utilities (#46719)
  • 作者: Bugen Zhao | +288/-346 | 9 个文件

  Native Rust chat renderers currently maintain renderer-specific JSON fixture DTOs for snapshot tests. This PR extracts a shared fixture loader for renderer tests and migrates the existing DeepSeek V3.2/V4 fixture tests to it, so future native renderers can reuse the same fixture format. It also makes two small renderer-facing improvements: - Allow multimodal prompt finalization to consume already-…

• f9e6844 #46602 — [Rust Frontend] Migrate gemma4 to unified parser (#46602)
  • 作者: Bugen Zhao | +1000/-394 | 23 个文件

  Migrate the Gemma4 Rust reasoning/tool parsing path to the unified parser interface introduced in #46583, by extending the winnow-based Gemma4 tool parser with reasoning states and events. Gemma4’s output syntax uses the same protocol stream for reasoning and tool calls. A native unified parser handles that shape more directly: - <|channel>thought\n is consumed as the atomic reasoning opener inste…

• c53994e #46665 — [Model Runner V2][Spec Decode] Use log1p to compute residual during rejection sampling (#46665)
  • 作者: Giancarlo Delfin | +7/-5 | 1 个文件

  Context target_log_probs + tl.log(1 - ratio) is currently used when calculating the log-residual while resampling rejected tokens. tl.log(1-x) is lossy because when x is small (less than 1 ULP), 1 - x gets rounded to 1.0, and tl.log(1.0) => 0. So for small x, you get a relatively large error. The consequence is that when q(x)/p(x) is small, meaning that the target model prefers a token more than…

• c5e3c40 #46628 — Fix P/D with DP Supervisor (#46628)
  • 作者: Robert Shaw | +2/-2 | 1 个文件

  SUMMARY: * prior to this PR, each DP rank in DP Supervisor mode ended up with the same engine_id * this caused garbage GSM8k

• e53a172 #46692 — [ROCm]: Bump aiter to 0.1.16.post2 (#46692)
  • 作者: Rohan Potdar | +2/-2 | 1 个文件

  AITER 0.1.16.post2; perf uplifts+bugfixes for gpt-oss, DSV4, Minimax M3, etc. ## Test Result —

• d490b98 #45237 — [Core] Avoid mixed length specdec batches via padding (#45237)
  • 作者: Yiliu Dong | +120/-0 | 2 个文件

  In P/D disaggregation with speculative decoding enabled, a request that arrives at the decode node via KVConnector has already had most of its prompt KV transferred and normally has only the residual decode-side token left to compute. Existing decode requests schedule 1 + N tokens with MTP/EAGLE/…, while the newly transferred request schedules only 1 token. That creates a non-uniform batch shape…

• 1744adc #45666 — [ROCM] [Communication] Add INT3 quantization method for quickreduce (#45666)
  • 作者: haoyangli0109 | +264/-20 | 7 个文件

  1.According to our kernel experiments, INT3 demonstrated better performance under TP2 (see figure below), but under TP=4 and TP=8 conditions, the performance of INT3 and INT4 was very close. This is likely because, as TP increases, the number of GPUs that need to communicate with each other grows exponentially, thereby requiring more synchronization and scheduling time; In this scenario, synchroni…

• 6f3da46 #46093 — [Pooling] Fix Cohere embed billed image token accounting for mixed-content inputs (#46093)
  • 作者: Taneem Ibrahim | +6/-1 | 1 个文件

  This PR fixes a small but user-visible accounting bug in the Cohere-compatible /v2/embed response. Cohere embed requests can provide image input in more than one shape. The existing process already counted top-level images as image input, but it missed images provided inside mixed-content inputs[].content[] items: On current main, this request can succeed but the response metadata reports the prom…

• 9bfd878 #43461 — [MoE] [MoE Refactor] Add moe kernel oracle abc 37753 (#43461)
  • 作者: Qiuyang Yue | +280/-6 | 5 个文件

  First PR of the series suggested by @robertgshaw2-redhat in #37776: introduces the MoEKernelOracle abstract base class and migrates the unquantized oracle as the reference implementation. Addresses #37753 in part. The remaining oracles (fp8, nvfp4, mxfp4, mxfp8, int8, int_wna16 — now 7 oracles, up from the 4 enumerated in the issue body when it was filed) are left for follow-up PRs. ## Approach Th…

• 15be787 #45019 — [NIXL][Mamba] Add Mamba1 support to NIXL P/D disaggregation (#45019)
  • 作者: Asaf Gardin | +104/-35 | 5 个文件

  Add Mamba1 hybrid-model support (e.g. Jamba) to the NIXL connector’s conv-state transfer for prefill/decode disaggregation, extending the existing Mamba2/GDN path (#41869, #37635). ## Test Result Successfully passed - pytest tests/v1/kv_connector/unit/test_nixl_connector_hma.py -k derive_mamba_conv_split -v ### Accuracy Test ## Accuracy Results Model: ai21labs/AI21-Jamba2-Mini (Mamba1) **Bench…

• 9222148 #46702 — [CPU][CI/Build] Allow more CPU CI agents (#46702)
  • 作者: Li, Jiang | +2/-1 | 1 个文件

  • Add a identifier to support more CPU CI agents on a node. ## Test Result —

• 638b1a9 #45269 — [CPU][RISC-V] Add RVV path for W4A8 INT4 GEMM (#45269)
  • 作者: wcy | +157/-7 | 5 个文件

  This PR adds an RVV backend for the CPU INT4 W4A8 GEMM operator on RISC-V and wires the existing int4_scaled_mm_cpu path to dispatch to it. 1. Add an RVV backend for CPU W4A8 GEMM The CPU W4A8 operator already has an optimized AVX512 path for x86. This PR adds the corresponding RISC-V implementation using RVV, while keeping the existing W4A8 packed weight layout, scale/zero-point buffers, and 32-c…

• 72adb20 #46605 — [Model] Remove AquilaForCausalLM, AquilaModel (#46605)
  • 作者: Tiezhen WANG | +2/-7 | 4 个文件

  Removing AquilaForCausalLM and AquilaModel (BAAI Aquila/AquilaChat series) as they’re obsolete now. Confirmed with BAAI. Adds both architectures to _PREVIOUSLY_SUPPORTED_MODELS pointing to vLLM v0.23.0. ## Test Result —

• 2396d91 #44029 — [CPU][Spec Decode] Enable DFlash SD for CPU (#44029)
  • 作者: guybd | +266/-8 | 8 个文件

  Enable DFlash speculative decoding on the vLLM CPU backend. Srart server: benchmark: ## Test Result Without this change, DFlash speculative decoding cannot run on the CPU backend because the DFlash input expansion path depends on GPU/Triton behavior and the CPU attention backend does not expose the hooks needed to precompute/store DFlash context KV cache. With this change, DFlash starts successful…

• 9b215ae #44610 — [Rust Frontend] Forward VLLM_ENGINE_READY_TIMEOUT_S via --args-json (#44610)
  • 作者: Kai | +16/-12 | 1 个文件

  When using the Rust frontend, setting export VLLM_ENGINE_READY_TIMEOUT_S=1800 has no effect. Models that require extended JIT compilation during startup (e.g., DeepSeek with DeepGEMM) can easily exceed the default 600 s engine-ready timeout window — even when the user has explicitly raised the limit via VLLM_ENGINE_READY_TIMEOUT_S. The Rust frontend silently ignores this env var, causing the serve…

🧪 CI/Tests

• 37ce349 #46735 — [CI] Fix failing CUDA graph capture in Triton MOE (#46735)
  • 作者: fxmarty-amd | +6/-1 | 2 个文件

  Following a change in https://github.com/vllm-project/vllm/pull/46254 that used self.a1_scale or self.a1_gscale as moe_kernel_quantize_input argument in Triton MOE, graph capture started failing with: resulting in the test CUDA_VISIBLE_DEVICES=0 pytest tests/models/quantization/test_gpt_oss.py -s -vvvvv -k “test_gpt_oss_attention_quantization and amd/gpt-oss-20b-MoE-Quant-W-MXFP4-A-FP8-KV-FP8-0.89…

• 302954e #46823 — [ROCm] [CI] fix transcription flakiness AMD: Entrypoints Integration (API Server OpenAI - Part 1) (mi325_1) (#46823)
  • 作者: TJian | +1/-0 | 1 个文件

  Fix AMD: Entrypoints Integration (API Server OpenAI - Part 1) (mi325_1) https://buildkite.com/vllm/ci/builds/74553/list?sid=019f026d-b093-4a93-bacc-e23d26bcb2db&tab=output The language autodetection might have failed. So, we tell the whisper that it is english audio explicitly. ## Test Result —

• 1a49845 #46792 — [Hardware][AMD][CI] Fix AMD CI image build (#46792)
  • 作者: Matt | +0/-1 | 1 个文件

  This PR fixes the AMD CI image build, which was broken after https://github.com/vllm-project/vllm/pull/46761 because it left an unused variable in the layernorm kernels. On AMD CI, the vLLM wheel is compiled with -Werror and -Wunused-variable, so this led to a csrc compilation failure. AMD CI will build a CI testing image. ## Test Result The CI testing image is successfully built. —

• dbc49b6 #45166 — [CI][NIXL] Fix NIXL EP import canary for the nixl 1.3.0 wheel and pin nixl==1.3.0 (#45166)
  • 作者: ovidiusm | +4/-24 | 2 个文件

  Bump NIXL version to 1.3.0 . Simplify NIXL EP import canary to not rely on checking internals of installations, instead check if EP module is importable. requirements/kv_connectors.txt is in run_all_patterns, so editing it triggers the full NixlConnector P/D sweep; the -nixlconnector-pd-accuracy- jobs run the import canary first. ## Test Result - Repro: build #71192 on this branch (CUDA 13 i…

• 27da2a2 #46691 — [Hardware][AMD][CI] Use Triton-based AITER MHA for LM Eval Qwen-3.5 Models Tests (#46691)
  • 作者: Matt | +9/-6 | 3 个文件

  Use Triton-based (rather than CK-based) AITER MHA in the LM Eval Qwen-3.5 35B AITER test. This sidesteps the issue reported in https://github.com/vllm-project/vllm/pull/46520, and also reverts the changes made in that PR. pytest -sv evals/gsm8k/test_gsm8k_correctness.py –config-list-file=configs/models-qwen35-mi355.txt This is run as part of the LM Eval Qwen-3.5 Models test group on MI355 in AMD …

• a2e8ec3 #46736 — [CI] Depend GPQA Eval DGX Spark job on arm64 image build (#46736)
  • 作者: Michael Goin | +2/-0 | 1 个文件

  DGX Spark is ARM, so the GPQA Eval job needs the arm64 CUDA image rather than the x86_64 image-build inherited from the group-level depends_on. The job-level depends_on overrides the group default, fixing the recurring nightly failure where the job runs before the arm64 image is ready. AI assistance (Claude) was used. 🤖 Generated with Claude Code

• 2a6f8f0 #39238 — [ROCm][CI] Fine-tuning queues and test names (#39238)
  • 作者: Andreas Karatzas | +43/-56 | 6 个文件

  This PR makes names of tests more descriptive on upstream. Also it fixes inconsistencies in the AMD CI pipeline configuration. It ensures agent_pool suffixes match the GPU count implied by step labels and num_gpus values, and adds missing num_gpus fields where the agent pool already implies multi-GPU execution. cc @kenroche

• 96eb8dd #46671 — [CI] Re-enable skipped glm and seedoss parser tests (#46671)
  • 作者: Flora Feng | +2/-2 | 2 个文件

  Re-enable the skipped parser tests on CI since they are fixed on main.

• cdfa2fd #46729 — [ROCm][CI] rm duplicate Distributed Torchrun ci test (#46729)
  • 作者: Divakar Verma | +0/-24 | 1 个文件

  This PR removes duplicate test Distributed Torchrun + Shutdown Tests (2 GPUs) from MI250 as it already exists on MI300.

• d3130d8 #38290 — [CI] Pin GitHub Actions to commit hashes in macos-smoke-test.yml (#38290)
  • 作者: Russell Bryant | +2/-2 | 1 个文件

  Pin actions/checkout and astral-sh/setup-uv to their full commit SHAs instead of mutable version tags. This is a security best practice recommended by GitHub because version tags (like v7) can be force-pushed to point at arbitrary code, whereas commit hashes are immutable. Pinning to hashes prevents a compromised or hijacked action repository from silently injecting malicious code into CI runs. Th…

• 2365b7a #46668 — [Hardware][AMD][CI] Mirror Basic Models (Others) and Weight Loading Multiple GPU test groups (#46668)
  • 作者: Matt | +18/-6 | 3 个文件

  This PR gates the Basic Models (Others) test group and mirrors the (optional) Weight Loading Multiple GPU test groups on AMD CI. These test groups have been unblocked after the mem access fault fix in https://github.com/vllm-project/vllm/pull/46548. The above test groups will be tested on AMD CI as part of the PR review process. ## Test Result The above test groups pass. cc @AndreasKaratzas —

• a6f41ab #46674 — [XPU][CI]Refine .buildkite/ci_config_intel.yaml for Intel GPU CI (#46674)
  • 作者: xiangdong | +2/-3 | 1 个文件

  Refine .buildkite/ci_config_intel.yaml for Intel GPU CI ## Test Result —

⚡ Performance

• c2507fb #46545 — [ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model (#46545)
  • 作者: Hongxia Yang | +110/-20 | 5 个文件

  Perf optimization MiniMax-M3 runs its single shared expert as a separate dense MLP every MoE layer (a gate_up GEMM + activation + down GEMM on a side stream, ×60 layers). Folding it into the routed grouped GEMM removes those per-layer launches, which is the dominant cost at low/medium concurrency (decode is launch-bound). The fusion is numerically equivalent to the separate-MLP path. - **r…

• 8b4d93b #46651 — [Perf] Remove redundant clone for GLM, Deepseek etc (#46651)
  • 作者: Wentao Ye | +4/-4 | 4 个文件

  These clone are not needed As in rms norm, when residual is not None, we will return a output instead of doing it inplace Covered in CI The larger tensor is, the more perf we get.

• c63cd49 #46546 — [ROCm][ [Perf] sparse attention optimization on minimax-m3 (#46546)
  • 作者: Hongxia Yang | +1238/-36 | 5 个文件

  Extend the work of PR https://github.com/vllm-project/vllm/pull/46035 (gfx942 focus) to address gfx950 platform. Validated on gfx950 to ensure Perf win on minimax-m3-mxfp8 model without accuracy loss before filing up the PR. Co-Authored-By: @yueliu14 yue.liu4@amd.com ## Test Result Perf (8k/1k, concurrency 64): | metric | base | opt | Δ | |—|—:|—:|—:| | Output throughput | 1269.98 | …

🔩 Misc

• 5e3dad0 #46783 — [Misc] Move the legacy api_server.py to the examples directory. (#46783)
  • 作者: wang.yuqi | +6/-6 | 5 个文件

  Move the legacy api_server.py to the examples directory. Eliminate the confusion caused by vllm/entrypoints/api_server.py and vllm/entrypoints/openai/api_server.py. This example can still works. ## Test Result —

🐛 Bug Fix

• 63e161f #46486 — [Bugfix][Tool Parser] PoolsideV1: fix string whitespace and required named tool choice (#46486)
  • 作者: Joe Rowell | +247/-11 | 2 个文件

  This PR brings some internal fixes for the PoolsideV1 tool parser into mainline vLLM. The main differences are that we handle the XML tags properly even if JSON is requested, and we handle whitespace more intelligently. ## Test Result —

• 35a49fc #46749 — [CI][Bugfix] Spawn engine in mm cache sleep test to fix ROCm HIP error (#46749)
  • 作者: peizhang56 | +3/-0 | 1 个文件

  tests/multimodal/test_cache.py::test_sleep_wake_preserves_mm_cache_consistency (added in #43001 as a regression test for #42995) fails on ROCm CI with: Root cause: the test instantiates an LLM, which launches EngineCore via the default VLLM_WORKER_MULTIPROC_METHOD=fork start method. When the test runs after other tests in the same process (it is collected into the cpu_test multimodal CI step, wher…

• 915e99e #46741 — [ROCm][Bugfix] Fix HIP fork re-init in multimodal offline examples (#46741)
  • 作者: peizhang56 | +14/-5 | 1 个文件

  The AMD/ROCm “Examples” CI step (.buildkite/test-amd.yaml, mi300 and mi355) fails with: vision_language_offline.py and vision_language_multi_image_offline.py import transformers before vllm. On ROCm, importing transformers calls torch.cuda.is_available(), which (default path) runs cuInit and creates a HIP context in the parent process ~@~T bb efore vllm sets PYTORCH_NVML_BASED_CUDA_CHECK=1 in vllm…

• d350fa8 #46733 — [Bugfix][Rust Frontend] Reject min_tokens above max_tokens (#46733)
  • 作者: Reid | +153/-28 | 6 个文件

  Rust frontend request paths did not consistently reject min_tokens > max_tokens. Python SamplingParams already treats this combination as invalid, and the Rust chat completions request type had a route-level check. However, other Rust paths such as /v1/completions and /generate could still reach the shared text lowering layer with an impossible sampling configuration: - max_tokens=4: generate at m…

• 3daea7c #46759 — [Bugfix][MRV2] Forward seq_lens_cpu_upper_bound for mamba hybrid models (#46759)
  • 作者: Michael Goin | +1/-0 | 1 个文件

  MambaHybridModelState.prepare_attn computes seq_lens_cpu_upper_bound but never forwards it to build_attn_metadata, so the mamba metadata builder receives None and fails assert seq_lens_cpu is not None during warmup. This blocks the V2 model runner (VLLM_USE_V2_MODEL_RUNNER=1) from starting for any mamba/conv-hybrid model. This one-line fix passes the already-computed tensor through, enabling MRV2 …

• ad28d60 #45544 — [Bugfix] Default tie_weights to sharing the weight (fix tied quantized embeddings, e.g. ModelOpt Gemma4) (#45544)
  • 作者: Mike G | +10/-3 | 1 个文件

  Fixes #45543. A model with tie_word_embeddings: true quantized via ModelOpt (NVFP4/FP8) — or whose lm_head is in exclude_modules and therefore receives UnquantizedLinearMethod — fails to load with NotImplementedError from tie_weights: After #39612, ParallelLMHead.tie_weights delegates to self.quant_method.tie_weights(…), and QuantizeMethodBase.tie_weights raised NotImplementedError. Only Unq…

• 8fa36fb #46506 — [Bugfix] FLASHINFER_MLA_SPARSE_SM120 compatibility with GLM-5 NVFP4 (#46506)
  • 作者: Gabriel Wu | +6/-4 | 1 个文件

  Models with an index_topk skip pattern (e.g. GLM-5.2, index_topk_freq=4) build a sparse-MLA indexer only on a subset of backbone layers; the remaining “skip” layers are constructed with indexer=None and instead receive the shared top-k indices buffer via MLAAttention (extra_impl_args[“topk_indices_buffer”], see mla_attention.py). FLASHINFER_MLA_SPARSE_SM120 asserted indexer is not None and only re…

• e45b279 #46316 — [Bugfix] Fix NVFP4+MTP crash: force unquantized mtp.fc for Qwen3Next (#46316)
  • 作者: Ranran | +11/-2 | 1 个文件

  Problem Serving an NVFP4 Qwen3-Next checkpoint (e.g. nvidia/Qwen3-Next-80B-A3B-Instruct-NVFP4) with MTP speculative decoding crashes while loading the draft model: AssertionError in load_column_parallel_weight — param [2048, 2048] vs checkpoint [2048, 4096]. mtp.fc is stored bf16 in the checkpoint, but the quant ignore glob “mtp.layers.0*” doesn’t match the sibling mtp.fc, so vLLM builds it…

🔧 Refactor

• cc79815 #46405 — [Refactor] Remove dead kernel code (#46405)
  • 作者: Wentao Ye | +0/-431 | 3 个文件

  Remove dead kernel code

🖥️ Kernel

• e8c24a7 #46643 — [Kernel] Vectorized fp32 moe_sum reduction and support any topk (#46643)
  • 作者: Michael Goin | +177/-51 | 2 个文件

  moe_sum only specialized topk ≤ 4 and fell through to at::sum_out for larger topk, so every DeepSeek-V3 / GLM / Qwen MoE (topk=8) paid PyTorch’s generic reduction on the critical path. The old topk ≤ 4 kernels also accumulated in bf16, which is lossy across 8 terms. This replaces the lot with one 16B-vectorized, fp32-accumulating kernel: - Compile-time unrolled for common topk (1/2/4/6/8/9), runti…

• e8e7b59 #46642 — [Kernel][MoE] Tune block-FP8 fused MoE for low-batch decode (#46642)
  • 作者: Michael Goin | +110/-88 | 10 个文件

  Justification Block-quantized FP8 fused MoE was locked to BLOCK_SIZE_N = block_shape[0] (128) at every batch size — in both the runtime heuristic (get_default_config) and the tuner search space. At decode the gate-up GEMM is SM-bound: a 128-wide N tile produces too few thread blocks to fill the GPU. This was never a kernel requirement. The Triton kernel indexes block scales per N element (offs_…