共 47 个 commit，涉及 452 个文件，+15064/-1335 行变动。

概要

统计项	数值
Commit 数	47
变更文件	452
新增行数	+15064
删除行数	-1335

Commit 列表

📦 Other

• 1ff9429 #48036 — [CI Bug] Fully solve accuracy issue for DSv3.2 + MTP + Sequence Parallel (#48036)
  • 作者: Wentao Ye | +34/-3 | 3 个文件

  A through fix for https://github.com/vllm-project/vllm/pull/47902 vllm serve deepseek-ai/DeepSeek-V3.2 –trust-remote-code –kernel-config.enable_flashinfer_autotune=False –enable-expert-parallel –tensor-parallel-size 8 –tokenizer-mode deepseek_v32 –tool-call-parser deepseek_v32 –enable-auto-tool-choice –reasoning-parser deepseek_v3 –port 9256 –speculative-config ‘{“method”:“mtp”,“num_spec…

• 7a74a96 #47021 — [NIXL] Avoid reading expired blocks in bidirectional turn-2 read (#47021)
  • 作者: tomerg-nvidia | +264/-22 | 10 个文件

  Currently with the multi-turn bidirectional mode, the nixl connector can result with garbage reads on turn 2 (P reading from D). This is because D sets a constant TTL for the blocks which is not extended by the heartbeat like the P TTL. When the blocks are expired they can be reused for other requests. The problem is that subsequent turns don’t have a way to tell that the blocks on D are no longer…

• b6754f5 #48594 — [Model] Enable LoRA support for tower and connector in LlavaNextVideo (#48594)
  • 作者: karthik | +149/-3 | 3 个文件

  Part of #31479 (enable LoRA for the tower and connector in more multimodal models). Adds SupportsLoRA, packed_modules_mapping, get_mm_mapping(), and get_num_mm_encoder_tokens() / get_num_mm_connector_tokens() to LlavaNextVideoForConditionalGeneration, following the pattern used for LlavaForConditionalGeneration in #31513. LlavaNextVideoForConditionalGeneration currently has no LoRA support at all …

• 894ebb2 #48291 — Add Cosmos3 Edge Reasoner model (#48291)
  • 作者: adsridhar | +1474/-27 | 13 个文件

  This PR brings in a new Cosmos3 Edge Reasoner model (not yet released) to vllm. Cosmos3 is a mixture of transformers model consisting of a Reasoner and a Generator tower. This PR introduces custom model layers, processors, and performs checkpoint mapping. Checkpoint loading and input processing tests have been added. ## Test Result The new tests pass.

• c9a788e #47595 — fix(security): guard lm-format-enforcer regex compile with timeout (#47595)
  • 作者: Juan Pérez de Algaba | +14/-1 | 1 个文件

  The lm-format-enforcer backend was left unguarded after the fix for GHSA-rwxx-mrjm-wc2m. Route the RegexParser call through compile_regex_with_timeout and validate regex patterns early in validate_structured_output_request_lm_format_enforcer, matching the xgrammar and outlines backends.

• 31be872 #48372 — [ROCm] Retune MI355 selective_state_update float16 config on the unified effective_batch grid (#48372)
  • 作者: vanshbhatia-amd | +32/-16 | 1 个文件

  Re-tunes the AMD Instinct MI355X selective_state_update (Mamba SSU decode kernel) cache_dtype=float16 launch config onto the same effective_batch grid already used by the MI300X (#47945) and MI350X (#48159) configs, so all AMD Instinct SSU configs are sampled on a uniform set of points. The original MI355 config (#47767) was keyed on a narrower, deployment-specific grid (10 … 5120). This P…

• 95aab66 #47984 — [ROCm][MiniMax-M3][Spec Decode] Support speculative decode with AITER sparse PA (#47984)
  • 作者: Tan Pin Siang | +182/-15 | 3 个文件

  This PR enables MiniMax-M3 speculative decode on the ROCm AITER sparse paged-attention path introduced in #47287. The core change maps flattened speculative decode rows back to their request-local query positions: Then it reuses the existing prefill sparse block-table builder to produce one compact page-16 AITER table row per query token. No new Triton/Gluon attention kernel is added. The existing…

• b50ef9c #44849 — [ROCm][MiniMax-M2] Dispatch fused QK-norm + AllReduce via AITER (#44849)
  • 作者: akii96 | +12/-0 | 1 个文件

  ⚠️ Requires: AITER build containing ROCm/aiter#3163 (Optionally ROCm/aiter#3189 for additional performance) Co-developed with @pbkowalski who wrote the initial dispatch re-wiring Adds ROCm dispatch for MiniMax-M2’s QK-norm custom op to use AITER’s fused kernel (custom_fused_qknorm_ar from ROCm/aiter#3163 + aiter#3189 further pushes the perf) Supersedes #42602 (closed as that used a compile-tim…

• 9e289c5 #44462 — up FI fp8 moe topk to 32 (#44462)
  • 作者: Dan Blanaru | +1/-1 | 1 个文件

  The FlashInfer backend now supports topk up to 32. See here The model I tested this on is not OSS so I cannot give a repro, but verified smoke tests work. While there isn’t some specific test to cover this, I’ve run:

• c4f5cd6 #47327 — [1/N] Add dense MHA path for sparse MLA short sequences (#47327)
  • 作者: Matthew Bonanni | +1991/-647 | 21 个文件

  Split up from https://github.com/vllm-project/vllm/pull/34744 Adds a fast path when all prefills in a batch have seq_len <= topk, routing to dense MHA instead of sparse MQA (subject to a q_len threshold) to obtain a substantial speedup. This is also a precursor to a masked MHA implementation (see https://github.com/vllm-project/vllm/pull/34744), which will enable speedups at seq_len > topk. ### La…

• 0b0ef8d #47521 — [Quantization][INC][ARK] Support INT2 XPU WOQ Linear (#47521)
  • 作者: Zhenzhong Xu | +158/-5 | 3 个文件
• 21472f3 #48287 — add pad-aware swiglu limit kernel (#48287)
  • 作者: gnovack | +140/-4 | 4 个文件

  When serving with expert-parallelism enabled, the MoE inputs and intermediate states are padded along the token dimension to handle the worst case (i.e. all tokens routed to a single rank; see https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/fused_moe/prepare_finalize/deepep_v2.py#L32-L33). Most MoE kernels have internal logic to ignore such padding tokens during GEMM exec…

• 8ac8375 #47782 — [Core] Preserve Marconi caching with selective hybrid cache retention (#47782)
  • 作者: Nick Hill | +502/-236 | 9 个文件

  The Marconi paper was implemented in https://github.com/vllm-project/vllm/pull/37898, which enables caching of mamba state for shared prefixes in “align” mode. This PR extends the shared prefix boundary caching to the retention-interval sparse caching added in https://github.com/vllm-project/vllm/pull/43447 and https://github.com/vllm-project/vllm/pull/45845 for both mamba states and SWA. In parti…

• 7dc447d #47568 — Added sliding window attention support for qwen-eagle3 architecture (#47568)
  • 作者: shanjiaz | +15/-0 | 2 个文件

  Added sliding window attention for eagle3 qwen3 architecture. Trained drafter models and tested locally. ## Test Result See results in comment here —

• 7fc9704 #48180 — Add DCP + Eagle support for Tokenspeed MLA backends (#48180)
  • 作者: Pavani Majety | +313/-23 | 8 个文件

  Adds support to run DCP + Tokenspeed_MLA with Eagle. Builds off of https://github.com/vllm-project/vllm/pull/47915/changes Validated with unit test and Kimi models and the lightseekai-org’s two eagle heads. ## Test Result GSM8k is validated. Perf (Green- this PR, Blue- No DCP) – validated with Agentic Benchmark: — BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing …

• 43c8cbf #47423 — [EC Connector] CPU Offloading EC Connector (#47423)
  • 作者: omerpaz95 | +3008/-0 | 18 个文件

  Add ECCPUConnector — CPU encoder-cache offloading connector This PR adds ECCPUConnector, a self-contained encoder-cache (EC) connector that offloads encoder outputs to a shared CPU mmap region on /dev/shm so a single ec_role=both vLLM instance can reuse them on a later request instead of re-running the vision encoder. The connector is intentionally minimal. Everything happens within one instance…

• c2c9f7c #48467 — remove force channels_last in Idefics3MultiModalProcessor (#48467)
  • 作者: Yan Ma | +4/-1 | 1 个文件

  After transformers bumped to 5.13.1(#47867), force channels_last would permute already-CHW tensors and cause below error: pytest -s -v tests/models/multimodal/processing/test_tensor_schema.py::test_model_tensor_schema[HuggingFaceTB/SmolVLM2-2.2B-Instruct] ## Test Result —

• 1be6e93 #48483 — lower memory required for capturing cudagraphs for large cudagraph sizes (#48483)
  • 作者: Omer Ullman Argov | +5/-1 | 1 个文件

  Fix an over-allocation during memory estimates. When attempting to capture CUDA graphs with large sizes, the dummy KV cache allocated 1 block per-token. However, we only ever need one block per-sequence. This over allocation makes it impossible to capture CUDA graphs for large sizes since it just OOMs. With this fix, the dummy KV cache allocates the true safe minimum of blocks, preventing the …

• b3cfca9 #48490 — [Mypy Fix] Split mypy work (#48490)
  • 作者: Wentao Ye | +24/-1 | 1 个文件

  There are 1700+ failures for “vllm/model_executor/models”,, this PR split the work so it should be easier to pick subtasks.

• 107a03b #48390 — [Core] Support fp32 lm_head for generation models via head_dtype (RFC #48305 §3.6) (#48390)
  • 作者: Karthik Kothuri | +249/-15 | 6 个文件

  RL training-inference consistency (RFC #48305 §3.6, “Dtype Replay”) requires vLLM’s rollout logits to match a trainer that computes the lm_head in fp32. vLLM already has ModelConfig.head_dtype (merged for pooling models in #23810), but it was restricted to pooling models — generation models silently fell back to the model dtype. This PR extends head_dtype to generation models (the goal of …

• bea70c7 #48011 — [Attention] Make sliding-window support an explicit backend capability (#48011)
  • 作者: Nicolò Lucchesi | +41/-0 | 10 个文件

  1/2 of https://github.com/vllm-project/vllm/pull/48012 , needed to get proper backend selection support. I split the PRs because this one is the one which has a ~very tiny impact on existing auto-selector logic, as explained below. cc @MatthewBonanni ### Problem Attention backend selection models every capability constraint explicitly (supports_sink, is_sparse, is_mla, …) except sliding window, so…

🐛 Bug Fix

• af453e5 #48262 — [Bugfix] Gemma4 parser: classify channel-less output consistently in streaming and non-streaming (#48262)
  • 作者: adhi29 | +124/-25 | 2 个文件

  Fixes #48217 With thinking enabled, the Gemma4 streaming parser pre initialized its engine to REASONING for any prompt ending in a new model turn. If the model then produced a direct answer with no channel markers, every streamed chunk was classified as reasoning and no content was ever emitted, while the non streaming path classified the exact same text as content. This PR makes the pre initializ…

• 32aef44 #48411 — [Bugfix] Include inline per-token-head scales in offloaded page transfer width (#48411)
  • 作者: Itay Etelis | +41/-6 | 3 个文件

  Per-token-head quantized KV pages (*_per_token_head kv-cache-dtypes) carry fp32 scales inline within each (head, token) cell, beyond real_page_size_bytes (budgeted in AttentionSpec.page_size_bytes). The offloading connector used real_page_size_bytes as the per-block transfer width, so offloaded pages were truncated mid-cell: tail heads/tokens (data and scales) were dropped and restores brought bac…

• 793cf79 #48583 — [Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation (#48583)
  • 作者: Juan Pérez de Algaba | +298/-13 | 5 个文件

  Serialize all uses of torch.sparse.check_sparse_tensor_invariants() behind a shared threading.Lock to prevent concurrent prompt-embedding parts from racing the process-global save/restore flag.

• 038ec29 #48473 — [Bugfix] Return 400 instead of 500 when multimodal data is sent to a text-only model (#48473)
  • 作者: Hoang Nguyen | +32/-2 | 2 个文件

  Sending multimodal data (e.g. an image_url) to a text-only model returns HTTP 500 InternalServerError. It is a client mistake and should be HTTP 400 BadRequest. BaseRenderer._process_multimodal reads self._mm_req_counter before resolving the processor: _mm_req_counter is only initialized inside the if mm_registry.supports_multimodal_inputs(…) block, so on a text-only model the attrib…

• af1f036 #48523 — [Bugfix] Skip minimax_m3 tool parser tests when Rust extension is absent (#48523)
  • 作者: Matt Woodson | +4/-0 | 1 个文件

  test_minimax_m3_tool_parser.py fails with RuntimeError: Rust tool parsing requires the vllm._rust_tool_parser PyO3 extension when the Rust extension is not built into the wheel. MinimaxM3ToolParser extends RustToolParser, so these tests require the PyO3 extension. test_rust_tool_parser.py already guards against this with pytest.importorskip(“vllm._rust_tool_parser”), but test_minimax_m3_tool_parse…

• fec64fe #40698 — [BugFix] Correct OTEL span start time for Dynamo compilation (#40698)
  • 作者: emricksini-h | +6/-3 | 1 个文件

  Fixes a bug where the Dynamo bytecode transform OpenTelemetry span would report a duration of ~56.3 years (starting in 1970). This issue is due to the fact that we were using time.perf_counter() to capture torch_compile_start_time. This returns a monotonic timestamp (seconds since system boot) rather than an absolute Unix epoch timestamp. When passed directly to OpenTelemetry (which expects Unix e…

• 9a21f0d #44863 — [BugFix] Initialize model_config for Qwen3-VL MoE (#44863)
  • 作者: wenpengw-nv | +1/-0 | 1 个文件

  Fix Qwen3-VL MoE initialization for encoder CUDA graph. Qwen3VLMoeForConditionalGeneration inherits encoder CUDA graph helper methods from Qwen3VLForConditionalGeneration, but its constructor skips Qwen3VLForConditionalGeneration.init while constructing the MoE language model. As a result, self.model_config is not initialized for the MoE subclass. When cudagraph_mm_encoder is enabled, EncoderC…

• 550218b #47606 — [Bugfix][Frontend] Flush engine reasoning parser at engine-reasoning → tool streaming boundary (#47606)
  • 作者: akii96 | +298/-11 | 2 个文件

  Edit: reworked to use the flush-based approach suggested by @bbrowning in review (previously decoded raw token ids, which is unsafe for multi-token characters). When an engine-based reasoning parser (e.g. qwen3) is paired with a text-based tool parser (e.g. hermes), a streamed tool call can be dropped. If one streamed delta contains both the end of reasoning and the start of the tool call (</t…

• 26587f9 #48261 — [BugFix][ModelRunner V2] Fix stale attn metadata in speculator prefill cudagraph capture (#48261)
  • 作者: Nick Hill | +72/-108 | 7 个文件

  The V2 speculator’s prefill FULL cudagraph capture reused the attention states built earlier during the target model’s capture. Those metadata objects are views into per-builder persistent buffers (e.g. FlashAttention’s AOT scheduler_metadata, dummy query_start_loc/seq_lens), which every subsequent build overwrites. By the time the speculator captured its prefill graphs, the buffer contents matche…

• 56a357e #48256 — [Bugfix][KV Cache] Don’t route uniform-page-size MLA+SWA models into DeepseekV4 packing (#48256)
  • 作者: Nicolò Lucchesi | +48/-0 | 2 个文件

  group_and_unify_kv_cache_specs() targets DeepseekV4, where MLA and sliding-window MLA layers have different page sizes and must be tuple-packed. It fired for any model containing a SlidingWindowMLASpec, so a non-DeepseekV4 model with a uniform page size was wrongly pushed into the packing path. This results in creating “one-element bins”, while unecessarily triggering separate paths for things lik…

🔩 Misc

• 50ac1c7 #45781 — [Misc] Rename VLLM_TRITON_ATTN_USE_TD to VLLM_TRITON_USE_TD (#45781)
  • 作者: Artur Fierka | +26/-7 | 3 个文件

  [Misc] Rename VLLM_TRITON_ATTN_USE_TD to VLLM_TRITON_USE_TD Follow-up to the tensor-descriptor (TD) pilot #40327 and the TD-adoption RFC #42545. The pilot shipped the TD opt-in behind the attention-specific VLLM_TRITON_ATTN_USE_TD; the RFC extends TD across vLLM’s Triton kernels behind a single, general flag. This renames the variable to VLLM_TRITON_USE_TD. The old name stays registered (so it d…

• 94c0ef3 #48549 — [Misc] Clean up “swap_space” (#48549)
  • 作者: wangxiyuan | +0/-11 | 1 个文件

  swap_space is deprecated from v0.18.0. It’s safe to clean up the warning now. ## Test Result —

• 6228630 #48057 — [Misc] Improve Matryoshka pooling dimensions validation (#48057)
  • 作者: Taneem Ibrahim | +17/-16 | 1 个文件

  PoolingParams.dimensions should only be set for Matryoshka embedding models. This PR keeps the non-Matryoshka error simple, while improving the Matryoshka path so out-of-range values report the valid model-specific range and explicit matryoshka_dimensions configs report the allowed values directly. ## Reproducer Tests Output on main: Output on Branch: For non-Matryoshka models, the error n…

🧪 CI/Tests

• f04d3f6 #47754 — [Test] Enable KV cache events for HMA models in CPU offloading test (#47754)
  • 作者: Itay Etelis | +14/-19 | 1 个文件

  KV cache events no longer force the hybrid KV cache manager off (the restriction was removed in #37688, with connector-side group support added in #43468), so this drops the stale uses_hma guard from test_cpu_offloading and enables the KV-event assertions for the HMA models as well.

• 0a9396a #47231 — [XPU][CI] Add tests/v1/e2e/general/test_correctness_sliding_window.py in Intel GPU CI (#47231)
  • 作者: xiangdong | +2/-0 | 2 个文件

  Test Result —

• 18c4067 #48513 — [ROCm][CI] Unblock AMD: Language Models Test (Extended Pooling) (#48513)
  • 作者: Micah Williamson | +4/-0 | 1 个文件

  Add a skip for the failing test models/language/pooling/test_token_classification.py::test_openai_privacy_filter[bfloat16-openai/privacy-filter] (https://buildkite.com/vllm/ci/builds/77673/list?sid=019f5553-8716-4596-8f4c-62c6657cef31&tab=output) until this issue is resolved https://github.com/vllm-project/vllm/issues/48510. One attempt to resolve is in progress here https://github.com/vllm-projec…

• 9427c45 #48258 — [ROCm][CI] Transformers: pass only one of input_ids/inputs_embeds (#48258)
  • 作者: stefankoncarevic | +4/-0 | 1 个文件

  The vLLM Transformers modeling backend can call the underlying HuggingFace model with both input_ids and inputs_embeds set. This happens during cudagraph capture / warmup, where the runner provides dummy input_ids together with precomputed inputs_embeds. HuggingFace models require exactly one of input_ids / inputs_embeds. For example Gemma3 raises: The failure surfaces through the backend …

• 93e3bc8 #48418 — [XPU][CI]Adjust timeout_in_minutes in Intel GPU CI (#48418)
  • 作者: xiangdong | +18/-18 | 7 个文件

  Adjust timeout_in_minutes in Intel GPU CI ## Test Result —

• 487dfb3 #48472 — [CI] Add SPDX license header to Rust/Protobuf sources (#48472)
  • 作者: Bugen Zhao | +813/-30 | 253 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Extend the SPDX license header pre-commit hook to cover Rust and Protobuf sources as well. ## Test Result —

✨ New Feature

• 0762f2a #42562 — [Perf][Feat] Add generic cuteDSL LL BF16 router (GEMM) (#42562)
  • 作者: Roberto L. Castro | +1812/-8 | 8 个文件

  vLLM already ships a LL router GEMM compatible with DeepSeek-V3/V4-Pro requirements, but it is locked to specific shapes: Models that fall outside these dimensions (e.g. DeepSeek-V4-Flash with K=14400, N=256) silently fall back to cuBLAS, leaving performance on the table. This PR adds a generic cuteDSL router GEMM that is competitive with dsv3_router_gemm on its home turf while being flexible enou…

• 7738ef3 #48463 — [Feat] Add Support for BertForMaskedLM to vLLM (#48463)
  • 作者: Snehlata | +158/-2 | 4 个文件

  Add Support for BertForMaskedLM to vLLM This PR adds support for BertForMaskedLM by introducing a dedicated model wrapper that integrates with vLLM’s encoder-only execution path and exposes per-token vocabulary logits through the masked language modeling (MLM) head. vLLM currently supports BERT encoder models but does not support BertForMaskedLM, preventing Hugging Face masked language modelin…

⚡ Performance

• dcf4072 #45000 — [Perf][ROCm] Fix GDN KKT warmup regression on RDNA by avoiding fp32 tl.dot (#45000)
  • 作者: nemanjaudovic | +19/-1 | 1 个文件

  After this PR: https://github.com/vllm-project/vllm/pull/42076, during running of Qwen3.5-9B model that uses GDN triton kernel on RDNA GPUs, warmup time went from ~150s to ~2400s. The problem was that operands of tl.dot() were casted to fp32, so that operation lowered to software fp32 matmul instead of wmma instuction. These changes are modifying kernel only for RDNA GPUs, while they are untouched…

• 75fe92a #48064 — [Distributed][Perf] Enable FlashInfer MNNVL allreduce RMS quant fusion (#48064)
  • 作者: Mohammad Miadh Angkad | +74/-22 | 2 个文件

  Enable AR + RMSNorm quant fusion to use the MNNVL backend, which current FI already supports QuantType)2 is the FP4 quant path https://github.com/flashinfer-ai/flashinfer/blob/release-v0.6.13/include/flashinfer/comm/trtllm_mnnvl_allreduce.cuh#L43-L47

🖥️ Kernel

• 382bbd5 #40977 — [ROCm][Kernel] Add HybridW4A16LinearKernel: Triton prefill + HIP skinny decode (#40977)
  • 作者: Matthias Gehre | +2165/-5 | 11 个文件

  Add a hybrid W4A16 linear kernel for ROCm gfx11/gfx12 that routes between two GEMM implementations based on batch size: - M <= 5: HIP wvSplitK_int4_g skinny GEMM (optimized for single-token decode) - M > 5: Triton fused dequant GEMM (optimized for prefill/batched inference) Both paths share a single weight tensor in the AWQ-style packed shuffle layout [N, K//8]. Supports both symmetric (uint4b8, b…

🦀 Rust Frontend

• 8b8af2c #43463 — [Frontend] Expose logprob_token_ids on Python OpenAI endpoints (#43463)
  • 作者: langzhao-netizen | +401/-7 | 9 个文件

  Closes #43466 SamplingParams.logprob_token_ids lets callers retrieve logprobs for a fixed set of vocabulary IDs. That is useful for scoring a small label vocabulary whose tokens may not appear in the model’s natural top-k. The capability was available through Python LLM and /generative_scoring, but not through the default Python OpenAI-compatible chat and completion routes. This PR exposes logprob…

📖 Documentation

• 5c34287 #48293 — [Doc] Add DeepseekV32ForCausalLM to supported_models.md (#48293)
  • 作者: Gavin Morris | +1/-0 | 1 个文件

  DeepseekV32ForCausalLM has been registered as a text-generation architecture since PR #25896 ("[New Model] DeepSeek-V3.2”), but the supported-models table was never updated to mention it: This adds it to the existing DeepseekV3 row, following the same alias pattern the file already uses, with the deepseek-ai/DeepSeek-V3.2-Exp example checkpoint (the test checkpoint listed in tests/models/registry….