共 22 个 commit，涉及 80 个文件，+4293/-630 行变动。

概要

统计项 数值
Commit 数 22
变更文件 80
新增行数 +4293
删除行数 -630

Commit 列表

📦 Other

  • 4dcd10e #44454 — [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config construction (#44454)
    • 作者: Lucas Wilkinson | +38/-49 | 1 个文件

    PR #42374 (first part of RFC #42082) has been split into 4 PRs: -> #44454 [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config #44455 [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends #44456 [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions #44458 [4/N][KV-Cache Layout Refactor] Standardize KV cache layout Extrac…

  • 3d3ba46 #43087 — Modify torch dependency in xpu.txt (#43087)
    • 作者: Bram Vanroy | +1/-1 | 1 个文件

    Updated torch version when requesting xpu. As per the official documentation, the way to install xpu versions is: Installing with +xpu leads to this error:

  • 66ecfd0 #42599 — [Dependency] Remove stale cuDNN frontend upper bound (#42599)
    • 作者: Mohammad Miadh Angkad | +1/-3 | 1 个文件

    Remove the stale upper bound on nvidia-cudnn-frontend since the breaking changes in 1.19.0 have been fixed. The constraint is now >=1.19.1, which excludes 1.19.0 while letting vLLM pick up newer cuDNN frontend fixes and improvements. This is compatible with FlashInfer’s >=1.13.0 requirement.

  • 6ac6920 #44417 — [videoloader] implement glm46v video loader (#44417)
    • 作者: Jared Wen | +259/-0 | 2 个文件

    echo #44126 implement a video loader for GLM-4.6V VideoProcessor

  • 1505b3d #44707 — [Cohere] Enable Cohere Mini Code model and update Command A-plus test registry (#44707)
    • 作者: Terrence Zhao | +7/-4 | 2 个文件

    Turns on Cohere North-Mini-Code model (official name) withing vLLM code. The official HF repo will be CohereLabs/North-Mini-Code. Cmd to run the model: Install vLLM and dependencies Start server Send request —

  • 6181e80 #44540 — [XPU] add xpu branch in compressed_tensors_moe_w4a4_mxfp4 (#44540)
    • 作者: zofia | +12/-1 | 2 个文件

    add xpu support in compressed_tensors_moe_w4a4_mxfp4 Test: INCModel/Qwen3-235B-A22B-Instruct-2507-MXFP4-LLMC

  • 8109664 #36423 — [XPU] Support cpu kv offloading and tiering offloading on XPU platform (#36423)
    • 作者: Chaojun Zhang | +86/-32 | 7 个文件

    Support CPU KV offloading with XPU swap_blocks kernel on XPU platform Need XPU kernel support cross layer KV layout with layer dimension: https://github.com/vllm-project/vllm-xpu-kernels/pull/335 pytest -s -v tests/v1/kv_offload pytest -s -v tests/v1/kv_connector/unit/offloading_connector/test_worker.py pytest -s -v tests/v1/kv_connector/unit/offloading_connector/test_scheduler.py ## Test Result Q…

  • 2a983c7 #44699 — [DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 (#44699)
    • 作者: Woosuk Kwon | +449/-333 | 7 个文件
  • bc5745a #42838 — [ROCm][MLA] Replace torch.cat in sparse-MLA forward_mqa with fused concat_mla_q (#42838)
    • 作者: Markus Hartikainen | +12/-5 | 2 个文件

    In the ROCm sparse-MLA backend, forward_mqa builds q = torch.cat([ql_nope, q_pe], dim=-1) once per layer per forward pass. On DeepSeek-V3.2 (61 sparse-MLA layers on the decode path) this adds up to 61 fresh tensor allocations and 61 aten::CatArrayBatchedCopy kernel launches per decode step, ≈ 360 µs at MC=4 (≈ 4% of a decode step measured at TP=4, ISL=1000, OSL=100 on MI355X). The fused ops.co…

🖥️ Kernel

  • 228bcc4 #44674 — [ROCm][Kernel] Enable permute_cols for ROCm (#44674)
    • 作者: Charlie Fu | +3/-8 | 3 个文件

    pytest -sv tests/kernels/core/test_permute_cols.py ================ 3 passed, 16 warnings in 1.81s ========================

🧪 CI/Tests

  • f0f6805 #44051 — [CI] Stabilize the multi-audio OpenAI server path (#44051)
    • 作者: Andreas Karatzas | +62/-59 | 3 个文件

    This PR stabilizes the multi-audio OpenAI server path when a chat request exceeds –limit-mm-per-prompt audio=2. The failing test intentionally sends too many audio inputs, expects a 400 BadRequestError, and then immediately checks that the same server still accepts a follow-up /v1/completions request: The over-limit request was already logically a client validation failure, but it was raised as a…

📖 Documentation

  • 15652a6 #44378 — [Doc] Fix multimodal torch.compile troubleshooting to not use removed VLLM_TORCH_COMPILE_LEVEL (#44378)
    • 作者: Daoyuan Li | +1/-1 | 1 个文件

    The “Compilation Errors” troubleshooting section in docs/design/torch_compile_multimodal.md tells users to disable compilation like this: VLLM_TORCH_COMPILE_LEVEL no longer exists anywhere in the codebase — torch.compile control was moved to CompilationConfig / the -O optimization levels. The env var is silently ignored, so following this step does not actually disable compilation: the model s…

🐛 Bug Fix

  • 51ef688 #44103 — [Bugfix][Mooncake] Fix per-group block_size/block_hash and group_idx in MooncakeStoreConnector KV events (#44103)
    • 作者: Yifan Qiao | +84/-20 | 5 个文件

    MooncakeStoreConnector emits BlockStored KV events so external subscribers can track which KV blocks are present in the store. In hybrid / multi-group (HMA) configurations — and whenever a group’s block_size is larger than the connector’s hash_block_size — three fields of the emitted event were wrong. 1. block_hashes used req_meta.block_hashes[chunk_idx], which indexes the request’s *hash-gran…

  • 9c7f774 #44041 — [Bugfix] Fix benchmark_moe.py after inplace mechanism removal (#44041)
    • 作者: Qiuyang Yue | +0/-3 | 1 个文件

    PR #43727 ([MoE] Remove inplace fused experts mechanism) removed disable_inplace() from vllm/model_executor/layers/fused_moe/utils.py and dropped the inplace parameter from both fused_experts() and FusedMoEKernel.init(). However, benchmarks/kernels/benchmark_moe.py was not updated, so running the benchmark on current main crashes with: This PR removes the three stale references: - Line 274: in…

  • 3b3d528 #44560 — [BugFix] Resolve multiple async kv load deadlock (#44560)
    • 作者: Nick Hill | +214/-1 | 6 个文件

    This is a different fix for the issue targeted by https://github.com/vllm-project/vllm/pull/40968. The changes are simpler and the resulting behavior should also be better - we avoid async-loading new kv if it would occupy blocks required for other in-flight chunked prefills or async-loading requests. Note: we apply this throttle to async kv loads only, rather than chunked-prefill requests in gene…

  • 67d3792 #44694 — [Bugfix] Fix Qwen3.5-FP8 nightly fail. Guard fused_add_rms_norm input/weight dtype mismatch in RMSNorm + quant fusion (#44694)
    • 作者: Vadim Gimpelson | +8/-2 | 1 个文件

    The nightly LM Eval Qwen3.5 Models (B200) — ci build #70074 job started failing on main, a regression introduced by #42646 “[perf] Add gemma RMS AR fusion”. The RMSNorm + quant fusion’s dtype guard only inspected rms_norm nodes, not fused_add_rms_norm nodes, so a GemmaRMSNorm (used by Qwen3.5 / Qwen3-Next) producing a bf16 input with an fp32 weight got fused into rms_norm_per_block_quant, which ab…

  • 00d1fb7 #43684 — [Bugfix][ROCm] ApplyRotaryEmb: fall back to native when flash_attn rotary grid would exceed the HIP per-dim limit (#43684)
    • 作者: Fuwei Yang | +23/-0 | 1 个文件

    On AMD ROCm/HIP the per-dimension launch grid is capped at 65,535. vllm/model_executor/layers/rotary_embedding/common.py::ApplyRotaryEmb.forward_hip delegates to the flash_attn Triton rotary kernel, whose grid is (cdiv(nheads, BLOCK_H), cdiv(seq_len, BLOCK_M), batch). With BLOCK_M = 8 (the default when rotary_dim <= 128) any seq_len above 65535 * 8 = 524,280 overflows gridY and hipModuleLaunch…

  • c9b4b18 #44559 — [Bugfix][Voxtral] Add fetch_audio to MistralCommonFeatureExtractor (transformers>=5.10 compat) (#44559)
    • 作者: Yadan-Wei | +177/-0 | 3 个文件

    Fixes #44554. Loading any voxtral model under transformers>=5.10 raises: transformers 5.10.0 added a ProcessorMixin.prepare_inputs_layout helper (processing_utils.py:704) that calls self.feature_extractor.fetch_audio(…) unconditionally — no hasattr guard. The method is implemented on SequenceFeatureExtractor, but vLLM’s MistralCommonFeatureExtractor is a duck-typed wrapper that inherits from obj…

✨ New Feature

  • 32f34d3 #44420 — [feature] add index share feature for DSA MTP (#44420)
    • 作者: Jared Wen | +114/-25 | 5 个文件

    This PR implements IndexCache top-k index reuse for DeepSeek Sparse MLA’s MTP Layer, inspired by IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse. The core idea is to avoid running the sparse attention indexer in every layer or every MTP speculative step when the top-k token selections can be reused. The implementation adds support for carrying topk_indices through the DeepSee…

  • 3bb4697 #37149 — [XPU][Feature] transparent sleep mode support for XPU platform (#37149)
    • 作者: Yan Ma | +410/-47 | 10 个文件

    Note that this depends on XPUPluggableAllocator and use_mem_pool API in torch-xpu 2.11. This PR is to enable transparent sleep mode on XPU platform. It introduces changes: - Add XpuMemoryAllocator and provide sleep/wake up interface for cuda parity - get_mem_allocator to support both cuda_alike path and xpu path - Add memory allocator UT coverage on XPU

⚡ Performance

  • 062b05f #44075 — [ROCm][Perf] Fused MoE W4A16 HIP kernel for AMD RDNA3 (gfx1100) (#44075)
    • 作者: JartX | +1906/-5 | 11 个文件

    Summary Native HIP kernel for W4A16 MoE on RDNA3 (gfx1100), replacing the Triton fused_moe_kernel_gptq_awq path. It uses the same dequant + dot primitives as the dense W4A16 kernel: v_dot2_f32_f16 / v_dot2_f32_bf16, exllama bit-trick dequant, and 64-bit CAS atomic output. ### What it does - Fused HIP kernel (csrc/rocm/moe_q_gemm_rdna3.cu): expert routing + W4A16 GEMM in a single kernel lau…

  • fa27d4e #44700 — [PERF] [Qwen3.5] Split mixed prefill+decode batches: route decodes to the recurrent kernel (#44700)
    • 作者: Vadim Gimpelson | +426/-31 | 5 个文件

    Description This PR has 3 changes: 1. Main: split mixed prefill+decode batches (see below). 2. Add MTP eval config for Qwen3.5-397B NVFP4. 3. Rename fast_kernel → aiter_kernel (fast_kernel was too general). ## Problem GDN attention ran all non-spec tokens through chunk_gated_delta_rule. Each decode becomes its own chunk padded to FLA_CHUNK_SIZE=64, so D decodes cost D near-empty …