共 38 个 commit，涉及 206 个文件，+4450/-1335 行变动。

概要

统计项 数值
Commit 数 38
变更文件 206
新增行数 +4450
删除行数 -1335

Commit 列表

⚡ Performance

  • fa27d4e #44700 — [PERF] [Qwen3.5] Split mixed prefill+decode batches: route decodes to the recurrent kernel (#44700)
    • 作者: Vadim Gimpelson | +426/-31 | 5 个文件

    Description This PR has 3 changes: 1. Main: split mixed prefill+decode batches (see below). 2. Add MTP eval config for Qwen3.5-397B NVFP4. 3. Rename fast_kernel → aiter_kernel (fast_kernel was too general). ## Problem GDN attention ran all non-spec tokens through chunk_gated_delta_rule. Each decode becomes its own chunk padded to FLA_CHUNK_SIZE=64, so D decodes cost D near-empty …

🐛 Bug Fix

  • 67d3792 #44694 — [Bugfix] Fix Qwen3.5-FP8 nightly fail. Guard fused_add_rms_norm input/weight dtype mismatch in RMSNorm + quant fusion (#44694)
    • 作者: Vadim Gimpelson | +8/-2 | 1 个文件

    The nightly LM Eval Qwen3.5 Models (B200) — ci build #70074 job started failing on main, a regression introduced by #42646 “[perf] Add gemma RMS AR fusion”. The RMSNorm + quant fusion’s dtype guard only inspected rms_norm nodes, not fused_add_rms_norm nodes, so a GemmaRMSNorm (used by Qwen3.5 / Qwen3-Next) producing a bf16 input with an fp32 weight got fused into rms_norm_per_block_quant, which ab…

  • 00d1fb7 #43684 — [Bugfix][ROCm] ApplyRotaryEmb: fall back to native when flash_attn rotary grid would exceed the HIP per-dim limit (#43684)
    • 作者: Fuwei Yang | +23/-0 | 1 个文件

    On AMD ROCm/HIP the per-dimension launch grid is capped at 65,535. vllm/model_executor/layers/rotary_embedding/common.py::ApplyRotaryEmb.forward_hip delegates to the flash_attn Triton rotary kernel, whose grid is (cdiv(nheads, BLOCK_H), cdiv(seq_len, BLOCK_M), batch). With BLOCK_M = 8 (the default when rotary_dim <= 128) any seq_len above 65535 * 8 = 524,280 overflows gridY and hipModuleLaunch…

  • c9b4b18 #44559 — [Bugfix][Voxtral] Add fetch_audio to MistralCommonFeatureExtractor (transformers>=5.10 compat) (#44559)
    • 作者: Yadan-Wei | +177/-0 | 3 个文件

    Fixes #44554. Loading any voxtral model under transformers>=5.10 raises: transformers 5.10.0 added a ProcessorMixin.prepare_inputs_layout helper (processing_utils.py:704) that calls self.feature_extractor.fetch_audio(…) unconditionally — no hasattr guard. The method is implemented on SequenceFeatureExtractor, but vLLM’s MistralCommonFeatureExtractor is a duck-typed wrapper that inherits from obj…

  • f87df1d #44613 — [Bugfix][MoE] Snapshot max_cudagraph_capture_size into FusedMoEConfig (#44613)
    • 作者: aoshen02 | +7/-16 | 5 个文件

    Several MoE kernels/quant-methods read get_current_vllm_config().compilation_config.max_cudagraph_capture_size inside their init: - FlashInferExperts.init (fused_moe/experts/flashinfer_cutlass_moe.py) - TrtLlmMxfp4ExpertsBase.init (fused_moe/experts/trtllm_mxfp4_moe.py) - GptOssMxfp4MoEMethod.init and Mxfp4MoEMethod.init (quantization/mxfp4.py) UnquantizedFusedMoEMethod.process…

  • ec0a31d #44692 — [Bugfix][Kernel] Fix mHC fused-RMSNorm big-fuse miscompile for hidden_size != 4096 (#44692)
    • 作者: Yongye Zhu | +1/-1 | 1 个文件

    mhc_pre_big_fuse_with_norm_tilelang (the RMSNorm-fused mHC pre big-fuse used by DeepSeek-V4 when norm_weight is supplied) pipelines its fused-RMSNorm pass at num_stages=3. Combined with the loop-carried sumsq reduction and the persistent output_shared staging buffer written at per-iteration offsets, the TileLang software pipeliner miscompiles the loop. The result is silently wrong layer_input …

  • 4765f0f #44130 — [Bugfix] Fix sequence_parallel_chunk_impl custom op aliasing its input (#44130)
    • 作者: Vadim Gimpelson | +4/-1 | 1 个文件

    Root cause sequence_parallel_chunk_impl() returns torch.narrow(y, …), which is a view of its input whenever the sequence length is already divisible by tp_size (the no-pad branch sets y = x). A functional custom op must not return an output that aliases one of its inputs, so under the AOT-autograd torch.compile path the runtime aliasing guard rejects it during the memory-profiling dummy run…

  • 703fb17 #44330 — [Bugfix] GPT-OSS instruction rendering (#44330)
    • 作者: yzong-rh | +275/-111 | 9 个文件

    Fix #33210: make GPT-OSS rendering follow HF for the first instruction block. (see examples below for side-by-side comparison) - Fold the first user system/developer instruction into the Harmony preamble. - Avoid rendering it as a separate extra message after the preamble. - Avoid emitting an empty developer block when there are no instructions/tools. ### Differences to HF rendering - Respect …

  • aa6fb8a #44648 — [Bugfix] [ROCm] [Critical] fallback to regular abi for ROCm (#44648)
    • 作者: TJian | +127/-25 | 6 个文件

    Build error message 1. Build successfully 2. Pass test_uva.py ## Test Result 1. Built successfully 2. pytest -svvvvv tests/kernels/core/test_uva.py Results: ## Extra details The silu_and_mul_per_block_quant that was moved into stable ABI in this PR https://github.com/vllm-project/vllm/pull/44334 works on ROCm. pytest tests/kernels/core/test_fused_silu_mul_block_quant.py test results: =============…

  • bbb6c27 #44615 — [Bugfix] Fix gemma4 crash on CPU: guard mem_get_info call (#44615)
    • 作者: adhithyamulticoreware | +5/-0 | 1 个文件

    What this fixes Fixes #44039 On CPU, current_platform.mem_get_info returns None (via Platform.getattr fallback — torch.cpu has no mem_get_info attribute). Both _process_image_input and _process_video_input in gemma4_mm.py called it unconditionally, crashing with TypeError: ‘NoneType’ object is not callable during model warmup. ## Root cause Platform.getattr delegates attribute lookups t…

  • d61d856 #44622 — [Bugfix] Update mistral tokenizer test for continue_final_message fix (#44622)
    • 作者: XuZhou | +2/-2 | 1 个文件

    Fix flaky test caused by upstream mistral-common bug fix. mistral-common PR #233 fixed a bug where Tekken normalizers (V7/V15) did not forward continue_final_message, causing a spurious EOS token (, token id 2) to be appended even when continue_final_message=True. After upgrading mistral-common, the expected values in tests/tokenizers_/test_mistral.py became stale. This PR removes the trailing…

  • 6542d48 #44618 — [Bugfix] Fix test_invocations flaky failure with newer openai SDK (#44618)
    • 作者: XuZhou | +8/-2 | 1 个文件

    Fix flaky test_invocations failure caused by openai SDK version mismatch. The test compared keys from client.chat.completions.create(…).model_dump() (openai SDK) against raw JSON from the /invocations endpoint (via requests). Starting with openai SDK >=2.32, model_dump() injects extra client-side fields (e.g. moderation) that are not present in the raw server response, causing: AssertionError: a…

  • ca73293 #44620 — [Bugfix][Rust Frontend] Fix UTF-8 char-boundary panic in incremental detokenizer (#44620)
    • 作者: Ting SUN | +25/-0 | 1 个文件

    The Rust frontend’s incremental detokenizer (DecodeStream::next_chunk in rust/src/tokenizer/src/incremental.rs) slices cumulative_output at a hold-back cutoff computed as a raw byte offset (len - min_bytes_to_buffer, where min_bytes_to_buffer comes from the request’s stop-string length), without aligning to a UTF-8 char boundary — unlike push_token and flush, which both use floor_char_boundary. Wh…

🔩 Misc

  • eafbb06 #44593 — [Misc] Replaced asserts with proper exceptions to improve UX for pooling (#44593)
    • 作者: Taneem Ibrahim | +44/-16 | 7 个文件

    Follow-up to #43286. Replaced the remaining ~10 assert statements across the pooling subsystem with proper exceptions to improve UX. - No actual functional changes. - Pre-commit hooks ## Test Result - Pre-commit hooks pass

📦 Other

  • c8beda4 #44213 — [Rust Frontend] Add Phi-4 mini JSON tool parser (#44213)
    • 作者: Devin Lai | +346/-4 | 6 个文件
    • Add a Rust frontend phi4_mini_json tool parser for Phi-4 mini functools[…] tool-call output. - Register the parser by explicit name and expose it through the chat tool parser factory. - Cover complete parsing, streaming deltas, split markers, parallel calls, nested arguments, parameters fallback, and malformed empty/truncated calls. ## Duplicate-work checks - gh pr list –repo vllm-project/vll…
  • 2f27c9a #44574 — Preserve layout-changing clones (#44574)
    • 作者: Mike G | +48/-0 | 2 个文件

    Preserve layout-changing aten.clone operations in the unsafe Inductor clone- elimination pass. The pass should still eliminate redundant clones, but it must not remove clones that materialize a different stride or storage offset from the input. This fixes the Mamba padded/sliced output case where .contiguous() is required to produce the compact layout expected by downstream custom ops. Validated o…

  • a50e675 #44021 — [Cohere] fix RoutingMethodType (#44021)
    • 作者: Terrence Zhao | +7/-3 | 3 个文件

    Cohere models have optional renormalize in router function, fix the RoutingMethodType if the router doesn’t have renormalize ## Test Result —

  • 4200f62 #42832 — [ROCm][GPT-OSS] Fuse RoPE + static Q FP8 quant on fused RoPE+KV path (#42832)
    • 作者: akii96 | +451/-1 | 2 个文件
    • Add a dedicated compile-time fusion for RoPE(q, k) -> static Q FP8 quant -> KV-cache update. - Reuse fused_rope_and_unified_kv_cache_update for the RoPE + KV-cache update portion, then apply static Q FP8 quantization to the fused Q output. - Keep ROCm AITER rotary pattern matching aligned with the lowered op signature by passing offsets=None only for the AITER triton rotary op. - Add targeted co…
  • c73b0d0 #44669 — [Core][Engine] allow DP ray placement groups to be set on specific nodes (#44669)
    • 作者: Walter Beller-Morales | +173/-11 | 3 个文件

    When running a DP deployment in raycreate_dp_placement_groups scans the whole Ray cluster and greedily packs DP ranks wherever there are free devices. When multiple independent DP engines share one Ray cluster (e.g. P/D disaggregation or multi-model serving under an external orchestrator), engines race for the same devices: one engine’s remote ranks land on another engine’s master node, which then…

  • e28e369 #44666 — Male Mergify comment less spammy (#44666)
    • 作者: Harry Mellor | +3/-0 | 1 个文件

    The comment became spammy when we added on: pull_request: types: [labeled] to trigger pre-commit. This is because there is often a pre-commit job ongoing when labels are added or removed, causing the ongoing job to be cancelled. Mergify considers a cancelled job as a failure, which triggers the comment. This PR adds more conditions so that the comment is only left on PRs that have either ready/ver…

  • b593396 #44621 — Upgrade tpu-inference to v0.21.0 (#44621)
    • 作者: Sting Lin | +1/-1 | 1 个文件

    Upgrade tpu-inference to latest stable release v0.21.0 Verified on tpu-inference CI. ## Test Result Success. —

  • 91e17d4 #38804 — Fix sarvam forward compatibility with transformers v5 (#38804)
    • 作者: Flame | +34/-1 | 1 个文件

    This PR fixes issue #38734. ### Root cause sarvam_mla has a custom PretrainedConfig that is incompatible with transformers v5 as the ignore_keys parameter which was supported in previous versions of transformers was removed from the validate_rope() method. This was replaced with a class var in PretrainedConfig ignore_keys_at_rope_validation. The problematic line is at line 134 in the sarvam config…

  • 6a89457 #41968 — Add objectstore as a secondary tier to multi-tier kv cache offloading (#41968)
    • 作者: Effi Ofer | +659/-2 | 6 个文件

    This PR adds an object store secondary tier to multi-tier kv cache offloading. The actual implementation uses Nvidia NIXL for accessing the object storage. Using the object store secondary tier, any s3 type object store can be used by passing the object store configuration to kv-connector-extra-config, for example: For more details on multi-tier see the multi-tier RFC here: https://github.com/vllm…

  • 7f003a1 #44609 — Support MiniCPMV batched preprocessing (#44609)
    • 作者: Yan Ma | +63/-56 | 1 个文件

    This PR refines the vendored MiniCPMVProcessor to support batched text+image preprocessing, referring the upstream MiniCPM-V processor code. It fixes below error when run python3 ./examples/generate/multimodal/vision_language_offline.py -m minicpmv: ## Test Result —

  • a80af24 #44635 — Speed up docs build (#44635)
    • 作者: Harry Mellor | +234/-159 | 32 个文件

    In my local the combination of these changes reduces the build time from 376s to 275s. This is a ~27% decrease which will noticeably improve build and queue times in CI. The performance affecting changes are: - Exclude vendored HF processor & config classes from API reference - these model specific classes are not important to include in the API reference - Removing separate_signature and show_sig…

  • 6a11d72 #44588 — [Reasoning][Structured Outputs] Add Command A plus tags for structural tags (#44588)
    • 作者: rishitdholakia13 | +1/-1 | 1 个文件

    This PR adds Command A+ to reasoning parser to the Cohere2MoeForCausalLM architecture.

  • 02d2da0 #44561 — [DSV4] Move more ops out of eager breakpoint (#44561)
    • 作者: Woosuk Kwon | +30/-14 | 1 个文件

    Same config each run (MTP-3, 8192-in/1024-out, low_entropy), 3 full sweeps per version, mean±std (population stdev):

  • 62215e7 #43167 — Remove KV cache scale boilerplate from model weight loading methods (#43167)
    • 作者: Harry Mellor | +88/-731 | 56 个文件

    The general changes in this PR are: - Converts get_cache_scale into get_cache_scale_mapper - Use this new mapper at the top level of AutoWeightsLoader.load_weights - Add KVCacheScaleParameter for BaseKVCacheMethod so that coersion to scalar happens automatically This allows us to: - Remove ~10 lines of boilerplate from every load_weights method in modelling code - Every new model going forward wil…

  • 7fe7800 #43150 — [BUG] Fix FP64 Gumbel precision coverage (#43150)
    • 作者: Tianyu Zhang | +391/-21 | 11 个文件

    The existing –use-fp64-gumbel flag only covered the explicit Triton Gumbel sampler. V1 sampling and spec decode also use the equivalent exponential-race form q.exponential_(); probs / q; argmax, so those paths still used fp32 exponential noise even when the precision flag was enabled. Thread use_fp64_gumbel through the Python V1 sampler, TopKTopPSampler, rejection sampler recovery sampling, and L…

  • 8a83e6f #44591 — [Rust Frontend] Batch auto-abort requests by engine (#44591)
    • 作者: HueCodes | +118/-25 | 2 个文件

    Coalesce Rust frontend auto-abort requests by engine before sending Abort messages. This reduces IPC round trips when many live streams are dropped at once. The abort worker still skips inactive requests and keeps the existing per-cause logging behavior. ## Test Result All passed. cargo test: 76 passed, 0 failed, 0 ignored, 0 doctests.

  • efc347f #44066 — docs: fix tokenizer optimization typo (#44066)
    • 作者: Chunyang Wen | +1/-1 | 1 个文件

    Fix a typo in the doc: Modes -> Models Build and check the pages. ## Test Result Expected —

  • d98b8f3 #43874 — [NixlConnector] Initiate deprecation cycle for kv_both role (#43874)
    • 作者: Nicolò Lucchesi | +94/-24 | 10 个文件

    Implement “Phase1” of the deprecation strategy described in https://github.com/vllm-project/vllm/issues/43807. That is a “soft deprecation” in which: - we change all official examples to refer users to using kv_producer/consumer roles - we warn users that kv_both is deprecated when detected - no effective functional change is expected with this PR, roles are not yet used/assumed We’ll follow up wi…

  • e64237a #44391 — [Rust Frontend] Support include_reasoning=false (#44391)
    • 作者: Chao-Ju Chen | +544/-26 | 4 个文件

    This PR adds Rust frontend support for include_reasoning=false in OpenAI chat completions. - Accept include_reasoning=false during chat request compatibility validation. - Preserve the request flag through chat request preparation. - Suppress reasoning output in non-streaming responses and streaming delta.reasoning chunks when disabled. - Add focused unit and route coverage for the non-streaming a…

  • d2f70da #44603 — fix: pad dummy run query_start_loc (#44603)
    • 作者: Uranus | +3/-0 | 1 个文件

    Hi from novita.ai team 👋 The decoder instance crashes when running GLM-5.1-FP8 in disaggregation way. Cuda coredump shows: By adding log before every torch.repeat_interleave call, I got: The root cause it that with dummy run, query_start_loc is not a monotonic sequence. Run same requests with the fixed decoder. ## Test Result The decoder worked fine for several hours. —

  • ef3af56 #44617 — Fix LLM.wait_for_completion output type docstring (#44617)
    • 作者: Vic Wen | +2/-1 | 1 个文件

    Fixes #44616. LLM.wait_for_completion() currently writes output_type as defaulting to RequestOutput, but the implementation accepts both RequestOutput and PoolingRequestOutput when output_type doesn’t provided: This PR updates the public API docstring so it matches the actual behavior. —

📖 Documentation

  • f6a708a #44435 — [Doc] Add Llama-3.2-3B-Instruct to batch-invariance tested models (#44435)
    • 作者: Daoyuan Li | +2/-5 | 1 个文件

    Adds meta-llama/Llama-3.2-3B-Instruct to the batch-invariance Tested Models list — the gap in the Llama 3.2 line (3.1-8B and 3.2-1B were already listed). Discussed in #27433 (@yewentao256 gave the go-ahead). ## Validation Validated on an RTX 4080 (Ada Lovelace, SM 8.9) with VLLM_BATCH_INVARIANT=1, vLLM v0.22.0, default V1 model runner: | Test | Backend | Result | | — | — | — | | test_log…

🧪 CI/Tests

  • ef0df7d #44647 — [CI] Bump mypy version 1.19.1 -> 1.20.2 (#44647)
    • 作者: Harry Mellor | +18/-34 | 9 个文件

    In my local environment this speeds up pre-commit run -a –hook-stage manual mypy-3.10 from ~11s to ~8s, which is a ~27% improvement. Also, the import follow skipping that happens in local mypy was removed a few weeks ago so it’s no longer true that mypy runs differently in CI. I’ve updated the documentation/comments/mypy.py to reflect this.

  • c66b198 #44649 — [CI] Bump mistral-common (#44649)
    • 作者: Harry Mellor | +7/-7 | 7 个文件

    This PR updates mistral-common in CI.