共 37 个 commit，涉及 169 个文件，+4236/-3092 行变动。

概要

统计项 数值
Commit 数 37
变更文件 169
新增行数 +4236
删除行数 -3092

Commit 列表

🧪 CI/Tests

  • 29c0ec4 #43164 — [ci] Move 3 entrypoints tests to h200_35gb queue (#43164)
    • 作者: Kevin H. Luu | +4/-1 | 2 个文件
    • Move Entrypoints Unit Tests, Entrypoints Integration (LLM), and Entrypoints Integration (Responses API) from the implicit gpu_1_queue path to the h200_35gb MIG queue. - Make the basic parser-context smoke request a concise answer so Qwen3 does not exhaust the 6,000-token test context while solving a multiplication prompt on H200. - Refresh the branch against current main while preser…
  • 088c0be #48771 — [CI] Fix macOS wheel release annotation context (#48771)
    • 作者: Kevin H. Luu | +1/-1 | 1 个文件
    • add the required release-wheels annotation context to the macOS arm64 wheel upload step - keep the macOS wheel artifact in the same dedicated annotation as the other wheel builds ## Root cause The macOS wheel upload step was added while the release-artifact annotation split was in flight. It retained the helper’s old two-argument invocation, but the merged helper requires a third context argumen…
  • 867ff69 #48772 — [CI] Gate non-default release wheel builds (#48772)
    • 作者: Kevin H. Luu | +69/-30 | 4 个文件

    Regular release-v2 builds currently build every wheel variant after each merge to main. This change limits the automatic wheel group to: - Linux x86_64, CUDA 13.0 - Linux aarch64, CUDA 13.0 CUDA 12.9, CPU, macOS, and ROCm wheel work remains available behind unblock steps for on-demand builds. Builds with NIGHTLY=1 bypass those blocks and run the full wheel set. The index scripts also avoid letting…

🐛 Bug Fix

  • c7ce03b #48988 — [Bugfix] Bump tml-fa4 for cutlass-dsl 4.6 API compatibility (#48988)
    • 作者: Michael Goin | +1/-1 | 1 个文件

    Bump tml-fa4 to include the cutlass-dsl 4.6 CuTe API migration (https://github.com/vllm-project/tml-fa4/pull/3) so Inkling Blackwell attention imports cleanly.

  • d96aee0 #48025 — [Bugfix] Re-sync parameter tp_rank after process_weights_after_loading (fix replicated / disable_tp weight reload) (#48025)
    • 作者: alexxu-roblox | +23/-7 | 3 个文件

    Weight reload can crash for replicated (disable_tp=True) parameters such as the DeepSeek-V2/V3 / GLM MLA fused a-projection (q_a_proj + kv_a_proj_with_mqa when q_lora_rank > 0). BasevLLMParameter.init stamps self.tp_rank with the global rank. It is reconciled to the layer’s tp_rank (0 for disable_tp) by update_param_tp_status(), but that only runs at construction. When a parameter is re-create…

  • f12b80c #43979 — [ROCm][Bugfix] Fix GPT-OSS Quark MXFP4 MoE loading - emulation buffer not block-aligned (#43979)
    • 作者: xuebwang-amd | +59/-7 | 3 个文件

    Purpose Quark-quantized MXFP4/FP8 GPT-OSS loading was first introduced in #29008, and has since been refactored and extended by several follow-ups. Those changes correctly added OCP-block-aligned TP slicing in gpt_oss.py, but the emulation path in quark_moe.py was left out of the size-rounding logic, so today the emulation backend is broken under TP (traceback below). ### Environment - ROC…

  • 41ea2dd #47680 — [Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680)
    • 作者: aoshen02 | +106/-40 | 11 个文件

    Fix prompt_logprobs to respect the logprobs_mode configuration setting across both V1 and V2 GPU model runners. Previously, prompt_logprobs always returned log_softmax results regardless of logprobs_mode. When a user set logprobs_mode=“raw_logits”, output logprobs correctly returned raw logits, but prompt logprobs silently ignored the mode and returned logprobs instead. ### Changes - **V1 runn…

  • c4cd2bd #46115 — [Bugfix] MoRIIO toy P/D proxy: fix DP-rank index aliasing + harden for high-concurrency bursts (#46115)
    • 作者: limeward | +268/-13 | 2 个文件

    Two independent fixes to the example MoRIIO disaggregated P/D proxy (examples/disaggregated/disaggregated_serving/moriio_toy_proxy_server.py). ### 1. DP-rank routing index aliasing (the main fix) The proxy chose a target instance and an in-instance DP rank from the same request counter: This indexing strategy leads to aliasing when len(prefill_instances) shares a common factor with dp_si…

  • bf578e1 #48729 — [Bugfix][GLM4V] Fix video dummy profiling and memory usage (#48729)
    • 作者: labAxiaoming | +72/-49 | 2 个文件

    [Bugfix][GLM4V] Fix video dummy profiling and memory usage This PR fixes GLM4V video profiling during multimodal startup. It addresses an unbounded frame search, incorrect selection on a non-monotonic token curve, video pixel-budget mismatch, and excessive dummy video memory usage. This follow-up was discovered while validating PR #47155. Startup was fast when video inputs were disabled: After e…

  • 11d2915 #48846 — [Bugfix][Tool Parser] Preserve whitespace in parameter values (MiniMax M2, Qwen3, MiniCPM5 XML) (#48846)
    • 作者: mosya415 | +164/-9 | 7 个文件

    Several tool parsers apply str.strip() to parameter values, silently corrupting any string argument whose value legitimately begins or ends with whitespace: leading indentation and trailing newlines are deleted. This breaks exact-string-match edit tools used by coding agents — an old_string like " if x:\n return 1\n" parses as “if x:\n return 1”, so the client-side match fails or the edit lands wi…

  • f38f3d1 #48596 — [Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
    • 作者: AlexHuang | +116/-31 | 3 个文件

    Fix: last KV block that fills at request finish time is never offloaded, causing prefix cache misses for that block. Also fixes a data corruption race when the block is reused. Bug principle: _build_store_jobs runs during schedule() — at that point the finishing token (EOS) hasn’t been processed yet, so the last block is still partial and skipped. request_finished is called after EOS is append…

  • 7b31925 #48699 — [Bugfix]Fix transformer backend failed: AttributeError: ‘Parameter’ object has no attribute ‘weight_loader’ (#48699)
    • 作者: Yejing Lai | +13/-12 | 1 个文件

    Reproduce: python3 -m vllm.entrypoints.openai.api_server –model deepseek-ai/DeepSeek-R1-Distill-Llama-8B –enforce-eager –port 8000 –host 0.0.0.0 –model_impl transformers –trust-remote-code –gpu-memory-util=0.9 –no-enable-prefix-caching –max-num-batched-tokens=8192 –max-model-len=8192 –quantization fp8 will meet transformer backend failed: AttributeError: ‘Parameter’ object has no attrib…

  • ce4bdcb #48855 — [Bugfix] Enable FlashAttention MLA prefill for Mistral Small 4 head dims (#48855)
    • 作者: Julien Denize | +10/-5 | 3 个文件

    Mistral Small 4 uses MLA head dimensions (qk_nope_head_dim=64, qk_rope_head_dim=64, v_head_dim=128). FlashAttnPrefillBackend.supports_mla_dimensions did not list these dims, so the model was excluded from the FlashAttention MLA prefill backend. This adds the config to the supported set (FA2/FA3/FA4), extends the selector unit test, and updates the attention backends doc. - A/B probe on H200 (FA3) …

  • 69d4f5e #46213 — [Bugfix][Multimodal] Fix Qwen3-Omni use_audio_in_video with mixed image/video inputs (#46213)
    • 作者: wendadawen | +266/-83 | 8 个文件

    Fix three bugs in Qwen3-Omni multimodal input validation when use_audio_in_video=True: 1. Single video + image inputs (V+I, I+V) crashed EngineCore because the deepstack interleaved branch only treated video positions as vision positions and missed image positions. 2. Interleaved embedding merge inferred modality from token counts and could misclassify image embeddings as video in mixed image/vide…

📦 Other

  • c233d90 #48496 — Remove even more unnecessary load_weights methods (#48496)
    • 作者: Harry Mellor | +639/-2076 | 44 个文件

    Aside from deleting boilerplate, the new things in this PR are: - Extract and deduplicate weight filters used by many models: - maybe_fuse_shared_experts - the same code was repeated in several models for enabling the AITER fused-shared-experts optimisation - get_spec_layer_idx_from_weight_name/skip_spec_layers - the same code was repeated in several models for skipping the loading of the spec dec…

  • da64db7 #48759 — [LoRA] Optimize TrtLlmLoRAExperts (#48759)
    • 作者: Jee Jee Li | +221/-41 | 2 个文件

    part of https://github.com/vllm-project/vllm/pull/47226 ## Test Result —

  • 425c4ea #48641 — [Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641)
    • 作者: Michael Goin | +85/-25 | 5 个文件

    apply_sampling_params copies the logits into a new fp32 tensor before applying sampling params. Generation models emit bf16 logits by default (except in RL), so this copy is a pure upcast that costs num_logits * vocab * 4 bytes. Under spec decode there are num_reqs * (1 + num_spec_tokens) logits instead of num_reqs, so the copy reaches multiple GiB. On Qwen3-8B + a dflash speculator it tries to al…

  • 02c01f4 #48990 — [Model] Use standard ModelOpt config for Inkling NVFP4 (#48990)
    • 作者: Michael Goin | +56/-110 | 7 个文件

    Use vLLM’s standard ModelOpt quantization infrastructure for Inkling NVFP4. - Detect legacy ModelOpt hf_quant_config.json files without producer.name. - Pass the global quantization config to Inkling MoE. - Remove the duplicate Inkling-specific NVFP4 config parser and per-layer config construction. - Preserve Inkling’s checkpoint-specific expert tensor loading. ## Test Result Validated on 4xB300 g…

  • fcd2255 #37524 — [Hardware][GPU] Profiler config additional to increase it scope and annotation details (#37524)
    • 作者: devalshahamd | +240/-26 | 4 个文件

    This pull request introduces enhanced configurability and profiling capabilities for CUDA graph capture and trace annotation in the codebase. The main improvements include new options for capturing profiler traces during CUDA graph capture, the ability to generate more detailed trace annotations with KV length information, and the integration of these features into the profiling and model capture …

  • cc25f02 #46868 — [Loader] Improve InstantTensor loading (#46868)
    • 作者: Michael Goin | +27/-14 | 9 个文件

    Keeps InstantTensor opt-in (load_format=“instanttensor”) but takes the good parts from #37309: the progress bar now reports load throughput in GB/s. We also bump to instanttensor>=0.1.9 and use copy=True so tensors own their memory, dropping the redundant vLLM-side .clone(). ### History and non-duplication This revives the existing InstantTensor implementation PR after merging current main. An ope…

  • 5784507 #48012 — [Attention] Allow selecting a different attention backend per KV-cache group (#48012)
    • 作者: Nicolò Lucchesi | +258/-3 | 4 个文件

    vLLM allows picking a single global attention backend for the whole model. Models that split layers across multiple KV-cache groups are forced to specify one backend, though the runtime (AttentionGroup) already supports heterogeneous backends, so all the plumbing is already in place, we just lack the UX to let the user enforce a choice. Most notably, hybrid SSM models is one example where diff…

  • c4dd6d7 #48849 — Fix: Restore data_parallel_size > 1 for use_sequence_parallel_moe (#48849)
    • 作者: passtoor-agi | +1/-0 | 1 个文件

    Fix memory regression for MoE models on pure TP configurations (DP=1). Root cause: Commit 1ff942965 (PR #48036) removed and self.data_parallel_size > 1 from use_sequence_parallel_moe in vllm/config/parallel.py, causing Sequence Parallel MoE to incorrectly enable for TP-only (DP=1) configurations. SP MoE is designed to avoid token replication across DP ranks (see comment lines 633-639 in pa…

  • ce2aecc #48417 — [Performance] Use CuTe-DSL for FlashInfer MXFP4 quantization (#48417)
    • 作者: JooHo Lee | +6/-2 | 2 个文件

    Use the CuTe-DSL backend for activation quantization in FlashInferMxFp4LinearKernel. The kernel is already selected only on SM100+ when FlashInfer CuTe-DSL is available, and its following MXFP4 GEMM already uses the CuTe-DSL backend. This change makes the activation quantization use the same backend while keeping backend selection explicit at the kernel call site. Fixes #48205. The default CUDA ba…

  • d4b4562 #48942 — [XPU] Bump vllm_xpu_kernels to v0.1.11.1 (#48942)
    • 作者: Artur Fierka | +1/-1 | 1 个文件

    Bumps vllm_xpu_kernels pin from v0.1.11 to v0.1.11.1. v0.1.11 registers moe_sum with the old 2-arg XPU schema. Commit f7aadae5e5 added 2 extra args (topk_ids, expert_map) to the CUDA-side moe_sum op, but the XPU-side registration was never updated to match — every XPU MoE forward pass now crashes: v0.1.11.1 includes vllm-xpu-kernels#462, which fixes this. Verified on B70: reproduced the crash on t…

  • 4c6e2e4 #47516 — [XPU][UT]fix _POSSIBLE_KERNELS error on XPU (#47516)
    • 作者: Yejing Lai | +2/-2 | 1 个文件

    Root cause: choose_scaled_mm_linear_kernel did a direct dict index possible_kernels [current_platform._enum]. POSSIBLE INT8 KERNELS has no PlatformEnum.XPU entry, so this raised an unhandled KeyError instead of the intended “no kernel found” error. Fix: Changed both lookups to .get(current_platform._enum, []), so an unsupported/unregistered platform falls through to the existing ValueError (“Faile…

  • 8502958 #47975 — [XPU] support HND layout (#47975)
    • 作者: liuzhenwei | +0/-8 | 1 个文件

    with PR https://github.com/vllm-project/vllm/pull/44455 and https://github.com/vllm-project/vllm-xpu-kernels/pull/432 Now we can support HND layout in XPU, this unblocks PD disaggregation with heterogeneous TP. ## Test Result —

  • d5b1ec2 #48828 — [XPU] allow forcing flash attn for mm_prefix (#48828)
    • 作者: liuzhenwei | +12/-6 | 1 个文件

    On XPU, multimodal prefix-LM models (e.g. Gemma4-26B-A4B-it) were unconditionally forced to Triton Attention (https://github.com/vllm-project/vllm/pull/47688) because XPU Flash Attention lacks FA4 and cannot apply the bidirectional mask for vision tokens. This prevented users from using Flash Attention even for text-only workloads for better performance where the mask is irrelevant. This PR allows…

  • 109b736 #48839 — [docs] preserve page path in stable-docs announcement link (#48839)
    • 作者: Sage | +1/-1 | 1 个文件

    the banner always sends users to the stable docs homepage instead of the matching page. verified locally

  • 426d48b #48281 — [KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
    • 作者: Chang Guo | +326/-10 | 11 个文件

    Add optional locality metadata to KV events emitted by the built-in FS and OBJ secondary tiers. locality describes storage placement relative to the publishing vLLM instance. LOCAL means the storage is local to that instance; REMOTE means it is not. The field does not define an access path, routability, or latency. - Accept LOCAL or REMOTE in FS and OBJ tier configuration. - Treat an omitted value…

  • 26c909e #41599 — [Model] Support TranslateGemma-12b-it (#41599)
    • 作者: Zhang Jian | +174/-3 | 2 个文件

    Closes https://github.com/vllm-project/vllm/issues/41540. Related to https://github.com/vllm-project/vllm/issues/32446 This PR is copied over from https://github.com/vllm-project/vllm/pull/32819, with comments addressed. - TranslateGemma’s chat template requires extra metadata (source_lang_code, target_lang_code) that is not compatible with exsiting OpenAI chat schema. This PR allows extra fie…

  • fb1d8cc #48042 — [rl] Stateful Trainer Send: New Abstractions [1/N] (#48042)
    • 作者: Aaron Hao | +644/-24 | 7 个文件

    Context This is the first of a three-PR split of the trainer-side weight-transfer rework originally proposed as one large PR. The split is: - PR 1 (this one): introduce the new trainer-side abstractions (WeightSource / ModuleSource, VLLMWeightSyncClient, TrainerWeightTransferEngine, WeightTransferTrainerFactory). Purely additive — the existing worker-side WeightTransferEngine and every back…

⚡ Performance

  • c71a583 #48110 — [Perf][Hybrid] Vectorize _copy_mamba_state_block to uint64 for temporal (#48110)
    • 作者: Francesco Fusco | +54/-16 | 1 个文件

    PR #40172 introduced the postprocess_mamba_fused_kernel in MRV1 to copy states in hybrid models when prefix caching is enabled in align mode (and under MTP). Every accepted draft step that crosses a block boundary triggers a full sweep of (num_reqs × total_states) state copies — for Qwen/Qwen3.5-9B that’s 24 linear-attention layers × (conv + temporal) = 48 state copies per accepted step, ~2 MiB te…

  • b5433b6 #48660 — [Perf] Optimize dsv4 routing using specialized kernel, 2.94% E2E TPOT improvement (#48660)
    • 作者: Wentao Ye | +264/-0 | 4 个文件

    Thanks the context from @zyongye ! vllm serve deepseek-ai/DeepSeek-V4-Flash -tp 4 -ep –attention-backend FLASHINFER_MLA_SPARSE_DSV4 –kv-cache-dtype fp8 –tokenizer-mode deepseek_v4 –all2all-backend allgather_reducescatter –port 8003 ### Acc lm_eval –model local-completions –model_args “base_url=http://127.0.0.1:8003/v1/completions,model=deepseek-ai/DeepSeek-V4-Flash,num_concurrent=1024” –ta…

  • efed8a1 #48788 — [ROCm][Perf][DSV4] Improve sparse decode reduction occupancy on gfx950 (#48788)
    • 作者: Fangzhou Ai | +2/-2 | 1 个文件

    The split-K sparse decode reducer currently processes 16 heads per workgroup, which keeps a [16, COMB_DIM] FP32 accumulator in each workgroup and limits workgroup-level parallelism. This change processes one head per reducer workgroup. It lowers per-workgroup accumulator/register pressure and exposes up to 16 times more independent workgroups while preserving the per-head reduction math and orderi…

🦀 Rust Frontend

  • fae5430 #48829 — [Frontend]Flatten beam-search beams with itertools.chain instead of sum (#48829)
    • 作者: Wang Xingda | +3/-1 | 1 个文件

    _beam_search_step flattened beams with sum(…, []), which is O(n²) in the number of beams and runs once per decode step. Replace it with itertools.chain.from_iterable, which is O(n).

🖥️ Kernel

  • c9be3a8 #48797 — [Kernel][Helion] Disable warp specialization in rms_norm_per_block_quant B200 configs (#48797)
    • 作者: Shangdi Yu | +45/-45 | 1 个文件

    Some autotuned configs for the rms_norm_per_block_quant Helion kernel set range_warp_specializes=true on the group loop. On B200/sm_100 this triggers a Triton compiler crash in TritonGPURemoveLayoutConversions: a convert_layout op synthesized after automatic warp specialization does not carry the expected ttg.partition attribute, so IR verification fails. See triton-lang/triton#10901 for the minim…

🔧 Refactor

  • 877dae9 #48780 — [Refactor] Remove deepseek dead code (#48780)
    • 作者: Wentao Ye | +3/-387 | 7 个文件

    Remove deepseek dead code