共 67 个 commit，涉及 256 个文件，+7810/-2424 行变动。

概要

统计项 数值
Commit 数 67
变更文件 256
新增行数 +7810
删除行数 -2424

Commit 列表

📦 Other

  • 6e96891 #48683 — [ROCm] Bump AITER to v0.1.16.post5 (#48683)
    • 作者: Fangzhou Ai | +1/-1 | 1 个文件

    Advance the ROCm image build from AITER v0.1.16.post3 to v0.1.16.post4. This brings the gfx950 MXFP4 MoE path, top-k and MLA metadata/reduction optimizations, and AITER’s FlyDSL 0.2.2 dependency. FlyDSL does not need a separate vLLM pin because the AITER build stage installs AITER’s requirements.txt before building its wheel. The tested AITER source commit was 72eedddd921d0fb8f274e7b9291dd04306d08…

  • 47f1b47 #49241 — Ci/add laguna xs gsm8k (#49241)
    • 作者: Michael Goin | +10/-0 | 2 个文件

    Just porting over https://github.com/vllm-project/vllm/pull/42071 to use the 1xB200 instead of 8xH200 runner ## Test Result —

  • c67650f #45991 — [XPU][DeepSeekV4]Add DeepSeek-V4 fuse_index_q SYCL kernel path (#45991)
    • 作者: Avery Miao | +150/-0 | 2 个文件

    This PR wires DeepSeek-V4 fused indexer Q (RoPE + quantization) to the SYCL kernel path on XPU, replacing the Triton fallback. ## Code change - vllm/models/deepseek_v4/common/ops/fused_indexer_q.py: - Add current_platform.is_xpu() dispatch before the existing has_cutedsl() / Triton branches. - XPU calls torch.ops._xpu_C.deepseek_fused_indexer_q_rope_fp8 (FP8 path) or torch.ops._xpu_C.deepseek_fuse…

  • 6700813 #44456 — [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions (#44456)
    • 作者: Lucas Wilkinson | +115/-225 | 10 个文件

    PR #42374 (first part of RFC #42082) has been split into 4 PRs: #44454 [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config #44455 [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends -> #44456 [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions #44458 [4/N][KV-Cache Layout Refactor] Standardize KV cache layout Move M…

  • 1134545 #48641 — Revert “[Sampler] Stop upcasting logits to fp32 in apply_sampling_params” (#48641) (#49033)
    • 作者: vllm-agent | +25/-85 | 5 个文件

    apply_sampling_params copies the logits into a new fp32 tensor before applying sampling params. Generation models emit bf16 logits by default (except in RL), so this copy is a pure upcast that costs num_logits * vocab * 4 bytes. Under spec decode there are num_reqs * (1 + num_spec_tokens) logits instead of num_reqs, so the copy reaches multiple GiB. On Qwen3-8B + a dflash speculator it tries to al…

  • adfbbc1 #49177 — Propagate Flash Attention cache configuration to Ray workers (#49177)
    • 作者: Stefan Kaestle | +1/-0 | 1 个文件

    This change adds FLASH_ATTENTION_ to the environment-variable prefixes propagated from the vLLM driver to Ray workers. Flash Attention’s compilation cache is configured through environment variables. Without propagating them, Ray workers do not use the configured cache and may repeat expensive compilation during worker startup or scaling operations. This keeps the cache configuration consistent be…

  • d9aa351 #49269 — Update BGE-M3 token expectations for leading spaces (#49269)
    • 作者: aoshen02 | +6/-6 | 1 个文件

    Update the BGE-M3 sparse plugin expectation for the leading spaces restored by #48674. The shared _check_sparse_embedding() expectation covers the online and offline return_tokens=True paths. The production helper and all other code remain unchanged in this PR. ## Root cause SentencePiece pieces such as ▁What now decode to " What" so distinct token IDs remain distinguishable. The old test expected…

  • ea0e9c8 #47985 — [MRV2] Add encoder cache profiling implementation (#47985)
    • 作者: Isotr0py | +93/-2 | 4 个文件

    Test Result —

  • 1940c84 #48992 — [Rust Frontend][gRPC] Add engine-aware health reporting (#48992)
    • 作者: Connor Carpenter | +333/-40 | 9 个文件

    Expose the standard grpc.health.v1.Health service on the existing Rust frontend gRPC listener. - Register health on the same listener as vllm.Generate. - Report the overall server and vllm.Generate as SERVING after startup. - Publish the engine client’s sticky healthy-to-unhealthy transition through a Tokio watch channel. - Report NOT_SERVING when the engine becomes unhealthy or shutdown begins. -…

  • 97a9800 #47879 — Update qutlass cmake for stable abi (#47879)
    • 作者: Chris Leonard | +11/-9 | 2 个文件

    Updates the vLLM QuTLASS CMake integration to build against a stable-ABI QuTLASS fork (see https://github.com/cleonard530/qutlass/pull/1 https://github.com/IST-DASLab/qutlass/pull/12), enabling the _qutlass_C library to become torch abi stable. ~~This is just to test the CI and make sure these updates work. We will need to figure out if we want to try and push these updates upstream to IST-DAS…

  • 0d9210a #47268 — Fixes non-coalesced HBM access in marlin_int4_fp8_preprocess_kernel_awq (#47268)
    • 作者: flutist-alibaba | +11/-7 | 1 个文件

    What does this PR do? Fixes non-coalesced HBM access in marlin_int4_fp8_preprocess_kernel_awq (AWQ int4-fp8 weight preprocessing kernel, model-load cold path). ### Problem Original thread mapping used threadIdx.x for the row dimension: Adjacent threads (tidx 0..31) read elements strided by size_n/8 int32 words — non-coalesced, ~12.5% sector utilization on large matrices. ### Fix Remap threadIdx…

  • af91f4b #49235 — [Cleanup] Remove unused StructuredOutputRequest.status field (#49235)
    • 作者: Nick Hill | +0/-4 | 1 个文件

    This was maybe left over from an earlier implementation; it’s never used. It just makes the code more difficult to follow!

  • 2396a61 #45964 — [Attention][MLA][DCP] Query replication for MLA decode (DeepSeek-V2/R1 + Kimi-K2.5) (#45964)
    • 作者: sungsoo ha | +157/-15 | 6 个文件

    With Decode Context Parallelism (DCP) the KV cache is sharded across the DCP group, so the standard MLA decode path all-gathers the query across the group every step so each rank can attend its KV shard with the full head set, then LSE-reduces the partials. That all-gather sits on the decode critical path. This PR adds an opt-in alternative: replicate the (small) MLA query projection *with…

  • 97a6681 #44214 — [RL Infra][FlashInfer] Enable router replay output from FlashInfer monolithic MoE kernel (#44214)
    • 作者: Xuanyu Zhang | +1099/-12 | 11 个文件

    Add routing_replay_out support to the FlashInfer monolithic MoE backend, enabling the RoutedExpertsCapturer to capture expert routing decisions from the monolithic kernel. Currently, routing capture only works through the modular kernel path via router.set_capture_fn(). The monolithic path fuses routing into the kernel itself, so routing decisions were lost. This PR enabled routing output by passi…

  • 58b2012 #49208 — [copy of #45208] CuMem slept-L1 fragmentation accounting (#49208)
    • 作者: Matthew Bonanni | +73/-46 | 4 个文件

    Co-authored with @terafin. Copy of #45508 with my edits applied.

  • b7c20d0 #48938 — [chore] adjust logo be more friendly to white background terminal (#48938)
    • 作者: Ning Xie | +4/-5 | 1 个文件

    Just a small chore change. By default vllm will log vllm logo and version + model info with colored schema. Currently, the color schema is more friendly to black background terminal. By now, the version and model name and most logo is white which is consistent with white background terminal, thus it is invisible. Change most color to blue to be more friendly to white background terminal NA ## Test…

  • 642076d #48639 — Support loading sample_from_anchor flag from speculators config (#48639)
    • 作者: Fynn Schmitt-Ulms | +24/-6 | 3 个文件

    Add support for loading the sample_from_anchor flag from speculators configs (added in https://github.com/vllm-project/speculators/pull/760). This flag is used for dflash/dspark models to indicate if the anchor slot in each prediction block generates a draft prediction or predicts itself. Support for the flag already exists in vllm, this pr just adds support for loading from speculators config. ##…

  • 9dd62d8 #48952 — Cosmos3 FP8 ModelOpt/Diffusers remapping (#48952)
    • 作者: wkutak | +43/-0 | 2 个文件

    ModelOpt/Diffusers FP8 Cosmos3 checkpoints are exported in a unified format: each quantized Linear ships ModelOpt’s native fake-quant buffers (*.input_quantizer._amax, *.weight_quantizer._amax, *.weight_quantizer._scale) so the checkpoint stays loadable by diffusers/transformers, alongside the weight_scale/input_scale sidecars that vLLM consumes. vLLM’s AutoWeightsLoader is fail-closed: it raise…

  • bd09107 #42569 — [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569)
    • 作者: Matthew Bonanni | +194/-33 | 8 个文件

    Depends on https://github.com/vllm-project/flash-attention/pull/166, which resolves an accuracy issue at long contexts. Implement FP8 KV cache support with FlashAttention 4 GSM8k on Nemotron 3 Nano with ## Test Result | Backend | GSM8K accuracy | Invalid | |—|—:|—:| | FlashInfer | 94.31% | 0.23% | | FA4 | 95.22% | 0.23% | —

  • b23bd73 #47245 — [XPU]add sycl path for Mhc (#47245)
    • 作者: Liangqiusong | +5/-20 | 1 个文件

    This PR replaces the native/Triton-based fallback implementations in the forward_xpu methods of the MHC (Multi-Head Chunking) layer with dedicated SYCL kernel calls for the DeepSeek V4 (DSv4) model on Intel XPU. Specifically, it: Replaces self.forward_native(…) calls with optimized SYCL kernel ops: - torch.ops._xpu_C.mhc_pre (in MHCPreOp.forward_xpu) - torch.ops._xpu_C.mhc_post (in MHCPostOp.for…

  • e2d7ade #49161 — [Rust Frontend] Bump xgrammar-structural-tag and enable local extension (#49161)
    • 作者: Bugen Zhao | +84/-85 | 20 个文件
    • Bump xgrammar-structural-tag to v0.2.0 (which mirrors upstream Python release v0.2.4). - Replace Parser::structural_tag_model() which returns an exhaustive enum Model from xgrammar-structural-tag, with structural_tag_builder() -> Option<&dyn StructuralTagBuilder> where upstream Model automatically implements, while still leave the room for us to add local extensions by creating new types impleme…
  • 15cb8e1 #49159 — [Multimodal] Allow keeping original image mode for ImageIO (#49159)
    • 作者: Isotr0py | +77/-15 | 5 个文件
    • Allow image_mode=None to keep image’s original mode during MediaIO ## Test Result —
  • f007cce #48679 — [KV Offload] Support self-describing KV events with TieringOffloadingSpec (#48679)
    • 作者: Chang Guo | +452/-65 | 6 个文件

    Related to #38260 and follows up on #43468. #43468 added opt-in self-describing KV events for GPU-to-CPU stores, but TieringOffloadingSpec still rejected the flag. Tier promotions also bypass the normal store path, so their CPU BlockStored events had only placeholder payloads. Those placeholders do not include token IDs, a parent hash, or a block size. Without that data, Dynamo cannot index the pr…

  • 530ee36 #49144 — fix(openai): reject non-numeric logprobs with 400 instead of 500 (#49144)
    • 作者: hcl | +49/-0 | 4 个文件

    check_logprobs is a mode=“before” validator, so it compares prompt_logprobs/logprobs (completion) and prompt_logprobs/top_logprobs (chat) against ints on the raw request body. A non-numeric JSON value hits the comparison and raises TypeError → HTTP 500. Repro: Type-guard the fields up front so a bad value returns a clean 400 instead. Numeric input is unchanged; added unit tests for both request ty…

  • 818cf61 #49042 — [Rust Frontend] Fix macro-based content format detection (#49042)
    • 作者: Reid | +34/-3 | 2 个文件

    The Rust chat template detector only recognized loops directly over message.content. It missed templates that pass the content into a macro and iterate with {% for item in content %}. Qwen3.5 uses the macro form, causing auto mode to incorrectly select the string content format. Multimodal messages consequently lost the <|vision_start|> and <|vision_end|> wrappers around <|image_pad|>, which could…

  • 823eaf6 #48334 — [XPU] FP8 o_proj with fp8_bmm and load-time scale transpose (#48334)
    • 作者: Xiaochang Wu | +124/-15 | 5 个文件

    Replace the BF16 reference _o_proj path (rocm_inv_rope_einsum) in XPU DeepSeek-V4 attention with a native FP8 BMM path using fp8_bmm, and optimize scale handling by moving transpose+reshape to load time. vllm/model_executor/kernels/linear/scaled_mm/xpu.py - Restore process_weights_after_loading in XPUFp8BlockScaledMMKernel: - Transpose weight_scale_inv from checkpoint layout [N/128, K/128] to …

  • f1f1259 #48781 — [Rust Frontend] Use zero-copy slicing for multimodal tensors (#48781)
    • 作者: Sage | +91/-99 | 4 个文件

    Summary After multimodal content is prepared, its batched results are separated into individual image or video patches before being sent onward. An extra copy was happening during this handoff. This change removes that redundant copy while preserving the same output. This is safe because it does not change how inputs are processed or what is sent onward; it only removes a temporary duplicate. …

  • df13b5a #47122 — [XPU] [MoE] add quant input when prepare for fusedmoe (#47122)
    • 作者: zofia | +37/-4 | 3 个文件

    Based on https://github.com/vllm-project/vllm/pull/47256 and https://github.com/vllm-project/vllm-xpu-kernels/pull/450

  • 4938d44 #47871 — [CPU] fixes heterogeneous NIXL KV transfer into CPU_ATTN decode workers (#47871)
    • 作者: Sihan Chen | +15/-13 | 2 个文件

    fix heterogeneous NIXL KV transfer into CPU_ATTN decode workers, and resolve Prefill(XPU) -> Decode(CPU) accuracy issue. The accuracy issue was fixed earlier in https://github.com/vllm-project/vllm/pull/38935. However after https://github.com/vllm-project/vllm/pull/44393 , the layout was changed again. This PR fixes the issue. Prefill (XPU) -> Decode (CPU) accuracy is validated locally in prefill …

🧪 CI/Tests

  • 5aab491 #49325 — [CI] Wire tests/models/inkling into a B200 job (#49325)
    • 作者: Nick Hill | +13/-0 | 1 个文件

    The tests/models/inkling suite (8 files: FA4 kernels, sconv metadata, MoE weight layout, contract validation, MTP input fusion, QKV prep, MM towers) was not referenced by any CI job, so none of it ran in CI - including the contract-validation tests that #48822 itself modified. Add a B200 job (the FA4 tests require SM100; they skip elsewhere) to the Models - Basic group. Suite runs in ~6.5 min on 1…

  • 5812e1a #41653 — [Test] Add DeepSeek MTP parallel-load tests (#41653)
    • 作者: Stefano Castagnetta | +265/-0 | 2 个文件

    test_mtp_correctness[deepseek] covers TP=1 only. This adds parallelism coverage for DeepSeek MTP weight loading and basic correctness: TP=2, EP=2, EP=2 + EPLB via the inline LLM constructor, plus DP=2 via AsyncLLM (mirrors tests/v1/distributed/test_eagle_dp.py). Each parametrize entry builds spec and no-spec engines at the same parallelism shape, runs greedy decode under VLLM_BATCH_INVARIANT=1, an…

  • 6bcda97 #49129 — [CI][NIXL] Isolate concurrent engine internal ports (#49129)
    • 作者: Andreas Karatzas | +13/-0 | 1 个文件

    The failing AMD CI run failed while starting the fourth NIXL configuration: concurrent prefiller and decoder EngineCores both selected internal TCPStore port 43991, producing EADDRINUSE and a 1,200-second readiness timeout. No commit in c233d90aa826df072872df47b201450059be8e71..b6ff8a2f509cc7ac9c58176f5115a836aa1e08bd changed the NIXL harness, connector, executor, or port selection. The bad endpoi…

  • 72d16ae #49231 — [CI] Exercise FA3 FP8 attention on SM90 (#49231)
    • 作者: Simon Mo | +6/-3 | 1 个文件

    Follow up on the review feedback in https://github.com/vllm-project/vllm/pull/43024#discussion_r3614823352. The H200 migration had skipped test_online_quantization[…, kv_cache_dtype=fp8] on SM90 with the claim that FlashAttention 3 did not support FP8 attention. FA3 does support FP8 query input. Its implementation selects BF16 output when Q is FP8 and validates a supplied output tensor against t…

  • 5feb395 #49234 — [ROCm][CI] fix test_rocm_quick_reduce.py (#49234)
    • 作者: Charlie Fu | +13/-3 | 1 个文件

    pytest -sv tests/distributed/test_rocm_quick_reduce.py =================== 40 passed, 17 warnings in 174.06s (0:02:54) ======================

🐛 Bug Fix

  • 8950394 #48860 — [Bugfix] Prefix-cache metrics double-counted when a KV connector defers requests (#48860)
    • 作者: Seiji Eicher | +204/-41 | 5 个文件

    Fixes #43736. Whenever the scheduler re-evaluates a waiting request that still has num_computed_tokens == 0, it re-runs the lookup and calls record again, adding the request’s num_tokens to queries and its num_hits to hits. The metrics skew toward the multiply-counted request’s statistics. Sequence: 1. The scheduler runs the prefix-cache lookup for a waiting request with num_computed_tokens == 0 a…

  • 7bb49be #49306 — [Bugfix] Handle MLA fallback during FA4 JIT warmup (#49306)
    • 作者: Roberto L. Castro | +5/-1 | 1 个文件

    Follow-up to #47451. As @mgoin correctly noted in https://github.com/vllm-project/vllm/pull/47451#discussion_r3616102551, JIT warmup must handle models for which no generic MLA prefill backend is available. Related auto-revert PR: #49268.

  • f890e1d #48843 — [BugFix] Set graph_pool_id before FULL CUDA graph capture in ModelRunner V2 (#48843)
    • 作者: Ilya Markov | +116/-0 | 2 个文件

    Fixes #48840: AssertionError: graph_pool_id is not set under graph capture when VLLM_USE_NCCL_SYMM_MEM=1 is enabled with ModelRunner V2 and FULL_DECODE_ONLY CUDA graphs. Add set_graph_pool_id() before torch.cuda.graph() in CudaGraphManager.capture() for the FULL capture path. Adds a CPU unit test that asserts graph_pool_id is set before entering torch.cuda.graph(). pytest tests/v1/cudagraph/test…

  • 5b3762a #49021 — [Bugfix][CPU] Fix Clang OpenMP build on macOS (#49021)
    • 作者: Sya | +3/-1 | 1 个文件

    Clang does not support capturing structured bindings in OpenMP regions, causing the CPU attention extension to fail to build on macOS. Replace the structured binding with regular local variables to fix. Tested on Apple M2,

  • 4d30c51 #49190 — [bugfix] Fix Cosmos3 Edge checkpoint weights filtering, video loading, prompt expansion (#49190)
    • 作者: bastefaniak | +30/-3 | 5 个文件
    1. Skip k_norm_und_for_gen weights in generator tower. 2. Route Cosmos3EdgeVideoProcessor through the Qwen3-VL video backend 3. Correct video placeholder expansion to produce timestamped, per-frame tokens Verify the new checkpoint weight is skipped in tests/models/multimodal/test_mapping.py Verify the processor produces the expected number of video tokens and frame wrappers in tests/models/multimo…
  • 3e0c887 #47298 — [Bugfix] Fix Ovis2_5 special tokens for transformers v5 (#47298)
    • 作者: Miłosz Grunwald | +12/-19 | 2 个文件

    Fix Ovis2_5 processor vllm serve AIDC-AI/Ovis2.5-2B –dtype bfloat16 –tensor-parallel-size 1 –max-model-len 8192 –trust-remote-code 2>&1 | tee “serve.log” ## Test Result Without the fix: (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] █ █ █▄ ▄█ (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀…

  • 8def3cd #48917 — [Bugfix] Propagate quant_config to LFM2 ShortConv projections (#48917)
    • 作者: Alex Yuan | +6/-0 | 3 个文件

    LFM2 / LFM2-MoE ShortConv decoder layers construct their projection linears (conv.in_proj, conv.out_proj) without the model’s quant_config, so on a quantized checkpoint those linears are built with no quantization method. Loading a quantized LFM2 checkpoint — e.g. a ModelOpt NVFP4 export, whose conv.in_proj/conv.out_proj carry weight_scale/weight_scale_2/input_scale — therefore crashes during …

  • f25953c #49113 — [Bugfix][Rust Frontend] Handle zero-column logprobs payloads without panicking (#49113)
    • 作者: Salt Sato | +59/-0 | 2 个文件

    The engine-core protocol can carry a logprobs payload with zero scored positions: the natural encoding of an empty position list is a pair of [0, 0] tensors, which is exactly what the crate’s own WireLogprobs::from_direct produces for Logprobs { positions: [] }. The decoder passes the column count straight into slice::chunks, which panics unconditionally on zero (chunk size must be non-zero), so s…

  • 94ed0bf #49146 — [Bugfix][KV Offloading] Handle queued request aborts without allocated KV blocks (#49146)
    • 作者: Chauncey | +49/-6 | 2 个文件

    Fix https://github.com/vllm-project/vllm/issues/49118 Handle queued request aborts without allocated KV blocks ## Test Result before: after: —

  • 0a684ab #48444 — [Bugfix] Fix WSL circular import from pin_memory warning_once (#48444)
    • 作者: Alejandro Paredes La Torre | +4/-2 | 2 个文件

    Fixes #48397. PR #46511 changed Platform.is_pin_memory_available() (interface.py) and CudaPlatformBase.is_pin_memory_available() (cuda.py) to log the WSL pinned-memory-disabled message via logger.warning_once() instead of logger.warning(). warning_once() dispatches through _should_log_with_scope(), which lazily imports vllm.distributed.parallel_state. vllm.utils.torch_utils computes PIN_MEMORY = i…

  • 2e2e626 #48317 — [Bugfix] Count per-group blocks in get_max_concurrency_for_kv_cache_config (#48317)
    • 作者: ormandj | +139/-12 | 2 个文件

    get_max_concurrency_for_kv_cache_config sums every group’s per-request memory and divides once by group 0’s page_size_bytes. For packed UniformTypeKVCacheSpecs, page_size_bytes is aggregated over each group’s actual members, so that normalization is wrong when aggregate group sizes differ. This change sums each group’s whole-block demand: The aggregated worker representation and representative sch…

  • a2b1f9f #49245 — [ROCm] [Release] [Bugfix] Fix the per commit wheel release pipeline. (#49245)
    • 作者: TJian | +2/-2 | 1 个文件

    Test Result —

  • 4ec199b #44492 — [Bugfix][Spec-Decode] Populate draft seq_lens_cpu_upper_bound for spec-decode attention metadata (#44492)
    • 作者: Oxana Korzh | +150/-0 | 4 个文件

    Purpose Fixes ROCm CI failures in speculative decoding (EAGLE / EAGLE3) caused by the draft attention metadata being built without seq_lens_cpu_upper_bound. When the GPU model runner’s draft speculators build the draft CommonAttentionMetadata, they did not populate seq_lens_cpu_upper_bound. Several consumers require this field for spec-decode batches — split_decodes_prefills_and_extends and the ML…

  • fbfe581 #48911 — [Bugfix][KV Offload] Preserve reachable tails for hybrid SWA groups (#48911)
    • 作者: coltonottley | +82/-12 | 2 个文件

    Fix CPU KV offload store pruning for hybrid sliding-window attention when: 1. the group is EAGLE/MTP and lookup requires one extra chunk before dropping the volatile draft tail; or 2. the final alignment segment is shorter than the configured full-attention alignment. The current store predicate retains only sliding_window_size_in_chunks at the end of every configured alignment segment: That disag…

  • f878367 #49193 — [Revert][Bugfix] Restore MiniCPM-V 4.6 ViT QKV weight loader (#49193)
    • 作者: tc-mb | +3/-3 | 1 个文件

    Purpose PR https://github.com/vllm-project/vllm/pull/47058 replaced the MiniCPM-V 4.6 ViT-specific weight loader with the generic WeightsMapper. The generic stacked mapping performs raw substring replacements and continues scanning after a match. As a result, q_proj can first become qkv_proj and then match v_proj inside the rewritten name, producing qkqkv_proj and an incorrect shard ID. This PR re…

  • 0a5069e #48563 — [Bugfix][Gemma4] Fix ModelOpt mixed-precision MoE config mapping (#48563)
    • 作者: wangqian | +45/-4 | 3 个文件

    Fix ModelOpt MIXED_PRECISION dispatch for Gemma4 checkpoints whose quantized_layers records the MoE parent module as model.language_model.layers.N.experts. The issue was reproduced and validated with wangqia0309/gemma-4-26B-A4B-it_nvfp4_experts_fp8_dense_fp8_attn-kv_fp8, which uses NVFP4 experts, FP8 dense and attention layers, and an FP8 KV cache. vLLM registers the runtime FusedMoE module under …

  • 8ce53a6 #47574 — [Bugfix] Zero new KV blocks for quantized + sliding-window hybrid caches (#47574)
    • 作者: EdalatiAli | +18/-1 | 1 个文件

    A hybrid KV cache that quantizes full-attention layers but keeps sliding-window layers in higher precision (–kv-cache-dtype fp8 –kv-cache-dtype-skip-layers sliding_window) produces NaN / all-zero output once the input exceeds the sliding window. Uniform fp8 and bf16 are fine; only the hybrid combo fails (seen on Blackwell + FlashInfer). ## Root cause Attention groups share one block pool. Th…

  • d835ad5 #49111 — [Bugfix][Rust Frontend] Map missing prompt logprobs for single-token prompts in chat and raw generate (#49111)
    • 作者: Salt Sato | +94/-53 | 4 个文件

    A single-token prompt has no scored prompt-logprobs positions, and the two model runners represent that case differently: Model Runner V1 returns an empty [0, k+1] tensor that decodes to zero scored positions, while Model Runner V2 omits the payload entirely (gpu/sample/prompt_logprob.py hits start_idx >= end_idx after excluding the unscored leading token and never writes the request into prompt_l…

  • 37bf988 #47295 — [XPU][Bugfix] Fix GroupCoordinator device_index (#47295)
    • 作者: Michał Ganczarenko | +4/-7 | 1 个文件

    Fix regression introduced in #45026 (ebfbcfe46) where GroupCoordinator.init makes all subgroups inherit device_index from _WORLD. For CUDA this is safe — device_index passes through logical_device_id_to_visible_device_id() before reaching torch.device(). For XPU and out-of-tree platforms, self.device is set directly as torch.device(f"{platform}:{self.device_index}") with no further remapping. …

  • 9459fc6 #45989 — [Bugfix][RL] Set vLLM config during weight reload (#45989)
    • 作者: aoshen02 | +45/-12 | 2 个文件

    RL weight updates re-run layer post-processing after model initialization. Those paths may call get_current_vllm_config(), but the worker reload APIs did not establish the config context, causing an assertion failure on the FP8 MoE + DeepEP path. This PR wraps the stable worker-level reload boundaries with set_current_vllm_config(): - regular reload_weights - RL start_weight_update, update_weights…

🔩 Misc

  • 040cbf9 #49214 — [Misc] Use VLLMValidationError in chat completion tool and batch validators (#49214)
    • 作者: Umut Polat | +16/-11 | 1 个文件

    Follow-up to #36254, continuing the migration of user-facing request validation in the OpenAI-compatible protocol layer to VLLMValidationError. A few checks in ChatCompletionRequest.check_tool_usage and BatchChatCompletionRequest.check_batch_mode still raised plain ValueError, while sibling checks in the same validators already use VLLMValidationError. The custom exception carries a parameter name…

  • 7a98c7a #49244 — [Misc] Remove old now unsupported max_num_partial_prefills and max_long_partial_prefills (#49244)
    • 作者: Nicolò Lucchesi | +4/-60 | 3 个文件

    Removes the max_num_partial_prefills and max_long_partial_prefills scheduler config fields. I believe we have been on v1 for 1+ year now, but we still have these args which are leftovers from v0/v1 hybrid period where an oracle was needed. These fields were introduced for the V0 scheduler and explicitly rejected by the V1 enablement oracle in #13726. With V0 removed, the fields are now dead config…

  • 0d9e606 #49299 — [Misc][Docs] Fix XPU compute-runtime driver link version mismatch (#49299)
    • 作者: Lena Onyshchenko | +1/-1 | 1 个文件

    The XPU installation guide (docs/getting_started/installation/gpu.xpu.inc.md) links the text “compute runtime 26.18” to release tag 26.14.37833.4, which matches neither the link text nor the driver version actually installed in docker/Dockerfile.xpu (26.18.38308.1). This updates the link to point at the matching 26.18.38308.1 release so users install the recommended driver version. Docs-only chang…

  • adc98f0 #49298 — [Misc] Add @esmeetu to codeowners for rust/src/bench (#49298)
    • 作者: Roy Wang | +1/-0 | 1 个文件

    #48107 ## Test Result —

  • 1d87486 #47212 — [Misc][Docs] Fix broken protocol link in speech_to_text doc (#47212)
    • 作者: Lena Onyshchenko | +1/-1 | 1 个文件

    vllm/entrypoints/openai/protocol.py no longer exists; point to speech_to_text/transcription/protocol.py. Made with Cursor

  • ae10e85 #47210 — [Misc][Docs] Remove duplicate CodeGeex4 row in XPU model table (#47210)
    • 作者: Lena Onyshchenko | +0/-2 | 1 个文件

    THUDM/CodeGeex4-All-9B was listed 3x in the XPU supported models table. Keep one.

  • 47d0597 #47211 — [Misc][Docs] Fix broken csrc kernel links in fusions doc (#47211)
    • 作者: Lena Onyshchenko | +2/-2 | 1 个文件

    Both kernels moved under csrc/libtorch_stable/; update the two 404’d links. Made with Cursor

🦀 Rust Frontend

  • eb44b3a #49295 — [Rust][Benchmark] Use async HTTP clients (#49295)
    • 作者: Bugen Zhao | +200/-122 | 15 个文件

    Uncached downloads created reqwest::blocking::Client while the benchmark was running inside a Tokio runtime. Dropping the blocking client could shut down its internal runtime from async context and panic. This PR changes to use async reqwest for dataset downloads and hf-hub access, so that we can remove its blocking feature from vllm-bench. The tokenizer interface remains synchronous. As a result,…

  • 616c9bd #45839 — [Frontend] Support additional sampling parameters for translation API (#45839)
    • 作者: Guan-Ming Chiu | +69/-1 | 1 个文件
    • Resolve # TODO support additional sampling parameters - Add top_p, top_k, min_p, frequency_penalty, repetition_penalty, presence_penalty, vllm_xargs to translation API - Verify new params propagate to SamplingParams and defaults stay neutral ## Test Result - Pass on T4. New params propagate —
  • 8688a06 #48937 — [Rust][Benchmark] Use tracing for logs (#48937)
    • 作者: Bugen Zhao | +498/-249 | 24 个文件

    Replace the raw println and eprintln calls in the ported Rust vllm-bench with tracing and structured fields, aligning with the convention used in the Rust frontend. Also changes the output destination for tracing in the Rust workspace to stderr. The final output will still be written directly to stdout, so compatibility is not broken if there’s any script parses the result. ## Test Result —

  • c01618f #48930 — [Rust][Benchmark] Integrate vllm-bench to vllm-rs & vllm CLI (#48930)
    • 作者: Bugen Zhao | +421/-280 | 12 个文件

    Signed-off-by: Bugen Zhao i@bugenzhao.com Integrate the Rust benchmark client vllm-bench introduced in #48107 with the main vLLM CLI (both vllm and vllm-rs). The bench crate has been refactored into a reusable library so that the vllm-rs binary can depend on it and integrate with it. The standalone vllm-bench binary is still kept as a thin wrapper around that binary to facilitate development and…

📖 Documentation

  • e78a0c8 #49148 — [XPU][Doc] Update XPU docker image documents (#49148)
    • 作者: Kunshang Ji | +44/-2 | 3 个文件

    Refine xpu documents ## Test Result —

✨ New Feature

  • 7ca0177 #47451 — [Feat][Perf] Add new warmup infrastructure for JITs (#47451)
    • 作者: Roberto L. Castro | +1855/-693 | 14 个文件

    This PR introduces a shared warmup infrastructure for JIT kernels in vLLM. The goal is to provide a standard, extensible way for kernels from different JIT backends, including Triton, CuTeDSL, TileLang, and potential future DSLs, to expose the set of specializations that should be compiled during engine startup. This is not intended to be a one-off warmup path for a specific kernel. Instead, it de…