vLLM Daily — 2026-06-22

共 45 个 commit，涉及 201 个文件，+6930/-4901 行变动。

概要

统计项	数值
Commit 数	45
变更文件	201
新增行数	+6930
删除行数	-4901

Commit 列表

📦 Other

• f2069b0 #46119 — [Pooling] Validate non-negative rerank top_n (#46119)
  • 作者: Taneem Ibrahim | +1/-1 | 1 个文件

  top_n represents a result count for rerank requests, so negative values are invalid input. Before this change, top_n=-1 was silently accepted and treated the same as top_n=0, returning all rerank results. This could hide client-side bugs and make request behavior harder to reason about. This PR adds request-level validation so top_n must be non-negative, while preserving the existing behavior that…

• ccd49f6 #41722 — [MyPy] Fix mypy for vllm/lora (#41722)
  • 作者: Martin Hickey | +140/-50 | 15 个文件

  Part of https://github.com/vllm-project/vllm/issues/26533 pre-commit run -a –hook-stage manual mypy-3.10 Before: After: 1. Test LoRA from docs here and in particular run Serving LoRA Adapters example 2. Run unit tests: $ pytest -s -v tests/lora ## Test Result 1. Server loads and can query the LoRA adapters successfully 2. Tests pass

• 3da4a1b #43404 — [XPU] add awq format for INCXPULinear (#43404)
  • 作者: Liangliang Ma | +89/-16 | 2 个文件

  convert AWQ to GPTQ, letting INCXPULinear can handle AWQ-format autoround models load. cc: @yiliu30 @jikunshang

• aa4990a #45111 — [Attention] Re-enable cross-layer KV cache layout for MLA via stride-aware kernels (#45111)
  • 作者: Yifan Qiao | +847/-21 | 13 个文件

  #37090 disabled the cross-layer (block-major) KV cache layout for all MLA backends after #37032 (GLM-4.7-Flash garbage output with KV offloading), attributing the bug to “MLA kernels requiring contiguous per-layer KV cache views”. The actual cause is narrower: a few kernels computed page addresses from block_size * entry_size instead of reading the cache tensor’s block-dim stride. The MLA write pa…

• a4610da #46373 — [docs] link security docs from AGENTS (#46373)
  • 作者: Simon Mo | +6/-0 | 1 个文件

  Add a security-review routing sentence to AGENTS.md that points agents to SECURITY.md, docs/usage/security.md, and docs/contributing/vulnerability_management.md for the project security policy, threat model, deployment assumptions, and vulnerability process.

• 09cdcf3 #46327 — [XPU] update nixl to v1.2.0 (#46327)
  • 作者: liuzhenwei | +10/-10 | 1 个文件

  follow https://github.com/vllm-project/vllm/pull/45843, update nixl to v1.2.0 in xpu dockerfile ## Test Result —

• d2c671c #44324 — [CPU][RISC-V] Add RVV micro GEMM for WNA16 (#44324)
  • 作者: wcy | +273/-2 | 4 个文件

  This PR adds an RVV-specific micro GEMM implementation for CPU WNA16 on RISC-V and wires the W4A16 GPTQ CPU path to use it. 1. Add an RVV-specific micro GEMM kernel for WNA16 The existing VEC path already uses the generic vector abstraction on RISC-V, but it still follows the generic FP32Vec16 tile shape. For the WNA16 micro-kernel this creates higher register pressure on current RVV targets. This…

• 89accad #45931 — [ROCm][DSV4] Disable TileLang MHC dispatch on gfx942 (#45931)
  • 作者: Tuukka Sarvi | +56/-22 | 4 个文件

  DeepSeek V4 can produce incorrect output on gfx942 when the ROCm MHC path selects TileLang. The non-TileLang fallback path produces coherent output, so gfx942 should not silently route through the known-incorrect TileLang MHC kernels until that path is fixed. This is a correctness guard, not a TileLang kernel fix. A future follow-up can re-enable TileLang MHC on gfx942 after those kernels are fixe…

• 3c8e495 #46108 — [Model] ColQwen3.5: fix retrieval correctness (bias + bidirectional) (#46108)
  • 作者: Athrael Soju | +79/-5 | 7 个文件

  Follow-up to #36887. The in-tree ColQwen3_5 model deviates from the native colpali ColQwen3_5Processor inference pipeline in three silent ways, none caught by the current sanity-level tests. Measured cost: ~2.5 ndcg@10 on Vidore3. This affects every ColQwen3.5 checkpoint — e.g. athrael-soju/colqwen3.5-4.5B-v3 and athrael-soju/VultronRetrieverPrime-Qwen3.5-8B — both of which MTEB loads thro…

• 1c4b51b #46352 — Temporarily skip M3 on CI (#46352)
  • 作者: Roger Wang | +2/-0 | 1 个文件

  Test Result —

• 80abe0d #46137 — [Rust Frontend] Support thinking_token_budget for chat and completions (#46137)
  • 作者: Chao-Ju Chen | +178/-7 | 13 个文件

  Add support for the thinking_token_budget request parameter in the Rust frontend, for both /v1/chat/completions and /v1/completions, reaching parity with the Python frontend (tracked in #44280, “Request compatibility and validation”). Previously the Rust frontend parsed thinking_token_budget on the chat endpoint but explicitly rejected it (“thinking_token_budget is not supported.”), and the comple…

• d14e551 #45993 — [Model] Remove MiniMaxText01, MiniMaxVL01, MiniMaxForCausalLM (#45993)
  • 作者: Tiezhen WANG | +10/-3881 | 20 个文件

  • Remove legacy MiniMax Text-01 and VL-01 model architectures: MiniMaxForCausalLM, MiniMaxText01ForCausalLM, MiniMaxM1ForCausalLM, MiniMaxVL01ForConditionalGeneration - These have been superseded by MiniMax M2 (MiniMaxM2ForCausalLM) and M3 (MiniMaxM3SparseForCausalLM, MiniMaxM3SparseForConditionalGeneration) which remain fully supported - Shared infrastructure (minimax_linear_attn.py, minimax_rms_…

• 68567ef #46216 — [CPUOffloadingManager] Maintain evictable list in LRUCachePolicy (#46216)
  • 作者: Varun Sundar Rabindranath | +64/-19 | 5 个文件

  Add evictable list to LRUCachePolicy for fast evictions. #### Why vllm serve command : guidellm command ##### main py-spy top –pid As can be seen from the py-spy output, the evict operation is very expensive. ##### PR There is still performance on the table. The lookup and get functions are interrupting the scheduler loop. ## Test Result

• 6bc6f2d #45939 — [1/N][Core] add partial prefix cache primitives (#45939)
  • 作者: Jiangyun Zhu | +719/-41 | 5 个文件

  This PR is the first, intentionally small, step toward fine-grained prefix cache hits for hybrid models. Hybrid full-attention + Mamba models often need the full-attention cache block size to align with the Mamba state block size. Since vLLM prefix-cache matching is block-size granular today, this makes the effective cache-hit granularity much coarser than what full attention can otherwise support…

• 9037498 #44517 — [DSV4][XPU] Pass gemm1_clamp_limit to XpuFusedMoe (#44517)
  • 作者: Ma Jian | +2/-0 | 1 个文件

  Pass quant_config.gemm1_clamp_limit to XpuFusedMoe so that the SwiGLU clamp limit is applied during MoE expert computation on XPU. ## Dependencies This PR depends on: - https://github.com/vllm-project/vllm/pull/42953 - https://github.com/vllm-project/vllm-xpu-kernels/pull/395

• db32b53 #43081 — [SpecDecode] Support DFlash with FlashInfer (#43081)
  • 作者: gq112 | +63/-16 | 3 个文件

  This PR is split out from #39995. It adds FlashInfer backend support needed for DFlash. Note: On RTX 4090, using FP8 KV cache requires the combination of DFlash + FlashInfer. ## Changes: - Allow FlashInfer to be selected for non-causal DFlash attention. - Route non-causal DFlash batches through FlashInfer native prefill with causal=False. - Keep non-causal FlashInfer away from unsupported DCP …

• 89bd2c1 #43132 — [Spec Decode] Add Qwen3 architecture support for EAGLE3 (#43132)
  • 作者: Benjamin Chislett | +523/-2 | 6 个文件

  As proven by DFlash, the Qwen3 base architecture is more stable for EAGLE3 training due to the extra norms in the attention module. We can improve performance of EAGLE3 and P-EAGLE by using Qwen3 as a base architecture and leveraging the higher learning rates now possible in training. I have manually ablated this and confirmed the results. This PR is experimental as no public checkpoints currently…

• 635c383 #45555 — [Multimodal] Add Qwen2-VL/Qwen2.5-VL processor-mapped video loader (#45555)
  • 作者: Ranran | +117/-4 | 4 个文件

  Qwen2-VL and Qwen2.5-VL currently fall back to the generic opencv video loader (uniform num_frames=32) because they ship no video_processor_type in their preprocessor config. This adds Qwen2VLVideoBackend, so the loader is auto-selected (#44126). Same pattern as the Qwen3-VL loader (#44412). This will be the baseline groundtruth of #45203 fps takes precedence over num_frames because vLLM’s VideoMe…

• c441ad1 #45957 — [KV Offloading] Add labeled metrics support (#45957)
  • 作者: Srinivas Krovvidi | +384/-123 | 3 个文件

  • add labelnames to offloading metric metadata - normalize offloading stats payloads to {metric_name: {labelvalues_tuple: value}} - add labeled aggregation/reduction and lazy Prometheus child creation for labeled values - keep existing unlabeled payloads compatible by mapping them to the empty label tuple

• 745bba5 #45935 — [Model]Fix MiniMaxM2ForCausalLM perf regression (#45935)
  • 作者: Jee Jee Li | +215/-16 | 2 个文件

  The root cause is that torch.compile can’t fuse these torch glue ops, which leads to the M25 perf regression. This PR fuses these torch ops manually with Triton. ## Test Result - PERF benchmark(nvidia/MiniMax-M2.5-NVFP4 TP4 on GB200*4) - Accuracy evaluation for nvidia/MiniMax-M2.5-NVFP4 /| GPQA Diamond | AIME 2025 | –|– | – | Official score| 0.839|0.853| This PR|0.833|0.858 —

• 2cac89f #45181 — [Spec Decode] Support mixed KV page sizes for DFlash (#45181)
  • 作者: Alex Steiner | +511/-141 | 8 个文件

  This PR addresses the KV-cache infrastructure gap needed by DFlash-style speculative decoding when the target and draft models have different KV page sizes. It adds: - A padding fallback in unify_kv_cache_spec_page_size for non-divisible page sizes. - Padded KV-cache reshape handling for FlashAttention-style layouts where the block dimension is not the first physical dimension. - A shared attentio…

• 3e6e335 #44638 — [Disagg] return routed_experts on streaming generate responses (#44638)
  • 作者: aoshen02 | +10/-0 | 2 个文件

  The disagg serve_tokens non-streaming path already returns per-token MoE routing in GenerateResponseChoice.routed_experts, but the streaming path (GenerateResponseStreamChoice) dropped it. This adds the field to the streaming choice and encodes it with the same base64(np.save(…)) transport used by the non-streaming path, populated on the chunk where the engine makes routing available…

• b91b772 #46039 — [ROCm][P/D] Support MiniMax-M3 mixed KV layouts in MoRIIO READ mode (#46039)
  • 作者: junkang1991 | +566/-72 | 4 个文件

  Fix MoRIIO READ-mode KV transfer when a model uses mixed per-layer KV cache layouts, observed with MiniMax-M3. MiniMax-M3 registers multiple KV cache layouts: - separated K/V cache: [2, num_blocks, block_size, num_kv_heads, head_dim] - ROCm interleaved K/V cache: [num_blocks, 2, block_size, num_kv_heads, head_dim] - 3D key-only / MLA indexer cache: [num_blocks, block_size, head_dim] MoRIIO previou…

🐛 Bug Fix

• 1c7bc18 #46365 — [Bugfix][CPU] Fix CPU model runner v2 (#46365)
  • 作者: Li, Jiang | +5/-0 | 1 个文件

  • Fix broken CPU model runner v2 test CI test ## Test Result —

• 78739e3 #46313 — [Bugfix] Reject matryoshka embedding dimensions above hidden size (#46313)
  • 作者: Maxwill Lin | +28/-0 | 2 个文件

  For matryoshka embedding models without an explicit matryoshka_dimensions list, PoolingParams._set_default_parameters only checked dimensions >= 1. A dimensions value above the model’s hidden size was then silently used to slice the embedding ([…, :d]), returning a hidden_size-length vector instead of rejecting the request. Add the upper-bound check (mirrors sglang’s _validate_for_matryoshka_dim…

• cec2ec1 #45100 — [Bugfix] Avoid racy accepted counts in async spec decode (#45100)
  • 作者: Weiwei Sun | +46/-5 | 3 个文件

  Fix an async speculative decoding race for hybrid non-align Mamba/GDN models (observed with Qwen3.5 MTP) where _prepare_inputs consumes a stale CPU copy of num_accepted_tokens. In the failing path, _update_states_after_model_execute writes accepted-token counts from GPU to input_batch.num_accepted_tokens_cpu_tensor with a non-blocking D2H copy. Under async scheduling, the next _prepare_inputs may …

• 435f82d #46341 — [Bugfix] Fix Llama4ForCausalLM initialization test failure (#46341)
  • 作者: liuzhenwei | +7/-1 | 1 个文件

  dummy_hf_overrides only set num_experts_per_tok=1 for Llama4ForConditionalGeneration, causing Llama4ForCausalLM and EagleLlama4ForCausalLM to get num_experts_per_tok=2. This conflicts with apply_router_weight_on_input=True which only supports topk=1. ## Test Result —

• 3112474 #46113 — [Bugfix] [Rust Frontend] Fix stop string truncation with repeated matches (#46113)
  • 作者: Reid | +8/-1 | 1 个文件

  Fix a Rust frontend stop-string truncation bug when the same stop string appears multiple times within the newly decoded text. The Rust implementation used rposition() in matches_stop_string(), which selected the rightmost match in the search window. Python V1 detokenization uses find(), which selects the leftmost match. This can produce incorrect output when a single decoded token contains repeat…

• 12fe2a9 #46305 — [Bugfix][Qwen3-VL] Fix multi-video crash with list-valued fps/num_frames (#46305)
  • 作者: Ting SUN | +59/-2 | 2 个文件

  A Qwen3-VL request that carries more than one video and passes per-video mm_processor_kwargs as a list (one value per video, e.g. fps=[2.0, 4.0] or num_frames=[8, 16]) crashes during preprocessing, before the request reaches inference. Qwen3VLMultiModalProcessor._call_hf_processor() processes videos in a per-item loop, but it copies the full mm_kwargs to every video without slicing the list-valued…

• 9c450b1 #45361 — [Kernel][Bugfix] Fix INT8 per-token-head KV cache rounding in Triton reshape-and-cache (#45361)
  • 作者: ZedongLiu | +65/-9 | 2 个文件

  What this PR does This PR fixes the Triton reshape-and-cache path for kv_cache_dtype=int8_per_token_head. The per-token-head KV cache kernel computes one absmax scale per (token, head), but the INT8 path previously clamped the scaled K/V values and then relied on the implicit float-to-int8 store. That store truncates toward zero, while INT8 absmax quantization should round to nearest before st…

• d3ad8e8 #46231 — [Bugfix] Defer offload reads while transfers are pending (#46231)
  • 作者: Palaiologos1453 | +106/-4 | 2 个文件

  Fixes #46014. This makes the offloading scheduler defer prefix-cache lookup for a request while that request still has in-flight transfer jobs. In the preemption/re-admission race described in the issue, this prevents the scheduler from issuing a load while a previously flushed store is still tracked in transfer_jobs. The request is retried on a later scheduling step after the worker completion is…

🧪 CI/Tests

• 9a938df #46355 — [Test][KV Offloading] Add unit tests for OffloadingSpecFactory and SecondaryTierFactory (#46355)
  • 作者: AlexHuang | +424/-0 | 4 个文件

  Add 21 unit tests for OffloadingSpecFactory and SecondaryTierFactory — the two factory classes that serve as the plugin entry points for KV offloading in vLLM. These factories had zero test coverage. Test coverage: - Pre-registration integrity sentinels (CI guard against stale module paths) - End-to-end factory → spec construction with real configs - spec_module_path dynamic import path (used …

• b5a2ade #46356 — [XPU][CI]Skip v1/spec_decode/test_speculators_correctness.py in intel GPU nightly (#46356)
  • 作者: xiangdong | +1/-1 | 1 个文件

  skip v1/spec_decode/test_speculators_correctness.py in nightly ## Test Result —

• b529bfd #45768 — [XPU][CI] Add agent_tags for Intel GPU CI (#45768)
  • 作者: xiangdong | +120/-0 | 10 个文件

  Add agent_tags for Intel GPU CI ## Test Result —

• f3df7a7 #45955 — [ROCm][CI] Enable kv_connector unit tests on ROCm (#45955)
  • 作者: Micah Williamson | +16/-0 | 4 个文件

  We are seeing failures in test_offloading_connector after https://github.com/vllm-project/vllm/pull/43458 enabled MRV2 for Llama models by default. here is an example from https://buildkite.com/vllm/amd-ci/builds/9582/list?sid=019ecfa8-eaee-4930-90f1-9c79dac59f06&tab=output It looks like we’re seeing a regression because V2 doesn’t yet have cross-layer KV cache support like V1 does. There are many…

• 485bbe1 #46163 — [CI] Fix missing tp_size attribute on RoutedExperts (#46163)
  • 作者: fxmarty-amd | +3/-4 | 1 个文件

  Fix a failing test in https://buildkite.com/vllm/amd-ci/builds/9636/canvas due to missing attribute. Fixes the error: - CUDA_VISIBLE_DEVICES=0 pytest tests/models/quantization/test_awq.py -s -vvvvv -k “test_awq_load” ## Test Result Test passing on MI300.

• a19ff22 #46018 — [Hardware][AMD][CI] Fix Spec Decode Eagle test group (#46018)
  • 作者: Matt | +17/-2 | 3 个文件

  This PR fixes the Spec Decode Eagle test group. pytest -v -s v1/e2e/spec_decode -k “eagle_correctness” ## Test Result The test group passes. It is run as part of AMD CI. cc @AndreasKaratzas —

• 4f0d004 #46080 — [Hardware][AMD][CI] Fix Kernels Attention test groups (#46080)
  • 作者: Matt | +68/-37 | 6 个文件

  This PR fixes both the gfx942 and gfx950-based Kernels Attention test groups. This PR is a re-do of https://github.com/vllm-project/vllm/pull/45786 with a different source branch, as that PR had merge conflicts but is based on a source branch that can no longer be written to. The following 3 test groups are now newly passing in AMD CI: MI300_1: Kernels Attention %N MI355_1: Kernels Attention %N MI…

• 13b83d7 #45967 — [ROCm][CI] skip test_double_aiter_rms_quant_fusion (#45967)
  • 作者: Charlie Fu | +9/-4 | 2 个文件

  The unit test tests/compile/passes/test_double_aiter_rms_quant_fusion.py is failing as follow: And this is because this unit test is testing a pattern with two consecutive torch.ops.vllm.rocm_aiter_group_fp8_quan ops: But the compiler does a optimization by removing one of the quant op: So we suppose we do not need this unit test.

• 5024160 #46298 — [Hardware][AMD][CI] Fix gfx942 Kernels MoE test group (#46298)
  • 作者: Matt | +29/-12 | 3 个文件

  This PR fixes and gates the gfx942-based Kernels MoE test group. A full fix for this test group on gfx950 is still in progress. pytest -v -s kernels/moe on a gfx942-based device. This is run as part of the Kernels MoE %N test group on AMD CI. ## Test Result The test group passes. cc @AndreasKaratzas —

• b80ce9d #46241 — [CI][test] Replace InternVL2-1B with InternVL3-1B in test_pipeline_parallel.py (#46241)
  • 作者: xiaolinchen | +2/-2 | 1 个文件

  OpenGVLab/InternVL2-1B (model_type=internvl_chat) cannot be loaded by AutoTokenizer on transformers >= 5.x: The error message is misleading — sentencepiece and tiktoken are both installed. The real cause is the combination of two facts: 1. transformers 5.x lists internvl_chat in MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS (in transformers/models/auto/tokenization_auto.py), so AutoTokenizer is forced…

📖 Documentation

• 6871738 #46376 — [Doc] Document pull request limit (#46376)
  • 作者: Simon Mo | +8/-2 | 1 个文件

  • Document that vLLM uses GitHub’s pull request limit for contributors without write access. - State the current cap of 6 open PRs. - Clarify bypass-list and expedited-review contact paths for critical contributions. ## Duplicate checks - gh pr list –repo vllm-project/vllm –state open –search “"pull request limit"”: no matches. - gh pr list –repo vllm-project/vllm –state open –search “"by…

• 2e2c479 #45940 — [Doc] Update MiniMax-M3 (#45940)
  • 作者: Jee Jee Li | +2/-2 | 2 个文件

  Test Result —

✨ New Feature

• a9f7b2d #43468 — [feature][kv_offload] Self-describing KV events for OffloadingConnector (#43468)
  • 作者: Change72 | +720/-62 | 9 个文件

  What This PR makes native OffloadingConnector CPU-offload KV-cache events self-describing behind an explicit opt-in: The flag is inert unless vLLM KV cache events are also enabled. With the flag off, the connector keeps the legacy placeholder payload (token_ids=[], block_size=0, parent_block_hash=None); note that stored events are now emitted one-per-offload-key rather than one-per-batch. The…

🦀 Rust Frontend

• 1eb2cc9 #46022 — [Frontend] Refactor ServingTokenization entrypoint. (#46022)
  • 作者: wang.yuqi | +322/-302 | 15 个文件

  following #41907 Refactor ServingTokenization entrypoint. 1. Remove the “OpenAI” prefix. > In the early days of vLLM (2023), there was only the OpenAI, so vLLM online serving was referred to as the OpenAI-Compatible Server. Remove the “OpenAI” prefix if this is not the official OpenAI API. - OpenAIServingTokenization -> ServingTokenization - openai_serving_tokenization -> serving_tokenization 2. B…