共 90 个 commit，涉及 534 个文件，+17083/-13015 行变动。

概要

统计项	数值
Commit 数	90
变更文件	534
新增行数	+17083
删除行数	-13015

Commit 列表

🐛 Bug Fix

• c8bdcc0 #47135 — [Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)
  • 作者: Michał Ganczarenko | +85/-1 | 2 个文件

  Fix ASRDataset.sample() raising “The decoder prompt cannot be empty” when benchmarking Cohere ASR models (e.g. CohereLabs/cohere-transcribe-03-2026). ASRDataset.sample() previously only built a decoder prompt for OpenAI/Whisper tokenizers, falling back to prompt = "" for all others. CohereAsrTokenizer sets skip_decoder_start_token=True and does not override skip_prompt_length_check, so the empty p…

• 63fcce4 #47031 — [Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031)
  • 作者: Michał Ganczarenko | +3/-3 | 1 个文件

  Fix KeyError when loading GraniteMoeSharedForCausalLM weights after the FusedMoE/MoERunner inversion refactor (#41184). After #41184, the parent _load_weights() expects expert keys of the form experts.{e}.w{n}.weight. GraniteMoeShared.load_weights() was still inserting routed_experts before delegating, producing a doubled path experts.routed_experts.routed_experts.{e}.w{n}.weight and a KeyError. R…

• 024b06b #42748 — [Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748)
  • 作者: Aleksei Ivashov | +4/-2 | 2 个文件

  serving.py already constructs UsageInfo (including prompt_tokens_details.cached_tokens when –enable-prompt-tokens-details is set) and passes it to the GenerateResponse constructor. However, the usage field was missing from the GenerateResponse model, so pydantic silently dropped it. Fix: add usage: UsageInfo | None = None to GenerateResponse, mirroring GenerateStreamResponse which already has…

• aa8bb55 #46730 — [ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730)
  • 作者: akii96 | +19/-8 | 1 个文件

  On gfx942 the DeepSeek-V4 Flash indexer quantizes Q and K to different FP8 types. K already uses the platform type (e4m3fnuz on gfx942, via current_platform.fp8_dtype()), but the fused RoPE+quant kernel in fused_indexer_q.py hardcodes Q to e4m3fn. The FP8 logits kernel then gets fnuz K with fn Q and falls back to a mixed-dtype path on every call. This change derives Q’s type from current_platform….

• fa4bec9 #47071 — [Bugfix] Fix pooled Whisper sliding-window KV sizing (#47071)
  • 作者: Andy Lo | +43/-2 | 2 个文件

  Voxtral Realtime’s causal Whisper encoder uses block_pool_size encoder tokens per pooled KV block, but its SlidingWindowSpec.sliding_window was still expressed in encoder-token units. The sliding-window KV cache manager then reserved about block_pool_size times too many blocks for the encoder cache. The fix converts the KV-cache spec’s sliding window into pooled units while leaving the attention k…

• 697c34b #47126 — [Bugfix] Fix beam search candidate indexing when logprobs count varies (#47126)
  • 作者: Chauncey | +119/-51 | 2 个文件

  [Bugfix] Fix beam search candidate indexing when logprobs count varies see https://buildkite.com/vllm/ci/builds/75148#019f1408-624f-485c-a3ea-762444b94b6a before: ## Test Result —

• b446792 #47209 — [ROCm][Bugfix] Fix Triton “out of resource: shared memory” Error In One-Shot LoRA MoE (#47209)
  • 作者: Micah Williamson | +9/-0 | 1 个文件

  We’re seeing the following error on MI300 and MI325: As the error suggests, we need to take the platform’s max shared mem into account when launching the LoRA one-shot MoE kernel _fused_moe_lora_one_shot_kernel. To handle this, I propose limiting num_stages to 2 if the platform doesn’t support the listed shared memory requirement 69632 (68KB). This shouldn’t impact the behavior of platforms that d…

• 3406e8f #47062 — [Bugfix][Frontend][gpt-oss] Return raw output when Harmony parser ends non-terminal (#47062)
  • 作者: achyuthan.s | +69/-17 | 2 个文件

  Fixes #45736. When GPT-OSS Harmony output ends in a non-terminal parser state — e.g. a malformed final channel that omits the <|message|> delimiter, trapping the body in the header — vLLM silently returned content: null with finish_reason=“stop” and completion_tokens > 0, dropping the generated answer with no exception, log, or recovery. This implements the consensus from #45736 (after #46437 …

• ac521f6 #45346 — [Bugfix][Structured Outputs] Reject degenerate structured_outputs that crash EngineCore (#45346)
  • 作者: Ting SUN | +33/-0 | 2 个文件

  A single request with structured_outputs={“json_object”: false} or {“json”: “”} crashes the EngineCore process: the request returns 500 EngineDeadError and every subsequent request fails (instance-wide DoS). StructuredOutputsParams counts constraints with is not None, so json_object=False (False is not None) and json="" ("" is not None) are treated as “a constraint is set” → a StructuredOutput…

• 2824282 #45657 — [Bugfix][Frontend] Normalize constrained Harmony recipients (#45657)
  • 作者: tarjan1 | +39/-2 | 2 个文件

  Normalize standalone <|constrain|>… markers that older openai-harmony versions misparse as recipients at the shared HarmonyParser boundary. This keeps constrained final output classified as regular content while exposing recipient=None through HarmonyParser.current_recipient and Segment.recipient. It covers non-streaming parsing, streaming parser state, and emitted segments. Related to #45570. T…

• 6829473 #47099 — [Bugfix] Align OpenCV video metadata timeline (#47099)
  • 作者: VectorPeak | +27/-4 | 2 个文件

  Fixes #46988. video_get_metadata() should describe the same source-video timeline that video_to_ndarrays() uses when it samples frames. Previously, video_to_ndarrays() sampled requested frames with: but video_get_metadata() reported fps as duration / num_frames and frames_indices as list(range(num_frames)). For sampled videos, this made metadata describe post-sampling positions instead of source-v…

• c8d2f3c #47154 — [Bugfix] compressed-tensors: allow int8 grouped WNA16 MoE on Marlin (#47154)
  • 作者: Joe Rowell | +0/-1 | 1 个文件

  The num_bits==8 branch of CompressedTensorsWNA16MarlinMoEMethod asserted group_size == -1, rejecting int8 experts quantized with a group size (e.g. group_size=128) even though Marlin supports them: uint8b128 is a supported quant type and MARLIN_SUPPORTED_GROUP_SIZES includes 128, and check_moe_marlin_supports_layer already admits group_size in {-1,32,64,128} (so the method is selected and then ass…

• 11b26c5 #47138 — [Bugfix][Tool Parser] PoolsideV1: fix logprobs AttributeError on Responses API (#47138)
  • 作者: Joe Rowell | +37/-2 | 2 个文件

  extract_tool_calls_streaming read request.logprobs, which ResponsesRequest lacks, raising AttributeError on /v1/responses. This was initially reported to us here ## Test Result —

• a773253 #47039 — [Bugfix] Restore part of bugfix #42650 after accidental deletion in #43241 (#47039)
  • 作者: Nikita Shapovalov | +36/-6 | 3 个文件

  This PR restores the upstream https://github.com/vllm-project/vllm/pull/42650 behavior that appears to have been accidentally reverted during https://github.com/vllm-project/vllm/pull/43241 final conflict-resolution/rebase cycle. ### Relevant upstream timeline: - vLLM #42650 merged on 2026-05-22 as c7624bea5 ([Bugfix] Source num_qo_heads from Attention layers in Flashinfer/Triton metadata builders…

• 25671cb #46875 — [Parser][Bugfix] Ensure tool call or other special tokens don’t leak in non-streaming tool parsing (#46875)
  • 作者: Ben Browning | +783/-219 | 14 个文件

  In the new parser engine, non-streaming tool parsing had several paths where tokens that should be consumed or suppressed leaked into response content. ### Auto-drop special tokens Special tokens (turn markers, channel delimiters, etc.) were only being filtered via token IDs, so they leaked into response content on the non-streaming text path of engine-based parsers (skip_special_tokens=False). Th…

• fcaa84e #47050 — [BugFix] Gate MRV2 mixed sparse-MLA warmup on max_num_seqs > 1 (#47050)
  • 作者: Nick Hill | +34/-4 | 3 个文件

  Thanks to @ZeldaHuang for finding this

• dc148dc #47157 — [CI][Bugfix] Fix Hybrid SSM NixlConnector PD prefix cache test (2 GPUs) (#47157)
  • 作者: Nicolò Lucchesi | +2/-0 | 1 个文件

  Fix flaky CI test https://buildkite.com/vllm/ci/builds/74969/list?sid=019f1009-547c-4b3c-9eae-3e500fda953a&tab=output, in which the output of two identical requests might be ~slightly different, causing the assertion to trigger. I currently suspect this might be due to a recent change to FA https://github.com/vllm-project/vllm/pull/36701, which should still be the default on hopper and that would …

• 7cf7cbc #45918 — [Bugfix] MiniCPM-V 4.6: fix grid rows/cols swap in placeholder generation (#45918)
  • 作者: tc-mb | +2/-2 | 1 个文件

  Fix rows/cols swap in MiniCPM-V 4.6 placeholder generation — grids[0] was read as cols, grids[1] as rows, but all transformers consumers (including HF’s own processor) use the opposite convention. Non-square grids (1×3, 2×3, etc.) produce wrong newline placement and incorrect token counts. Square grids (2×2, 3×3) are unaffected. ## Why only V4.6 V2.0–V4.5 and O series call image_processor.get_slic…

• 00ebf19 #46230 — [Bugfix][Quant] Raise actionable error instead of bare assert for group-size/TP mismatch (#46230) (#46236)
  • 作者: Arsalan Shakil | +37/-12 | 6 个文件
• ded6676 #45960 — [Bugfix] Seed RayExecutorV2 TCPStore port by DP rank to avoid collisions (#45960)
  • 作者: Seiji Eicher | +64/-1 | 2 个文件

  Under data parallel with the Ray V2 executor, each DP engine runs its own RayExecutorV2 and selects the torch.distributed TCPStore port with an independent random get_open_port() in _init_workers_ray. Co-located engines can land on the same port through the pick-release-bind window. A worker then connects to another engine’s store on that port and the handshake value does not match, so distributed…

• aab7af0 #46997 — [Bugfix][ROCm][MLA] Pass q/kv dtypes to get_mla_metadata_v1 in FP8 decode (#46997)
  • 作者: peizhang56 | +209/-0 | 3 个文件

  Fix a silent accuracy regression for MLA models served with an FP8 KV cache on gfx950 (MI355X) using the AITER MLA backend. With amd-aiter == 0.1.16.post2, Kimi-K2.5 at TP2 (32 q-heads/rank) produces garbage — gsm8k drops from ~0.93 to ~0. ### Root cause AiterMLAMetadataBuilder builds the persistent split/reduce metadata for decode via get_mla_metadata_v1, but did not forward dtype_q /…

• 1907d38 #44002 — [Bugfix] Reject negative values for max_logprobs and long_prefill_token_threshold (#44002)
  • 作者: JianweiZheng | +1/-1 | 1 个文件

  Two CLI-settable integer config fields silently accept negative values that no validator rejects: - ModelConfig.max_logprobs (vllm/config/model.py:234) — declared int = 20 with no constraint. _validate_logprobs only rewrites the == -1 sentinel; every other negative survives. For logprob-requesting traffic the error message reflects the malformed cap back at the user (“max allowed: -5”); for logpro…

• aed541d #46945 — [Bugfix][Responses] Set completed status for Harmony function calls (#46945)
  • 作者: Aman Paswan | +2/-0 | 2 个文件

  Fixes #46940. Harmony /v1/responses currently emits completed function_call output items with status: null because _parse_function_call() constructs ResponseFunctionToolCall without setting the status field. This change sets status=“completed” for completed Harmony function calls, matching the existing non-Harmony Responses path and the streaming done-event behavior. ## Test Result - Set status=“c…

📦 Other

• f5a8d73 #46995 — [Spec Decode] DSpark (#46995)
  • 作者: Benjamin Chislett | +1821/-94 | 24 个文件

  Adds support for DSpark speculative decoding, using both DeepSeek-V4 DSpark models as well as DeepSeek-trained Qwen3-DSpark models. ## Design DSpark uses non-causal sliding-window attention. To implement this, instead of manually reimplementing the MLA attention, we instead utilize the existing SparseMLA backends with an expanded topk size, in which the queries all include each other in their to…

• c638f92 #47265 — [Rust Frontend] Split engine core DTOs into separate modules (#47265)
  • 作者: Bugen Zhao | +806/-789 | 54 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Pure code structure refactoring without any logic changes, making the engine-core-client crate cleaner than mixing all definitions into mod.rs. Now we have: - request - output - sampling - structured_outputs This also helps if we want to isolate the northbound-facing shape from the wire representation of some type. Also clean up crate-root-level re-expor…

• 13c49f9 #45368 — [xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368)
  • 作者: Chaojun Zhang | +182/-35 | 3 个文件

  This PR fixes several bugs after #42290 and #41184 . ### Fixes 1. Rank dimension mismatch in _apply_expand for TP > 1 (non-fully-sharded LoRA) 2. Hardcoded add_inputs=True in add_lora_linear The add_inputs kwarg was popped but add_expand was always called with add_inputs=True, ignoring the caller’s value. This broke dual-stream LoRA which passes add_inputs=False. **3. add_inputs not forwar…

• a78c156 #30966 — Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend (#30966)
  • 作者: Harry Mellor | +4/-681 | 5 个文件

  Migrate support of GPTBigCode and Starcoder2 to use the Transformers modeling backend instead of having duplicated implementations in vLLM. These architectures are ~2 years old and do not feature in the top 100 most used architectures in vLLM. This migration: - Reduces the number of model implementations maintained in vLLM - Adds support for the following features that the Transformers modelling b…

• 5c4db60 #45903 — docs(security): document gRPC interface as insecure for private use only (#45903)
  • 作者: Juan Pérez de Algaba | +21/-0 | 1 个文件

  Add a new section to the security guide documenting that the gRPC interface does not implement authentication, authorization, or encryption and must be protected via network-level access controls.

• 4e5ca89 #47269 — [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269)
  • 作者: Fangzhou Ai | +41/-1 | 1 个文件

  Adds opt-in cross-layer sharing of the lightning-indexer top-k block selection for MiniMax-M3 sparse attention. When use_index_cache is set, only 1 of every index_topk_freq sparse-attention layers recomputes the indexer score + top-k selection; the rest reuse the selection the preceding compute layer wrote into the shared topk_indices_buffer in the same forward pass. Adjacent sparse layers sel…

• a22e0df #47263 — [Model] Remove AyaVision, MusicFlamingo (#47263)
  • 作者: Harry Mellor | +3/-1422 | 12 个文件

  Reasons: - Both of these architecture classes recieved very low vLLM usage - MusicFlamingo has been superseded by AudioFlamingo3 (new MusicFlamingo checkpoints use AudioFlamingo3) This PR removes both architecture classes and adds them to _PREVIOUSLY_SUPPORTED_MODELS

• cc56379 #47192 — [Model] Support Hy3 token suffix and JSON Schema array types (#47192)
  • 作者: stevenkuang | +24/-11 | 2 个文件

  Support Hy3 token suffix and JSON Schema array types. Tests for the hy_v3 reasoning parser and tool parser.​ ## Test Result Passed — - Hy3 tokenizers append a suffix to the special tokens. Read the suffix from tokenizer.init_kwargs[“token_suffix”] and apply it to the parser’s special tokens. - Support JSON Schema array types, i.e. union-type declarations such as “type”: [“string”, “object”].

• ed41aa2 #43950 — [ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950)
  • 作者: Fangzhou Ai | +32/-39 | 3 个文件

  Re-enable the aiter multi-head-consensus (mHC) pre/post ops as the preferred ROCm path for DeepSeek V4. #43679 introduced the tilelang fused post+pre mHC kernel and explicitly left a hook to switch back to the (faster) aiter mHC kernels once an aiter release containing the mhc_pre_gemm_sqrsum_kernel race-condition fix was available. This is that follow-up. ### Dispatch / fallback path Selectio…

• 77a9c5a #44353 — Weight sync refactor + move sparse nccl engine (#44353)
  • 作者: Aaron Hao | +953/-1046 | 32 个文件

  Current state of the weight sync path requires non weight transfer engine components to assume too much about weight sync internals. Gpu workers should not need to worry about whether to run layerwise reload, or how to construct weight loaders for weight transfer engines. These concepts are weight transfer engine specific, and should be taken care of by the engine. Also, NCCL sparse weight update …

• f651a8a #42486 — [XPU][UT]Enable ut qk_norm_rope_fusion (#42486)
  • 作者: Yejing Lai | +10/-7 | 2 个文件

  Removed ut qk_norm_rope_fusion cuda hard code, added xpu support.

• 8f82be5 #47242 — [CI/Build] Fix LoRA testing (#47242)
  • 作者: Jee Jee Li | +17/-2 | 1 个文件

  Test Result —

• a461070 #47243 — [Core] Make sleep-mode backend capability flags communicator-agnostic (#47243)
  • 作者: Nils Matteson | +15/-14 | 2 个文件

  Follow-up to #44074. Renames the two SleepModeBackend capability flags that named NCCL specifically, so the interface stays generic across communicator backends: - preserves_nccl → preserves_communicators - preserves_graphs_with_nccl → preserves_graphs_with_communicators This addresses @galletas1712’s review feedback on #44074 (NVIDIA Dynamo team): NCCL is not the only comms backend in vLLM, so a …

• 4470ae8 #46806 — Remove mantis (#46806)
  • 作者: Tiezhen WANG | +8/-203 | 10 个文件

  We are running a sprint to reduce the technical debt of the vLLM code base starting by removing obsolete model architectures based on vLLM internal usage tracking. The usage of both Grok models is very low, less than 20 hours in the past 3 months. Confirmed with Prof. Wenhu Chen from Tiger research ## Test Result —

• 5b431b9 #47166 — [Rust Frontend] Coerce completion max_tokens: null to default (#47166)
  • 作者: Blas Rodriguez Irizar | +34/-1 | 2 个文件

  An explicit “max_tokens”: null in a /v1/completions request is silently treated as “no limit” instead of the default of 16, causing runaway generation. CompletionRequest::max_tokens uses #[serde(default = “default_completion_max_tokens”)] (→ Some(16)). Serde’s field default only applies when the JSON key is absent — an explicit {“max_tokens”: null} deserializes Option to None. That None f…

• df802a8 #47162 — [CPU] Remove speculative decoding stream overrides from CPUModelRunner (#47162)
  • 作者: Jonathan Mamou | +3/-63 | 2 个文件

  • Remove 4 CPU-specific method overrides (_copy_draft_token_ids_to_cpu, _get_draft_token_ids_cpu, _copy_valid_sampled_token_count, _get_valid_sampled_token_count) from CPUModelRunner that are no longer needed thanks to the monkey patches in vllm/v1/worker/cpu/shm.py - The GPU code path now works correctly on CPU — all CUDA stream/event operations are handled as no-ops by _StreamPlaceholder and _Ev…

• 93d8f83 #44074 — [Core] Pluggable sleep-mode backend abstraction (RFC #34303) (#44074)
  • 作者: Nils Matteson | +309/-4 | 4 个文件

  This is PR 1 of the two-PR plan discussed in RFC #34303: a thin backend abstraction in front of the sleep/wake-up GPU path, with the existing cumem mechanism wrapped as the default backend. It is a no-op for every existing user and unblocks the cuda_checkpoint backend (Phase 1/2, #37921/#37925) and out-of-tree backends to land as siblings without touching the public API. This implements th…

• aeb35b9 #46512 — [Rust Frontend] Add error context in tool parser failures (#46512)
  • 作者: Maria Guevara | +57/-19 | 12 个文件

  Adds 80-char buffer snippet to parse_buffered_event to enrich error messages on failed parsing. #44624 added the Python bridge python/src/lib.rs that converts ToolParserError to PyValueError via error.to_report_string() and surfaces it through logger.exception in rust_tool_parser.py. The error string is currently passed via verbatim “{}”, with no structured fields or further logging. - The winnow …

• 9a08a51 #47164 — fix: skip cooperative top-K on SM120 (#47164)
  • 作者: Gabriel Wu | +1/-0 | 1 个文件

  Disable cooperative top-K on SM120, where clustered kernel launches fail with cudaErrorInvalidValue. SM120 will use the existing persistent top-K fallback instead. ## Testing Manually verified the failure on an SM120 GPU for cluster sizes 4, 8, and 16, and verified successful server startup with the fallback.

• c5200d3 #46076 — [Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE (#46076)
  • 作者: Jiangyun Zhu | +1996/-98 | 12 个文件

  add dcp support for GLM 5.2 logs ## Test Result |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr| |—–|——:|—————-|—–:|———–|—|—–:|—|—–:| |gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.9340|± |0.0068| | | |strict-match | 5|exact_match|↑ |0.9333|± |0.0069| ## Perf test GB200 concurrency 256 decode performance But dcp prefill speed is only 1/2 of tp, so the…

• 9969466 #46104 — [Spec Decode] Support SWA + DFlash for MiMo (#46104)
  • 作者: Benjamin Chislett | +243/-25 | 4 个文件

  Adds support for various combinations of causality and sliding-window for DFlash layers. Enables all-causal SWA and all-non-causal SWA for DFlash, and sets up a model-implementation foundation from which to build out hybrid SWA+Full DFlash support in MRV2. Also adds support for loading the mask embedding from a file (“mask_embedding.pt”) in the model checkpoint path; since the MiMo checkpoint does…

• a264e41 #47219 — [Distributed] Default FlashInfer allreduce to mnnvl on single node (#47219)
  • 作者: Woosuk Kwon | +38/-22 | 1 个文件

  The single-node default for the FlashInfer fused allreduce backend was pinned to trtllm to work around an mnnvl cudagraph capture/replay hang (#35772). That hang was root-caused and fixed upstream in FlashInfer (flashinfer-ai/flashinfer#3304, released in >= 0.6.12; vLLM currently pins flashinfer-python==0.6.13), so the workaround is no longer needed. This PR makes VLLM_FLASHINFER_ALLREDUCE_BACKEND…

• f098ee7 #47090 — [GLM5] Support FlashMLA FP8 KV cache (Hopper & Blackwell) (#47090)
  • 作者: Woosuk Kwon | +307/-44 | 3 个文件

  Adds FlashMLA backend support to the new GLM5/DSV3.2 model definition. This PR modifies the existing fused kernels to support FlashMLA’s FP8 KV cache layout and support BF16 query (which is required for Hopper).

• 9294dd2 #46255 — fix(reasoning): guard rfind in ernie45 streaming branch (#46255)
  • 作者: hcl | +2/-1 | 1 个文件

  The elif self.response_end_token_id in delta_token_ids branch does content[:content.rfind(self.response_end_token)] with no != -1 check. The other two slice sites in the same method already guard it. rfind can be -1 here: the token id shows up in delta_token_ids while incremental detokenization hasn’t emitted the string into delta_text yet. Then content[:-1] silently drops …

• c8f9c15 #46993 — [ROCm][V1][MLA] Clone prefill backend state per metadata builder (#46993)
  • 作者: Andreas Karatzas | +37/-1 | 3 个文件

  This prevents V1 MLA prefill metadata from being shared across metadata builders. DBO creates metadata builders per ubatch. MLA prefill backends store prepared prefill metadata on the backend object. Reusing the same backend instance from the static forward context lets one ubatch overwrite another ubatch’s prepared metadata. Proposed changes: - Add MLAPrefillBackend.clone(). - Have MLACommonMetad…

• 3cecee4 #47066 — [Model Runner V2][Spec Decode] Fix stale values in idx_mapping from CG num reqs padding (#47066)
  • 作者: Giancarlo Delfin | +15/-6 | 3 个文件

  Context During full cudagraphs, num_reqs can be padded up to the next capture size. Currently, the speculators can run sample_draft as part of a cudagraph, using the padded num_reqs first dimension size for idx_mapping. This is problematic because stale values in the idx_mapping within the padding region can be used during gumbel sampling to write to the draft_logits tensor. The consequence is t…

• 727971f #41396 — Add Medusa speculative decoding e2e test (#41396)
  • 作者: Rishi Puri | +85/-3 | 4 个文件

  Add an e2e test for Medusa speculative decoding in v1. Medusa is the only spec decode proposer in vLLM v1 without any functional test coverage — EAGLE, EAGLE3, MTP, NGram, Suffix, Draft Model, and Hidden State Extraction all have existing e2e tests. The Medusa code path has unique logic for hidden state extraction that is completely disjoint from the other proposers. Without a test, regressions in…

• 7a341fa #47105 — [XPU] Support ZE_AFFINITY_MASK passthrough in xpu_disagg_acc_test (#47105)
  • 作者: liuzhenwei | +57/-24 | 3 个文件

  Allow the XPU disagg accuracy test to respect an externally provided ZE_AFFINITY_MASK for device selection ## Test Result —

• e840f0d #47140 — [Platform] Replace torch.cuda.Event with torch.Event (#47140)
  • 作者: Kunshang Ji | +60/-70 | 31 个文件

  part of https://github.com/vllm-project/vllm/issues/30679 CI ## Test Result —

• d8f483d #46301 — [Spec Decode] Fix hidden-state extraction block size for hybrid verifiers (#46301)
  • 作者: Igor Margulis | +187/-15 | 3 个文件

  Purpose extract_hidden_states + ExampleHiddenStatesConnector silently produces all-zero hidden states on hybrid (Mamba + attention) verifiers such as Qwen3.5-35B-A3B, and crashes with a CUDA index out of bounds at scale. Training on the zero/garbage states “converges” (validation looks fine) but the resulting draft model has ~0% real acceptance — see vllm-project/speculators#613. This supe…

• c231d1f #47007 — fix(security): bound tokenizer work when explicit truncation_side is set (#47007)
  • 作者: Juan Pérez de Algaba | +94/-9 | 2 个文件

  When both truncation_side and truncate_prompt_tokens are set, get_encode_kwargs() disabled tokenizer-level truncation entirely, allowing attacker-controlled prompts to force unbounded tokenization before post-tokenization slicing. Restore a tokenizer-level max_length bound (context window + 1) in the explicit-side branch, and add a defense-in-depth character-level pre-truncation in _text_len_check…

• db808b3 #46781 — [Model Runner V2][Spec Decode] Implement block verification for rejection sampling (#46781)
  • 作者: Giancarlo Delfin | +641/-85 | 4 个文件

  Context Block verification as described in https://arxiv.org/pdf/2403.10444v3 is an approach to rejection sampling that is expected to produce acceptance rates at least as high as standard (per-token) rejection sampling, for a comparable cost. This feature is currently not supported in Model Runner V2, but is in progress for V1. The main idea of block verification is that we look at the joint pr…

• 7a327f0 #47125 — [Rust Frontend] Simplify unit tests with shared TestTokenizer (#47125)
  • 作者: Bugen Zhao | +682/-828 | 28 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Introduce a shared, general & configurable TestTokenizer for Rust frontend tests, which encodes and decodes UTF-8 at the byte level, with pluggable multi-character special tokens. Migrate most ad-hoc tokenizer stubs to it, reducing maintenance burden as the codebase grows. This also tightens the Tokenizer trait by requiring id_to_token implementations, s…

• 1ab9522 #47058 — Remove more unnecessary load_weights methods (#47058)
  • 作者: Harry Mellor | +1028/-3778 | 63 个文件

  • FusedMoE now takes ckpt_names instead of expert_mapping and stores them individually in RoutedExperts - RoutedExperts.load_weights then calls RoutedExperts.get_expert_mapping which uses these names to construct the expert mappings on the fly - RoutedExperts.load_weights now handles checkponts like Qwen3 VL MoE where the fused expert weights may be transposed - RoutedExperts.load_weights now brea…

• 0fc2512 #46450 — [KV Offload] Pass ScheduleEndContext to on_schedule_end hook (#46450)
  • 作者: Ronen Schaffer | +42/-14 | 11 个文件

  • Extend on_schedule_end() to accept a ScheduleEndContext dataclass containing new_req_ids and preempted_req_ids, enabling managers to react to per-step scheduling decisions - The new dataclass uses Iterable[str] field types for flexibility - Updates both OffloadingManager and SecondaryTierManager base classes and all implementations. ## Test Result —

• 62c7d80 #47151 — Forward fix nightly errors from #44589 (#47151)
  • 作者: Harry Mellor | +27/-17 | 3 个文件

  This PR forward fixes the failures introduced to main by #44589. Supersedes #47096. Fixes for Language Models Test (MTEB): - Move the hf_to_vllm_mapper so it belongs to Gemma3Model so that it can load itself. This is necessary because Gemma3Model is registered directly as Gemma3TextModel - Reinstate the prefixing of model. to JinaEmbeddingsV5Model weights so that it can load correctly Fix for Mult…

• ab80b3d #47139 — [CI/Build] Bump PyNvVideoCodec version (#47139)
  • 作者: Isotr0py | +1/-1 | 1 个文件

  PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED. - Fix https://github.com/vllm-project/vllm/pull/44465#issuecomment-4841638946 ## Test Result — BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

• 91055ef #47134 — [XPU] C++ implementation for get_memory_info (#47134)
  • 作者: Qiming Zhang | +72/-0 | 1 个文件

  Torch.xpu.mem_get_info returns wrong memory info with latest UMD. More details please refer to https://github.com/vllm-project/vllm-xpu-kernels/blob/70387bfdcc15f85628ed31b8faf62af537096435/tests/test_get_memory_info.py#L16 Use API of kernels to replace it. Refer to https://github.com/jikunshang/vllm/commit/bd4f96f97fa03cf70dda72ac9098b72f3fdb8477. Add some bounds and type check.

• 3675bcf #47101 — [Rust Frontend] Refactor TLS serve path with unified MaybeTlsListener (#47101)
  • 作者: Bugen Zhao | +138/-180 | 6 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Introduce a MaybeTlsListener as the only listener type the server will operate on, which can either be a plain TCP/UDS listener, or a SSL wrapper over that. Implement axum::serve::Listener and Stream<Item = std::io::Result> for that so it can be used both for axum HTTP server and tonic gRPC server. cc @tahsintunan Cargo tests ## Test Resu…

• bdbd727 #47110 — [Rust Frontend] Extend renderer/parser roundtrip tests to support token ids (#47110)
  • 作者: Bugen Zhao | +160/-79 | 2 个文件

  Some model paths are token-id native: renderers may directly yield token ids, and output processors may directly consume token ids. This PR extends the roundtrip tests to support raw token ids in those cases, demonstrated by the GPT-OSS/Harmony path. ## Test Result All passed locally. I also verified that roundtrip_gpt_oss fails if generated chunk token ids are cleared, which confirms the test cov…

• 5dc36a4 #47143 — [Model] Remove Tarsier, Tarsier2 (#47143)
  • 作者: Harry Mellor | +2/-872 | 12 个文件

  These models are: - >1.5 years old - See almost no usage in vLLM - Are only downloaded by vLLM CI so the download stats you see on HF are misleading This PR removes them both and adds entries to _PREVIOUSLY_SUPPORTED_MODELS for them. Note that Tarsier2 has a previously supported version of v0.23.0 as that was the last version of vLLM to support Transformers v4 Closes https://github.com/vllm-proje…

• 5360477 #33057 — Bump actions/checkout from 6.0.1 to 7.0.0 (#33057)
  • 作者: dependabot[bot] | +2/-2 | 2 个文件

  Bumps actions/checkout from 6.0.1 to 7.0.0.

• 8cf7c4d #46020 — [Attention Backend] add HPC-Ops Attention backend (#46020)
  • 作者: Cheng Jiang | +978/-14 | 10 个文件

  Support hpc-ops attention backend. HPC-Ops is a production-grade, high-performance, and easy-to-use operator library for LLM inference, developed by the Tencent Hunyuan AI Infra team. You can enable hpc-ops attention backend by specify –attention_backend HPC_ATTN –kv-cache-dtype fp8_e4m3 –block-size 64. WARNING: The HPC attention backend currently supports only the Hy3-FP8 model. If you wan…

• 364ee36 #47010 — fix(security): prevent image decompression bomb OOM denial of service (#47010)
  • 作者: Juan Pérez de Algaba | +115/-12 | 7 个文件

  Add VLLM_MAX_IMAGE_PIXELS env var (default 178,956,970 – matching PIL’s 2x decompression-bomb threshold) and enforce a pixel-count check in ImageMediaIO.load_bytes() before calling image.load(), so raster memory is never allocated for oversized images. Additionally, replace the global Image.MAX_IMAGE_PIXELS = None in the NemotronVL processors with a finite 300M-pixel cap to restore PIL’s built-in…

• 14f8660 #47032 — [CI/Build] Add CPU test dependency pre-commit hooks (#47032)
  • 作者: Li, Jiang | +1306/-21 | 3 个文件

  • Add CPU test dependency pre-commit hooks to pin package versions with CUDA ## Test Result —

• 8cc2423 #46433 — [XPU] Optimize XPU worker shutdown logic to prevent resource leak (#46433)
  • 作者: Chaojun Zhang | +41/-5 | 7 个文件

  Purpose: Optimize the XPU shutdown flow to properly release oneCCL/Level Zero resources when vLLM server stops. This also fixes a hang issue where repeated server startups on the same XPU devices would deadlock during oneCCL initialization. ## Problem: With the default shutdown_timeout=0 (abort mode), worker processes are force-killed without executing cleanup code. On XPU, this leaves Level Ze…

🧪 CI/Tests

• f1cf6b0 #47299 — [CI] Fix segfault in tracing test (#47299)
  • 作者: Nick Hill | +4/-5 | 1 个文件

  This was actually already fixed for rocm case but it is not rocm-specific and the regular metrics test fails intermittently due to this. E.g. https://buildkite.com/vllm/ci/builds/75740/list?jid=019f1d4f-1e55-4e15-b6db-fd8f495f8bef&tab=output

• e7d0fcb #47197 — [CI] Fix various failures on main (#47197)
  • 作者: Harry Mellor | +18/-14 | 5 个文件

  • https://github.com/vllm-project/vllm/pull/43586 missed Gemma3ForConditionalGeneration when adding path to the interface for various methods (Multi-Modal Models (Extended Generation 1)) - https://github.com/vllm-project/vllm/pull/47000 used torch.accelerator.set_device which doesn’t exist (Distributed Tests (2xH100-2xMI300)) - https://github.com/vllm-project/vllm/pull/46635 added Tensor.split cal…

• dee5da1 #47250 — [Test] Run SageMaker handler-override tests in-process via TestClient (#47250)
  • 作者: Jyothirmai Kottu | +227/-333 | 1 个文件

  The SageMaker handler-override integration tests in tests/entrypoints/serve/sagemaker/test_sagemaker_handler_overrides.py each launched a full model server via RemoteOpenAIServer, only to exercise customer handlers that fully replace the /ping and /invocations endpoints. No inference engine is needed to validate override behavior. Beyond being slow, starting a server made these tests sensitive t…

• c3b1f9e #47193 — [ROCm][CI] Enable LoRA TP Distributed Test Group In AMD CI (#47193)
  • 作者: Micah Williamson | +21/-2 | 2 个文件

  There were a couple of issues in this test group. First, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True was copied over from the upstream CI definition which was included as a workaround for a bug which only impacts CUDA CI, not ROCm. It was causing an issue with custom all reduce in pytest -v -s tests/lora/test_chatglm3_tp.py::test_chatglm3_lora_tp4_fully_sharded_loras (example log here https:/…

• 3c1396b #47222 — [Hardware][AMD][CI] Toggle test coredumps on ROCm debug agent (#47222)
  • 作者: Matt | +16/-1 | 2 个文件

  We are observing some cases where AMD CI tests hang after memory access faults. The cause appears to be linked to the GPU coredump handler failing to attach/exit cleanly in some cases. In particular, the coredump handler currently always fails inside the ROCm test container with GPU coredump: execvp failed: No such file or directory due to https://github.com/rocm/rocm-systems/issues/6206. As such,…

• 345b28f #47195 — [Hardware][AMD][CI] Bump timeouts of various test groups on AMD CI (#47195)
  • 作者: Matt | +14/-14 | 1 个文件

  We are seeing some failures on AMD CI caused by overly-short timeouts. Most of these errors are caused by slow Docker image pulls on the MI300 cluster. While we investigate these further, we’re bumping timeouts of various MI300-based test groups by 45 minutes to ensure we still get meaningful signals by allowing test groups to run to completion. It is worth noting that this change does not affect …

• 27d5f78 #47048 — [CI] Move distributed small LM eval to B200 (#47048)
  • 作者: Lucas Wilkinson | +68/-14 | 4 个文件

  • Rename the single-device B200 small-model LM eval step to 1xB200. - Move the distributed small-model LM eval step from 2xL4 to 2xB200. - Rename the distributed step/key to reflect its 2xB200 placement. ## Duplicate-work checks - No issue number was provided, so no issue comment check was applicable. - Checked open PRs for lm-eval-small-models-distributed-2xb200, DiffusionGemma GSM8K B200, models…

• f41e8dd #47065 — [ROCm][CI] Move PyTorch Compilation Unit Tests to MI300(gfx942) (#47065)
  • 作者: Charlie Fu | +20/-20 | 1 个文件
• ea9ddf5 #45977 — [XPU][CI] Enable shared loader test (#45977)
  • 作者: Chaojun Zhang | +28/-1 | 2 个文件

  This PR improves stability of test_sharded_state_loader on XPU by forcing max_num_seqs=1 in the XPU test path. - Apply max_num_seqs=1 only when running on XPU. - Keep behavior unchanged for non-XPU platforms.

⚡ Performance

• 89e9920 #44639 — [CPU][Perf]Added tanh AOR for faster gelu activations. (#44639)
  • 作者: almayne | +328/-3 | 9 个文件

  Co-authored-by: GPT 5.4 Add the Arm Optimised Routine for tanh on SIMD architectures. This is enabled for gelu activations where the tanh approximation is used. - Add a unit test and ensure it passes. - Compare performance of new implementation and PyTorch’s F.gelu(tanh). ## Test Result The below passes: python -m pytest -x -v -s tests/kernels/core/test_cpu_activation.py::test_cpu_gelu_tanh_and_mu…

• 92c7fac #45739 — [Perf] Restore zero-init of swizzled NVFP4 scale buffer to recover Blackwell decode throughput (#45739)
  • 作者: Albert Cheng | +7/-1 | 1 个文件

  Fixes a severe NVFP4 decode regression on Blackwell (B300 / GB300) introduced by #42988 ("[Perf] zeros -> empty to remove additional fill”, commit b29cbf0652). create_fp4_scale_tensor (vllm/_custom_ops.py) allocates the swizzled scale-factor buffer with shape (round_up(m, 128), round_up(n // 16, 4) // 4), i.e. padded in both the M (to 128) and packed-K (to 4) dimensions. The NVFP4 quant kernel…

• 953bba4 #46703 — [PERF] Extend NCCL symmetric memory to AllGather and ReduceScatter (#46703)
  • 作者: Woosuk Kwon | +405/-112 | 6 个文件

  Enable NCCL symmetric memory and NVLS (NVLink SHARP) for AllGather and ReduceScatter collectives (used by MoE dispatch/combine in Expert Parallel deployments), extending the existing symmetric-memory support that was limited to AllReduce. Also fixes per-communicator window registration so multiple NCCL communicators can share the symmetric memory pool without double-registering segments. All chang…

🔧 Refactor

• b1190d0 #47185 — [Refactor][GPT-OSS] Harmony Responses API Refactor to use HarmonyParser (#47185)
  • 作者: yzong-rh | +424/-854 | 9 个文件

  Second pass at https://github.com/vllm-project/vllm/pull/46102 - Make gpt-oss in Responses API use the unified HarmonyParser - Consolidate HarmonyContext and HarmonyStremaingContext - Minimal changes to _process_harmony_streaming_events (Harmony -> Responses translation) – just enough to fix bug, no refactor to consolidate harmony vs non-harmony and streaming vs non-streaming Bugs fixed: - _proce…

• 9e84ec8 #46842 — [Refactor] Remove dead minimax allreduce rms kernel (#46842)
  • 作者: Wentao Ye | +0/-52 | 4 个文件

  We are using minimax_allreduce_rms_qk instead.

✨ New Feature

• 248d1fb #46182 — [Feat][1/N] CuTeDSL warmup infrastructure, FA4 MLA (#46182)
  • 作者: Roberto L. Castro | +518/-2 | 9 个文件

  Depends on: https://github.com/vllm-project/vllm/pull/46167 and https://github.com/vllm-project/flash-attention/pull/150 This is the first PR in a broader CuTeDSL warmup series. The initial study focuses only on the FA4 MLA prefill path. This PR adds a generic CuTeDSL warmup hook and one provider for FA4 MLA prefill. The goal is to compile the relevant FA4 CuTeDSL kernels during model warmup, befo…

• 20434c4 #46621 — [Feat] Improve Triton JIT diagnostics (#46621)
  • 作者: Roberto L. Castro | +86/-9 | 1 个文件

  BEFORE: AFTER:

• 3a9784b #47076 — [Feature] DP supervisor using rust frontend (#47076)
  • 作者: Wentao Ye | +60/-3 | 2 个文件

  A better way for https://github.com/vllm-project/vllm/pull/44915 The DP supervisor is quite thin, we can still use Python, but get the performance using Rust for DP sub ranks. Then we exit elegantly.

• 245888f #43637 — [Feature] Detect all2all peer fault with fault tolerance backend and prevent corrupted output (#43637)
  • 作者: fangyuchu | +82/-4 | 5 个文件

  Some all2all backends (nixl_ep, deepep_low_latency) support fault-tolerant communication: when a peer rank fails, instead of hanging indefinitely like traditional collectives, the healthy rank sets a mask to exclude the failed rank after a configurable timeout and continues running. However, the ongoing forward pass that triggered the timeout already produced a corrupted token (computed with one r…

🖥️ Kernel

• 8e9d70f #45140 — [Kernel][XPU] Adjust kernel unit tests for XPU (#45140)
  • 作者: Agata Dobrzyniewicz | +5/-5 | 1 个文件

  Adjustments to kernel unit tests so they pass on Intel XPU. More may be added as they come up. ## Adjustments tests/kernels/mamba/test_mamba_ssm.py — widen bf16 atol for XPU. The selective_state_update Triton kernels accumulate internally in fp32, so on rare elements they are ~1 ULP more accurate than the pure-PyTorch bf16 reference (which accumulates in bf16). With the default bf16 toleranc…

🔩 Misc

• 06fae69 #47132 — [Misc] Mistral label alert (#47132)
  • 作者: Nicolò Lucchesi | +1/-1 | 1 个文件

🦀 Rust Frontend

• 2bc20e8 #46610 — [Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser (#46610)
  • 作者: Chauncey | +397/-570 | 6 个文件

  [Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser ## Test Result —