共 45 个 commit，涉及 240 个文件，+7617/-2888 行变动。

概要

统计项 数值
Commit 数 45
变更文件 240
新增行数 +7617
删除行数 -2888

Commit 列表

🧪 CI/Tests

  • 3775d5f #47479 — [ROCm][CI] Adding test groups for parity with upstream (#47479)
    • 作者: Andreas Karatzas | +189/-2 | 2 个文件

    Adds missing MI300 AMD CI parity jobs for MRCR evals, vLLM IR kernels, KDA kernels, GGUF plugin coverage, and Model Runner V2. It also bumps the MI355 examples timeout to reduce false failures on the longer examples suite. The KDA path includes a ROCm-specific Triton workaround that avoids num_stages=4 for chunk_delta_h. These additions bring the ROCm pipeline closer to the upstream parity coverag…

  • b790c84 #45246 — [CI] Enable sccache for Rust build under CUDA/ROCm (#45246)
    • 作者: Bugen Zhao | +67/-33 | 3 个文件

    Signed-off-by: Bugen Zhao i@bugenzhao.com We’ve encountered several cases where BuildKit cache is not reliable for Rust builds, especially when including the target directory for incremental build artifacts. To mitigate the issue, we - added sharing=locked in #43815 (for ROCm) and #44852 (for CPU) - excluded target dir in #44624 On the other side, sccache seems more reliable when spanning differ…

  • 2dfaae7 #47510 — [XPU][CI]Fix dependency typo in Intel GPU CI (#47510)
    • 作者: xiangdong | +1/-1 | 1 个文件

    Test Result — - [X] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”. - [X] The test plan, such as providing test command. - [X] The test results, such as pasting the results comparison before and after, or e2e results - [X] (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

  • 3f0b773 #47405 — [XPU][CI]Mv huggingface cache to larger disk in Intel GPU CI (#47405)
    • 作者: xiangdong | +2/-1 | 2 个文件

    Test Result —

  • d85601c #47465 — [CI] Pin modelscope version to fix test breakage (#47465)
    • 作者: Nick Hill | +7/-7 | 6 个文件

    We can update the code to properly tolerate more versions after this. But this is to get CI green, we should be pinning all of our dependencies regardless.

  • 34bf7b4 #46456 — [CI] intel CI: add quantization and awq case for xpu (#46456)
    • 作者: wenjun liu | +30/-0 | 3 个文件

    Test Result —

  • 442ccc6 #47482 — [ROCm][CI] Adding extract hs 2gpu (#47482)
    • 作者: Andreas Karatzas | +4/-2 | 2 个文件

    Updates the Extract Hidden States Integration CI job to request the 2-GPU MI300 pool and declare two GPUs. The underlying TP2 extraction test already requires two GPUs, so this aligns the Buildkite agent allocation with the test requirement. The TP2 test timeout is also increased on ROCm while preserving the existing shorter timeout on other platforms. This should reduce ROCm-only timeout failures…

  • 6768fbc #47480 — [ROCm][CI] Adding qwen3 dp4 eplb (#47480)
    • 作者: Andreas Karatzas | +25/-0 | 2 个文件

    Adds the Qwen3-30B-A3B FP8 DP4 async EPLB accuracy job to the ROCm AMD CI pipeline on the 4-GPU MI300 pool. The job runs the existing scheduled integration script with ROCm-specific coverage in test-amd.yaml. The script now disables ROCm MoE padding when ROCm is detected, avoiding the EPLB failure mode seen on AMD. This keeps the change scoped to the Qwen3 DP4 async EPLB path.

  • 407f406 #47477 — [ROCm][CI] Adding metadata (#47477)
    • 作者: Andreas Karatzas | +62/-0 | 3 个文件

    Adds a ROCm Docker build metadata job to AMD CI on the MI250 pool. The job runs the existing Docker build metadata argument tests from the ROCm image and declares the Docker/bake inputs it depends on. Dockerfile.rocm now copies the Docker and bake files into the ROCm image so those tests can inspect the ROCm bake path. The tests also assert the ROCm ci-base content hash label and metadata input co…

  • 8357226 #47376 — [XPU][CI] Split test_punica_ops into separate pytest invocations for stability (#47376)
    • 作者: Chaojun Zhang | +3/-1 | 1 个文件

    Stabilize the XPU LoRA Punica kernel CI by splitting test_punica_ops.py into three separate pytest invocations, rather than running the full suite in a single process. This avoids the test failures that occur when all tests are executed together, and removes the need for an ever‑growing list of –deselect exclusions.

📦 Other

  • d7192cf #47539 — [CI Bugfix] Lazily import Qwen warmup dependencies (#47539)
    • 作者: Roberto L. Castro | +21/-13 | 1 个文件

    Qwen warmup imported CUDA-touching FLA/Mamba helpers at module load time. Since gpu_worker -> kernel_warmup -> qwen_triton_warmup, this initialized CUDA before fork and broke engine tests: https://buildkite.com/vllm/ci/builds/75088#019f12dc-d9e0-48ce-a925-f64180ca7359 Fix: move those imports inside the warmup functions so the module is import-safe; CUDA is only touched when warmup actually runs. #…

  • 400a9c3 #47530 — [Rust Frontend] Bump llm-multimodal version (#47530)
    • 作者: Isotr0py | +48/-24 | 6 个文件
  • bbdcbe4 #47452 — Move Roberta remaining nn.Embedding to VocabParallelEmbedding (#47452)
    • 作者: Max de Bayser | +3/-5 | 1 个文件

    Roberta models have 3 embedding modules. 1 already was a VocabParallelEmbedding but the other 2 weren’t. nn.Embedding is not a PluggableLayer so does not allow platform specific customizations. cc: @noooop

  • 1f486d9 #47102 — Add Triton Backend for Unlimited-OCR R-SWA (#47102)
    • 作者: Dakai An | +68/-17 | 5 个文件

    This PR adds explicit TritonAttention support for Unlimited-OCR’s R-SWA decode mask. The Triton path implements the mask with: This keeps prompt/image prefix tokens globally visible, while generated tokens use a causal sliding window. The implementation reuses existing rswa_prefix_lens metadata and applies the mask inside Triton unified attention. It also allows the R-SWA path to use TritonAttenti…

  • 6429d5f #46684 — [Rust Frontend] add repetition_detection support to sampling params (#46684)
    • 作者: yangyang-cs95 | +177/-20 | 14 个文件
    • Added RepetitionDetectionParams struct to the engine-core-client protocol layer with max_pattern_size, min_pattern_size, and min_count fields - Added repetition_detection field to EngineCoreSamplingParams for wire transport - Wired the field through vllm-text SamplingParams and lowered to engine params - Enabled repetition_detection in the chat completions API (types, convert, validate) - Added …
  • fbc9ba6 #46656 — New stable abi cleanup (#46656)
    • 作者: Chris Leonard | +96/-44 | 50 个文件

    This is to cleanup a bit after migrating vLLM to the libtorch stable ABI. This includes: - moving all headers that are only used by files in the libtorch_stable directory to that directory. - Adding device guards that we didn’t add during migration because they weren’t there before, but probably should be there. - removing an import warning for _qutlass_C in cuda.py Other things that could be impr…

  • bd8d902 #47467 — [CPU][Build] Enable oneDNN ITT task collection by default for CPU primitive-level profiling (#47467)
    • 作者: Evgeny Parshutin | +1/-1 | 1 个文件

    This PR enables oneDNN ITT API task collection in the CPU backend by flipping the build-time CMake option ONEDNN_ENABLE_ITT_TASKS from OFF to ON in cmake/cpu_extension.cmake. When building the CPU extension, vLLM fetches and statically links oneDNN for its matmul/reorder kernels. oneDNN’s ITT API integration tags each primitive execution with an ITT task, which lets Intel VTune Profiler (and any o…

  • 9b8e765 #47289 — [Rust Frontend] Recover buffered text from incomplete tool calls at EOS (#47289)
    • 作者: Reid | +61/-9 | 1 个文件

    When a Gemma4 tool-call stream is truncated before the closing marker, Gemma4UnifiedParser::finish() returns an incomplete-tool-call error while the raw tool-call text remains buffered inside the parser. The unified stream wrapper previously closed parser state without recovering that buffer, so clients could receive only Done and an empty assistant message. This PR mirrors the existing mid-stream…

  • 4c3c64f #46853 — Add Laguna XS.2.1 DFlash drafter support (#46853)
    • 作者: adamkbaranowski | +494/-47 | 7 个文件

    Add vLLM model support for Poolside Laguna XS.2.1 DFlash draft checkpoints. This registers DFlashLagunaForCausalLM and implements the Laguna-specific DFlash drafter used with method=“dflash” speculative decoding. The drafter consumes verifier auxiliary hidden states from the Laguna target model, precomputes context K/V for DFlash attention, and shares the target model token embeddings and LM head….

  • e24d1b2 #47472 — Fix Transformers modeling backend usage stats (#47472)
    • 作者: Harry Mellor | +8/-1 | 1 个文件

    Any use of the Transformers modeling backend previously obfuscated what was being wrapped. This PR fixes that so that: - All usage of the Transfomers modeling backend is prefixed with the wrapper class (fixes entries in _TRANSFORMERS_SUPPORTED_MODELS) - Models loaded with the fallback behaviour include the wrapped architecture in the name

  • d29125c #43232 — Xqa decode kernels (#43232)
    • 作者: Dan Blanaru | +538/-143 | 7 个文件

    Enable the FlashInfer TRTLLM/XQA decode path on Hopper (SM90) for vLLM V1 FlashInfer attention. This makes TRTLLM attention support phase-aware, so decode can use FlashInfer’s trtllm_batch_decode_with_kv_cache XQA path while prefill can remain on the existing FlashInfer native path when TRTLLM prefill is not supported. The branch also tracks prefill and decode query dtypes separately, since SM90 X…

  • d715b3a #47361 — Delete PagedAttention (#47361)
    • 作者: Michael Goin | +90/-1472 | 9 个文件

    It is time. ## Test Result —

  • ec0ffaa #47435 — [Rust Frontend] Improve scheduler stats logging parity (#47435)
    • 作者: Bugen Zhao | +521/-30 | 6 个文件

    Signed-off-by: Bugen Zhao i@bugenzhao.com This PR fills several straightforward Rust frontend periodic log-stats parity gaps with Python: - deferred requests, preemptions, and external prefix cache hit rate - spec decoding: accepted/draft throughput, accepted/drafted token counts, average draft acceptance rate, and per-position acceptance rate - mfu: existing estimated FLOPs/read/write counters …

  • 178fd56 #47410 — support GLM-5.2 gate use FP32 (#47410)
    • 作者: Yuxuan Zhang | +19/-0 | 2 个文件
  • 3e158ae #47428 — [ModelRunner V2] Fix Mamba2 crash on non-spec-decode (#47428)
    • 作者: Nick Hill | +1/-1 | 1 个文件

    MambaHybridModelState.build_metadata populated num_accepted_tokens on every forward pass, so the Mamba2 attention builder passed a wrongly shaped tensor (num_reqs, not num_decodes) straight into selective_state_update, tripping assert num_accepted_tokens.shape == (N,). This crashed hybrid Mamba2 + MoE models (e.g. GraniteMoeHybrid) now routed to V2. Gate the population on speculative decoding bein…

  • a2f7130 #44443 — [ModelRunner V2] Enable by default for all dense models (#44443)
    • 作者: Wentao Ye | +55/-17 | 6 个文件

    Part of https://github.com/vllm-project/vllm/issues/41286 Covered in CI

  • de2a8fc #47128 — [ROCm] [PyTorch] Move to stable abi since ROCm upgraded to torch 2.11 (#47128)
    • 作者: TJian | +22/-131 | 6 个文件

    Since ROCm has upgraded to torch 2.11 in PR https://github.com/vllm-project/vllm/pull/45362 , this PR follow up to clean up the previous fallback loop https://github.com/vllm-project/vllm/pull/44648 . So we are moving back to stable_abi. ## Test Result - pytest -svvvvv tests/kernels/core/test_uva.py - python3 -m pytest tests/lora/test_moe_lora_align_sum.py -q - Moe test —

  • 84b9c27 #47304 — Update DeepGEMM tag to point to latest nv-dev branch for sm120 support (#47304)
    • 作者: Michael Goin | +4/-2 | 2 个文件

    This PR https://github.com/vllm-project/vllm/pull/43477 landed in vLLM without bumping up our commit tag for DeepGEMM for the SM120 support, which resulted in many failures for models that trigger DeepGEMM kernels i.e. https://github.com/vllm-project/vllm/issues/47169, https://github.com/vllm-project/vllm/issues/47266 For now we should bump the DeepGEMM commit for nv-dev (which includes https://gi…

  • 25fcb65 #47283 — [Rust Frontend] Use enum-backed domain types for engine outputs and structured outputs (#47283)
    • 作者: Bugen Zhao | +760/-603 | 16 个文件

    Several engine-core protocol payloads arrive as product-shaped Python/msgpack structs, but Rust code wants finite semantic variants. Encoding those invariants in the public Rust types makes invalid states harder to construct and keeps validation closer to the serde boundary. Refactor some engine-core protocol types to expose enum-backed domain types northbound, while still perserving the existing …

  • 08a8a4a #46306 — feat(rust): expose profiler control routes in Rust frontend (#46306)
    • 作者: Pranav Thakur | +381/-2 | 11 个文件

    This adds the /start_profile and /stop_profile HTTP routes to the Rust frontend when profiling is enabled via –profiler-config. The routes forward the call to the Python engine’s profile utility RPC. Fixes #46092 Checks performed: - Duplicate PRs checked: None found addressing the Rust frontend profiler routes. - Tests run locally: - cargo test -p vllm-cmd –bin vllm-rs - cargo test -p vllm-serve…

  • b0b8a28 #44785 — [Model] Add LLaVA-OneVision-2 (LlavaOnevision2ForConditionalGeneration) (#44785)
    • 作者: chengzheng345 | +2593/-0 | 6 个文件

    Add support for LLaVA-OneVision-2 as a single multimodal architecture implemented in one model file (vllm/model_executor/models/llava_onevision2.py). Supported inputs: text + image + video (T + I+ + V+). - Image: AnyRes dynamic tiling + 2x2 spatial merge with exact placeholder accounting. - Video: three preprocessing backends selected through a single mm_processor_kwargs entry point (video_backend…

🐛 Bug Fix

  • 978de83 #47447 — [Bugfix][CPU] Ship examples/ in the CPU release image (#47447)
    • 作者: AgenticSpark | +6/-0 | 1 个文件

    Fixes #47401. The vllm/vllm-openai-cpu image is missing examples/ (e.g. examples/tool_chat_template_gemma4.jinja), so –chat-template examples/*.jinja and the documented example paths don’t work out of the box on CPU. The vllm-openai release stage in docker/Dockerfile.cpu is FROM base, and never copies examples/. The ADD ./examples/ ./examples/ that already exists in the file is in the vllm-test s…

  • 18f658b #47384 — [Bugfix][Frontend] Fix batch chat endpoint corrupting logprobs when return_token_ids is set (#47384)
    • 作者: David Feng | +80/-1 | 3 个文件

    Summary The /v1/chat/completions/batch endpoint wired logprobs rendering to request.return_token_ids instead of return_tokens_as_token_ids. As a result, setting return_token_ids (which should only add a separate token_ids list) also silently rendered every logprob token as a “token_id:{id}” placeholder. BatchChatCompletionRequest was missing the return_tokens_as_token_ids field entirely (prese…

  • 1aeabec #44682 — [Bugfix][Rust Frontend] Tolerate out-of-vocab prompt ids in detokenizer (#44682)
    • 作者: Ting SUN | +103/-10 | 1 个文件

    A pre-tokenized /v1/completions prompt can contain token ids in [tokenizer_vocab, model_vocab). The model can embed those ids, but the Rust frontend also uses prompt ids as left context for streaming detokenization. If the tokenizer cannot map one prompt id back to token text, DecodeStream prefix seeding fails the stream with a tokenizer error before generation can continue. This keeps Tokenizer::…

  • 979f551 #47217 — [Bugfix][Gemma4] Keep image bidirectional attention within the sliding window (#47217)
    • 作者: Luciano Martins | +89/-9 | 8 个文件

    Gemma4 vision models that use blockwise bidirectional image attention (use_bidirectional_attention=“vision” → 12B, 26B-A4B, 31B) silently fell back to causal-only attention over image tokens whenever a single image’s soft-token span exceeded the text sliding window (e.g. ~1100 soft tokens at max_soft_tokens=1120 vs sliding_window=1024). This degraded vision/OCR quality — catastrophic on the en…

  • 41de138 #47485 — [BugFix] Derive FlashInfer Q dtype from resolved per-group builder state (#47485)
    • 作者: Michael Goin | +21/-19 | 1 个文件

    Fixes the Quantization CI failures from #43232: get_q_data_type decides from the global config, so FP8-Q gets selected on archs with no fp8 tensor-core path (L4 crashes with fp8 tensor core is not supported in fa2 backend) and for unquantized –kv-cache-dtype-skip-layers groups. Use the builder’s group-aware self.cache_dtype instead and only quantize Q on SM90/SM100. Verified with the previously-f…

  • 276b837 #47483 — [ModelRunner V2][BugFix] Free all model refs on shutdown (#47483)
    • 作者: Nick Hill | +4/-0 | 1 个文件

    There are some remaining references to the model that aren’t cleared on shutdown meaning that memory might not be released. Fixes CI failure https://buildkite.com/vllm/ci/builds/76036/list?jid=019f2439-b7d0-4fa7-868d-c2d44c78d50e&tab=output which started after https://github.com/vllm-project/vllm/pull/44443 that made MRV2 the default for all dense models.

  • 258f8de #47311 — [Bugfix][Tool Parser] poolside_v1: accept tool calls without newline after function name (#47311)
    • 作者: Joe Rowell | +103/-1 | 2 个文件

    Problem poolside_v1’s non-streaming extract_tool_calls uses a func_detail_regex that requires a literal newline between the function name and the <arg_key> block: That matches GLM-4’s format, but some models emit the tool call with no newline after the name — e.g. <tool_call>get_weather<arg_key>city</arg_key><arg_value>Paris</arg_value></tool_call>. For that output func_detail_regex.se…

  • e392bf7 #46974 — [BugFix][MRV2] Ensure all req slots are accounted for when scheduling (#46974)
    • 作者: Nick Hill | +223/-5 | 2 个文件

    MRV2 assumes that there can be at most max_num_seqs active requests, which corresponds to the maximum running queue length in the scheduler. However, there are two edge cases where MRV2 slots can still be occupied by requests not in the running queue: 1. Those waiting for streaming input (in WAITING_FOR_STREAMING_REQ state) 2. Those which have been preempted “out of band” - in particular during fo…

  • 443e68c #47437 — [Bugfix] Fix pooled Whisper encoder sliding-window kernel size (#47437)
    • 作者: Nick Hill | +27/-13 | 2 个文件

    The causal Whisper encoder pools block_pool_size encoder tokens per KV block, so its SlidingWindowSpec.sliding_window is expressed in pooled units for the KV cache manager. The attention kernel, however, reads its window from the same spec field but runs on the block_pool_sizex-expanded (unpooled) sequence, so it interpreted the pooled window as an unpooled one and attended to block_pool_sizex too…

  • a47f38f #47383 — [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in block verification kernels (#47383)
    • 作者: Woosuk Kwon | +148/-4 | 2 个文件

    _compute_cumulative_log_p_kernel and _compute_local_residual_mass_kernel (used when rejection_sample_method=“block”) multiply int32 logit/request-state indices by vocab-scale strides before int64 promotion. With a GLM-scale vocab (~155k tokens) the byte offset wraps once logit_idx * vocab >= 2**31 (logit_idx >= ~13.8k, reachable with large max_num_seqs × 1 + num_speculative_tokens), or once req_st…

🦀 Rust Frontend

  • a14f57a #47498 — [Frontend] Refine the entrypoint class’s inheritance hierarchy. (#47498)
    • 作者: wang.yuqi | +90/-191 | 23 个文件

    The entrypoint class’s inheritance hierarchy ├─BaseServing <- Used to handle basic functions such as _is_model_supported, via OpenAIServingModels. │ ├─GenerateBaseServing <- Used to handle streaming output, as well as BeamSearch (via BeamSearchOnlineMixin). │ │ ├─OpenAIServingChat │ │ │ └─AnthropicServingMessages │ │ ├─OpenAIServingCompletion │ │ ├─OpenAIServingResponses │ │ └─ServingTokens │ ├─Po…

📖 Documentation

  • 4875b44 #47517 — [Doc] Fix VLM2Vec benchmark chat template path (#47517)
    • 作者: Kalyanam Dewri | +1/-1 | 1 个文件

    Summary: - Fix the VLM2Vec benchmarking example to point at the checked-in examples/pooling/embed/template/vlm2vec_phi3v.jinja template. - Keep the benchmarking docs consistent with the pooling model docs and example script. - Checked open issues and PRs for this path and did not find an existing duplicate. - AI assistance was used to identify and prepare this docs-only change; the final diff and …

⚡ Performance

  • 320ee28 #47285 — [Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel (#47285)
    • 作者: Chauncey | +7/-0 | 1 个文件

    [Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel ## Test Result before after —

✨ New Feature

  • 3af8789 #38174 — [Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) (#38174)
    • 作者: Wonderful | +364/-5 | 8 个文件

    Implements Token-Level Intersection (TLI) speculative decoding, allowing target and draft models to have different (but overlapping) vocabularies. Closes #38173 ## Algorithm Based on the ICML 2025 oral paper: > Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies > Timor et al., https://arxiv.org/abs/2502.05202 How it works: 1. At startu…