共 30 个 commit，涉及 141 个文件，+4322/-1275 行变动。

概要

统计项 数值
Commit 数 30
变更文件 141
新增行数 +4322
删除行数 -1275

Commit 列表

🐛 Bug Fix

  • 0cd6f76 #47379 — [Bugfix][Frontend][gpt-oss] Recover raw tail when Harmony parser ends non-terminal (#47379)
    • 作者: yzong-rh | +145/-83 | 4 个文件

    co-author: @shenoyvvarun @Achyuthan-S Following https://github.com/vllm-project/vllm/pull/47062 and https://github.com/vllm-project/vllm/pull/46437, addresses issue https://github.com/vllm-project/vllm/issues/45736 for Responses API. Similar to https://github.com/vllm-project/vllm/pull/47061 Unlike #47061, this PR doesn’t implement raw output tracking for Responses API. Instead it consolidates raw…

  • e7c9df9 #44297 — [Bugfix][Structured Output][Spec Decode] Constrain bitmask and trim grammar advance at the reasoning boundary (#44297)
    • 作者: yue.yu | +480/-16 | 6 个文件

    Fixes #44006. When VLLM_ENFORCE_STRICT_TOOL_CALLING=1 is combined with MTP/EAGLE speculative decoding, the engine intermittently rejects token 248069 (Qwen’s ) and the request finishes with 500 “Failed to advance FSM … grammar rejected tokens”. There are two related defects in StructuredOutputManager.grammar_bitmask, a hardening follow-up to the first fix, and a scheduler-side fix requir…

  • 26eb872 #45844 — [Bugfix] Fix CPU split-KV scratchpad sizing (#45844)
    • 作者: gausah01 | +57/-15 | 2 个文件

    Fix a CPU split-KV hang seen with google/gemma-4-26B-A4B-it. The worker could get stuck in reduce_splits waiting for a split completion flag. The root cause was the split-KV scratchpad was sized using the final output element size / smaller fallback q-head count, while the runtime split path writes temporary split outputs as float and can process more q-heads per split. This could under-size the s…

  • f329ce4 #47536 — [ROCm][CI][Bugfix] Use VllmRunner for voxtral_realtime tests to avoid OOM on AMD GPU (#47536)
    • 作者: Shanshan Shen | +64/-55 | 1 个文件

    On ROCm CI, test_voxtral_realtime.py fails with: Root cause: The current engine and async_engine fixtures do not clean up GPU memory thoroughly enough between tests. ROCm reclaims VRAM lazily, so back-to-back model loads OOM. VllmRunner.exit (conftest.py:1298) already has the correct cleanup sequence: 1. engine_core.shutdown() with extended 60s timeout on ROCm 2. del self.llm 3. torch._dynamo….

  • 3799501 #47566 — [Bugfix][Multimodal] Normalize direct PIL image inputs (#47566)
    • 作者: Ting SUN | +8/-0 | 1 个文件

    vLLM normalizes image bytes from URL/base64/file inputs before they reach the model: JPEG EXIF orientation is applied to pixels and images are converted to the default RGB mode with transparency composited against the configured background. The supported direct PIL.Image path used by offline LLM.generate and LLM.chat skipped that preprocessing and passed the raw PIL object into ImageProcessorItems…

  • 978de83 #47447 — [Bugfix][CPU] Ship examples/ in the CPU release image (#47447)
    • 作者: AgenticSpark | +6/-0 | 1 个文件

    Fixes #47401. The vllm/vllm-openai-cpu image is missing examples/ (e.g. examples/tool_chat_template_gemma4.jinja), so –chat-template examples/*.jinja and the documented example paths don’t work out of the box on CPU. The vllm-openai release stage in docker/Dockerfile.cpu is FROM base, and never copies examples/. The ADD ./examples/ ./examples/ that already exists in the file is in the vllm-test s…

  • 18f658b #47384 — [Bugfix][Frontend] Fix batch chat endpoint corrupting logprobs when return_token_ids is set (#47384)
    • 作者: David Feng | +80/-1 | 3 个文件

    Summary The /v1/chat/completions/batch endpoint wired logprobs rendering to request.return_token_ids instead of return_tokens_as_token_ids. As a result, setting return_token_ids (which should only add a separate token_ids list) also silently rendered every logprob token as a “token_id:{id}” placeholder. BatchChatCompletionRequest was missing the return_tokens_as_token_ids field entirely (prese…

🧪 CI/Tests

  • f1445f6 #47551 — [CI] Bump huggingface-hub from v1.10.2 to v1.22.0 (#47551)
    • 作者: Harry Mellor | +49/-45 | 25 个文件

    This release includes https://github.com/huggingface/huggingface_hub/pull/4394: > snapshot_download now caches a repository’s file listing on disk under a new trees/ folder, so re-downloading a commit that’s already cached costs a single network call — resolving the branch or tag to a commit hash — instead of one metadata request per file. The listing is immutable per commit and shared by both sna…

  • f006e5a #47554 — [CI][AMD] Allow git operations on previously created work trees (#47554)
    • 作者: Tres | +4/-2 | 1 个文件

    The buildkite runs git checkout as root but the job command as a different user (e.g. buildkite-agent/uid=1001); git 2.35.2+ rejects the repo with “fatal: detected dubious ownership” unless the directory is explicitly allowed. This failure is specific to runs merging main and could not be reproduced with runs on main Testing was confirmed with https://buildkite.com/organizations/vllm/pipelines/ci/…

  • 3775d5f #47479 — [ROCm][CI] Adding test groups for parity with upstream (#47479)
    • 作者: Andreas Karatzas | +189/-2 | 2 个文件

    Adds missing MI300 AMD CI parity jobs for MRCR evals, vLLM IR kernels, KDA kernels, GGUF plugin coverage, and Model Runner V2. It also bumps the MI355 examples timeout to reduce false failures on the longer examples suite. The KDA path includes a ROCm-specific Triton workaround that avoids num_stages=4 for chunk_delta_h. These additions bring the ROCm pipeline closer to the upstream parity coverag…

🔩 Misc

  • 1d354c6 #46966 — [Misc] Validate Pooling cache_salt Values (#46966)
    • 作者: Taneem Ibrahim | +15/-0 | 1 个文件

    Pooling requests expose cache_salt, but unlike chat completions, completions, and responses, it does not validate that the value is a non-empty string. This adds the same validation to the shared pooling requests so embedding, classification, scoring/rerank, and generic pooling requests all reject invalid cache_salt values consistently before they enter request processing. This existing valid cach…

  • 2f21224 #47333 — [Misc] Update request-extras parity for batch chat completion (#47333)
    • 作者: Taneem Ibrahim | +5/-1 | 2 个文件

    Currently batch chat completions convert each conversation into a normal ChatCompletionRequest, but BatchChatCompletionRequest did not explicitly declare several compatible request extras that normal chat completions already support. This adds explicit batch request fields for: - media_io_kwargs - mm_processor_kwargs - priority - cache_salt ### Main vs Branch Behavior Reproducer: On main: …

  • fa1fa96 #46939 — [Misc] Forward request-level prompt extras for cross-encoder scoring (#46939)
    • 作者: Taneem Ibrahim | +13/-1 | 1 个文件

    Cross-encoder scoring already collected request extras like cache_salt, mm_processor_kwargs, and chat_template_kwargs, but _pre_process only used chat_template_kwargs. The engine-level extras were dropped before process_for_engine, so cross-encoder score/rerank requests did not consistently honor request-level cache salt or multimodal processor overrides. This PR forwards the engine prompt extras,…

  • 6eac8e0 #47082 — [Misc] Preserve cross-encoder pooling extra kwargs (#47082)
    • 作者: Taneem Ibrahim | +4/-1 | 1 个文件

    Cross-encoder scoring adds compressed_token_type_ids to PoolingParams.extra_kwargs when token type IDs are produced during preprocessing. Before this change, that assignment replaced the entire extra_kwargs dict. That could drop existing request/pooling extras such as cache_salt, mm_processor_kwargs, or future scoring extras. This PR preserves existing extra_kwargs and adds only the internal compr…

📦 Other

  • 1a308c4 #43645 — [XPU] Add W8A8 FP8 linear kernel with multi-granularity quant support (#43645)
    • 作者: Chaojun Zhang | +188/-12 | 6 个文件

    This PR adds XPUW8A8FP8LinearKernel and XPUW8A16FP8LinearKernel for XPU, replacing the previous XPUFP8ScaledMMLinearKernel with proper support for multiple weight/activation quantization granularities. ### Changes - Add XPUW8A8FP8LinearKernel and XPUW8A16FP8LinearKernel, replacing XPUFP8ScaledMMLinearKernel - Select W8A8 or W8A16 kernel based on whether activation quantization is present - Ensure …

  • 4c3c17d #47567 — [ROCm] Disable persistent sparse-MLA kernel for chunked-prefill continuations (#47567)
    • 作者: Rohan Potdar | +12/-2 | 1 个文件

    [ROCm] Disable persistent sparse-MLA kernel for chunked-prefill continuations Fixes #47042. GLM-4.6/4.5 / DeepSeek-V3.2-style sparse-MLA (DSA) models on ROCm produce correct output up to ~20K tokens and then collapse into repetition / garbage at longer contexts. Root cause: the AITER persistent MLA work-stealing kernel (get_mla_metadata_v1 + the work_meta_data persistent path in aiter.mla.ml…

  • 07516fd #45953 — [MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953)
    • 作者: Ekagra Ranjan | +436/-36 | 8 个文件

    Followup to DSD: https://github.com/vllm-project/vllm/pull/32374 which ran only with MRv1 and Piecewise graph. This PR mainly enables DSD with MRv2 + FCG. DSD + MRv1 is still on PW. Currently, we capture FCG for decode only for given K say K=3 which meant only multiple of K+1=4 size of graph are captured. However, in DSD the chosen optimal draft k can be <K during scheduling which would lead to mi…

  • 67ff0ae #42890 — Support nvfp4 kv with kv-cache-dtype-skip-layers sliding_window (#42890)
    • 作者: sychen52 | +227/-4 | 6 个文件

    NVFP4 kv does not support –kv-cache-dtype-skip-layers sliding_window before this PR - [reverted] Remove the padding and scaling logic of kv cache in mamba - Update the kv cache scaling and padding logic in unify_kv_cache_spec_page_size. New logic: - Find the “primary” by smallest bytes per token in all AttentionSpec - Set the target page byte size as a multiple of primary page bytes that barely l…

  • d6d39c1 #47155 — [GLM4V] Avoid GLM4V processor init during startup metadata reads (#47155)
    • 作者: labAxiaoming | +67/-8 | 1 个文件

    GLM4V startup budget paths only need processor metadata such as processor_class and size.longest_edge, but they were initializing the full HF processor via get_hf_processor(). This happens independently in APIServer, EngineCore, and worker processes, adding startup latency and host memory usage. Read processor_class and image/video size from config files instead, and build dummy placeholder text f…

  • 576bf75 #47220 — [AMD][EPLB] Enable EPLB for Quark OCP MXFP4 MoE (#47220)
    • 作者: Oxana Korzh | +5/-0 | 1 个文件

    QuarkOCP_MX_MoEMethod inherited supports_eplb=False, so enabling EPLB on Quark MXFP4 MoE models (e.g. Kimi-K2.6-MXFP4) raised NotImplementedError at init. The AITER weight shuffle (layout=(16,16)) only tiles the trailing two dims and keeps the expert dimension outermost, so EPLB’s per-expert row relocation (weights + MX block scales) is layout-safe. Override supports_eplb to True. Accuracy test fo…

  • f63dca6 #47035 — [ROCm] Fix encoder-decoder cross-attention KV layout aliasing (#47035)
    • 作者: djramic | +91/-43 | 3 个文件

    Encoder-decoder models (Whisper, Nemotron-Parse, …) share one raw KV allocation between decoder self-attention (ROCM_ATTN, K/V-first) and cross-attention (AITER/TRITON, blocks-first). The mismatched stride conventions over the same bytes cause aliasing → garbage / repeated-token output. Changes: - Reconcile layouts once at allocation (gpu_model_runner.py): when shared attention groups disagr…

  • 8651f04 #47523 — [Rust Frontend] Speed up chat roundtrip tests (#47523)
    • 作者: Bugen Zhao | +34/-19 | 2 个文件

    Signed-off-by: Bugen Zhao i@bugenzhao.com Rendering-parsing roundtrip tests now take a long time when running all unit tests in the Rust frontend, resulting in more than 20 seconds of wall time. This PR speeds up roundtrip tests by… 1. reducing repeated tokenizer/backend setup via grouping the tests by model 2. compiling tokenizer hot-path dependencies with optimization in dev builds case | re…

  • d7192cf #47539 — [CI Bugfix] Lazily import Qwen warmup dependencies (#47539)
    • 作者: Roberto L. Castro | +21/-13 | 1 个文件

    Qwen warmup imported CUDA-touching FLA/Mamba helpers at module load time. Since gpu_worker -> kernel_warmup -> qwen_triton_warmup, this initialized CUDA before fork and broke engine tests: https://buildkite.com/vllm/ci/builds/75088#019f12dc-d9e0-48ce-a925-f64180ca7359 Fix: move those imports inside the warmup functions so the module is import-safe; CUDA is only touched when warmup actually runs. #…

  • 400a9c3 #47530 — [Rust Frontend] Bump llm-multimodal version (#47530)
    • 作者: Isotr0py | +48/-24 | 6 个文件
  • bbdcbe4 #47452 — Move Roberta remaining nn.Embedding to VocabParallelEmbedding (#47452)
    • 作者: Max de Bayser | +3/-5 | 1 个文件

    Roberta models have 3 embedding modules. 1 already was a VocabParallelEmbedding but the other 2 weren’t. nn.Embedding is not a PluggableLayer so does not allow platform specific customizations. cc: @noooop

  • 1f486d9 #47102 — Add Triton Backend for Unlimited-OCR R-SWA (#47102)
    • 作者: Dakai An | +68/-17 | 5 个文件

    This PR adds explicit TritonAttention support for Unlimited-OCR’s R-SWA decode mask. The Triton path implements the mask with: This keeps prompt/image prefix tokens globally visible, while generated tokens use a causal sliding window. The implementation reuses existing rswa_prefix_lens metadata and applies the mask inside Triton unified attention. It also allows the R-SWA path to use TritonAttenti…

🦀 Rust Frontend

  • ab3b6d9 #47529 — [Frontend] Limit SO_REUSEPORT to multi-worker serving (#47529)
    • 作者: Bugen Zhao | +12/-7 | 3 个文件

    Only enable SO_REUSEPORT when the server socket is created for multiple API servers. In other words, the in-proc Python API server (hope it was not intentional) and the Rust frontend won’t share a socket, avoiding confusion when multiple vLLM instances are launched on the same port. ## Test Result —

  • fb5291b #45877 — [Frontend] [Parser] Port DeepSeek V4 to streaming parser engine framework (#45877)
    • 作者: Ben Browning | +1900/-671 | 22 个文件

    Replace the old DeepSeekV4ToolParser (V32 subclass) and DeepSeekV3ReasoningParser delegation with a unified ParserEngine subclass, matching the pattern used by Gemma4, Qwen3, MiniMax M2, and NemotronV3. The new DeepSeekV4Parser implements the full DSML tool call format as a declarative state machine config with custom arg conversion for the string=“true”/“false” parameter tag semantics and wrapper…

  • a14f57a #47498 — [Frontend] Refine the entrypoint class’s inheritance hierarchy. (#47498)
    • 作者: wang.yuqi | +90/-191 | 23 个文件

    The entrypoint class’s inheritance hierarchy ├─BaseServing <- Used to handle basic functions such as _is_model_supported, via OpenAIServingModels. │ ├─GenerateBaseServing <- Used to handle streaming output, as well as BeamSearch (via BeamSearchOnlineMixin). │ │ ├─OpenAIServingChat │ │ │ └─AnthropicServingMessages │ │ ├─OpenAIServingCompletion │ │ ├─OpenAIServingResponses │ │ └─ServingTokens │ ├─Po…

📖 Documentation

  • 4875b44 #47517 — [Doc] Fix VLM2Vec benchmark chat template path (#47517)
    • 作者: Kalyanam Dewri | +1/-1 | 1 个文件

    Summary: - Fix the VLM2Vec benchmarking example to point at the checked-in examples/pooling/embed/template/vlm2vec_phi3v.jinja template. - Keep the benchmarking docs consistent with the pooling model docs and example script. - Checked open issues and PRs for this path and did not find an existing duplicate. - AI assistance was used to identify and prepare this docs-only change; the final diff and …