共 33 个 commit，涉及 159 个文件，+14411/-1533 行变动。

概要

统计项 数值
Commit 数 33
变更文件 159
新增行数 +14411
删除行数 -1533

Commit 列表

📦 Other

  • 0d80979 #45548 — [Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
    • 作者: Flora Feng | +47/-45 | 4 个文件

    Remove separate self.reasoning_parser_cls and self.tool_parser attributes from OpenAIServingChat, consolidating all parser access through the existing self.parser_cls

  • fa63bb9 #43914 — Remove redundant Triton KV cache dtype asserts and enforce architectural support (fp8 >= sm89) (#43914)
    • 作者: Mike G | +33/-39 | 2 个文件

    On the Triton attention backend (TRITON_ATTN), an fp8 KV cache is stored via current_platform.fp8_dtype() == float8_e4m3fn (Triton fp8e4nv), which has no lowering before SM89; a bfloat16 KV cache needs native bf16 (SM80+). Neither is caught early today: on older GPUs the engine loads and then dies deep in inductor autotuning with type fp8e4nv not supported in this architecture, surfacing o…

  • 5ed15f4 #43557 — Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel (#43557)
    • 作者: Xin He | +253/-17 | 3 个文件

    Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel, which currently produces severely incorrect quantization results. Related PR: #37463 ## Root Cause The original code computes the E8M0 scale factor via: This is mathematically wrong because: floor(log2(a / b)) != floor(log2(a)) - floor(log2(b)) Dividing by 6.0 before extracting the exponent shifts the mantissa boundary, c…

  • b997071 #45510 — (security) Enforce audio upload size limit before full file materialization (#45510)
    • 作者: Juan Pérez de Algaba | +210/-2 | 4 个文件

    The speech-to-text routes (/v1/audio/transcriptions, /v1/audio/translations) called UploadFile.read() without bounds before checking VLLM_MAX_AUDIO_CLIP_FILESIZE_MB, allowing oversized uploads to fully materialize in memory before rejection. This adds a bounded chunked read that rejects over-limit files early, preventing memory exhaustion from malicious uploads.

  • 1d88c4d #45676 — [Docs] Update the online serving docs. (#45676)
    • 作者: wang.yuqi | +83/-37 | 4 个文件

    Update the online serving docs. ## Test Result —

  • 9872921 #44423 — [XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423)
    • 作者: Yejing Lai | +4/-0 | 1 个文件

    XPU matmul/attention kernels are not batch-invariant across chunked-vs-non-chunked prefill and sync-vs-async scheduling shapes, so token outputs can diverge slightly between configs.

  • 40eac9a #44760 — [Rust Frontend] Support parallel_tool_calls = false (#44760)
    • 作者: FAUST | +135/-23 | 8 个文件

    part of https://github.com/vllm-project/vllm/issues/44280 Test | Purpose – | – structured_stream_suppresses_later_tool_calls_when_parallel_disabled | Verifies that when parallel_tool_calls is false, structured_chat_event_stream emits only the first tool call’s events and the final AssistantMessage contains a single tool call, suppressing all subsequent tool calls. prepare_chat_request_maps_paral…

  • b5adb02 #45200 — [Models] Fix MiMo v2.x QKV TP sharding + FP4 support (#45200)
    • 作者: Giancarlo Delfin | +170/-5 | 2 个文件

    Context MiMo v2.5 Pro is currently not supported for TP != 8 because of an issue with the fused QKV projection weights sharding. Additionally, MiMo v2.5 Pro FP4 (used as the base model for the recently released DFlash repo), is not supported because the incorrect quantization config is currently used for MoE (MXFP4 is required for the expert weights). Attempting to serve either model does not re…

  • 64833f8 #45137 — [Rust Frontend] Add external→internal request-id map for abort() (#45137)
    • 作者: Sahil Singh | +447/-27 | 13 个文件

    Adds abort() by external (user-supplied) request id to the Rust frontend’s Llm, TextLlm, and ChatLlm facades, mirroring Python AsyncLLM.abort(). This PR introduces InflightRequests, a refcounted index of external_id → {internal_id → live_guard_count} that Llm maintains across active streams. Changes: - vllm-llm: new inflight.rs module (InflightRequests, RequestGuard); Llm::generate() tracks th…

  • 7df4fe1 #45638 — [Model] Remove XverseForCausalLM (#45638)
    • 作者: Tiezhen WANG | +1/-15 | 4 个文件

    XverseForCausalLM has been a registry alias to LlamaForCausalLM since PR #10814 (Dec 2024) deleted the standalone xverse.py implementation. Per vllm-release telemetry over the last 6 months: 38 instances / 0.028 GPU-hours total. The arch is effectively unused. Adds XverseForCausalLM to _PREVIOUSLY_SUPPORTED_MODELS so users get a clear error pointing at vLLM v0.23. Users with XVERSE checkpoints can…

  • e8d3e22 #45629 — Fix included router missing path for FastAPI >=0.137 (#45629)
    • 作者: Roger Wang | +38/-1 | 1 个文件

    FastAPI just released 0.137 which stores lazy _IncludedRouter objects (BaseRoute subclasses with no .path) in app.routes. prometheus-fastapi-instrumentator (<=8.0.0) reads route.path unconditionally in _get_route_name, so every request raises AttributeError in the metrics middleware and the server returns 500 – e.g. /health never becomes ready and the engine looks hung at startup. Patch the instr…

  • 8760f97 #45391 — [CPU] Refine CPU attention frontend (#45391)
    • 作者: Li, Jiang | +384/-217 | 4 个文件
    • Remove SDPA - Move ISA check to the init - Add head_dim 48 CI tests ## Test Result —
  • 2725c84 #38608 — [XPU] Enable sequence parallel support for XPU (#38608)
    • 作者: Chaojun Zhang | +128/-19 | 5 个文件

    Test Result UT : pytest -s -v tests/compile/correctness_e2e/test_sequence_parallel.py pytest -s -v pytest -s -v tests/compile/correctness_e2e/test_sequence_parallel.py ### Accuracy (GSM8K) #### meta-llama/Llama-2-13b-chat-hf Case | Strict | Flexible – | –: | –: Enable SP | 0.360 | 0.364 Disable SP | 0.372 | 0.376 Eager | 0.368 | 0.372 #### Qwen/Qwen3-32B Configuration | Strict | Flexible – …

  • 2c764c0 #45173 — Added real /v1/embeddings support for messages + chat_template_kw (#45173)
    • 作者: Taneem Ibrahim | +400/-10 | 5 个文件

    The motivation is to support embedding models whose input needs to be rendered through a chat template and controlled with model-specific chat_template_kwargs, such as instruction-style embedding prompts. Before this PR, vLLM had an embedding chat path for the top-level messages extension, but /v1/embeddings requests with chat-message-shaped input were rejected during request validation. That mean…

  • 9548a18 #45136 — [XPU] Support int4 group_size=32 W4A16 MoE (#45136)
    • 作者: Marceli Fylcek | +5/-1 | 1 个文件

    The XPU WNA16 fused-MoE expert backend (XPUExpertsWNA16) accepted only the catch-all int4 weight scale key kInt4Static (GroupShape(1, -1)). In the compressed-tensors WNA16 mapping, this catch-all key is assigned to every int4 symmetric scheme except group_size=32 — i.e. per-channel as well as group_size 64/128/256 all collapse into kInt4Static. Only group_size=32 is carved out into its own distinc…

  • 4ef4492 #32374 — [V1][Spec Decode] Add Dynamic SD (#32374)
    • 作者: Ekagra Ranjan | +586/-7 | 23 个文件

    Why is Dynamic SD needed? SD methods need to verify K tokens for each sequence during decoding. As BS increases, the effective BS becomes BS * K which increases the compute requirement during verification. When this BS*K goes beyond a critical BS then SD negatively impacts the TPOT. DSD helps by tuning down the K to an optimal value such that we continue to reap the benefits from SD. ## Use cas…

🐛 Bug Fix

  • 588db18 #44409 — [Bugfix] Two-phase KV allocation for cross-group prefix cache hits (supersedes #33775) (#44409)
    • 作者: Saddss | +255/-34 | 4 个文件

    Hybrid models (e.g. Gemma-4) with local prefix hits + external KV could assign the same physical block twice: per-group touch → extend → get_new_blocks(external) let group i’s external alloc evict group j’s untouched hit blocks → bad ref_cnt / duplicate block IDs → downstream #43884 assert in OffloadingConnectorScheduler. Fix: coordinator runs all groups’ local (add_local_compu…

  • 6c5872e #45417 — [Bugfix] Unset HF’s default max_new_tokens for DiffusionGemma (#45417)
    • 作者: Martin Kukla | +14/-0 | 1 个文件

    DiffusionGemma model’s generation_config.json sets default max_new_tokens to 256. The rationale being that the HuggingFace users should never hit OOMs when running the model. For vLLM implementation, we don’t need to limit the max length by default in order to limit OOMs. Run the model without specifying max len, and generate more than one canvas. ## Test Result Generates more than one canvas — …

  • c17e2f7 #45465 — [Bugfix][Rust Frontend] Make metrics respect –served-model-name (#45465)
    • 作者: Reid | +118/-9 | 2 个文件

    The Rust frontend currently accepts –served-model-name, but frontend Prometheus metrics can still be labeled with the backend model path instead of the public served model name. For example, serving Qwen/Qwen3-0.6B with: This change resolves served model names once in server state setup and uses the primary served name for frontend-side metrics labels. If no served model name is configured, it pr…

  • ddad5db #45557 — [Bugfix][Rust] Sync EngineCoreReadyResponse with the Python dataclass (#45557)
    • 作者: Will Eaton | +54/-1 | 4 个文件

    The Rust EngineCoreReadyResponse had fallen three fields behind the Python dataclass: block_size (required) plus the optional kv_cache_size_tokens and kv_cache_max_concurrency. Registration is a field-keyed map, so a Rust engine encoding the response omits block_size and the Python frontend rejects it on decode. Adds the three fields and a python_compat.py fixture asserting the Rust and Python fie…

  • ebb0a71 #44965 — [Bugfix] Reject out-of-range temperature values in SamplingParams (#44965)
    • 作者: Peter Pan | +6/-0 | 1 个文件

    The Python frontend of SamplingParams._verify_args only checked the lower bound of temperature (>= 0), so values like temperature=3 silently flowed through to the engine and produced a normal response. The Rust frontend already enforces the OpenAI-spec range [0, 2] via #[validate(range(min = 0.0, max = 2.0))] on ChatCompletionRequest.temperature. This PR tightens the Python check to 0.0 <= tempera…

  • b8336c3 #45564 — [Bugfix][V1] Split V2 model-runner attention groups on num_heads_q (#45564)
    • 作者: Roger Wang | +7/-3 | 1 个文件

    The V2 model runner’s init_attn_backend (vllm/v1/worker/gpu/attn_utils.py) keys attention metadata-builder groups on (backend, kv_cache_spec) only. The V1 runner (gpu_model_runner.initialize_attn_backend) additionally keys on per-rank num_heads_q, with a comment explaining why: layers with different Q-head counts (e.g. a spec-decode draft head vs its target) must get separate metadata builders…

  • e3e3cd5 #45602 — [Bugfix][CI] Update Dockerfile dependency graph PNG (#45602)
    • 作者: Flora Feng | +0/-0 | 1 个文件
    • Update docs/assets/contributing/dockerfile-stages-dependency.png to match the current docker/Dockerfile. - PR #44795 added two new COPY –from=build lines for wheel checksum cache-busting, which introduced new edges in the Dockerfile stage dependency graph. The PNG was not regenerated in that PR, causing the update-dockerfile-graph pre-commit hook to fail for anyone whose changes touch docker/Do…
  • b675cb7 #45086 — [Bugfix][CPU] Honor cgroup memory limit when computing KV cache size (#45086)
    • 作者: maobaolong | +52/-0 | 1 个文件

    When vLLM runs inside a container (Docker / Kubernetes pod) with a memory cgroup limit, the CPU worker still reads NUMA MemTotal from /sys/devices/system/node/node*/meminfo, which reports host-wide numbers and ignores the cgroup limit. As a result, gpu_memory_utilization is applied to the host’s total RAM rather than the container’s quota, which leads to KV-cache allocation failures such as: V…

  • 1801fad #44645 — [Bugfix] Stream Llama4 weight loading to avoid host-OOM with copy-returning loaders (#44645)
    • 作者: Noa Neria | +64/-72 | 2 个文件

    [Bugfix] Stream Llama4 weight loading to avoid host-OOM with copy-returning loaders Loading Llama-4-Scout/Maverick (Llama4ForConditionalGeneration) with –load-format runai_streamer OOMs the host during weight loading: each TP worker transiently holds ~the entire language-model checkpoint in host RAM, so on a multi-GPU node the workers collectively exceed available memory and get OOM-kil…

  • 3d6ce81 #45291 — [Bugfix][Model] Validate runai_streamer model_loader_extra_config (#45291)
    • 作者: Ting SUN | +75/-5 | 2 个文件

    RunaiModelStreamerLoader reads model_loader_extra_config with isinstance guards that silently drop anything unexpected: So invalid configs are accepted silently instead of being reported: - unknown keys (e.g. {“typo_key”: 1}) are ignored; - a wrong type (concurrency=“16”) is dropped, so the requested concurrency is never applied; - a negative value (concurrency=-1) sets RUNAI_STREAMER_CONCURRENCY=…

  • c621af1 #45383 — [BugFix] Fix prompt_embeds for multimodal models (#45383)
    • 作者: Michael Ma | +53/-7 | 2 个文件

    Fix prompt_embeds for multimodal models (e.g. Gemma3ForConditionalGeneration). Related to #44842. With LLM(…, enable_prompt_embeds=True).generate({“prompt_embeds”: …}), a multimodal model currently produces crashes the EngineCore with a CUDA index-out-of-bounds assertion (vectorized_gather_kernel: index out of bounds → EngineDeadError). Text-only models are unaffected. There are two indepe…

  • 9fd737b #45487 — [Bugfix][DCP] Fix illegal memory access in DCP a2a decode under full CUDA graphs (#45487)
    • 作者: Jeff (Junze) Ma | +10/-11 | 1 个文件

    –dcp-comm-backend a2a crashes every worker with a CUDA illegal memory access on the first request when full CUDA graphs are enabled (cudagraph_mode=FULL_AND_PIECEWISE) for DCP4 when running model nvidia/Kimi-K2.5-NVFP4. The DCP a2a combine drew its all-to-all send/recv buffers from the shared, growable WorkspaceManager. A full CUDA graph bakes in those buffer addresses at capture; vLLM’s post-cap…

📖 Documentation

  • 25c53d1 #45671 — [ROCm][Doc] Add installation notes about python version requirement (#45671)
    • 作者: vllmellm | +13/-0 | 1 个文件

    Added a note to the ROCm pre-built wheel installation section about python version requirement. Addressing #44660. ## Test Result —

✨ New Feature

  • 48df95c #45458 — [Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
    • 作者: Ting SUN | +93/-14 | 3 个文件

    usage.prompt_tokens already includes the image / audio / video placeholder tokens, but clients cannot tell how many tokens each modality contributed. This adds image_tokens, audio_tokens and video_tokens to usage.prompt_tokens_details, following sgl-project/sglang#27122. audio_tokens mirrors the OpenAI field of the same name; image_tokens and video_tokens are multimodal extensions beyond the OpenA…

🦀 Rust Frontend

  • c4a3f9d #45413 — [Frontend] Add Streaming Parser Engine and new Qwen3 Parser (#45413)
    • 作者: Ben Browning | +8492/-902 | 33 个文件

    Fixes https://github.com/vllm-project/vllm/issues/44873 Introduce a declarative, config-driven streaming parser engine that replaces hand-rolled per-model streaming logic with a shared state machine. The engine handles reasoning/content/tool-call transitions, incremental JSON argument extraction, and detokenizer hold-back recovery through a single ParserEngineConfig. This wires up Qwen3 as the fir…

⚡ Performance

  • e2bf2b3 #45566 — [Perf] Use bisect for mm feature lookup in model runner v2 (#45566)
    • 作者: Roger Wang | +8/-10 | 1 个文件

    Port changes in #44212 to the model runner v2 encoder runner. ## Test Result —

  • 725c3bc #44400 — [ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400)
    • 作者: Amanzhol Salykov | +2173/-0 | 15 个文件

    This PR replaces w4a16 MoE GPTQ Triton kernel with custom FlyDSL w4a16 MoE kernel if 1. weights are int 4-bit quantized 2. group_size==32 3. arch==gfx950 (currently limited to gfx950, as the MoE was tuned only for gfx950) 4. –moe-backend=flydsl 5. lora is disabled targeting Kimi K2.5 INT4 model. The kernel differs from aiter.fused_moe implementation, as we pre-shuffle/flatten weights+scales durin…