共 17 个 commit，涉及 80 个文件，+3543/-358 行变动。
概要
| 统计项 | 数值 |
|---|---|
| Commit 数 | 17 |
| 变更文件 | 80 |
| 新增行数 | +3543 |
| 删除行数 | -358 |
Commit 列表
🐛 Bug Fix
- c621af1 #45383 — [BugFix] Fix prompt_embeds for multimodal models (#45383)
- 作者: Michael Ma | +53/-7 | 2 个文件
Fix prompt_embeds for multimodal models (e.g. Gemma3ForConditionalGeneration). Related to #44842. With LLM(…, enable_prompt_embeds=True).generate({“prompt_embeds”: …}), a multimodal model currently produces crashes the EngineCore with a CUDA index-out-of-bounds assertion (vectorized_gather_kernel: index out of bounds → EngineDeadError). Text-only models are unaffected. There are two indepe…
- 9fd737b #45487 — [Bugfix][DCP] Fix illegal memory access in DCP a2a decode under full CUDA graphs (#45487)
- 作者: Jeff (Junze) Ma | +10/-11 | 1 个文件
–dcp-comm-backend a2a crashes every worker with a CUDA illegal memory access on the first request when full CUDA graphs are enabled (cudagraph_mode=FULL_AND_PIECEWISE) for DCP4 when running model nvidia/Kimi-K2.5-NVFP4. The DCP a2a combine drew its all-to-all send/recv buffers from the shared, growable WorkspaceManager. A full CUDA graph bakes in those buffer addresses at capture; vLLM’s post-cap…
- 54bbf51 #44795 — [Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28 (#44795)
- 作者: achyuthan.s | +67/-1 | 2 个文件
Fixes #44759 The nightly images have been broken since May 28 with this at startup: ImportError: cannot import name ‘AnthropicOutputConfig’ from ‘vllm.entrypoints.anthropic.protocol’ The source on main is fine — both protocol.py and serving.py were updated together in #42396. The issue is in the Dockerfile. vllm-base was installing the wheel via –mount=type=bind,from=build, and bind mounts aren’t…
- 521b88c #45468 — [Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468)
- 作者: WEI CHENG CHIU | +66/-1 | 2 个文件
Fixes #45436. nvidia/diffusiongemma-26B-A4B-it-NVFP4 (DiffusionGemma, added in #45163) fails with an HTTP 500 when a structured output constraint is supplied (response_format: json_schema or top-level structured_outputs): the request starts generating and is then terminated mid-flight by an xgrammar FSM rejection deep in the scheduler. This PR rejects structured output requests against diffusion l…
⚡ Performance
- e2bf2b3 #45566 — [Perf] Use bisect for mm feature lookup in model runner v2 (#45566)
- 作者: Roger Wang | +8/-10 | 1 个文件
Port changes in #44212 to the model runner v2 encoder runner. ## Test Result —
- 725c3bc #44400 — [ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400)
- 作者: Amanzhol Salykov | +2173/-0 | 15 个文件
This PR replaces w4a16 MoE GPTQ Triton kernel with custom FlyDSL w4a16 MoE kernel if 1. weights are int 4-bit quantized 2. group_size==32 3. arch==gfx950 (currently limited to gfx950, as the MoE was tuned only for gfx950) 4. –moe-backend=flydsl 5. lora is disabled targeting Kimi K2.5 INT4 model. The kernel differs from aiter.fused_moe implementation, as we pre-shuffle/flatten weights+scales durin…
- 71b961d #44572 — [Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement (#44572)
- 作者: Wentao Ye | +87/-159 | 3 个文件
A follow up for https://github.com/vllm-project/vllm/pull/43706, we now directly implement in kernel level. The swap_ab logic is from SM100/SM120 currently Acc covered in unit tests Perf: Using this script we can get
📦 Other
- 9548a18 #45136 — [XPU] Support int4 group_size=32 W4A16 MoE (#45136)
- 作者: Marceli Fylcek | +5/-1 | 1 个文件
The XPU WNA16 fused-MoE expert backend (XPUExpertsWNA16) accepted only the catch-all int4 weight scale key kInt4Static (GroupShape(1, -1)). In the compressed-tensors WNA16 mapping, this catch-all key is assigned to every int4 symmetric scheme except group_size=32 — i.e. per-channel as well as group_size 64/128/256 all collapse into kInt4Static. Only group_size=32 is carved out into its own distinc…
- 4ef4492 #32374 — [V1][Spec Decode] Add Dynamic SD (#32374)
- 作者: Ekagra Ranjan | +586/-7 | 23 个文件
Why is Dynamic SD needed? SD methods need to verify K tokens for each sequence during decoding. As BS increases, the effective BS becomes BS * K which increases the compute requirement during verification. When this BS*K goes beyond a critical BS then SD negatively impacts the TPOT. DSD helps by tuning down the K to an optimal value such that we continue to reap the benefits from SD. ## Use cas…
- 78e7293 #45277 — [Build] Fix CUDA arch build coverage gaps (#45277)
- 作者: Shengqi Chen | +239/-131 | 14 个文件
This PR fixes several CUDA architecture build/dispatch gaps found while auditing the CUDA arch matrix in #45260. The main theme is making top-level arch requests, per-component CMake arch filters, and runtime support checks agree with each other. Before this change, some paths either carried misleading PTX settings that were filtered out, built for arch families that the runtime never used, or adv…
- cf027b8 #45442 — [Core] Simplify MRV2 async output handling (#45442)
- 作者: Nick Hill | +8/-14 | 3 个文件
MRV2 currently resolves ModelRunnerOutput from AsyncModelRunnerOutput depending on whether async scheduling is enabled. However the executors already handle the output type appropriately depending on what is required from the upstream caller, so it’s simpler for the model runner to not have to care about this.
- b3f0a0a #45536 — Fix docs build on
main(#45536)- 作者: Harry Mellor | +1/-2 | 2 个文件
We will have to do this periodically now that we’ve stopped building docs on every PR
- 470229c #45252 — [Security] Fix DoS via prompt_embeds on M-RoPE models (#45252)
- 作者: Juan Pérez de Algaba | +94/-4 | 2 个文件
Replace the fatal assertion in _init_mrope_positions that crashes the EngineCore when prompt_token_ids is None. When only prompt_embeds are provided (e.g. /v1/completions with prompt=None + prompt_embeds), derive the sequence length from the embeddings tensor and use dummy token IDs. This is safe because get_mrope_input_positions only needs len(input_tokens) when there are no multimodal features (…
- 2b30060 #45118 — [Security] Add timeout guard for regex compilation in structured outp… (#45118)
- 作者: Juan Pérez de Algaba | +126/-4 | 5 个文件
- Add compile_regex_with_timeout() utility that wraps regex compilation in a thread with a deadline, preventing ReDoS attacks where adversarial patterns cause exponential DFA state-space explosion. - Apply the timeout to compiler.compile_regex() in the xgrammar backend, oc.Index() in the outlines backend, and xgr.Grammar.from_regex() during validation. - Introduce VLLM_REGEX_COMPILATION_TIMEOUT_S …
- 96fa5cd #45478 — [CI Bug] Fix
ValueError: There is no module or parameter named 'model.vision_tower.vision_model'(#45478)- 作者: Wentao Ye | +10/-6 | 2 个文件
Fixes https://buildkite.com/vllm/ci/builds/71235#019ebd0b-5a0c-44e8-9f0c-7a7965833cc8 We reuse the renaming logic from huggingface. Now
- 9261dbb #45491 — Treat null completion max_tokens like the default (#45491)
- 作者: Andreas Karatzas | +8/-0 | 1 个文件
This fixes a Schemathesis failure in the OpenAI API server job: Buildkite: The generated request was: For that exact request, the pre-fix failure is deterministic. The CI job only looks flaky because this is a schema-fuzzing test: Schemathesis does not generate that exact max_tokens: null completion body on every run. The serving code later asserts that CompletionRequest.max_tokens is not None. Th…
- 2ecf7d0 #45467 — [Model Runner V2] Fix
openai.InternalServerError: Error code: 500 - 'list index out of range'(#45467)- 作者: Wentao Ye | +2/-0 | 1 个文件
VLLM_USE_V2_MODEL_RUNNER=1 pytest tests/entrypoints/openai/chat_completion/test_chat_echo.py::test_top_logprobs -xvs Will raise error This is becuase for -1 for logprobs meaning all instead of None, this PR fixed the bug Now