共 45 个 commit，涉及 182 个文件，+5270/-1257 行变动。

概要

统计项 数值
Commit 数 45
变更文件 182
新增行数 +5270
删除行数 -1257

Commit 列表

🐛 Bug Fix

  • d02df74 #48973 — [Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
    • 作者: Thomas Fahrner | +47/-5 | 2 个文件

    Data URLs with RFC 2397 parameters between the mediatype and the base64 marker (e.g. data:image/jpeg;charset=utf-8;base64,…) are rejected with an opaque HTTP 501 (“Only base64 data URLs are supported for now”), because MediaConnector._load_data_url splits the header on the first ; and requires the remainder to equal base64. RFC 2397 explicitly allows such parameters, and clients that emit them…

  • a454a1d #49180 — [Bugfix][Benchmarks] Restore –skip-tokenizer-init with custom dataset (#49180)
    • 作者: mgazz | +185/-9 | 3 个文件

    benchmarks with custom datasets cannot use –skip-tokenizer-init. PR #39896 ([MyPy] Fix mypy for vllm/benchmarks) added a blanket assert tokenizer is not None before get_samples() in serve.py to satisfy the TokenizerLike type annotation. This does not allow the execution of models that do not like use a tokenizer like Prithvi. Error: Can be reproduced with the following command:

  • 163ecba #49586 — [Bugfix] Skip linear bias in layerwise reload to avoid corruption (#49586)
    • 作者: li-jinpeng | +3/-0 | 1 个文件

    Linear layers that carry a bias are corrupted when loaded through the online quantization + layerwise-reload path (e.g. online FP8). After loading, bias holds uninitialized memory instead of the checkpoint values. Root cause. The linear base class creates bias after create_weights(). Online quantization calls initialize_online_processing() from inside create_weights(), i.e. before bias exist…

  • 589a5b8 #49221 — [PD][NixlPush][Bugfix] Fix blocking handshake call on writer thread (#49221)
    • 作者: Nicolò Lucchesi | +196/-80 | 4 个文件

    Address S2 here https://github.com/vllm-project/vllm/issues/48633. P->D Handshakes on Push mode are initiated by the writer thread and they are currently blocking (that’s not the case for D->P ones which are correctly async with _send_registration_to_p). This is an issue under load when one or more new D instances are being spun up (eg as a result of auto-scaling), leading to noticeable perfor…

  • 7bdf8cc #48769 — [Bugfix] Fix humming kernel crash when layer.has_bias is None (#48769)
    • 作者: Kyle Sayers | +1/-1 | 1 个文件

    Purpose ## * Fix HYV3 model with CT inference ## Testing ## * Validated that RedHatAI/Hy3-NVFP4-FP8 runs ## RCA - LinearBase.init sets self.has_bias = bias, and some custom model layers (e.g. Hy3-NVFP4-FP8) pass bias=None. This None propagates as has_bias=None into HummingLayerMeta. - BaseHummingConfig.to_cpp_str() skips any field whose value fails isinstance(value, (bool, int, Enum)) — Non…

  • 275556c #49623 — [Bugfix] Detect mixed precision in packed KV cache specs (#49623)
    • 作者: Michael Goin | +40/-5 | 2 个文件

    Fix KV-cache zeroer initialization when mixed-precision attention specs are packed in UniformTypeKVCacheSpecs. The worker-side KVCacheConfig.has_mixed_precision_kv_cache check only inspected top-level AttentionSpec objects. Packed specs were skipped, while generate_scheduler_kv_cache_config unpacked them. This let the scheduler mark blocks for zeroing without the worker creating a zeroer, causing …

  • 0231dd5 #49385 — [BugFix][LoRA] Skip marlin-backend gpt-oss LoRA tests on XPU (#49385)
    • 作者: Chaojun Zhang | +4/-2 | 1 个文件

    Purpose ## Fix XPU crashes in test_gpt_oss_lora_tp2 and test_gpt_oss_lora when mxfp4_use_marlin=True due to an unsupported MXFP4 Marlin backend. ## Root cause ## The tests explicitly request moe_backend=“marlin” and linear_backend=“marlin”, but Marlin MXFP4 kernels are CUDA‑only and not registered for XPU. Explicit requests fail instead of falling back, unlike the auto path which correctly skip…

  • e18f003 #48776 — [Bugfix][KV cache] Support sparse-MLA targets with SWA drafts (#48776)
    • 作者: Michael Goin | +181/-61 | 2 个文件

    Serving a sparse-MLA target such as nvidia/GLM-5.2-NVFP4 with a regular sliding-window DSpark draft could fail during KV-cache planning because the target MLA/indexer pages cannot be unified with the draft page. ## Design Keep the existing grouping path unchanged when page sizes can be unified. If page-size unification fails for the narrow MLA + regular-SWA case, promote only the draft’s cache-all…

  • b91a40e #49626 — [Bugfix] Restore structured output logger initialization (#49626)
    • 作者: Chang Guo | +4/-0 | 1 个文件

    Restore the module-level logger in vllm/v1/structured_output/init.py, fixing a merge regression that breaks pre-commit for the whole repository. On main @ 46f01a50a, StructuredOutputManager._create_grammar calls logger.exception(…) (line 185), but the module has no logger binding — both from vllm.logger import init_logger and logger = init_logger(name) are absent. This has two consequenc…

  • 46f01a5 #49609 — [CI][Bugfix] Fix test isolation in block_int8/ptpc_fp8 MoE kernel tests (#49609)
    • 作者: Nick Hill | +4/-4 | 2 个文件

    test_block_int8.py and test_triton_moe_ptpc_fp8.py set the default device in an autouse fixture scoped to “module”, so it runs only once. When an earlier suite module (test_batched_moe) leaves torch’s default device dirty, the fixture never re-runs, so only the first test gets a CUDA default and the rest allocate on CPU and fail with: NotImplementedError: Could not run ‘_moe_C::topk_softmax’ with …

  • 0416dab #44993 — [Bugfix][Structured Output][Spec Decode] Advance grammar across reasoning boundary (#44993)
    • 作者: yue.yu | +146/-54 | 3 个文件

    Purpose Fixes #43388. Fixes #48228. Fixes #34650. Under async scheduling + speculative decoding + a Qwen-style reasoning parser, should_advance in vllm/v1/structured_output/init.py silently fails for JSON/regex/choice constraints. Two distinct bugs interact: Bug 1: the delta window misses the marker. The delta window computed from num_computed_tokens - num_output_placeholders skips </t…

  • 638d6e9 #44239 — [Bugfix][CI/Build] Fix Plamo2 HF runner crash on transformers v5 (_tied_weights_keys list→dict) (#44239)
    • 作者: Nikhil Kulkarni | +30/-7 | 2 个文件

    Fixes the Plamo2ForCausalLM item in #38379. ## Root cause pfnet/plamo-2-1b remote code defines _tied_weights_keys = [“lm_head.weight”] — the transformers v4 list[str] format. Transformers v5 changed this attribute to dict[str, str]. With tie_word_embeddings=True in the model config, get_expanded_tied_weights_keys() crashes during HfRunner.from_pretrained(): ## Fix Adds _fix_v4_tied_weights_keys(mo…

  • 1ad84fe #49391 — [Bugfix][Spec Decode] Select earliest-completing stop string in check_stop_strings (#49391)
    • 作者: Junpu Yu | +105/-11 | 2 个文件

    Fix a streaming↔non-streaming correctness divergence in stop-string handling that surfaces under speculative decoding. check_stop_strings returns the first stop string in list order that is found in the newly generated text, rather than the stop string that completes earliest in the text. Without speculative decoding each engine step appends ~one token, so at most one stop string is in the…

  • 12213c6 #47312 — [Bugfix] handle grammar compilation failures to avoid engine crash (#47312)
    • 作者: zhrrr | +123/-41 | 7 个文件

    This PR resolves the long-standing TODO in StructuredOutputManager._create_grammar: > # TODO: we still need to handle xgrammar compilation failures, though it should be unlikely as we test that up front as well. A structured-output request whose grammar fails to compile currently crashes the whole EngineCore instead of failing just that request. _create_grammar re-raises, and because it runs i…

  • 10c7547 #45224 — [Bugfix][Core] shm_broadcast: bound idle reader waits and release read slots (#45224)
    • 作者: chaeminlim-mb | +155/-23 | 2 个文件

    MessageQueue local readers can park indefinitely on the best-effort ZMQ notify path even after the writer has written to shared memory. This PR: - caps idle local-reader waits at SHM_READER_RECHECK_INTERVAL_MS (5s), so a reader always wakes to re-read the authoritative SHM written-flag even when a notify ping is dropped (PUB SNDHWM=1 drops silently, SUB is CONFLATE). Warning cadence via VLLM_RINGB…

📦 Other

  • 453f017 #49124 — [UX] Improve data-parallel launch validation (#49124)
    • 作者: Taneem Ibrahim | +52/-23 | 1 个文件

    [UX] Raise actionable errors for invalid data-parallel launch options Replacing data-parallel launch combination assertions with plain ValueError at their existing validation stages, using a consistent prefix, the relevant CLI flags and values, and a corrective action. It also validates the lower bound of –node-rank before constructing ParallelConfig. This does not duplicate #45150: that PR det…

  • 7b40fb9 #49247 — [UX] Reject incompatible nested runtime overrides (#49247)
    • 作者: Taneem Ibrahim | +56/-23 | 4 个文件

    Runtime configuration overrides validated nested values only by checking whether they were any dataclass, so an unrelated dataclass could silently replace the expected nested config type. Other invalid overrides used assertions that disappeared under python -O, producing low-level attribute errors instead. This change accepts only mappings or instances of the expected nested dataclass, reports the…

  • 8eac21a #49673 — [ROCM] Fix AITER Fused AllReduce RMSNorm for Transformers Backend (#49673)
    • 作者: BadrBasowid | +18/-4 | 2 个文件

    Fix AITER fused AllReduce+RMSNorm dispatch for 3D Transformers inputs by calculating tokens across all leading dimensions. The previous logic incorrectly selected the 1-stage kernel, causing: Also adds Transformers backend coverage to the TP2 AR+RMS fusion test. - lm_eval Qwen/Qwen3-32B-FP8 with transformers backend and TP 2 - pytest tests/compile/fusions_e2e/test_tp2_ar_rms.py ## Test Result - lm…

  • 833483f #48218 — Encoder cache extension hooks (#48218)
    • 作者: hotTea | +106/-15 | 8 个文件

    This PR adds extension hooks for the V1 encoder cache lifecycle so downstream or out-of-tree implementations can customize encoder cache behavior without changing the default vLLM path. Concretely, this PR: - Adds EncoderCacheManagerConfig and EncoderCacheManagerMetadata. - Allows VllmConfig to provide a custom scheduler-side encoder cache manager class via ec_manager_config.encoder_cache_manager_…

  • 5c5434e #49693 — Remove Quantization test parallelism (#49693)
    • 作者: Kevin H. Luu | +2/-4 | 1 个文件
    • Remove parallelism: 8 from the H200 Quantization test. - Remove parallelism: 3 from the H200 Quantized Models Test. - Remove the shard arguments that depend on BUILDKITE_PARALLEL_JOB and BUILDKITE_PARALLEL_JOB_COUNT. This reduces fan-out pressure on the h200_35gb queue. The tests still run in full, but each step runs as one job instead of sharded parallel jobs. ## Why this is not duplicate work …
  • 0d77325 #49223 — Bump Transformers version to 5.14.1 (#49223)
    • 作者: Harry Mellor | +8/-8 | 8 个文件

    No fixes needed for this one, it just worked!

  • bf27e34 #41276 — [CompressedTensors] DeepSeek4 CT Quantization Support (#41276)
    • 作者: Kyle Sayers | +4/-1 | 1 个文件

    DeepSeek-V4-Flash-NVFP4-FP8 ## Model Optimizations This model was obtained by using the following branch with LLM Compressor: https://github.com/vllm-project/llm-compressor/pull/2647 ## Deployment ## Accuracy Evaluation | Benchmark | deepseek-ai/DeepSeek-V4-Pro-Base | deepseek-ai/DeepSeek-V4-Pro | RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8 | | - | - | -| - | | GPQA | | 90.1 | 0.93 (330/792 samples) | | GS…

  • d65acd8 #49258 — [Model] Support llm-compressor Inkling NVFP4 weights (#49258)
    • 作者: Michael Goin | +83/-10 | 3 个文件
    • Normalize nested llm-compressor Inkling expert names to the existing fused-MoE parameter names while retaining flattened-name support. - Load compressed-tensors per-expert global scales into the existing w13/w2 layouts. This is not a duplicate of #48876: it is a minimal alternative built on the landed general compressed-tensors infrastructure and omits synthetic tiny-checkpoint config and routin…
  • 80c9d5d #48050 — [ROCm][Quantization] Add Quark W4A8 (INT4-FP8) MoE CI coverage (#48050)
    • 作者: amd-sourjya | +37/-1 | 1 个文件

    Add vLLM CI coverage for the Quark W4A8 (INT4 weight + FP8 activation) fused-MoE path (QuarkW4A8Fp8MoEMethod), which dispatches through the ROCm AITER fused MoE kernel on gfx942/gfx950. Complements the earlier W4A8 method refactor (#39136). - New tests/quantization/test_quark.py::test_quark_w4a8_fp8_moe: loads a tiny Qwen3-MoE reference model (amd/tiny-qwen3-moe-w4a8), asserts routed experts use Q…

  • da54a5b #49654 — [Docs] Fix broken anchor links in serving/pooling/MoE docs (#49654)
    • 作者: Euisuh Jeong | +7/-7 | 4 个文件

    Four in-page anchors in the docs point at fragments that do not exist on the rendered site, so the links land at the top of the target page instead of the intended section. | Link | Problem | Fix | | — | — | — | | scoring.md#rerank-api (4x) | heading is ### Cohere Rerank API | #cohere-rerank-api | | moe_kernel_features.md#fused-moe-experts-kernels | heading is ## Fused Experts Kernels | #fus…

  • b354734 #49603 — [Bug] Fix batch invariance rms norm comparison (#49603)
    • 作者: Wentao Ye | +35/-45 | 1 个文件

    Currently we compare batch invariance with itself, this PR fixes the issue by comparing with a PyTorch native implementation. Covered in CI

  • 75ccdf3 #48155 — [Core] Update PyTorch to 2.13.0, torchvision to 0.28.0, triton to 3.7.1 (#48155)
    • 作者: Andrey Talman | +69/-37 | 15 个文件

    Update the PyTorch ecosystem to the released 2.13.0 on the production channel (download.pytorch.org/whl/…). Everything is released, so there are no test-channel index URLs — pure production bump, same approach as the 2.12.1 production PR #45082. - torch: 2.11.0 → 2.13.0 - torchvision: 0.26.0 → 0.28.0 - triton: 3.6.0 → 3.7.1 (matches torch 2.13.0 Requires-Dist: triton==3.7…

  • b0cb1da #49486 — [DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case (#49486)
    • 作者: Wentao Ye | +43/-0 | 1 个文件

    Skip topk and router when not needed Originally: Now Part of https://github.com/vllm-project/vllm/issues/45861 vllm serve deepseek-ai/DeepSeek-V4-Flash –tensor-parallel-size 4 –enable-expert-parallel –attention-backend FLASHMLA_SPARSE_DSV4 –attention-config ‘{“use_fp4_indexer_cache”:true}’ –kv-cache-dtype fp8 –tokenizer-mode deepseek_v4 –all2all-backend allgather_reducescatter –port 8003 #…

  • 494845e #49364 — Revert “[MRV2] Always build attn metadata at capture time” (#49364) (#49451)
    • 作者: vllm-agent | +21/-35 | 2 个文件
  • c8db00b #48816 — Fix GPTQ quantized Qwen3.5 MTP weight loading with spec decode (#48816)
    • 作者: vllmellm | +12/-2 | 1 个文件

    when loading Qwen/Qwen3.5-397B-A17B-GPTQ-Int4 with speculative decoding, weight loading fails with: The issue is that the MTP weight is not quantized, but the weight loading expects every layer to be quantized. This fix detects skip applying the quantized config to the MTP layers. ## Test Result |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr| |—–|——:|—————-|—–:|—–…

  • ac36a7a #48630 — [MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630)
    • 作者: Michael Goin | +390/-67 | 12 个文件

    Replacement for https://github.com/vllm-project/vllm/pull/48037 where we cap the intermediate memory required for large batch rejection sampling by using a fixed size scratch buffer and simply go through the sampling process multiple times. The loop is sync-free since chunk bounds come from cu_num_logits_np. ## Test Result | | peak activation | available KV cache | result | |—|—|—|—| | mai…

  • 521aa80 #48399 — [Core] Simplify KVBlockZeroer index tensor handling (#48399)
    • 作者: Nick Hill | +6/-55 | 4 个文件

    Simplification related to recent race condition fix https://github.com/vllm-project/vllm/pull/48085. There’s no need to prealloc a dedicated circular list of buffers just for the indices that are transferred, we don’t do this for other similar tensors in other places.

  • a76df87 #49481 — [MooncakeStore] Re-derive full external hits on stored boundaries (#49481)
    • 作者: Dao007forever | +110/-38 | 8 个文件

    When an external hit covers the full prompt, the final token must still be recomputed. The scheduler previously rounded that hit down arithmetically, which could select an interior fine-grained boundary that no producer persisted and cause repeated failed loads. Send the raw request length to the worker, derive the aligned lookup bound there, and re-run cache-hit discovery below the request end ag…

  • f83de6d #49523 — [CPU][Docs] Update docs and dockerfile for s390x (#49523)
    • 作者: Rehan Khan | +111/-22 | 3 个文件

    Update the docs for s390x to reflect the updated N/A ## Test Result N/A —

⚡ Performance

  • dd72658 #48597 — [Perf][GLM-5.2] Blackwell decode optimizations (#48597)
    • 作者: xiaozhoupy | +2139/-138 | 29 个文件

    Proposal to improve performance GLM-5.2 / DeepSeek-V3.2 Blackwell (SM100 / GB200 / GB300 sm_103) decode-latency optimizations — this branch’s delta vs main. ## Reproducing Instructions ### Branch ## PRs ### Router / MoE GEMM ### MLA / Attention / DSA (sparse) ### MTP / Spec decode ### Fusion / All-reduce / Norm ### Quantization / PDL ### Long context ## Notes - Several pieces extend already-me…

  • 80c7683 #49477 — [Perf] Defer MM embeds loading off the event loop (#49477)
    • 作者: Guan-Ming Chiu | +68/-22 | 2 个文件
    • With –enable-mm-embeds, loading multi-MB embedding tensors blocks the API server’s event loop, stalling all concurrent requests - Defer the loading to the shared thread pool, the same way every other modality is already handled - Error semantics and the sync path are unchanged - Not a duplicate: issue #49317 covers a different stage; no open PR touches this path - pytest tests/entrypoints/unit_…
  • a4904ba #48531 — [Perf][KVConnector][Mooncake] Vectorize prepare_value on the KV load path (#48531)
    • 作者: Summer Yang | +211/-21 | 4 个文件

    For a long-context request, 289K tokens is about 2,258 blocks. ChunkedTokenDatabase.prepare_value spends about 35 ms of GIL-held time per request computing RDMA target addresses on the KV load path. This cost is amplified by the receive-thread pool in #45971: four concurrent key builds take 208 ms wall time, compared with 174 ms when run serially. This PR vectorizes the computation with NumPy and …

🧪 CI/Tests

  • 2ac1251 #49513 — [CI] Use explicit devices in IR tests (#49513)
    • 作者: Andreas Karatzas | +66/-31 | 2 个文件

    Motivation: vLLM IR Tests The IR RMSNorm and LayerNorm cases relied on class-level default-device state that a function-scoped fixture reset before later parameterized cases. That left generated inputs on CPU while the tested operators were compiled for ROCm. The IR input generators now accept a device and every affected accelerator test passes it explicitly. This fixes the vLLM IR Tests test …

  • 1479bd9 #49270 — [ROCm][CI] Prepare AMD mirrors for regating (#49270)
    • 作者: Andreas Karatzas | +364/-292 | 23 个文件

    Prepare the existing AMD CI mirrors for regating without adding any new mirror definitions. This change: - reroutes existing mirrors from MI325 to MI300 or MI250 where appropriate - disables DinD for applicable MI300 mirrors - updates mirror timeouts based on observed runtime - moves CPU-only workloads to MI250 - fixes ROCm test commands to use HIP_VISIBLE_DEVICES - updates AMD pytest and faulthan…

  • 2659467 #49593 — [CI][PD] Add hybrid SSM P_TP>D_TP accuracy sweep entry (#49593)
    • 作者: Nicolò Lucchesi | +1/-0 | 1 个文件

    Adding CI coverage mentioned here https://github.com/vllm-project/vllm/pull/49297#pullrequestreview-4743067791

  • a49d37c #49511 — [CI] Disable reasoning in Responses smoke test (#49511)
    • 作者: Andreas Karatzas | +1/-0 | 1 个文件

    Motivation: Entrypoints Integration (Responses API) The basic Responses API smoke test can intermittently let Qwen3 spend its output budget on a reasoning trace and finish as incomplete. The assertion only checks basic response completion, so it now requests reasoning effort none explicitly. Dedicated cases in the same suite continue to exercise reasoning output. This fixes the **Entrypoints Integ…

  • 4501a6d #49551 — [ROCm][CI] Language Models tests tiny-mixtral with aiter fix (#49551)
    • 作者: music-dino | +0/-8 | 1 个文件

    Fix failing TitanML/tiny-mixtral aiter using tests in mi355_1:Language Models Tests (Standard): - models/language/generation/test_common.py::test_models[True-True-5-32-TitanML/tiny-mixtral] - models/language/generation/test_common.py::test_models[False-True-5-32-TitanML/tiny-mixtral] The tiny-mixtral model is untrained and is randomly initialized, the logprobs distribution is nearly uniform, and t…

  • c6fe94b #49606 — [CI] Bump PyTorch Compilation Unit Tests timeout to 150 min (#49606)
    • 作者: Andrey Talman | +1/-1 | 1 个文件

    The “PyTorch Compilation Unit Tests” job runs the whole compile/ suite serially (find compile/ -maxdepth 1 -name ’test_*.py’ | xargs -n1 pytest), loading and torch.compile-ing ~102 models (incl. 7-8B) across large parametrized matrices (test_config, test_dynamic_shapes_compilation, test_aot_compile). It has grown to sit right at the 110 min ceiling on every lane: | Build | torch | Result | Duratio…

  • f00efc5 #49510 — [CI] Isolate cudagraph tests in child processes (#49510)
    • 作者: Andreas Karatzas | +3/-24 | 1 个文件

    Motivation: Cudagraph The cudagraph cases reused the pytest process for engine lifetimes and then polled weak references, making VRAM release depend on nondeterministic cyclic-reference collection. Marking each case as forked makes child-process exit the deterministic GPU-memory ownership boundary. The manual weak-reference polling and duplicated cleanup are no longer needed. This stabilizes the *…

  • 0e36e3b #49512 — [CI] Use explicit devices in quantization tests (#49512)
    • 作者: Andreas Karatzas | +22/-20 | 2 个文件

    Motivation: Kernels Quantization Test 1 and Kernels Quantization Test 2 These tests relied on mutable global Torch default-device state, which a function-scoped fixture reset between parameterized cases and caused CPU tensors to reach ROCm/Triton kernels. The affected inputs, scales, weights, scores, and IDs now use an explicit accelerator device. This preserves the existing parameter coverage wit…