共 43 个 commit，涉及 134 个文件，+3425/-1528 行变动。

概要

统计项 数值
Commit 数 43
变更文件 134
新增行数 +3425
删除行数 -1528

Commit 列表

📦 Other

  • c8db00b #48816 — Fix GPTQ quantized Qwen3.5 MTP weight loading with spec decode (#48816)
    • 作者: vllmellm | +12/-2 | 1 个文件

    when loading Qwen/Qwen3.5-397B-A17B-GPTQ-Int4 with speculative decoding, weight loading fails with: The issue is that the MTP weight is not quantized, but the weight loading expects every layer to be quantized. This fix detects skip applying the quantized config to the MTP layers. ## Test Result |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr| |—–|——:|—————-|—–:|—–…

  • ac36a7a #48630 — [MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630)
    • 作者: Michael Goin | +390/-67 | 12 个文件

    Replacement for https://github.com/vllm-project/vllm/pull/48037 where we cap the intermediate memory required for large batch rejection sampling by using a fixed size scratch buffer and simply go through the sampling process multiple times. The loop is sync-free since chunk bounds come from cu_num_logits_np. ## Test Result | | peak activation | available KV cache | result | |—|—|—|—| | mai…

  • 521aa80 #48399 — [Core] Simplify KVBlockZeroer index tensor handling (#48399)
    • 作者: Nick Hill | +6/-55 | 4 个文件

    Simplification related to recent race condition fix https://github.com/vllm-project/vllm/pull/48085. There’s no need to prealloc a dedicated circular list of buffers just for the indices that are transferred, we don’t do this for other similar tensors in other places.

  • a76df87 #49481 — [MooncakeStore] Re-derive full external hits on stored boundaries (#49481)
    • 作者: Dao007forever | +110/-38 | 8 个文件

    When an external hit covers the full prompt, the final token must still be recomputed. The scheduler previously rounded that hit down arithmetically, which could select an interior fine-grained boundary that no producer persisted and cause repeated failed loads. Send the raw request length to the worker, derive the aligned lookup bound there, and re-run cache-hit discovery below the request end ag…

  • f83de6d #49523 — [CPU][Docs] Update docs and dockerfile for s390x (#49523)
    • 作者: Rehan Khan | +111/-22 | 3 个文件

    Update the docs for s390x to reflect the updated N/A ## Test Result N/A —

  • 9a698f3 #49487 — [Performance][Model] Avoid transient Inkling result allocations (performance, and OOM prevention on smaller memory configurations) (#49487)
    • 作者: Mike G | +2/-2 | 2 个文件

    Inkling currently performs two required output operations using out-of-place expressions. In the tested H100 configuration, each allocates an unnecessary temporary tensor: - Applying global_scale to the dense-MLP output allocates an 86 MiB temporary. - Adding the sink-expert output to the routed MoE output allocates a 94 MiB temporary. Both destination buffers are private and have no remaining con…

  • fc5fda1 #49474 — [Docs] Re-add Reo.dev analytics beacon (#49474)
    • 作者: jcotant-inferact | +4/-0 | 2 个文件

    Re-add the Reo.dev documentation analytics beacon to docs.vllm.ai. vLLM POC’d Reo.dev analytics earlier this year (vllm-project/vllm_website#13 / vllm-project/vllm#33957) and rolled it back (vllm-project/vllm_website#16 / vllm-project/vllm#36528) while we evaluated it further. Re-adding for docs.vllm.ai as we are exploring Reo.dev again. Adds enableThirdPartyTracking: true, which wasn’t present in…

  • 27ffbfd #48044 — Fused Shared Expert Support for AMD Quark DeepSeek-V4 Model Checkpoints (#48044)
    • 作者: Colin Z | +175/-13 | 3 个文件

    Enable serving AMD Quark DeepSeek-V4-MXFP4 checkpoints (AMD-Quark quantized, allowing MXFP4 shared expert fusion) in vLLM. The runtime layout implemented in this PR reuses the existing deepseek_v4_fp8 path. vllm/models/deepseek_v4/quant_config.py - DeepseekV4FP8Config.override_quantization_method now also claims quant_method=“quark” for model_type=deepseek_v4, guarded by _is_quark_mxfp4_ocp so…

  • 191146d #49492 — Add quantization label automation (#49492)
    • 作者: Michael Goin | +25/-1 | 2 个文件
    • label PRs that touch the core quantization directory or contain quant in the title - label issues containing the whole words quantization or quantized ## Why The existing quantization label had no automatic PR or issue application, so relevant work was under-labeled. Searches for open PRs about quantization label and issue-autolabel automation found no duplicate change. ## Impact Quantization-re…
  • f3a920a #48993 — [Core][DSV4] Compact MXFP4 indexer KV cache and packed group overlays (#48993)
    • 作者: Summer Yang | +216/-84 | 3 个文件

    Context When DeepSeek V4 uses MXFP4 indexer K values, engine reserves the larger FP8 row for them. Its packed KV planner also buckets layers by page size, leaving avoidable holes when different cache groups have different mixtures of page sizes. This PR: - sizes MXFP4 indexer K rows from their packed values and UE8M0 scales (head_dim / 2 + head_dim / 32 bytes), reducing the 128-dimensional row …

  • 61a0953 #48914 — Bump Flashinfer version to 0.6.15 (#48914)
    • 作者: Wei Zhao | +4/-4 | 3 个文件

    Bump flashinfer version to 0.6.15 and re-enable persistent cache for autotuning using set_autotune_process_group, which supports distributed auto-tuning across ranks with synchronization to avoid timeout caused by straggler. There has been observation that the current version (0.6.14) may cause hang when using nvfp4 MoE, see https://github.com/flashinfer-ai/flashinfer/issues/3971. This is addr…

  • 37e370f #48957 — [DSv4 Perf] Skip empty c128 kernel launch, around 2x kernel performance improvement. (#48957)
    • 作者: Wentao Ye | +53/-2 | 2 个文件

    We do compress in SparseAttnCompressC128Block8Kernel and store in SparseAttnNormRopeStoreFullKernel We will skip it inside the kernel by taking a look at boundary, but this skip could be move ahead to bypass the full kernel. Acc covered in unit test Perf through this AI generated script And we get

  • c79ff5f #49326 — [Build] Bump vllm-flash-attn to C++20-compatible commit for torch-nightly (#49326)
    • 作者: Andrey Talman | +1/-1 | 1 个文件

    Bumps the vllm-flash-attn pin so vLLM builds against torch nightly, which now requires C++20. ed4b7342 = flash-attention#168 (“Require C++20 to match PyTorch ATen headers”), which sets CMAKE_CXX_STANDARD 20 in the vllm-flash-attn build. ## Why Recent PyTorch added a hard guard in ATen.h: vLLM’s main extensions already build at C++20, but the vendored vllm-flash-attn subproject pinned C++…

  • 1a659a0 #49431 — Upgrade tpu-inference to v0.25.0 (#49431)
    • 作者: Teresa Chen | +1/-1 | 1 个文件

    Upgrade tpu-inference to latest stable release v0.25.0 Verified on tpu-inference CI. ## Test Result Success. —

  • 0f6cf7f #49045 — [Rust Frontend] Extract request preparation from the inference path (#49045)
    • 作者: Sage | +190/-109 | 2 个文件

    Extracts the existing chat and text request preparation into engine-free processors for normal inference and a future renderer. This implements Stage 1 of #49047.

  • c79ad3a #49255 — [Rust Frontend][gRPC] Add abort control RPC (#49255)
    • 作者: Connor Carpenter | +177/-29 | 5 个文件

    Add an idempotent gRPC control RPC for aborting active Rust frontend requests by ID. - Add vllm.Control.Abort with repeated request IDs and an empty response. - Register vllm.Control on the existing gRPC listener. - Resolve caller-facing request IDs through the frontend’s external-to-internal request-ID mapping before aborting EngineCore work. - Treat empty, unknown, repeated, and completed reques…

  • d6dbdb9 #49408 — [XPU] WA of topk_softplus_sqrt arg mismatch on XPU (#49408)
    • 作者: Liangqiusong | +15/-0 | 1 个文件

    This PR adds a workaround to fall back to old interface for XPU, This will be removed once xpu-kernels is updated.

  • 06da482 #49395 — [XPU] WA of topk_softmax arg mismatch on XPU (#49395)
    • 作者: liuzhenwei | +11/-0 | 1 个文件

    PR #48979 added the is_padding argument to _moe_C.topk_softmax, but the current XPU kernel does not yet support it. This PR adds a workaround to fall back to old interface for XPU, This will be removed once xpu-kernels is updated. ## Test Result —

⚡ Performance

  • 80c7683 #49477 — [Perf] Defer MM embeds loading off the event loop (#49477)
    • 作者: Guan-Ming Chiu | +68/-22 | 2 个文件
    • With –enable-mm-embeds, loading multi-MB embedding tensors blocks the API server’s event loop, stalling all concurrent requests - Defer the loading to the shared thread pool, the same way every other modality is already handled - Error semantics and the sync path are unchanged - Not a duplicate: issue #49317 covers a different stage; no open PR touches this path - pytest tests/entrypoints/unit_…
  • a4904ba #48531 — [Perf][KVConnector][Mooncake] Vectorize prepare_value on the KV load path (#48531)
    • 作者: Summer Yang | +211/-21 | 4 个文件

    For a long-context request, 289K tokens is about 2,258 blocks. ChunkedTokenDatabase.prepare_value spends about 35 ms of GIL-held time per request computing RDMA target addresses on the KV load path. This cost is amplified by the receive-thread pool in #45971: four concurrent key builds take 208 ms wall time, compared with 174 ms when run serially. This PR vectorizes the computation with NumPy and …

🐛 Bug Fix

  • 638d6e9 #44239 — [Bugfix][CI/Build] Fix Plamo2 HF runner crash on transformers v5 (_tied_weights_keys list→dict) (#44239)
    • 作者: Nikhil Kulkarni | +30/-7 | 2 个文件

    Fixes the Plamo2ForCausalLM item in #38379. ## Root cause pfnet/plamo-2-1b remote code defines _tied_weights_keys = [“lm_head.weight”] — the transformers v4 list[str] format. Transformers v5 changed this attribute to dict[str, str]. With tie_word_embeddings=True in the model config, get_expanded_tied_weights_keys() crashes during HfRunner.from_pretrained(): ## Fix Adds _fix_v4_tied_weights_keys(mo…

  • 1ad84fe #49391 — [Bugfix][Spec Decode] Select earliest-completing stop string in check_stop_strings (#49391)
    • 作者: Junpu Yu | +105/-11 | 2 个文件

    Fix a streaming↔non-streaming correctness divergence in stop-string handling that surfaces under speculative decoding. check_stop_strings returns the first stop string in list order that is found in the newly generated text, rather than the stop string that completes earliest in the text. Without speculative decoding each engine step appends ~one token, so at most one stop string is in the…

  • 12213c6 #47312 — [Bugfix] handle grammar compilation failures to avoid engine crash (#47312)
    • 作者: zhrrr | +123/-41 | 7 个文件

    This PR resolves the long-standing TODO in StructuredOutputManager._create_grammar: > # TODO: we still need to handle xgrammar compilation failures, though it should be unlikely as we test that up front as well. A structured-output request whose grammar fails to compile currently crashes the whole EngineCore instead of failing just that request. _create_grammar re-raises, and because it runs i…

  • 10c7547 #45224 — [Bugfix][Core] shm_broadcast: bound idle reader waits and release read slots (#45224)
    • 作者: chaeminlim-mb | +155/-23 | 2 个文件

    MessageQueue local readers can park indefinitely on the best-effort ZMQ notify path even after the writer has written to shared memory. This PR: - caps idle local-reader waits at SHM_READER_RECHECK_INTERVAL_MS (5s), so a reader always wakes to re-read the authoritative SHM written-flag even when a notify ping is dropped (PUB SNDHWM=1 drops silently, SUB is CONFLATE). Warning cadence via VLLM_RINGB…

  • 76bf552 #49415 — [Bugfix] Fix DeepSeek-V4 DSpark draft shared-expert padding for TP > 8 (#49415)
    • 作者: Mike G | +31/-11 | 3 个文件

    Add DeepSeek-V4-Pro DSpark shared-expert padding for built-in MTP loading when tensor parallelism requires block-aligned shared-expert padding to support sharding. This is needed to serve DSV4-Pro with a 16x H100 sharded configuration using TP=16. Use in conjunction with https://github.com/vllm-project/vllm/pull/49133 which provides the basic enablement for DSV4-NVFP4 with DSpark-MXFP4 draft enabl…

  • 4080263 #49485 — [Bugfix][Model] Remove SciPy dependency from Inkling scale planning (#49485)
    • 作者: Mike G | +32/-2 | 2 个文件

    Inkling’s HMLP vision scale planning calls scipy.optimize.linear_sum_assignment, but SciPy is not a core vLLM dependency; it is installed only through optional extras such as audio and bench. Consequently, constructing the released Inkling vision tower in a standard vLLM environment fails with ModuleNotFoundError before weight loading begins. This PR replaces that single startup-only SciPy depende…

  • b07ec92 #49489 — [Bugfix] Make shared NVFP4 MoE scales writable (#49489)
    • 作者: Matej Sirovatka | +46/-4 | 2 个文件

    FlashInfer NVFP4 backends broadcast shared input scales with expand, leaving registered parameters backed by overlapping stride-zero storage. Layerwise reload cannot copy regenerated values into those parameters. Materialize independent per-expert scale storage and cover the copy-back operation with a focused regression test. Assisted-by: OpenAI Codex Without this, NVFP4 online quantization of Qwe…

  • 229e01e #48425 — [BugFix] Handle per-group prefix-hit divergence for hybrid models with KV connector (#48425)
    • 作者: Nick Hill | +279/-40 | 4 个文件

    This is a fix for https://github.com/vllm-project/vllm/issues/46453. cc @ZhanqiuHu @howarlii

  • 149daf0 #47573 — [Bugfix] Exclude location-derived path vars from torch.compile cache factors (#47573)
    • 作者: Nils Matteson | +69/-0 | 2 个文件

    The bug VLLM_XLA_CACHE_PATH (derived from XDG_CACHE_HOME) and VLLM_CONFIG_ROOT (derived from XDG_CONFIG_HOME/HOME) are hashed into the torch.compile cache key (envs.compile_factors() → backends.py hash_key and caching.py aot_compile_hash_factors). Where a cache or config directory lives carries no information about compiled artifacts, but because these derived paths are hashed, **relocating HOM…

  • 917fdb5 #49467 — [Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel (#49467)
    • 作者: Michael Goin | +14/-4 | 1 个文件

    Fix an issue where Hopper + FP8 models regress by up to 20% after https://github.com/vllm-project/vllm/pull/41652/changes#diff-5ee97c25141448b7fd638e73ebcca7d3eead061d22b538e7983f4cdd81ac8f69 landed due to skipping deepgemm warmup ## Test Result —

  • 4b594b4 #49452 — [Bugfix][CI] Fix topk_softplus_sqrt no-op on non-XPU platforms (#49452)
    • 作者: stefankoncarevic | +1/-1 | 1 个文件

    Fixes a regression that made the fused topk_softplus_sqrt kernel a no-op on all non-XPU platforms (ROCm and CUDA). The XPU workaround in #49408 placed a return at function-body scope inside topk_hash_softplus_sqrt (vllm/_custom_ops.py). As a result, the real is_padding-aware call to torch.ops._moe_C.topk_softplus_sqrt(…) became dead code and was never reached on non-XPU platforms. The output ten…

  • 7d10a4c #49001 — [Bugfix] Retry config read to survive concurrent HF cache refresh (#49001)
    • 作者: peizhang56 | +11/-7 | 1 个文件

    tests/v1/distributed/test_external_lb_dp.py intermittently fails on ROCm CI (Distributed DP Tests (2 GPUs)) with the [4] (api_server_count=4) cases erroring as Exception: Servers failed to start. The real cause, hidden one level down in the server log, is one API-server process dying during startup with pydantic … Value error, Unrecognized model in ibm-research/PowerMoE-3b. Should have a model_t…

  • 910cc85 #49427 — [Bugfix] Restore gather_and_maybe_dequant_cache OOB guard (#49427)
    • 作者: Nick Hill | +8/-3 | 2 个文件

    The gather_and_maybe_dequant_cache rewrite in #28029 was based on pre-#28760 code and merged five days after it, silently reverting the bound check that #28760 had added: when seq_starts pushes the block index past the end of the batch’s block table row, the kernel reads block_table out of bounds (#27909). This PR restores the guard by skipping such tokens. Also ports test_gather_cache_oob to the …

  • 3de4b2b #48748 — [Bugfix][Parser] Fix special tokens (EOS/BOS) leaking into reasoning content (#48748)
    • 作者: Ben Browning | +72/-15 | 3 个文件

    Special tokens (like <｜end▁of▁sentence｜> for deepseek_v4) leaked into reasoning content when generation ended during the parser’s REASONING state without emitting a closing tag. The skip_tool_parsing guard preserved them for a later tool-call pass that either never ran or would independently drop them via its own engine. The reason this is safe is all tokens used for transitions and tool …

  • 2dc5a72 #49400 — [Bugfix][Renderer] Rebuild vision chunk UUIDs in async render path (#49400)
    • 作者: Guan-Ming Chiu | +2/-0 | 1 个文件
    • HfRenderer.render_messages_async skips the rebuild_mm_uuids_from_mm_data call that the sync path performs for unified-vision-chunk models (Kimi-K2.5) - Online serving therefore attaches stale pre-chunking UUIDs to multi_modal_uuids, corrupting mm-cache identity for chunked videos - Add the missing call, mirroring the sync path - Not a duplicate: no open PR touches this path - Verified with a tem…
  • 7c21548 #49297 — [PD][Bugfix] Fix NIXL hybrid MLA+mamba heterogeneous TP (#49297)
    • 作者: Ziming Huang | +10/-8 | 2 个文件

    Fixes KV transfer for hybrid MLA + SSM (Mamba) models under heterogeneous TP (prefill TP > decode TP) with the NIXL connector. ## Test Result —

🔩 Misc

  • 239fc73 #49217 — [Misc] Use VLLMValidationError in chat_utils content-part validation (#49217)
    • 作者: Umut Polat | +27/-13 | 1 个文件

    Continuing the migration of user-facing request validation in the entrypoints layer to VLLMValidationError (following #36254 and #49214). Several content-part validation checks in chat_utils.py still raised plain ValueError, while other checks in the same module already use VLLMValidationError (e.g. the unsupported-type check right next to the ones migrated here). VLLMValidationError carries a par…

🧪 CI/Tests

  • 4319345 #49423 — [CI] Fix stale/fragile untethered kernels-root tests (#49423)
    • 作者: Nick Hill | +65/-18 | 6 个文件

    Test-only fixes for files excluded as broken when wiring the kernels-root catch-all CI job (#49340). These tests were unintentionally not running in the CI and had become stale/broken. Runtime bugs found in the same audit will be fixed in separate PRs. - test_fused_recurrent_packed_decode / test_fused_sigmoid_gating_delta_rule: since #39064 the FLA kernels reserve state index 0 as NULL_BLOCK_ID (C…

  • b44311b #49388 — [CI] stabilize GDN prefill CuTeDSL test (#49388)
    • 作者: Thien Tran | +14/-19 | 1 个文件

    This was originally from #49143 This PR makes RNG inputs for test_gdn_prefill_cutedsl.py deterministic to make the test deterministic (different RNG inputs may shift the tolerance a bit. In the original test, I intentionally put the tolerance quite tight). ## Test Result —

  • b0d7875 #49450 — [CI] Increase timeout of pytorch-compilation-unit-tests (#49450)
    • 作者: Nick Hill | +1/-1 | 1 个文件

    Similar to https://github.com/vllm-project/vllm/pull/49374. This one is also taking longer, e.g. https://buildkite.com/vllm/ci/builds/79339#019f88d2-9b09-4081-a473-5e9f6bd9416c

  • 53c2f20 #49350 — [ROCm][CI] skip moe weight padding for eplb (#49350)
    • 作者: Divakar Verma | +4/-1 | 1 个文件

    This PR skips moe padding for eplb as contiguous weights are required, following a refactoring. pytest -s -v tests/distributed/test_elastic_ep.py::test_elastic_ep_scaling[sync_eplb] This PR is the 3rd piece for solving this test case. 1. https://github.com/vllm-project/vllm/pull/47206 -> Fixes the weight transfer issue. 2. https://github.com/vllm-project/vllm/pull/49251 -> Resolves the final accur…

  • 2f75e7f #49374 — [CI] Increase timeouts for jobs exceeding current limits (#49374)
    • 作者: Kevin H. Luu | +14/-14 | 8 个文件

    Increase CI timeouts for jobs that exceeded their configured limits in Buildkite builds 79241 and 79299. For each job with a true Buildkite timed_out state, the new timeout is ceil(current_timeout * 1.5) because the pipeline schema requires integer minutes: - 45 -> 68: Basic Correctness, OpenAI Part 1, MTEB, Multimodal Standard 1 - 30 -> 45: Benchmarks CLI - 35 -> 53: e2e Scheduling - 50 -> 75: AP…

🦀 Rust Frontend

  • 61c9ef9 #49153 — [Frontend] Parallelize preprocessing within the same request for pooling models online serving. (#49153)
    • 作者: wang.yuqi | +530/-812 | 17 个文件

    TL;DR - Use multithreading to accelerate preprocessing. You can specify the number of threads using renderer_num_workers. note: - We have not implemented tiling to overlap preprocessing and computation for online serving as we did for offline inference (#47699), in order to avoid unnecessary complexity. Therefore, it is not recommended to submit an extremely large number of requests at once. Yo…