共 50 个 commit，涉及 202 个文件，+7319/-1390 行变动。

概要

统计项 数值
Commit 数 50
变更文件 202
新增行数 +7319
删除行数 -1390

Commit 列表

📦 Other

  • 3ee9eea #47457 — [macOS][CPU][Installation] Fix the broken installation of vllm 0.24.0 in macos + cpu (#47457)
    • 作者: Ranran | +1/-1 | 1 个文件

    Problem vllm-metal installation is broken, because it cannot compile the vllm dependency on mac before this PR. tokenspeed-mla==0.1.2 in requirements/cuda.txt has no platform marker, but the package itself only publishes manylinux wheels (no sdist). Since setup.py’s get_requirements() copies this file’s lines verbatim into vLLM’s published install_requires, the unconditional requirement ships i…

  • 5bce653 #47187 — Make the Transformers modeling backend as fast as native vLLM (#47187)
    • 作者: Harry Mellor | +2679/-160 | 23 个文件

    For the Transformers modelling backend this PR: - enables the use of MergedColumnParallelLinear and QKVParallelLinear - fully leverages MoERunner to enable: - async shared experts - DBO - Sequence parallel - tp_plan becomes optional because the fusion passes know correct the TP orientations of the resulting fusions - pp_plan becomes optional because vLLM now checks for the nn.ModuleList in the tex…

  • 07f9baf #47140 — Revert “[Platform] Replace torch.cuda.Event with torch.Event (#47140)” (#47668)
    • 作者: Kunshang Ji | +70/-60 | 31 个文件

    part of https://github.com/vllm-project/vllm/issues/30679 CI ## Test Result —

  • 8f4c69b #47444 — [Rust Frontend] Cache metric handles for scheduler & request stats (#47444)
    • 作者: Bugen Zhao | +343/-237 | 9 个文件

    This PR reduces repeated get_or_create lookups on scheduler & request-level metrics paths. Instead of calling get_or_create every time when we want to access the global metrics handle for a request, we pre-resolve per-engine metrics handles once ClientInner and GenerateOutputStream are created, and cache them for future use. Still keep dynamic-label metrics, such as spec decoding accepted tokens p…

  • 8b79971 #43597 — attention: pass None for unused args in unified attention TD path (#43597)
    • 作者: Artur Fierka | +29/-20 | 1 个文件

    Performance follow-up to #40327 (UA+TD). On XPU we measured +1-10% throughput in some configurations. Co-authored with: @quinnlp

  • 740f379 #46065 — [ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator (#46065)
    • 作者: BadrBasowid | +499/-148 | 12 个文件

    Previously, with the AITER fused AR-RMSNorm enabled, two separate custom-allreduce instances were live: vLLM’s CustomAllreduce for the plain allreduce, and a standalone AITER CustomAllreduce (rocm_aiter_ops._CUSTOM_ALL_REDUCE) for the fused op, each with its own IPC buffers, both registered during CUDA graph capture. Performance was the main reason it’s done this way rather than by dropping the AI…

  • ba22152 #47259 — fix(security): block request-level GPU video backend selection withou… (#47259)
    • 作者: Juan Pérez de Algaba | +131/-8 | 4 个文件

    Prevent request-level media_io_kwargs from selecting GPU video backends (e.g. pynvvideocodec) when the engine did not reserve VRAM for them at startup. VideoMediaIO.merge_kwargs() now strips GPU backend overrides from runtime kwargs unless the static config already named that backend, closing a resource exhaustion vector where unreserved CUDA context and decoder-surface allocations could starve th…

  • 736f1a5 #47688 — [XPU] Route mm_prefix models to Triton attention backend (#47688)
    • 作者: liuzhenwei | +9/-0 | 1 个文件

    mm_prefix (prefix-LM bidirectional mask) is only supported by FA4, which is unavailable on XPU, so fall back to Triton attention. pytest -s -v tests/models/multimodal/generation/test_mm_prefix_lm.py::test_mm_prefix_lm_e2e ## Test Result —

  • 344609a #47695 — [CI/Build] Fix pre-commit check (#47695)
    • 作者: Li, Jiang | +7/-8 | 1 个文件

    Test Result —

  • 2fa1056 #47420 — [Core][DP] Rotate load-balancer tie-break to avoid systematic engine bias (#47420)
    • 作者: Qiming Zhang | +6/-0 | 1 个文件

    Motivation In DPLBAsyncMPClient.get_core_engine_for_request, the internal DP load balancer selects the engine with the lowest score (waiting * 4 + running), scanning from a fixed eng_start_index and using a strict < comparison. When engines are tied, the winner is always the first one scanned — i.e. eng_start_index, which is constant for a single front-end client (0). Between coordinator stats…

  • 990c2a0 #45243 — [RISC-V] Enable BF16 on VLEN=256 hardware (#45243)
    • 作者: velonica0 | +24/-9 | 2 个文件

    #36578 added find_isa(${CPUINFO} “zvfbfmin” RVV_BF16_FOUND) to detect BF16 support via /proc/cpuinfo. However, the latest hardware currently available (Spacemit X100 / K3) does not report zvfbfmin in /proc/cpuinfo due to a kernel/firmware limitation, causing BF16 to be silently disabled. VLLM_CPU_RVV_BF16=1 force-enables the RVV BF16 (Zvfbfmin) path on hardware that supports BF16 but under-reports…

  • e433634 #47538 — [Performance][Hardware][RISC-V] Reduce LMUL pressure in INT4 LUT dequant (#47538)
    • 作者: LiuTong | +23/-10 | 1 个文件

    The FP32Vec16(int64_t, FP32Vec16&) LUT constructor extracts 16 four-bit nibble indices from a packed int64 using u64 vectors at LMUL_1024. Using u64 to process an int64 input is natural — on fixed-width SIMD like x86 AVX, element width has no impact on register file pressure. However, RVV’s variable-length register grouping (LMUL) means element width directly determines how many physical registers…

  • d9c1767 #46361 — [INC][ARK] Direct Register Custom Op for ARK (#46361)
    • 作者: Zhenzhong Xu | +152/-43 | 6 个文件

    Purpose: * Direct Register Custom Op for ARK * Support torch compile computation graph log:

  • e9cc1fd #47687 — [CI/Build][CPU] Remove global extra index (#47687)
    • 作者: Li, Jiang | +9/-18 | 6 个文件

    Remove global extra index to avoid flaky package downloads. ## Test Result —

  • 394edc8 #47682 — [XPU] limit max-num-seqs in test_lmeval.py for XPU (#47682)
    • 作者: Qiming Zhang | +3/-2 | 1 个文件

    Default number of max-num-seqs would cause the UT failure on XPU. Limit it to 64 to avoid device error.

  • 95a248f #47433 — [Attention Backend] HPC_ATTN backend support mtp and dynamic scheduled attention (#47433)
    • 作者: Cheng Jiang | +123/-30 | 3 个文件

    Test Result —

  • d2ec433 #43957 — [XPU] Fix Eagle3 initialization on XPU (#43957)
    • 作者: Chaojun Zhang | +44/-1 | 3 个文件

    Purpose: Fix Eagle3 draft model embedding sharing when the target and draft models have different embedding widths. ## Why tests/models/test_initialization.py uses a patched _initialize_kv_caches_v1 to skip the warmup path. That patch works under fork with create_process_for_each_test because the child inherits the test process state, but it does not work under spawn because the child starts fr…

  • 78a04c2 #43092 — [XPU] Fix CUDA API shims breaking Torch Dynamo during AOT compile (#43092)
    • 作者: Łukasz Ślusarczyk | +56/-7 | 2 个文件

    On XPU, XPUModelRunner applies _torch_cuda_wrapper(), which patches torch.cuda.* to delegate to torch.xpu.*. Previously, callables were assigned by direct alias (e.g. torch.cuda.current_stream = torch.xpu.current_stream), so CUDA and XPU shared the same function object. During EngineCore startup, profile_run() triggers AOT compile. Torch Dynamo builds its in-graph handler table (TorchInGraphFu…

  • cc1d020 #46942 — [MRV2] Enable mm prefix bidi attention support on MRV2 (#46942)
    • 作者: Isotr0py | +165/-3 | 5 个文件
    • Wire mm prefix attn mask computation to MRV2 - Actually, tests/models/multimodal/generation/test_common.py’s gemma3 test can’t catch the difference of with/without mm_prefixlm attention, because their logprobs are still quite similar. - This PR adds test to cover this regression. ## Test Result Test should only pass with this PR’s fix. —

⚡ Performance

  • 5ad1117 #47416 — [perf]Add fused Kimi image preprocessing (#47416)
    • 作者: Kevin_Xiong | +402/-2 | 5 个文件

    This PR adds a fused CPU image preprocessing path for Kimi-K2.5/K2.6 vision chunks. - Adds KimiK25FusedVisionProcessor, a vLLM-local processor that preserves the Kimi NaViT resize/token semantics while fusing padding, normalization, and patchification through numba. - Uses the fused processor automatically when numba is available. - Falls back to the remote HF image processor when numba is unavail…

  • f70caef #47474 — [Perf] Cache token_to_req_indices for dsv4, 5x~6x kernel performance improvement (#47474)
    • 作者: Wentao Ye | +34/-25 | 4 个文件

    So we save two additional copy from CPU to GPU ### Acc ### Perf Running this AI-generated microbenchmark Script And we will get 5x~6x kernel level improvement.

🐛 Bug Fix

  • 8d8ec38 #47429 — [Bugfix][Spec Decode] Add missing draft_id_to_target_id to DSparkDeepseekV4ForCausalLM (#47429)
    • 作者: Laurent-Zhang | +2/-0 | 1 个文件

    What this PR does Fixes #47418 DSparkDeepseekV4ForCausalLM is a full-vocab draft model (draft ids are target ids, no d2t remapping). However, DSparkSpeculator.load_draft_model accesses model.draft_id_to_target_id unconditionally, causing an AttributeError when running DeepSeek-V4-Flash with DSpark. ### Root Cause In vllm/v1/worker/gpu/spec_decode/dspark/speculator.py:86: This line assumes the d…

  • 598d511 #47589 — [Bugfix][Distributed] Delegate MNNVL allreduce one-shot selection (#47589)
    • 作者: jesco | +60/-13 | 2 个文件

    Fixes #47284. After #47219, vLLM can default FlashInfer fused allreduce/RMSNorm to the mnnvl workspace backend on single-node setups. The fusion call still forced use_oneshot from vLLM’s legacy per-rank threshold table, which can disagree with FlashInfer MNNVL’s AUTO workspace sizing. When vLLM forces use_oneshot=True for a shape that FlashInfer sized for AUTO/two-shot, FlashInfer reports an insuf…

  • 095adf1 #47671 — [Bugfix] Fix int32 overflow in triton_decode_attention page offsets (#47671)
    • 作者: Yifan Qiao | +2/-2 | 1 个文件

    _fwd_kernel_stage1 / _fwd_grouped_kernel_stage1 load kv_page_number as int32 and multiply it by the KV buffer page stride in int32. With MLA c_kv (576 dims) and PAGE_SIZE 480 the page stride is ~276k, so page numbers above ~7.7k overflow to negative offsets → CUDA illegal memory access. Hybrid models hit this quickly: all layer groups share one block pool, so a 90GB KV buffer already holds ~163k p…

  • 373eb31 #46066 — [Bugfix][Core] Fix num_output_placeholders underflow with async scheduling + spec decode (#46066)
    • 作者: Ting SUN | +58/-3 | 3 个文件

    Under async scheduling with speculative decoding, reset_prefix_cache(reset_running_requests=True) force-preempts the running requests, zeroes each request’s num_output_placeholders, and records the in-flight frame count in async_tokens_to_discard so those now-stale frames are dropped when they return. This path is reached by the /reset_prefix_cache dev endpoint and, more commonly, by RLHF weight u…

  • 7a90eb9 #47091 — [Bugfix] [Gemma4] Fix Gemma4 MTP draft model layers ignoring quant_config (#47091)
    • 作者: Ayushman Singh | +13/-4 | 1 个文件

    This PR fixes quantization support for the Gemma4 Eagle (MTP) draft model in speculative decoding. Previously, all layers in the Gemma4 MTP draft model were hardcoded to quant_config=None, causing quantized draft checkpoints to silently load incorrectly and produce 0% draft token acceptance. This PR addresses the following: - Use get_draft_quant_config in Gemma4MultiTokenPredictor and Gemma4MTP to…

  • 98e4726 #47697 — [fix][run_batch]: respect proxy env vars when downloading media URLs (#47697)
    • 作者: Qiming Zhang | +4/-12 | 1 个文件

    aiohttp.ClientSession does not read HTTP_PROXY/HTTPS_PROXY environment variables by default. This causes download_bytes_from_url to fail with a connection timeout in network environments that require a proxy (e.g., corporate networks), even though tools like curl/wget work fine. Add trust_env=True to aiohttp.ClientSession() in download_bytes_from_url so that aiohttp respects the system proxy setti…

  • 40cc2e8 #47165 — [Bugfix] Return HTTP 422 for unprocessable image URLs instead of 500 (#47165)
    • 作者: Alexis K. | +288/-12 | 5 个文件

    Associated issue Issue 47163 - [Bug]: Image URL errors return HTTP 500 instead of 422 for unprocessable content This PR fixes the issue where vLLM returns HTTP 500 (Internal Server Error) when image URLs in requests point to non-existent or inaccessible resources. The fix ensures these client-side errors properly return HTTP 422 (Unprocessable Entity), making it clear that the issue is with the…

  • 90ce3a0 #47607 — [bugfix] fix MOSS-Audio deepstack_input_embeds initialization in PP (#47607)
    • 作者: Yan Ma | +4/-0 | 1 个文件

    After PR #44443, all dense models use ModelRunnerV2 as default and so does MOSS-Audio. And will run into below errors: verified on L20. pytest -s -v tests/models/multimodal/generation/test_moss_audio.py::test_moss_audio_parallel_smoke[pp2] ## Test Result —

  • 26c754d #47116 — [XPU][Bugfix] Do not transpose weight_scale_inv at load time (#47116)
    • 作者: Ma Jian | +2/-24 | 1 个文件

    The MLA attention’s _o_proj path (via _get_cached_wo_a_bf16 in rocm_aiter_mla_sparse.py) directly reads weight_scale_inv and assumes the original [N/128, K/128] checkpoint layout for manual dequantization. Pre-transposing to [K/128, N/128] in process_weights_after_loading caused _expand_2d_block_scales to apply wrong scale values (row/col blocks swapped), silently degrading GSM8K accuracy from 96….

  • d039c17 #47448 — [Bugfix] Recycle post-final-norm hidden in GLM MTP (single norm) (#47448)
    • 作者: xiaozhoupy | +16/-7 | 1 个文件

    The compile-free DeepseekV32 MTP draft layer returned the pre-final-norm hidden as both the draft-logits hidden and the recycled previous_hidden_states. Recycling the pre-final-norm hidden mismatches the draft model’s hnorm and drops MTP acceptance; the reference deepseek_mtp.py (PR #45895) recycles the post-final-norm hidden. Fix by computing the post-final-norm hidden once – fusing the residual…

  • 16f8110 #47532 — [Bugfix][CPU][RISC-V] Fix VLEN detection for RVV attention path (#47532)
    • 作者: LiuTong | +26/-18 | 3 个文件

    Three bugs in the RISC-V VLEN detection chain caused the RVV-optimized attention kernel to be silently disabled or the build to fail: 1. cpu_attn_has_isa(“rvv”) only checked __riscv_v_min_vlen == 128, excluding VLEN=256 hardware (e.g. Spacemit X100). This is inconsistent with cpu_attn_rvv.hpp which supports both 128 and 256. 2. CMake VLEN auto-detection read /proc/cpuinfo unconditionally, which de…

  • f1073c0 #46739 — [CPU][BugFix] Multiple fixes to w4a8_int8 CPU MoE path (#46739)
    • 作者: Fadi Arafeh | +192/-190 | 10 个文件

    w4a8 MoE refactor in https://github.com/vllm-project/vllm/pull/42789 broke parts of this path. This PR: - fixes the group size passed to C++ kernel - registers fake impl for torch compile - adds support for fp32, bf16, fp16 pre quant activations - restores channelwise support, which was dropped by previous refactor - extends int4 MoE tests, accelerate reference path and add them to Arm CI - moves …

  • 8974ed8 #44461 — [Bugfix][Voxtral Realtime] Fix token feedback timeout silent hang (#44461)
    • 作者: Ting SUN | +3/-6 | 1 个文件

    Closes #36015. Closes #35863. Voxtral realtime has a background feed_tokens() task that moves engine output tokens from input_stream into the realtime transcription buffer. That task currently wraps input_stream.get() in asyncio.wait_for(…, VLLM_ENGINE_ITERATION_TIMEOUT_S). A normal idle interval with no engine output can therefore raise TimeoutError and kill the feedback task while the websocke…

  • fb2face #46037 — [Bugfix][Model] Fix crash loading Mamba/Mamba2 checkpoints without an architectures field (#46037)
    • 作者: Ting SUN | +9/-0 | 1 个文件

    Loading a Mamba/Mamba2 checkpoint whose config.json omits the top-level architectures field crashes engine core init with a bare, message-less AssertionError: mamba_block_size is not None. This hits the official HF mamba2 conversions such as AntonV/mamba2-130m-hf and AntonV/mamba2-370m-hf. Root cause is an arch-name mismatch inside vLLM itself: 1. When architectures is absent, vLLM injects the bas…

  • fa4321d #47609 — [Bugfix][TurboQuant] Preserve KV cache dtype in backend shape (#47609)
    • 作者: Lucas Wilkinson | +6/-1 | 1 个文件

    Fix TurboQuant KV cache shape setup so TQFullAttentionSpec carries its real cache dtype (for example turboquant_k8v4) into backend shape/layout computation instead of being treated as auto. This prevents TurboQuant backends from seeing auto and failing with an unknown TurboQuant cache dtype during startup. ## CI Failure This fixes the Buildkite LM Eval TurboQuant KV Cache failure from build #7…

🔧 Refactor

  • b1c6dba #47329 — [Refactor] Remove multiple dead code (#47329)
    • 作者: Wentao Ye | +0/-186 | 7 个文件

    Remove multiple dead code

🧪 CI/Tests

  • 51ee564 #47748 — [CI] Skip test for checkpoint that was deleted (#47748)
    • 作者: Harry Mellor | +1/-0 | 1 个文件

    Tencent just deleted tencent/HunyuanOCR and most of vLLM’s CI runners have broken HF caches. This PR marks the checkpoint as not available online so that it deosn’t block the rest of CI.

  • f676808 #47730 — [CI] Use TTY for AMD CI tests for colored buildkite logs (#47730)
    • 作者: Nick Hill | +4/-1 | 2 个文件

    For other tests we launch the docker container with a tty, unless there’s a reason not to I’m unaware of it would be good to do so for the AMD tests too for consistency. This also means we’ll get nicer logs in builkite with the ansi colors.

  • cdab283 #47675 — [XPU][CI]Add agent tags for Basic Models Tests (Initialization) in Intel GPU CI (#47675)
    • 作者: xiangdong | +4/-0 | 1 个文件

    Test Result —

  • fb265fc #47591 — [ROCm][CI] Increasing parallelism in Basic Models Tests (Extra Initialization) (#47591)
    • 作者: Andreas Karatzas | +6/-6 | 1 个文件

    Increasing parallelism in Basic Models Tests (Extra Initialization) %N to avoid timeouts like: - https://buildkite.com/vllm/amd-ci/builds/10424/list?jid=019f29a6-adae-4dae-9bbe-dd8444e163df&tab=output

  • 8f0e75e #47481 — [ROCm][CI] Adding nixl multiconn (#47481)
    • 作者: Andreas Karatzas | +88/-7 | 4 个文件

    Adds MI300 ROCm CI jobs for Hybrid SSM NixlConnector prefix cache coverage and MultiConnector NIXL plus offloading accuracy and edge case coverage. The new jobs run on the 2-GPU MI300 pool and use the TRITON_ATTN backend. The NIXL integration scripts now accept ATTENTION_BACKEND and VLLM_SERVE_EXTRA_ARGS so the same scripts can be reused across backend-specific ROCm coverage. The MultiConnector sc…

  • 6971582 #47406 — [Test][XPU] Skip fork in kv_sharing_fast_prefill test on XPU (#47406)
    • 作者: Liangliang Ma | +3/-1 | 1 个文件

    Pytest has already touched XPU before this decorator runs (via torch.xpu.is_available() in conftest.py), so os.fork() in the fork_new_process_for_each_test decorator triggers ‘Cannot re-initialize XPU in forked subprocess’. ROCm bypasses the decorator for the same reason; extend the escape hatch to XPU.

  • 6569df6 #47534 — [Test][LoRA] Use lightweight CPU reference and skip heavy cleanup in punica ops tests (#47534)
    • 作者: Chaojun Zhang | +79/-49 | 1 个文件

    Optimize tests/lora/test_punica_ops.py by replacing the GPU-based torch_ops.sgmv_shrink/expand reference (which uses torch.einsum) with lightweight CPU per-LoRA matmul loops, and overriding the heavy per-test cleanup fixtures that are unnecessary for pure kernel tests. 1. CPU matmul reference: (_cpu_bgmv_shrink / _cpu_bgmv_expand): Replace torch.einsum(“bi, boi -> bo”, …) with a simple per-L…

  • b712181 #46944 — [ROCm][Test] Fix test_per_token_group_quant_fp8 tolerance for 1-ULP FP8 rounding on gfx950 (#46944)
    • 作者: Spandan Tiwari | +19/-1 | 1 个文件

    test_per_token_group_quant_fp8 fails consistently on MI355 (gfx950) with rtol=0.15. This is potentially a test tolerance issue, may not be a kernel bug. ## Root Cause The test compares the output of the Triton-based per_token_group_quant_fp8 kernel against a PyTorch reference (native_per_token_group_quant_fp8). One input element lands almost exactly at the **midpoint between two adjacent e4m3fn re…

📖 Documentation

  • 641cb59 #45813 — [Doc] Clarify fastokens availability (#45813)
    • 作者: LiJzd | +3/-2 | 1 个文件

    Fixes #43446 - Clarifies that VLLM_USE_FASTOKENS is available in vLLM v0.23.0 and later. - Tells users to upgrade vLLM if their installed version does not recognize the environment variable. ## Duplicate-work check - Checked vllm-project/vllm#43446 comments. - Checked open PRs with 43446 in:body: no matches. - Checked open PRs with VLLM_USE_FASTOKENS fastokens: no matches. ## Validation - git diff…

  • 3d7f357 #47701 — [Doc] docs: fix note formatting for pooling models (#47701)
    • 作者: Sungjae Lee | +2/-3 | 1 个文件

    The purpose of this PR is fixing note part of pooling models docs. ## Test Result —

🦀 Rust Frontend

  • 98ba9b9 #47024 — [Frontend] Support OpenAI Responses API namespace tools (#47024)
    • 作者: Zhong | +383/-22 | 8 个文件

    Fixes #46737. Namespace tools are expanded to an internal namespace__tool representation for prompting and parsing, but the Responses API should expose the original tool name expected by clients. This PR introduces a consistent mapping between the external tool name and the internal namespace-qualified representation throughout the Responses pipeline. The mapping is applied when: * expanding names…

✨ New Feature

  • f2aaf59 #44880 — [Feature] Support MTP speculative decoding for Bailing hybrid models (#44880)
    • 作者: alex101-ops | +1200/-6 | 11 个文件

    Add MTP (Multi-Token Prediction) speculative decoding support for Bailing MoE hybrid models, and enhance the linear attention backend to support speculative decode with CUDAGraph. Key changes: - New BailingMoeV25MTPModel: Add the draft model implementation for Bailing MoE V2.5 MTP speculative decoding (bailing_moe_mtp.py), including BailingMoeV25MultiTokenPredictor with shared embedding/lm…

  • b6cc46e #47070 — [Feature] Support sequence parallel without the need for DP, 1.9%~5.0% E2E Throughput Improvement (#47070)
    • 作者: Wentao Ye | +33/-22 | 7 个文件

    Support sequence parallel without the need for DP Originally we must have DP to eanble sequence parallel, but this constraint is not needed, this PR fixes the issue. vllm serve zai-org/GLM-5.2-FP8 –kv-cache-dtype fp8_e4m3 –enable-expert-parallel –tensor-parallel-size 8 –tool-call-parser glm47 –enable-auto-tool-choice –reasoning-parser glm45 –port 9256 –profiler-config.profiler=torch –prof…