共 46 个 commit，涉及 272 个文件，+8006/-5406 行变动。

概要

统计项 数值
Commit 数 46
变更文件 272
新增行数 +8006
删除行数 -5406

Commit 列表

✨ New Feature

  • 0b0bd2b #44428 — [Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428)
    • 作者: fangyuchu | +1088/-7 | 27 个文件

    Add fault tolerance (FT) framework for DP+EP (Data Parallelism + Expert Parallelism) MoE deployments. When one DP rank dies, the EP all2all operation on surviving ranks blocks indefinitely, causing a full cluster hang. This framework detects faults, aborts in-flight requests, and allows an external orchestrator to trigger coordinated recovery via a REST API. Key design: - Sentinel pattern: All…

🐛 Bug Fix

  • 33ef67e #49403 — [BugFix] Increase the max supported duration for MOSS-TD (#49403)
    • 作者: Canlin Guo | +2/-3 | 1 个文件

    Previously, MOSS-TD incorrectly treated Whisper’s default 30-second chunk length as the maximum duration of a complete audio item. As a result, it reported only about 375 embedding tokens as max_tokens_per_mm_item. Consequently, the encoder cache size usually fell back to max_num_batched_tokens (for example, 8,192). Longer audio producing more embedding tokens than this cache capacity was rejected…

  • d1a8ba6 #49149 — [Bugfix][MiniMax-M3] Fix token-major top-k buffer handling in Triton … (#49149)
    • 作者: rongfu.leng | +13/-3 | 2 个文件

    Fixs: https://github.com/vllm-project/vllm/issues/49147 1. In the vllm/models/minimax_m3/nvidia/model.py+790, model assign topk_indices_buffer shape is [token, head, topk]. 2. But Triton indexer vllm/models/minimax_m3/common/indexer.py+423 still use [head, token, topk] this shape, 3. Triton sparse attention also reads data using the [head, token, topk] layout. vllm/models/minimax_m3/common/sparse_…

  • 1423569 #48852 — [Bugfix][Tool Parser] Fix dropped streaming arguments in Jamba and InternLM2 parsers (#48852)
    • 作者: mosya415 | +99/-8 | 4 个文件

    When streaming tool calls, the first time a call’s arguments become non-empty, the Jamba and InternLM2 parsers computed the streamed delta by locating the raw model delta text inside json.dumps(arguments) with str.index: The raw delta need not appear verbatim in the re-serialized JSON. It can carry structural text such as , “arguments”: {…}, or the model may emit compact JSON whose spacing diffe…

  • 0b1a8bb #49802 — [Bugfix][CI] Fix stale Mooncake lookup expectation broken by a merge race (#49802)
    • 作者: Harry Mellor | +4/-2 | 1 个文件

    tests/v1/kv_connector/unit/test_mooncake_store_worker.py::test_lookup_rejects_boundary_missing_one_mamba_shard fails on main, breaking the V1 Core + KV + Metrics step for every unrelated PR (e.g. build 80149): This is a merge race between two Mooncake PRs, not a functional regression: - #49481 (a76df87) changed MooncakeStoreWorker.lookup so a hit covering the whole request is re-derived below the …

  • 70052fb #49499 — [Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of the KV-head dedup (#49499)
    • 作者: Yifan Qiao | +311/-93 | 3 个文件

    Fix silent Mamba/linear-attention state corruption in the Mooncake store connector when serving hybrid models with TP > 1. With MLA (or GQA with num_kv_head < tp_size) all TP ranks hold identical attention KV, so the store dedups: ranks share one tp_rank key namespace and stripe PUTs across blocks (put_step). That assumption is wrong for hybrid models’ Mamba/linear-attention KV-cache groups, whose…

  • 6a1acac #49655 — [BUGFIX] Fix log capture in KV test (#49655)
    • 作者: liuzhenwei | +2/-2 | 1 个文件

    UT failed (after https://github.com/vllm-project/vllm/issues/48776) with following msg: Root cause: vLLM’s logger sets propagate=False, so pytest’s caplog fixture sees no records and caplog.text is empty. causing this error. Change: This PR switches to the caplog_vllm fixture, which temporarily enables propagation during the test, ensuring the “attention compute is unchanged” warning i…

  • 972848f #49704 — [Bugfix] Support non-uniform page sizes in KVBlockZeroer (#49704)
    • 作者: Elvir Crnčević | +92/-31 | 3 个文件

    Models with multiple KV cache groups of different dimensions (e.g. GLM-5.2 which has MLA + DSA indexer attention) produce different page sizes per group. KVBlockZeroer previously asserted a single uniform page size, crashing during initialization. Group segments by page size and launch the Triton zeroing kernel once per group. The kernel is unchanged. I vibe coded this to fix #49696, but it’s uncl…

  • 2279575 #47206 — [AMD][Bugfix][EPLB] Fix elastic EP scaling accuracy on ROCm (#47206)
    • 作者: Oxana Korzh | +89/-21 | 2 个文件

    Two fixes for distributed/test_elastic_ep.py, which collapsed GSM8K accuracy at the initial and scaled checkpoints on ROCm: Bug 1 Scale-up produces garbage output on ROCm (accuracy=0) Root cause. New elastic-EP ranks are created with load_dummy_weights=True. On ROCm, initialize_single_dummy_weight() .zero_()s every non-floating-point tensor in model.state_dict(). The FusedMoE expert-topology buffe…

  • e222c33 #49727 — [Bugfix] Register axk1 config to fix A.X-K1 init (#49727)
    • 作者: djramic | +4/-3 | 3 个文件

    AXK1ForCausalLM (skt/A.X-K1) failed to init because AXK1Config was never registered in _CONFIG_REGISTRY, so config parsing fell back to HF AutoConfig which doesn’t know model_type: axk1. Register axk1=“AXK1Config” and fix the “AXK1”→“axk1” casing mismatch in the config and MLA-detection checks. pytest -v -s “models/test_initialization.py::test_can_initialize_large_subset[AXK1ForCausalLM]” ## Test …

  • d02df74 #48973 — [Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
    • 作者: Thomas Fahrner | +47/-5 | 2 个文件

    Data URLs with RFC 2397 parameters between the mediatype and the base64 marker (e.g. data:image/jpeg;charset=utf-8;base64,…) are rejected with an opaque HTTP 501 (“Only base64 data URLs are supported for now”), because MediaConnector._load_data_url splits the header on the first ; and requires the remainder to equal base64. RFC 2397 explicitly allows such parameters, and clients that emit them…

  • a454a1d #49180 — [Bugfix][Benchmarks] Restore –skip-tokenizer-init with custom dataset (#49180)
    • 作者: mgazz | +185/-9 | 3 个文件

    benchmarks with custom datasets cannot use –skip-tokenizer-init. PR #39896 ([MyPy] Fix mypy for vllm/benchmarks) added a blanket assert tokenizer is not None before get_samples() in serve.py to satisfy the TokenizerLike type annotation. This does not allow the execution of models that do not like use a tokenizer like Prithvi. Error: Can be reproduced with the following command:

  • 163ecba #49586 — [Bugfix] Skip linear bias in layerwise reload to avoid corruption (#49586)
    • 作者: li-jinpeng | +3/-0 | 1 个文件

    Linear layers that carry a bias are corrupted when loaded through the online quantization + layerwise-reload path (e.g. online FP8). After loading, bias holds uninitialized memory instead of the checkpoint values. Root cause. The linear base class creates bias after create_weights(). Online quantization calls initialize_online_processing() from inside create_weights(), i.e. before bias exist…

  • 589a5b8 #49221 — [PD][NixlPush][Bugfix] Fix blocking handshake call on writer thread (#49221)
    • 作者: Nicolò Lucchesi | +196/-80 | 4 个文件

    Address S2 here https://github.com/vllm-project/vllm/issues/48633. P->D Handshakes on Push mode are initiated by the writer thread and they are currently blocking (that’s not the case for D->P ones which are correctly async with _send_registration_to_p). This is an issue under load when one or more new D instances are being spun up (eg as a result of auto-scaling), leading to noticeable perfor…

📦 Other

  • 3e74c60 #49587 — [Docs] Use gen-files for generated docs content (#49587)
    • 作者: Harry Mellor | +522/-868 | 34 个文件

    Until now we have had a mixture of methods for generating content in the docs using: - mkdocs hooks to generate hidden *.inc.md files and then including them in near empty files using snippets - pre-commit hooks to generate a whole page that was tracked in git This PR ports both of these methods to use gen-files, which: - Generates virtual files for the CLI reference, just like the API reference -…

  • b9b6306 #39330 — feat[vLLM × v5]: Add audio support for the Transformers backend (#39330)
    • 作者: Harshal Janjani | +680/-150 | 8 个文件

    What does this PR do? → This PR adds support for v5 Transformers audio encoder models in the vLLM Transformers backend. These changes are deliberate and are blocked by this Transformers PR which adds prerequisite compatibility to the supported models for vLLM. Once that PR is merged, this PR will be marked ready for review! → Outlining the design choices of one PR without contex…

  • ca0defa #49726 — Make bare hugging_face imports forbidden (#49726)
    • 作者: Harry Mellor | +199/-106 | 29 个文件

    This PR adds huggingface_hub as a forbidden import so all requests will go through hf_api and hf_fs.

  • fe51457 #48796 — [Core] Keep attention backends eligible for text-only serving of prefix-LM models (#48796)
    • 作者: Tri Vo | +165/-7 | 4 个文件

    Prefix-LM multimodal models set is_mm_prefix_lm=True from static model configuration (e.g. Gemma 4, Gemma 3, Molmo2). This flag is evaluated at server startup and tells backend selection that some multimodal tokens may need bidirectional (prefix) attention, so backends without supports_mm_prefix() are rejected. That constraint is correct while vision inputs can still appear. The problem is…

  • dbcc1cd #49786 — [Model] Remove Ouro (#49786)
    • 作者: Harry Mellor | +1/-451 | 4 个文件

    This model is: - ~1 year old and has been superseded by Plamo3 - See very little usage in vLLM - Has been marked for deletion This PR removes it and adds it to _PREVIOUSLY_SUPPORTED_MODELS.

  • 190be7d #49781 — [Docs] Fix confusing docstring indentation in nemotron_h.py (#49781)
    • 作者: Johnny-Liou | +1/-2 | 1 个文件

    Follow-up to #48018, requested by @hmellor: a docstring it added fails the docs build on main. The wrapped bullet sat at 6 spaces, between griffe’s item indent (4) and continuation indent (8). .readthedocs.yaml sets fail_on_warning: true, so that single warning aborts the build (RTD 33746915 on 866fea2b9). This collapses the bullet onto one line. Docstring text only. No other PR addresses this war…

  • 94682b7 #49753 — [multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
    • 作者: Brandon Pelfrey | +190/-24 | 7 个文件

    Follow up on RFC #30839 and #44465 by making the number of retained PyNvVideoCodec hardware decoders configurable. In frontend heavy workloads (low OSL, large videos), moving to additional decoders can improve throughput by up to 94% in our testing. The decoder pool was previously fixed at one slot, which serialized concurrent GPU video decoding even when the GPU had additional decode capacity. Th…

  • 0ba2aa3 #49242 — Stabilize GPU memory teardown between ROCm CI tests (#49242)
    • 作者: Aarushi Jain | +126/-93 | 3 个文件
    • Harden ROCm GPU memory cleanup between heavy back-to-back CI tests (LoRA Qwen-VL and Nixl connector PD accuracy). - Default GPU memory settle waits on ROCm in pytest via conftest, with explicit teardown for heavy Qwen-VL LoRA tests and shell-driven Nixl accuracy runs. ## Problem Build 11036 failures on mi300-1-lora-1 and mi300-4-distributed-nixlconnector-pd-accuracy-4-gpus were caused by residua…
  • 318b527 #49419 — [XPU] add warning for xpu graph limitations (#49419)
    • 作者: liuzhenwei | +10/-0 | 1 个文件

    Add a warning when XPU Graph is enabled to highlight its experimental status and known limitations: - only single-GPU execution is supported - FLASH_ATTN supports PIECEWISE mode only; use TRITON_ATTN for FULL mode - graph capture may use significantly more memory than CUDA ## Test Result —

  • caa9cad #49737 — [ROCm][Docker] Drop MORI_GPU_ARCHS so MoRI autodetects the device arch (#49737)
    • 作者: Rohan Potdar | +1/-1 | 1 个文件

    Remove ENV MORI_GPU_ARCHS=gfx942;gfx950 from Dockerfile.rocm_base. MoRI JIT-compiles at runtime and autodetects the GPU arch, but mori.jit.config.detect_gpu_arch() treats MORI_GPU_ARCHS as an override that returns the first entry of its _SUPPORTED_ARCHS (gfx942). So on a gfx950 (MI350) host it builds a gfx942 code object → device kernel image is invalid, crashing MoRI at startup. #44374 fixes this…

  • 84d26b9 #49729 — [Model] Remove Plamo2 (#49729)
    • 作者: Harry Mellor | +6/-1014 | 13 个文件

    This model is: - ~1 year old and has been superseded by Plamo3 - See very little usage in vLLM - Is only downloaded by vLLM CI so the download stats you see on HF are misleading This PR removes it and adds it to _PREVIOUSLY_SUPPORTED_MODELS.

  • 5d8e90a #45321 — [WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321)
    • 作者: Tyler Michael Smith | +338/-78 | 13 个文件

    Update DeepEP to https://github.com/deepseek-ai/DeepEP/commit/d4f41e4e93602a15e95f55f6ee8df8f1aaa0e4bb so that we can pick up DeepEPv2. Missed this in https://github.com/vllm-project/vllm/pull/41183 Ideally we’d have a single source of truth for the commit across both the install script and the Dockerfile but I don’t see a sane way to do it at the moment PyTorch ships an earlier version of NCCL. P…

  • c064fa5 #49484 — Fix GLM-4.1V video placeholder token ID handling. (#49484)
    • 作者: Aarushi Jain | +24/-43 | 2 个文件

    Fix GLM-4.1V video frame placeholder token IDs to match HuggingFace Glm4vProcessor, resolving generation parity failures in ROCm multimodal CI. ## Problem mi300_1: Multi-Modal Models (Extended Generation 3) failed on: test_custom_inputs_models[glm4_1v-video-test_case7] At video scale 0.2, vLLM described different video content than HF (e.g. “book on floor” vs “laptop on bed”) because frame placeho…

  • 8c13ee5 #49387 — Add sm_107 for Rubin (#49387)
    • 作者: Tyler Michael Smith | +60/-13 | 6 个文件
    • enable CUDA 13.4’s native SM107 target for Vera Rubin - allow SM107 to reuse compatible SM100-family kernels across the CUDA, DeepGEMM, FlashMLA, and QuTLASS builds - resolve exact architecture matches before family fallbacks, preventing 10.0f from masking 10.7f This follows PyTorch #190654. I also checked its review feedback and mirrored the corrected boundaries here: CUDA 13.4 is included with…
  • 453f017 #49124 — [UX] Improve data-parallel launch validation (#49124)
    • 作者: Taneem Ibrahim | +52/-23 | 1 个文件

    [UX] Raise actionable errors for invalid data-parallel launch options Replacing data-parallel launch combination assertions with plain ValueError at their existing validation stages, using a consistent prefix, the relevant CLI flags and values, and a corrective action. It also validates the lower bound of –node-rank before constructing ParallelConfig. This does not duplicate #45150: that PR det…

  • 7b40fb9 #49247 — [UX] Reject incompatible nested runtime overrides (#49247)
    • 作者: Taneem Ibrahim | +56/-23 | 4 个文件

    Runtime configuration overrides validated nested values only by checking whether they were any dataclass, so an unrelated dataclass could silently replace the expected nested config type. Other invalid overrides used assertions that disappeared under python -O, producing low-level attribute errors instead. This change accepts only mappings or instances of the expected nested dataclass, reports the…

  • 8eac21a #49673 — [ROCM] Fix AITER Fused AllReduce RMSNorm for Transformers Backend (#49673)
    • 作者: BadrBasowid | +18/-4 | 2 个文件

    Fix AITER fused AllReduce+RMSNorm dispatch for 3D Transformers inputs by calculating tokens across all leading dimensions. The previous logic incorrectly selected the 1-stage kernel, causing: Also adds Transformers backend coverage to the TP2 AR+RMS fusion test. - lm_eval Qwen/Qwen3-32B-FP8 with transformers backend and TP 2 - pytest tests/compile/fusions_e2e/test_tp2_ar_rms.py ## Test Result - lm…

  • 833483f #48218 — Encoder cache extension hooks (#48218)
    • 作者: hotTea | +106/-15 | 8 个文件

    This PR adds extension hooks for the V1 encoder cache lifecycle so downstream or out-of-tree implementations can customize encoder cache behavior without changing the default vLLM path. Concretely, this PR: - Adds EncoderCacheManagerConfig and EncoderCacheManagerMetadata. - Allows VllmConfig to provide a custom scheduler-side encoder cache manager class via ec_manager_config.encoder_cache_manager_…

  • 5c5434e #49693 — Remove Quantization test parallelism (#49693)
    • 作者: Kevin H. Luu | +2/-4 | 1 个文件
    • Remove parallelism: 8 from the H200 Quantization test. - Remove parallelism: 3 from the H200 Quantized Models Test. - Remove the shard arguments that depend on BUILDKITE_PARALLEL_JOB and BUILDKITE_PARALLEL_JOB_COUNT. This reduces fan-out pressure on the h200_35gb queue. The tests still run in full, but each step runs as one job instead of sharded parallel jobs. ## Why this is not duplicate work …

🧪 CI/Tests

  • 9a50464 #49800 — [CI] Stop flaky test from downloading model every time (#49800)
    • 作者: Harry Mellor | +16/-20 | 1 个文件

    Presumably the original intent of force downloading to a tmpdir was so the model was deleted after the test. This is not necessary because we have a HF cache.

  • aaaeda9 #49770 — [CI] fix compile test | refactor VLLM_DISABLE_COMPILE_CACHE for tests (#49770)
    • 作者: Divakar Verma | +23/-14 | 4 个文件

    Fixes tests/compile/test_compile_ranges.py tests that flakily failed on persistent CI agents when a warm vLLM compile cache caused graphs to load from disk, skipping the custom post-grad passes the tests count. Adds a new disable_vllm_compile_cache fixture (fresh Inductor cache + VLLM_DISABLE_COMPILE_CACHE=1) so compilation always runs, and migrates the compile-ranges and attention-fusion tests to…

  • d9cd774 #49763 — [ROCm][CI] Force native compile caches onto local disk (#49763)
    • 作者: Aarushi Jain | +6/-4 | 1 个文件
  • 33c4f35 #49739 — [ROCm][CI] Wait for ROCm VRAM to settle between compiled and eager LL… (#49739)
    • 作者: Aarushi Jain | +2/-0 | 1 个文件

    Wait for ROCm VRAM to settle between compiled and eager LLM runs in test_dynamic_shapes_compilation, fixing mi300_1: PyTorch Compilation Unit Tests. ## Problem Build 11128 failed on: test_dynamic_shapes_compilation[False-True-0-backed-Qwen/Qwen2-7B-Instruct] After the compiled Qwen2-7B run, starting an eager LLM in the same process hit a GPU fault in V2 model runner (buffer_utils.py → states.py → …

  • 7513d07 #49733 — [ROCm][CI] Fix XPASS(strict) on mixed audio embeds test (#49733)
    • 作者: djramic | +2/-0 | 1 个文件

    The text-then-audio_embeds test is expected to fail with PyTorch 2.12 and newer, so it is marked xfail(strict=True). Current ROCm CI still uses PyTorch 2.11, where the test passes; an unconditional strict xfail therefore becomes XPASS(strict) and fails the job. Apply the strict xfail only when is_torch_equal_or_newer(“2.12.0”). PyTorch versions below 2.12 continue to run the test normally, while a…

  • 89f6aa3 #49734 — [KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flaky api-server test (#49734)
    • 作者: Harry Mellor | +109/-5 | 5 个文件

    Fixes two environment-dependent CI failures observed in build 79959 (also intermittently elsewhere). Both are real bugs, not just test noise. ### 1. FS KV-offload tier silently fails without O_DIRECT (15 failures) vllm/v1/kv_offload/tiering/fs/io.py always opened block files with O_DIRECT. Several filesystems reject O_DIRECT with EINVAL: - overlayfs — backs /tmp inside many CI Docker container…

  • 9e6746b #49749 — [CI] Stabilize memory-sensitive compile and structured output tests (#49749)
    • 作者: Jiangyun Zhu | +4/-0 | 2 个文件

    Stabilize two memory-sensitive CI tests: - Give BAAI/bge-multilingual-gemma2 more of the L4’s memory budget in the compile-correctness test. After the PyTorch 2.13 upgrade, the VLLM_COMPILE + inductor path has about -0.45 GiB available for KV cache at the default utilization, versus 1.21 GiB in the earlier nightly. Setting –gpu-memory-utilization 0.98 keeps this test configuration viable without …

  • 7e51939 #49508 — [CI] Avoid unnecessary Hugging Face metadata requests (#49508)
    • 作者: Andreas Karatzas | +7/-4 | 2 个文件

    Motivation: Engine (1 GPU) and Distributed Torchrun + Shutdown Tests (2 GPUs) The Engine job timed out in an unnecessary image-processor lookup for a text-only model, while Torchrun shutdown waited on an optional remote safetensors-index lookup after Hub throttling. ModelConfig now fetches image metadata only after multimodal support is established. The default loader also skips the index probe wh…

  • 9863102 #49509 — [CI] Reuse loaded config for cached tokenizer (#49509)
    • 作者: Andreas Karatzas | +9/-0 | 2 个文件

    Motivation: Rust Frontend OpenAI Coverage CachedHfTokenizer loaded model configuration a second time after its caller had already loaded it, so concurrent cache replacement could break the redundant read. This change passes the existing config into the cached tokenizer and avoids that second Hub/cache access. Callers that do not provide a config retain the existing fallback behavior. It stabilizes…

  • 4179806 #49257 — [CI][AMD] Deprecate DinD for MI355 tests (#49257)
    • 作者: Andreas Karatzas | +39/-0 | 1 个文件

    This deprecates DinD for all MI355 CI jobs and runs them directly in AMD Kubernetes pods; no open PR duplicates this change. Validation: pre-commit run –files .buildkite/test-amd.yaml and rendered-pipeline checks passed; AI assistance was used.

⚡ Performance

  • a82f1b3 #48017 — [Perf][V1] Skip LRU hash-split in free_blocks when prefix caching is off (#48017)
    • 作者: Agata Dobrzyniewicz | +5/-3 | 1 个文件

    BlockPool.free_blocks() runs once per engine step. Since #42656 it always partitions freed blocks into blocks_with_hash / blocks_without_hash (two list builds + a per-block block_hash is None branch) and issues two queue ops (prepend_n + append_n). When prefix caching is disabled (enable_caching=False), no block ever carries a hash, so blocks_with_hash is always empty and the partition is …

  • 213f681 #49768 — Revert “[Perf][GLM-5.2] Blackwell decode optimizations” (#49768)
    • 作者: Woosuk Kwon | +138/-2139 | 29 个文件

    Reverts vllm-project/vllm#48597

🖥️ Kernel

  • 866fea2 #48018 — [Kernel] ReplaySSM: cache SSM inputs for faster Mamba2 standard decode (#48018)
    • 作者: Johnny-Liou | +2954/-31 | 23 个文件

    Background This is the first sub-PR split out from the large draft PR #47576, which presents the full ReplaySSM design across Mamba2 and Gated DeltaNet for both standard and speculative decode. To keep review tractable, this PR lands only the first stage, Mamba2 standard decode. ReplaySSM caches recent SSM inputs instead of writing the recurrent state back to HBM on every step. It is a collabor…