共 39 个 commit，涉及 194 个文件，+8011/-2798 行变动。

概要

统计项 数值
Commit 数 39
变更文件 194
新增行数 +8011
删除行数 -2798

Commit 列表

🐛 Bug Fix

  • 7154856 #47791 — [Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)
    • 作者: Daniel Socek | +5/-1 | 1 个文件

    PR Purpose Fixes IndexError in kv_postprocess_layout_on_receive() for 5D blocks-first KV caches. kv_postprocess_layout_on_receive() (added in #30275) permutes a received KV cache from HND to NHD layout. It assumes the cache is 4D. But since #42095 is recently merged, non-MLA backends allocate a 5D cache (num_blocks, 2, block_size, H, D). This causes the following error: (This was also flagged a…

  • 0da6e7f #49134 — [Bugfix] Reject contradictory custom-op directives (#49134)
    • 作者: Taneem Ibrahim | +59/-11 | 3 个文件

    CompilationConfig.custom_ops accepted an operation that was both explicitly enabled and disabled, deferring the conflict to a runtime assert in CustomOp.enabled(). The existing all/none conflict check was also an assert; under python -O, both checks disappeared and the invalid configuration ran with the disable directive silently. Fusion settings could also append an enable directive after validat…

  • 5559679 #49052 — [Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)
    • 作者: coltonottley | +1/-0 | 1 个文件

    Fixes #48959. CPU KV offload can fatally abort EngineCore on a valid unaligned sliding-window external load because the scheduler sanity-checks pending physical GPU blocks using a chunk-aligned footprint. For the production geometry: - sliding window: 4,096 tokens; - GPU block: 32 tokens; - offload chunk: 256 tokens (blocks_per_chunk=8); - cached prefix: 98,012 tokens; - valid pending span: 129 GP…

  • 7a29a3c #49440 — [Bugfix][KV Offload] Namespace persistent cache by model runner (#49440)
    • 作者: Jonguk Cheong | +22/-1 | 2 个文件

    V1 and V2 model runners can currently share a persistent native KV-offload namespace at single-rank parallelism even though their serialized cache layouts are incompatible. V1 uses a packed cross-layer representation while V2 uses per-layer tensors. The tested layouts have the same byte count, so size checks cannot reject V1 pages loaded by V2 and inference silently returns incorrect output. This …

  • 1240c74 #49372 — [Bugfix] Respect declared attention contract for ColQwen3.5 retrievers (#49372)
    • 作者: Athrael Soju | +135/-16 | 2 个文件

    Fix the attention mismatch between vLLM and released ColQwen3.5 retrieval checkpoints. Assisted-by: OpenAI Codex #46108 made ColQwen3.5 pooling models unconditionally bidirectional. The VultronRetriever checkpoints were trained and evaluated with causal attention, while the older athrael-soju/colqwen3.5-4.5B-v3 checkpoint was trained with bidirectional attention. Applying one attention mode to eve…

  • 48ebd6f #49226 — [Bugfix][KVConnector] Disable cross-layer KV blocks for per-token-head quant (#49226)
    • 作者: achyuthan.s | +4/-0 | 1 个文件

    Fixes #48412. Combining OffloadingConnector with a per-token-head quantized KV cache (fp8_per_token_head / int8_per_token_head / int4_per_token_head) corrupts generations on the first forward pass, before any offloaded block is ever restored. ### Root cause OffloadingConnector.prefer_cross_layer_blocks is hardcoded True. For single-group models on backends with indexes_kv_by_block_stride (e.g….

  • d30b1ec #49671 — [Bugfix][KV Offloading] Defer request finalization until final store (#49671)
    • 作者: Rui Yin | +111/-58 | 3 个文件

    Fixes #49635. OffloadingConnectorScheduler.request_finished() currently notifies the manager before the scheduler prepares a finished request’s deferred final store. TieringOffloadingManager can then delete its request state, and the next prepare_store() crashes EngineCore with KeyError. This change: - builds all final store jobs before signaling manager.on_request_finished(); - tracks whether the…

  • 9321aff #49805 — [Bugfix] Wait for the linear bias before layerwise online processing (#49805)
    • 作者: Harry Mellor | +67/-10 | 4 个文件

    Fixes the test_fp8.py::test_online_quantization[-True-] failures currently on main (e.g. build 80095, Quantization job), which fail with: Root cause. #49586 added “bias” to SKIP_TENSORS, but that set is consumed for two unrelated purposes: 1. keeping tensors off the meta device (capture_layer_to_meta / restore_layer_on_meta / materialize_layer) — what #49586 needed, and 2. deciding which ten…

  • 33ef67e #49403 — [BugFix] Increase the max supported duration for MOSS-TD (#49403)
    • 作者: Canlin Guo | +2/-3 | 1 个文件

    Previously, MOSS-TD incorrectly treated Whisper’s default 30-second chunk length as the maximum duration of a complete audio item. As a result, it reported only about 375 embedding tokens as max_tokens_per_mm_item. Consequently, the encoder cache size usually fell back to max_num_batched_tokens (for example, 8,192). Longer audio producing more embedding tokens than this cache capacity was rejected…

  • d1a8ba6 #49149 — [Bugfix][MiniMax-M3] Fix token-major top-k buffer handling in Triton … (#49149)
    • 作者: rongfu.leng | +13/-3 | 2 个文件

    Fixs: https://github.com/vllm-project/vllm/issues/49147 1. In the vllm/models/minimax_m3/nvidia/model.py+790, model assign topk_indices_buffer shape is [token, head, topk]. 2. But Triton indexer vllm/models/minimax_m3/common/indexer.py+423 still use [head, token, topk] this shape, 3. Triton sparse attention also reads data using the [head, token, topk] layout. vllm/models/minimax_m3/common/sparse_…

  • 1423569 #48852 — [Bugfix][Tool Parser] Fix dropped streaming arguments in Jamba and InternLM2 parsers (#48852)
    • 作者: mosya415 | +99/-8 | 4 个文件

    When streaming tool calls, the first time a call’s arguments become non-empty, the Jamba and InternLM2 parsers computed the streamed delta by locating the raw model delta text inside json.dumps(arguments) with str.index: The raw delta need not appear verbatim in the re-serialized JSON. It can carry structural text such as , “arguments”: {…}, or the model may emit compact JSON whose spacing diffe…

  • 0b1a8bb #49802 — [Bugfix][CI] Fix stale Mooncake lookup expectation broken by a merge race (#49802)
    • 作者: Harry Mellor | +4/-2 | 1 个文件

    tests/v1/kv_connector/unit/test_mooncake_store_worker.py::test_lookup_rejects_boundary_missing_one_mamba_shard fails on main, breaking the V1 Core + KV + Metrics step for every unrelated PR (e.g. build 80149): This is a merge race between two Mooncake PRs, not a functional regression: - #49481 (a76df87) changed MooncakeStoreWorker.lookup so a hit covering the whole request is re-derived below the …

📦 Other

  • 3f1d409 #49285 — [KV Offload] Fix num_tokens_after_batch for different termination types (#49285)
    • 作者: AlexHuang | +4/-2 | 1 个文件

    Fix incorrect num_tokens_after_batch computation in offloading scheduler by using num_computed_tokens for aborted requests (where partial KV data may exist) instead of always using num_tokens. ## Problem The original code used num_tokens for all finished requests: For aborted requests, num_computed_tokens < num_tokens (partial KV data). Using num_tokens would attempt to store chunks without actual…

  • da3a252 #48021 — [KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)
    • 作者: liranschour | +2673/-396 | 11 个文件

    Adds a generic peer-to-peer (P2P) secondary tier for KV-cache offloading, letting a vLLM engine both fetch KV blocks from and serve KV blocks to remote peers over a NIXL data transport with a ZMQ control channel. It generalizes the existing prefill/decode (PD) path into a symmetric P2P model: a consumer issues a LookupMsg for the blocks it needs; the producer answers against its local tier…

  • 21fd9e8 #45429 — [Model] Support top_k and top_p sampling for DiffusionGemma (#45429)
    • 作者: Guan-Ming Chiu | +18/-2 | 1 个文件

    Support per-request top_k/top_p for DiffusionGemma by filtering logits before the compiled denoise step, mirroring the AR sampler. Masked tokens become -inf; committed argmax (top-1) unaffected. Padding switched to masked_fill_ to avoid -inf * 0 = NaN on truncated canvases. Related: #45163. Validation relaxation lives in the companion PR. GPU behavioral check: per-request filtering, default fast p…

  • 7eca0e1 #48906 — [KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
    • 作者: Chang Guo | +1135/-498 | 14 个文件

    This implements the first shared CPU-region replica-reduction path from #47929. For pure MLA tensor parallelism, each TP rank holds a replica of the latent KV payload. This duplicate host-copy path is present in both V1 and V2: both runners hand per-rank canonical MLA tensors to the same offloading connector, which previously reserved and stored one host slot per rank. The runners organize their G…

  • dbd80cc #49777 — [UX] DCP Topology Validation (#49777)
    • 作者: Taneem Ibrahim | +23/-17 | 1 个文件

    ModelConfig.verify_with_parallel_config() validates three model-specific decode-context-parallel constraints for non-MLA GQA/MQA models with assert. Invalid user-provided –tensor-parallel-size and –decode-context-parallel-size combinations therefore raise AssertionError, and the validation disappears entirely under python -O, allowing invalid topologies into engine initialization. This replaces …

  • 70009fb #46837 — [MM][CG] Support ViT CUDA Graph for Gemma-4 (#46837)
    • 作者: anthonsu | +476/-1 | 4 个文件

    This PR implements static CUDA graph support for the Gemma-4 vision encoder. This PR introduces full SupportsEncoderCudaGraph support for Gemma4ForConditionalGeneration, enabling 100% static compilation for the vision encoder. The core optimization bypasses dynamic slicing in the pooler. Instead of relying on data-dependent control flow, the host calculates static pointwise mapping indices (gather…

  • ee1d996 #49814 — [Build] Fix for DeepEP manylinux pidfd sycall usage (#49814)
    • 作者: Tyler Michael Smith | +27/-0 | 1 个文件

    Fix the release pipeline. I think this comment explains the issue pretty well

  • 26d725c #49803 — [Model] Add VaultGemma via Transformers modeling backend (#49803)
    • 作者: Harry Mellor | +8/-17 | 6 个文件
    • Fixes bug in EngineArgs where full attention models were being treated as sliding if both of these conditions were true: - Has layer_types with all full_attention (therefore is_interleaved=False) - Has sliding_window: int present in the config - Adds VaultGemma (which exhibited this) to the model registry via the Transformers modeling backend Closes https://github.com/vllm-project/vllm/issues/49…
  • 2e0da24 #45117 — Mergify message not on cancelled (#45117)
    • 作者: Harry Mellor | +1/-0 | 1 个文件

    This feature was added in response to https://github.com/Mergifyio/mergify/issues/5172. This should remove any spam comments caused by cancelled (i.e. superseded by GitHub Actions concurrency rules) jobs

  • 3e74c60 #49587 — [Docs] Use gen-files for generated docs content (#49587)
    • 作者: Harry Mellor | +522/-868 | 34 个文件

    Until now we have had a mixture of methods for generating content in the docs using: - mkdocs hooks to generate hidden *.inc.md files and then including them in near empty files using snippets - pre-commit hooks to generate a whole page that was tracked in git This PR ports both of these methods to use gen-files, which: - Generates virtual files for the CLI reference, just like the API reference -…

  • b9b6306 #39330 — feat[vLLM × v5]: Add audio support for the Transformers backend (#39330)
    • 作者: Harshal Janjani | +680/-150 | 8 个文件

    What does this PR do? → This PR adds support for v5 Transformers audio encoder models in the vLLM Transformers backend. These changes are deliberate and are blocked by this Transformers PR which adds prerequisite compatibility to the supported models for vLLM. Once that PR is merged, this PR will be marked ready for review! → Outlining the design choices of one PR without contex…

  • ca0defa #49726 — Make bare hugging_face imports forbidden (#49726)
    • 作者: Harry Mellor | +199/-106 | 29 个文件

    This PR adds huggingface_hub as a forbidden import so all requests will go through hf_api and hf_fs.

  • fe51457 #48796 — [Core] Keep attention backends eligible for text-only serving of prefix-LM models (#48796)
    • 作者: Tri Vo | +165/-7 | 4 个文件

    Prefix-LM multimodal models set is_mm_prefix_lm=True from static model configuration (e.g. Gemma 4, Gemma 3, Molmo2). This flag is evaluated at server startup and tells backend selection that some multimodal tokens may need bidirectional (prefix) attention, so backends without supports_mm_prefix() are rejected. That constraint is correct while vision inputs can still appear. The problem is…

  • dbcc1cd #49786 — [Model] Remove Ouro (#49786)
    • 作者: Harry Mellor | +1/-451 | 4 个文件

    This model is: - ~1 year old and has been superseded by Plamo3 - See very little usage in vLLM - Has been marked for deletion This PR removes it and adds it to _PREVIOUSLY_SUPPORTED_MODELS.

  • 190be7d #49781 — [Docs] Fix confusing docstring indentation in nemotron_h.py (#49781)
    • 作者: Johnny-Liou | +1/-2 | 1 个文件

    Follow-up to #48018, requested by @hmellor: a docstring it added fails the docs build on main. The wrapped bullet sat at 6 spaces, between griffe’s item indent (4) and continuation indent (8). .readthedocs.yaml sets fail_on_warning: true, so that single warning aborts the build (RTD 33746915 on 866fea2b9). This collapses the bullet onto one line. Docstring text only. No other PR addresses this war…

⚡ Performance

  • 8d28b48 #49524 — [Perf] Isolate MM preprocessing on its own executor (#49524)
    • 作者: Guan-Ming Chiu | +31/-20 | 3 个文件
    • One in-flight image/video preprocessing currently blocks every concurrent request’s tokenization for its full duration (1s+ for large images), because both share a single-worker executor - Give MM preprocessing its own executor so the two never contend; composes with #44786 - Narrow the #38418 guard to pooling models, whose preprocessing still runs on the renderer workers - pytest tests/renderer…
  • 30b0714 #49531 — [Perf] DeepSeek-OCR-2 TTFT Optimize (#49531)
    • 作者: RED | +32/-36 | 1 个文件

    create_custom_4d_mask in deepseek-ai/DeepSeek-OCR-2 was CPU-bound, ## What changed Optimize create_custom_4d_mask to computes the mask in one shot and caches it: - Mask is batch-invariant (token_type_ids is the same pattern for every sample), so we compute one [1, 1, S, S] mask and broadcast over the batch. - Cached by (sequence_length, dtype, device); ≤2 tiny entries (<1 MB total). ## Benchmark H…

  • 7fe6d3c #48763 — [Perf] Fix moe reduce_scatter perf regression by removing additional comm, 5% E2E throughput gain back. (#48763)
    • 作者: Wentao Ye | +13/-37 | 2 个文件

    Part of https://github.com/vllm-project/vllm/issues/46654 Following up PR for https://github.com/vllm-project/vllm/pull/48036 and alternative for https://github.com/vllm-project/vllm/pull/48657 Originally: So all-gather(2H) + all-reduce(H) -> all-gather(H) vllm serve zai-org/GLM-5.2-FP8 –kv-cache-dtype fp8_e4m3 –enable-expert-parallel –tensor-parallel-size 4 –tool-call-parser glm47 –enable-au…

  • a82f1b3 #48017 — [Perf][V1] Skip LRU hash-split in free_blocks when prefix caching is off (#48017)
    • 作者: Agata Dobrzyniewicz | +5/-3 | 1 个文件

    BlockPool.free_blocks() runs once per engine step. Since #42656 it always partitions freed blocks into blocks_with_hash / blocks_without_hash (two list builds + a per-block block_hash is None branch) and issues two queue ops (prepend_n + append_n). When prefix caching is disabled (enable_caching=False), no block ever carries a hash, so blocks_with_hash is always empty and the partition is …

🧪 CI/Tests

  • 0164022 #49853 — [CI] Fix speech correctness check rejecting improved WER (#49853)
    • 作者: Taneem Ibrahim | +6/-1 | 1 个文件

    The short-form speech correctness test uses a symmetric closeness check for WER, even though lower WER is better. In Buildkite build #80224, Cohere produced a WER of 11.698929 versus the 11.92 baseline. The result was better, but it fell just below the symmetric accepted interval of [11.7008, 12.1392] and failed CI. This change makes the short-form check one-sided while preserving the existing upp…

  • b153ae6 #49651 — [XPU][CI] add heterogeneous TP UT (#49651)
    • 作者: liuzhenwei | +7/-4 | 1 个文件

    Add heterogeneous TP ut, which requires 4 cards of Intel GPU. ## Test Result —

  • 7a6a5b3 #49773 — [CI] Compute speech WER directly with jiwer (#49773)
    • 作者: Chang Guo | +3/-5 | 1 个文件

    Closes #49771. The speech correctness tests use evaluate.load(“wer”). With the current test dependencies, evaluate==0.4.3 calls the removed huggingface_hub.hf_api.HfFolder API and fails after inference completes. This change computes WER directly with jiwer, which is already a test dependency and the backend used by the Hugging Face WER metric. It preserves the existing per-sample alignment and gl…

  • 6b0103d #49822 — [CI] Stabilize Pooling Rerank Equivalence Test (#49822)
    • 作者: Taneem Ibrahim | +4/-2 | 1 个文件

    This fixes a recurring numerical flake in test_rerank_api_instruction_field_matches_chat_template_kwargs, observed on main in Buildkite failures in #80147, #80164, and #80202. ## Reproducer ### Output on main: ### Output on this branch:

  • 9a50464 #49800 — [CI] Stop flaky test from downloading model every time (#49800)
    • 作者: Harry Mellor | +16/-20 | 1 个文件

    Presumably the original intent of force downloading to a tmpdir was so the model was deleted after the test. This is not necessary because we have a HF cache.

📖 Documentation

  • 2e860de #49782 — [Doc] Add compile cache volume example to the Docker deployment page (#49782)
    • 作者: Nils Matteson | +20/-0 | 1 个文件

    The Docker page shows how to persist the Hugging Face cache across containers but has no example for the compile cache, so a fresh container recompiles the model’s torch.compile artifacts even when the weights are mounted. The mechanism is already documented in Faster Startup (#47374): VLLM_CACHE_ROOT can be persisted or baked into an image. This adds the concrete docker recipe next to the existin…

🖥️ Kernel

  • 0111002 #46340 — [Kernel] TD operand loads for batched MoE GEMM (moe_mmk) on XPU (#46340)
    • 作者: Lena Onyshchenko | +331/-33 | 9 个文件

    Purpose Add a Tensor-Descriptor (TD) operand-load path to moe_mmk / expert_triton_kernel (the batched MoE expert GEMM) and enable it on XPU. On XPU, masked tl.load feeding tl.dot bypasses the Xe XMX 2D-block-read path; loading both operands via tl.make_tensor_descriptor restores it. Output is bit-identical to the non-TD path. The batched kernel is the one used by *low-latency Expert-Parallel…

✨ New Feature

  • 0b0bd2b #44428 — [Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428)
    • 作者: fangyuchu | +1088/-7 | 27 个文件

    Add fault tolerance (FT) framework for DP+EP (Data Parallelism + Expert Parallelism) MoE deployments. When one DP rank dies, the EP all2all operation on surviving ranks blocks indefinitely, causing a full cluster hang. This framework detects faults, aborts in-flight requests, and allows an external orchestrator to trigger coordinated recovery via a REST API. Key design: - Sentinel pattern: All…