vLLM Daily — 2026-06-25

共 78 个 commit，涉及 382 个文件，+14959/-6491 行变动。

概要

统计项	数值
Commit 数	78
变更文件	382
新增行数	+14959
删除行数	-6491

Commit 列表

📦 Other

• d490b98 #45237 — [Core] Avoid mixed length specdec batches via padding (#45237)
  • 作者: Yiliu Dong | +120/-0 | 2 个文件

  In P/D disaggregation with speculative decoding enabled, a request that arrives at the decode node via KVConnector has already had most of its prompt KV transferred and normally has only the residual decode-side token left to compute. Existing decode requests schedule 1 + N tokens with MTP/EAGLE/…, while the newly transferred request schedules only 1 token. That creates a non-uniform batch shape…

• 1744adc #45666 — [ROCM] [Communication] Add INT3 quantization method for quickreduce (#45666)
  • 作者: haoyangli0109 | +264/-20 | 7 个文件

  1.According to our kernel experiments, INT3 demonstrated better performance under TP2 (see figure below), but under TP=4 and TP=8 conditions, the performance of INT3 and INT4 was very close. This is likely because, as TP increases, the number of GPUs that need to communicate with each other grows exponentially, thereby requiring more synchronization and scheduling time; In this scenario, synchroni…

• 6f3da46 #46093 — [Pooling] Fix Cohere embed billed image token accounting for mixed-content inputs (#46093)
  • 作者: Taneem Ibrahim | +6/-1 | 1 个文件

  This PR fixes a small but user-visible accounting bug in the Cohere-compatible /v2/embed response. Cohere embed requests can provide image input in more than one shape. The existing process already counted top-level images as image input, but it missed images provided inside mixed-content inputs[].content[] items: On current main, this request can succeed but the response metadata reports the prom…

• 9bfd878 #43461 — [MoE] [MoE Refactor] Add moe kernel oracle abc 37753 (#43461)
  • 作者: Qiuyang Yue | +280/-6 | 5 个文件

  First PR of the series suggested by @robertgshaw2-redhat in #37776: introduces the MoEKernelOracle abstract base class and migrates the unquantized oracle as the reference implementation. Addresses #37753 in part. The remaining oracles (fp8, nvfp4, mxfp4, mxfp8, int8, int_wna16 — now 7 oracles, up from the 4 enumerated in the issue body when it was filed) are left for follow-up PRs. ## Approach Th…

• 15be787 #45019 — [NIXL][Mamba] Add Mamba1 support to NIXL P/D disaggregation (#45019)
  • 作者: Asaf Gardin | +104/-35 | 5 个文件

  Add Mamba1 hybrid-model support (e.g. Jamba) to the NIXL connector’s conv-state transfer for prefill/decode disaggregation, extending the existing Mamba2/GDN path (#41869, #37635). ## Test Result Successfully passed - pytest tests/v1/kv_connector/unit/test_nixl_connector_hma.py -k derive_mamba_conv_split -v ### Accuracy Test ## Accuracy Results Model: ai21labs/AI21-Jamba2-Mini (Mamba1) **Bench…

• 9222148 #46702 — [CPU][CI/Build] Allow more CPU CI agents (#46702)
  • 作者: Li, Jiang | +2/-1 | 1 个文件

  • Add a identifier to support more CPU CI agents on a node. ## Test Result —

• 638b1a9 #45269 — [CPU][RISC-V] Add RVV path for W4A8 INT4 GEMM (#45269)
  • 作者: wcy | +157/-7 | 5 个文件

  This PR adds an RVV backend for the CPU INT4 W4A8 GEMM operator on RISC-V and wires the existing int4_scaled_mm_cpu path to dispatch to it. 1. Add an RVV backend for CPU W4A8 GEMM The CPU W4A8 operator already has an optimized AVX512 path for x86. This PR adds the corresponding RISC-V implementation using RVV, while keeping the existing W4A8 packed weight layout, scale/zero-point buffers, and 32-c…

• 72adb20 #46605 — [Model] Remove AquilaForCausalLM, AquilaModel (#46605)
  • 作者: Tiezhen WANG | +2/-7 | 4 个文件

  Removing AquilaForCausalLM and AquilaModel (BAAI Aquila/AquilaChat series) as they’re obsolete now. Confirmed with BAAI. Adds both architectures to _PREVIOUSLY_SUPPORTED_MODELS pointing to vLLM v0.23.0. ## Test Result —

• 2396d91 #44029 — [CPU][Spec Decode] Enable DFlash SD for CPU (#44029)
  • 作者: guybd | +266/-8 | 8 个文件

  Enable DFlash speculative decoding on the vLLM CPU backend. Srart server: benchmark: ## Test Result Without this change, DFlash speculative decoding cannot run on the CPU backend because the DFlash input expansion path depends on GPU/Triton behavior and the CPU attention backend does not expose the hooks needed to precompute/store DFlash context KV cache. With this change, DFlash starts successful…

• 9b215ae #44610 — [Rust Frontend] Forward VLLM_ENGINE_READY_TIMEOUT_S via --args-json (#44610)
  • 作者: Kai | +16/-12 | 1 个文件

  When using the Rust frontend, setting export VLLM_ENGINE_READY_TIMEOUT_S=1800 has no effect. Models that require extended JIT compilation during startup (e.g., DeepSeek with DeepGEMM) can easily exceed the default 600 s engine-ready timeout window — even when the user has explicitly raised the limit via VLLM_ENGINE_READY_TIMEOUT_S. The Rust frontend silently ignores this env var, causing the serve…

• 4d3b4b9 #46584 — [Rust Frontend] Make ToolParserOutput a seq of ToolParserEvent to preserve order (#46584)
  • 作者: Bugen Zhao | +1153/-818 | 32 个文件

  Previously ToolParserOutput is a flattened representation like: So if there are normal texts and calls in a single output simultaneously, we cannot tell their order, especially if some text appears before calls while other text appears after calls. This PR refactors ToolParserOutput into a Vec, where ToolParserEvent is Text | ToolCallDelta, so the order is preserved. For compatibi…

• 36fd7e8 #46394 — [SimpleCPUOffloadConnector] Fix remaining global→block conversions under PCP/DCP (#46394)
  • 作者: Jeff (Junze) Ma | +94/-6 | 2 个文件

  PR #39831 ("[SimpleCPUOffloadConnector] PCP + DCP support", merged as c88d3d477) added context-parallel support by scaling KV block sizes by cp_world_size = pcp_world_size * dcp_world_size. Under CP, each physical block holds block_size per-rank tokens but represents block_size * cp_world_size global tokens, while the scheduler counts global tokens. Every place that converts a global-token c…

• 1aad125 #46202 — [CPU] Enable chunked prefill and prefix caching for qwen3.5 (#46202)
  • 作者: Tianmu Li | +388/-32 | 7 个文件

  • Removes the guard in vllm/platforms/cpu.py that unconditionally disabled enable_prefix_caching and enable_chunked_prefill for linear-attention models on AMX CPU platforms. - Fix CPUModelRunner._zero_block_ids which is not full-attention blocks in hybrid models (e.g. Qwen3.5). A newly allocated block may be partially written during a chunked-prefill step. Mirror GPU behavior ## Test Result —

• 76c3c4f #46583 — [Rust Frontend] Introduce unified parser interface & combined parser (#46583)
  • 作者: Bugen Zhao | +1204/-1745 | 74 个文件

  Move tool-parser and reasoning-parser crates into modules of a unified parser crate (parser/src/tool and parser/src/reasoning). Introduce a UnifiedParser interface, which combines ToolParser and ReasoningParser: it takes decoded text and emits Text | Reasoning | Tool events directly. The chat crate now operates on UnifiedParser directly. The only implementor of UnifiedParser for now is CombinedPar…

• 6e3a983 #46655 — [ROCm] Remove erroneous inclusion of gptq_marlin as supported quant scheme on ROCm (#46655)
  • 作者: Micah Williamson | +0/-1 | 1 个文件

  Here we remove it. It will be added back in the future if we do indeed add support for gptq_marlin.

• dda3aca #46488 — [Speculative Decoding] Propagate norm_output and fc_norm config for Eagle3 speculators (#46488)
  • 作者: Orestis Zambounis | +2/-0 | 1 个文件

  • Add missing norm_output and fc_norm config propagation in the speculators Eagle3 config handler (algos.py) - The Eagle3 model code already reads both fields via getattr defaults, but the config handler never transferred them from the checkpoint config dict to the PretrainedConfig - Without this fix, Eagle 3.1 checkpoints (which set norm_output=True and/or fc_norm=True) silently fall back to Fals…

• d6696e2 #46636 — [ROCm] Begin Deprecation Window for CUDA_VISIBLE_DEVICES on ROCm (#46636)
  • 作者: Micah Williamson | +8/-0 | 1 个文件

  ROCm currently respects CUDA_VISIBLE_DEVICES in vLLM. This has been a recurring source of bugs, especially in distributed environments, e.g. when using Ray. Examples: https://github.com/vllm-project/vllm/pull/45998, https://github.com/vllm-project/vllm/pull/35069, https://github.com/vllm-project/vllm/pull/33305, https://github.com/vllm-project/vllm/pull/33308 Follow up: https://github.com/vllm-pro…

• d511b5b #40469 — Chore: Fix minor doc sentence, grammar, quote errors (#40469)
  • 作者: Ashwin Phadke | +4/-4 | 3 个文件

  Found one small documentation error on kv_cache_manager docs. Others were caught by Claude code.

• 56ca599 #46389 — Humming support for 2/3/5/6/7-bit pack-quantized weight-only inference (#46389)
  • 作者: HDCharles | +40/-22 | 4 个文件

  separating out wNa16 support for humming per discussion in https://github.com/vllm-project/vllm/pull/45185 note: while humming can support per-tensor quantization, since marlin cannot, i didn’t want to enable it in this pathway so that we wouldn’t run into situations where per-tensor quantized 4/8 bit weight inference would get routed to marlin and error. AI assistance was used (Claude). 🤖 Generat…

• cf57311 #46386 — Run DeepSeek-V2-Lite prefetch-offload eval eager on ROCm (#46386)
  • 作者: Aarushi Jain | +8/-0 | 1 个文件

  The mi300_1: deepseek-v2-lite-prefetch-offload-accuracy nightly is flaky on MI300X — ~1 in 5 fresh servers emits ~45% garbage decode tokens, dropping GSM8K accuracy below the 0.25 threshold. The corruption is nondeterministic across identical runs and reproduces on the plain non-offload baseline. Root cause is a HIP graph capture/replay regression in the ROCm runtime (ROCm/clr), not vLLM or the of…

• e7df232 #46252 — [KV Offload] Gate packed HMA KV cache on cross-layer config (#46252)
  • 作者: Lucas Wilkinson | +42/-33 | 4 个文件

  • Use kv_connector_extra_config[“enable_cross_layers_blocks”] to opt multi-group HMA layouts into packed KV allocation as laid out in https://docs.vllm.ai/en/stable/features/nixl_connector_usage/#cross-layers-blocks - Keep DeepSeek V4-style UniformTypeKVCacheSpecs layouts on the packed path by default. - Remove the extra VLLM_USE_PACKED_HMA_KV_CACHE environment flag. - Canonicalize packed KV cache…

• b3a688c #46548 — [ROCm] Fix OOB During Model Warmup With ROCM_ATTN and MRV2 (#46548)
  • 作者: Micah Williamson | +8/-8 | 3 个文件

  Resolves https://github.com/vllm-project/vllm/issues/46179 (as well as numerous other failing tests in AMD CI, e.g. pytest -v -s tests/compile/fullgraph/test_full_graph.py::test_full_graph[neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic-model_kwargs1-2]). This PR resolves a latent seg fault in the ROCM_ATTN that was exposed when Llama was added to the default architectures for MRV2 in https://github…

• 1cd3e0e #46627 — [Bug] Fix IndentationError: expected an indented block after 'with' statement (#46627)
  • 作者: Wentao Ye | +2/-7 | 1 个文件

  Seems to be introduced by https://github.com/vllm-project/vllm/pull/44483 This PR fixes the issue

• f889325 #45850 — [KV Offload] Use background thread for mmap / cpu_tensors pinning (#45850)
  • 作者: Yiwei Hu | +89/-38 | 1 个文件

  This pull request introduces improvements to how CPU tensors / mmap are pinned for CUDA transfers in vllm/v1/kv_offload/cpu/gpu_worker.py, including asynchronous pinning for better startup performance. The changes add a dedicated function for pinning CPU tensors, enable background pinning on CUDA platforms, and ensure logging and synchronization of the pinning process at the end of CpuGpuOffloadin…

• bb61177 #46363 — [KV Offloading] Replace bool|None lookup return with LookupResult enum (#46363)
  • 作者: Ronen Schaffer | +323/-172 | 14 个文件

  Replace the ambiguous bool | None return type of OffloadingManager.lookup() with a LookupResult enum that distinguishes between two previously conflated None cases: - HIT_PENDING — block exists in the primary tier but its write is still in-flight. Should count as a hit in _sliding_window_lookup’s consecutive-hit streak. - RETRY — block found on a secondary tier (promotion started) or async…

• 007b5a5 #46511 — [Log] Update to log once (#46511)
  • 作者: Wentao Ye | +23/-21 | 4 个文件

  Some infos are not needed to replicate, this PR fixes the issue

• 61ee183 #46414 — [ROCm] Fix AITER FP8 quantization schema tests (#46414)
  • 作者: djramic | +29/-59 | 1 个文件

  The AITER FP8 quant schema tests fail on MI300: - test_per_tensor_quant_torch_compile errors with error_on_custom_op_aliasing does not exist (a torch >= 2.12 config). - test_per_tensor_quant_matches_native[dynamic] fails because AITER’s dynamic scale uses fp8 max 240 while native uses 224, landing on different fp8 grids. Fix: Schema tests now use the shared fp8-safe opcheck helper, which patch…

• 84c62e1 #46535 — [Model Runner V2][MM] Support EVS (#46535)
  • 作者: Nick Hill | +245/-50 | 10 个文件

  This also changes the Qwen VL recompute_mrope_positions methods to accept input_ids as a device tensor rather than python list, to avoid cpu roundtrip.

• 563c628 #46607 — [XPU] bump up vllm_xpu_kernels to v0.1.10.1 (#46607)
  • 作者: Kunshang Ji | +1/-1 | 1 个文件

  bump up vllm_xpu_kernels to v0.1.10.1 ## Test Result —

• 62890e2 #46467 — Fix duplicated logging when loading a corrupt or partial video (#46467)
  • 作者: Tae Jeong | +6/-6 | 1 个文件

  Loading a corrupt/partial video logged one WARNING per broken frame in OpenCVVideoBackend, duplicating the single summary line. Demote the per-frame messages to debug; keep the summary at warning. Verbosity-only. ## AI assistance Prepared with AI assistance (Claude); reviewed by submitter. — - [v] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”. - [v] …

• cf9fd64 #46284 — Fix KV offload request-finished lifecycle contract (#46284)
  • 作者: Rui Yin | +336/-114 | 5 个文件

  Fixes #46027. This aligns the KV offload on_request_finished() lifecycle contract across layers: - OffloadingConnectorScheduler now calls OffloadingManager.on_request_finished() when the request stops and no more GPU<->primary submit-side work will be issued, while retaining request state for later complete_store() / complete_load() callbacks from already-submitted jobs. - TieringOffloadingManager…

• d4448b5 #45973 — [XPU][Docker] switch to ubuntu 24.04 as base image (#45973)
  • 作者: Kunshang Ji | +21/-11 | 2 个文件

  switch base image to ubuntu 24.04, this can reduce image size ## Test Result —

• 160c80a #46582 — [Rust Frontend] Raise frontend JSON body limit (#46582)
  • 作者: Roy Wang | +44/-0 | 2 个文件

  The Rust frontend OpenAI routes use Axum’s JSON extractor, which has a 2 MiB default request body limit. Large valid requests from 1M context length support models can be rejected before validation with Failed to buffer the request body: length limit exceeded. This raises the router JSON body limit and adds a regression test for a request larger than Axum’s default. ## Duplicate Check Searched ope…

• f237e16 #45053 — [KV Offload] Replace OffloadingHandler with OffloadingWorker (#45053)
  • 作者: Martin Hickey | +176/-465 | 13 个文件

  Makes KV transfer direction explicit and removes the medium routing dispatcher in the offloading worker path. Today, every transfer goes through a concrete OffloadingWorker in kv_offload/worker/worker.py that routes each job by (src_medium, dst_medium) to a registered handler. The OffloadingHandler.transfer_async is a single direction agnostic entry point. But each worker owns exactly one offloade…

• d20dbf9 #46412 — [Mooncake] Only check and store new KV cache range (#46412)
  • 作者: Wei Zhao | +487/-60 | 6 个文件

  This PR optimizes CPU overhead in Mooncake Store KV cache store path. Existing code checks KV cache range from the beginning in every store call. This PR only checks the range for the newly generated KV cache during chunked prefill. This saves various CPU overhead, including batch_is_exist lookup. ### Performance benchmark Deepseek v4 on 8xB300 ## Test Result —

• ede54b9 #46555 — set AttentionCGSupport.UNIFORM_BATCH for fa2 on xpu (#46555)
  • 作者: Xinyu Chen | +1/-1 | 1 个文件

  FA2 on XPU will decide grid size by max_query_len, it won’t support ALWAYS with mix-prefill-decode batch. ## Test Result —

• dc0d318 #46189 — [Attention] Add FLASH_ATTN_MLA_SPARSE backend for Hopper sparse MLA (#46189)
  • 作者: Dakai An | +292/-0 | 4 个文件

  This PR adds a Python-only FLASH_ATTN_MLA_SPARSE backend for sparse MLA models on Hopper GPUs. The backend reuses vLLM’s existing FlashAttention MLA wrapper and sparse MLA index conversion path, then calls flash_attn_varlen_func(…, fa_version=3) for BF16 KV-cache sparse decode. The tested GLM-5.1-FP8 checkpoint has: - num_attention_heads = 64 - kv_lora_rank = 512 - qk_rope_head_dim = 64 - compre…

• d7c1821 #45810 — [Model][MiniMax-M3] Add pipeline parallelism support (#45810)
  • 作者: soaringk | +124/-49 | 3 个文件

  Adds pipeline-parallelism (PP) support to MiniMax-M3. Fixes #45455. Relates to #45668. - Implement SupportsPP for MiniMaxM3SparseForCausalLM and MiniMaxM3SparseForConditionalGeneration. - Add PP first/last-rank handling for embed_tokens, final norm, and lm_head. - Thread intermediate_tensors through the MiniMax-M3 text and VL wrappers. - Update docs/models/supported_models.md for MiniMax-M3 PP sup…

• 4cd1a84 #46362 — [Model] Remove BaiChuanForCausalLM and BaichuanForCausalLM (#46362)
  • 作者: Tiezhen WANG | +2/-536 | 10 个文件

  • Remove BaiChuanForCausalLM and BaichuanForCausalLM model architectures (Baichuan 7B/13B/Baichuan2) - Adds both architectures to _PREVIOUSLY_SUPPORTED_MODELS ## Removed files - vllm/model_executor/models/baichuan.py — custom model implementation (~490 LOC) - examples/template_baichuan.jinja — chat template - rust/src/chat/tests/templates/vllm_examples/template_baichuan.jinja — Rust snapshot ## No…

• 191826e #46550 — [CI/Build] Fix topk histogram build on SM75 (#46550)
  • 作者: Mohammad Miadh Angkad | +13/-4 | 1 个文件

  Fix release build failure introduced by #43008, where redux.sync.add.u32 was emitted for sm_75 even though it requires sm_80+ CI/Build

• 489abad #44124 — feat: support to OpenMOSS-Team (#44124)
  • 作者: hurukawa | +2926/-2 | 11 个文件

  Closes #43431. Also manually smoke-tested four MOSS-Audio models with both audio_url and input_audio. E2E audio input: - Source: existing vLLM public test asset from vllm/assets/audio.py - Asset: AudioAsset(“winning_call”), resolved by vLLM from s3://vllm-public-assets - File: multimodal_asset/winning_call.ogg - URL: https://vllm-public-assets.s3.us-west-2.amazonaws.com/multimodal_asset/winning_ca…

• 96de8bb #46549 — [MoE] Free unused MXFP4 scales in OAI Triton Backend (#46549)
  • 作者: Woosuk Kwon | +20/-4 | 2 个文件

  When using the OAI Triton backend, the original mxfp4 block scales are swizzled into the precision configs and are no longer read by the kernel, so they can be dropped. This eliminates the extra GPU memory (~6 GB) consumed by the Triton MoE backend.

🧪 CI/Tests

• cdfa2fd #46729 — [ROCm][CI] rm duplicate Distributed Torchrun ci test (#46729)
  • 作者: Divakar Verma | +0/-24 | 1 个文件

  This PR removes duplicate test Distributed Torchrun + Shutdown Tests (2 GPUs) from MI250 as it already exists on MI300.

• d3130d8 #38290 — [CI] Pin GitHub Actions to commit hashes in macos-smoke-test.yml (#38290)
  • 作者: Russell Bryant | +2/-2 | 1 个文件

  Pin actions/checkout and astral-sh/setup-uv to their full commit SHAs instead of mutable version tags. This is a security best practice recommended by GitHub because version tags (like v7) can be force-pushed to point at arbitrary code, whereas commit hashes are immutable. Pinning to hashes prevents a compromised or hijacked action repository from silently injecting malicious code into CI runs. Th…

• 2365b7a #46668 — [Hardware][AMD][CI] Mirror Basic Models (Others) and Weight Loading Multiple GPU test groups (#46668)
  • 作者: Matt | +18/-6 | 3 个文件

  This PR gates the Basic Models (Others) test group and mirrors the (optional) Weight Loading Multiple GPU test groups on AMD CI. These test groups have been unblocked after the mem access fault fix in https://github.com/vllm-project/vllm/pull/46548. The above test groups will be tested on AMD CI as part of the PR review process. ## Test Result The above test groups pass. cc @AndreasKaratzas —

• a6f41ab #46674 — [XPU][CI]Refine .buildkite/ci_config_intel.yaml for Intel GPU CI (#46674)
  • 作者: xiangdong | +2/-3 | 1 个文件

  Refine .buildkite/ci_config_intel.yaml for Intel GPU CI ## Test Result —

• e2af449 #46686 — [Hardware][AMD][CI] Move Metrics, Tracing (2 GPUs) & make optional (#46686)
  • 作者: Matt | +3/-2 | 2 个文件

  Move the Metrics, Tracing (2 GPUs) test group to MI325 and make optional for investigation purposes, as we’re seeing an unusually high rate of CPU-side segfaults on this test group currently. —

• 3f5a1e1 #46573 — [ROCm][CI] Expand basic correctness target suites (#46573)
  • 作者: Andreas Karatzas | +174/-60 | 4 个文件

  This PR splits the basic-correctness target-suite change out of #39238. It updates distributed basic-correctness parametrization so ROCm suites can run the intended two-GPU cases instead of being limited to L4/A100. The target suite now defaults from the detected ROCm accelerator, while preserving the existing TARGET_TEST_SUITE override and adding VLLM_TARGET_TEST_SUITE. This keeps MI250, MI300, M…

• dc55936 #46650 — [AMD][CI] Fix Pipeline + Context Parallelism test group (#46650)
  • 作者: Aarushi Jain | +7/-6 | 1 个文件

  The Pipeline + Context Parallelism (4 GPUs) group fails on ROCm at test_pp_cudagraph[FLASH_ATTN-2-JackFram/llama-160m]. The test hardcodes –attention-backend=FLASH_ATTN, which resolves to the unified CUDA FlashAttentionBackend on every platform. That backend is not runnable on ROCm: - get_flash_attn_version() intentionally returns None on ROCm (“ROCm doesn’t use vllm_flash_attn”), but FlashAttent…

• 2801b11 #45914 — [Test] Pin block_size in auto-fit max_model_len test (#45914)
  • 作者: Liangliang Ma | +4/-2 | 1 个文件

  The kv_cache_memory_bytes=1MB budget assumed CUDA’s default block_size=16. On platforms whose default block_size is larger (e.g. XPU defaults to 64) a single KV block already exceeds 1MB, so auto-fit fails with ’not enough GPU memory available to serve even a single token’. Pin block_size=16 in the test so the budget is independent of the platform default.

• 549c707 #46580 — [ROCm][CI] Skip the MoE Marlin tile-padding helper assertion (#46580)
  • 作者: Andreas Karatzas | +4/-0 | 1 个文件

  Skip the MoE Marlin tile-padding helper assertion on ROCm, where production code intentionally keeps MoE Marlin disabled and falls back to ROCm/native MoE paths. test_check_moe_marlin_supports_layer_padding was added in #45703 with CUDA Marlin selection semantics, but the helper it exercises still returns False immediately on ROCm. The AMD kernel-quantization lane was already printing this failure…

⚡ Performance

• c63cd49 #46546 — [ROCm][ [Perf] sparse attention optimization on minimax-m3 (#46546)
  • 作者: Hongxia Yang | +1238/-36 | 5 个文件

  Extend the work of PR https://github.com/vllm-project/vllm/pull/46035 (gfx942 focus) to address gfx950 platform. Validated on gfx950 to ensure Perf win on minimax-m3-mxfp8 model without accuracy loss before filing up the PR. Co-Authored-By: @yueliu14 yue.liu4@amd.com ## Test Result Perf (8k/1k, concurrency 64): | metric | base | opt | Δ | |—|—:|—:|—:| | Output throughput | 1269.98 | …

• 77c1d9f #40784 — [ROCm][Perf] Tune wvSplitK on gfx1151 (#40784)
  • 作者: Matthias Gehre | +53/-21 | 1 个文件

  Tune wvSplitK skinny GEMM kernel for gfx1151. ## Kernel benchmarks bf16 wvSplitK kernel benchmark on Strix Halo (gfx1151, 20 CUs): - N=1 average: +18.7% bandwidth improvement - N=2 average: +19.8% - N=3 average: +15.2% - N=4 average: +13.0% fp16 wvSplitK kernel benchmark: - N=1 average: +6.1% bandwidth improvement ## End-to-end benchmark AMD Strix Halo (gfx1151, LPDDR5X…

• fc61c6f #46392 — [Perf] Enable + tune FlashInfer fused allreduce at world_size=16 on SM 10.3 (GB300) (#46392)
  • 作者: Jeff (Junze) Ma | +13/-4 | 3 个文件

  FlashInfer fused allreduce+RMSNorm is gated by world size in two places, and neither currently admits world_size=16: - PassConfig.flashinfer_max_size() returns None unless the world size is in FI_SUPPORTED_WORLD_SIZES (was [2, 4, 8]), and - it then looks up FI_ALLREDUCE_FUSION_MAX_SIZE_MB[capability], which for SM 10.3 (GB300) had no 16 entry. When either is missing, flashinfer_max_size(16) is Non…

• 9e88e96 #45971 — [Perf][KVConnector][Mooncake] Parallelize KV load with a receive-thread pool (#45971)
  • 作者: Yifan Qiao | +54/-27 | 3 个文件

  In MooncakeStoreConnector, each KV-load request pays control overhead before any RDMA transfer begins: Python-side request preparation (token/block walk, slice list construction, tp-rank rotation, disk-offload budgeting) plus the master key lookup that runs inside batch_get_into_multi_buffers. With a single receive thread, that overhead is serial dead time on the NIC between consecutive transfers …

• 7f99e80 #46425 — [Perf][ThinkingBudget] reduce search space for thinking tokens (#46425)
  • 作者: Walter Beller-Morales | +61/-4 | 2 个文件

  Currently the thinking token budget logic uses _find_last_sequence_index to scan the entire generated sequence on every single generated token. This causes the scan time to blow up on longer generations because it’s basically a O(n^2) nested loop. One way to fix this time complexity is to keep track of the sequences that have already been scanned in the TB state and only search for new tokens. Thi…

• 061043e #46353 — [CPU][Perf] Accelerate unquantized MoE for AArch64 (#46353)
  • 作者: Fadi Arafeh | +672/-32 | 15 个文件

  Accelerate unquantized MoE for AArch64 - Enable FusedMoE kernel for AArch64 - Implement AdvSIMD BFMMLA interface to accelerate w13 and w2 GEMMs - Extend generic micro kernel interface and MoE kernel to support packing input matrix - Abstract sleef.h includes and tanh symbol for x86 under the AVX vectorizer class ## Performance 2.1x higher throughput for gpt-oss and 2.34x higher through…

• 0bc479e #46542 — [Perf][LoRA] Replace O(n) list.index() with a dict in convert_mapping (#46542)
  • 作者: Lynn | +11/-3 | 1 个文件

  convert_mapping (in vllm/lora/punica_wrapper/utils.py) builds the per-token LoRA index lists by calling lora_index_to_id.index(…) for every prompt token and every batch token: Each list.index() is an O(num_loras) linear scan, so the construction is O(num_tokens × num_loras) every time LoRA metadata is rebuilt — which is on the per-forward path via update_metadata. The code already flagge…

• 52fbe12 #46543 — [Perf][Multimodal] Avoid building a full timestamps list in video frame sampling (#46543)
  • 作者: Lynn | +2/-4 | 1 个文件

  The GLM-4V / GLM video frame-sampling backends (GLM46VVideoBackend and GLMGAVideoBackend in vllm/multimodal/video.py) build a Python list with one float timestamp per source frame: and then only ever read it as timestamps[frame_index] inside the sampling loop. Two issues: 1. The list is O(total_frames_num) to allocate, but is never read when the total_frames_num < extract_t branch is taken…

🐛 Bug Fix

• 710ebaa #46114 — [ROCm][Bugfix] Fix chunk alignment when using context parallelism with TRITON_MLA (#46114)
  • 作者: Micah Williamson | +53/-39 | 2 个文件

  There is a bug in mla_attention when using context parallelism on ROCm. max_context_chunk is being aligned properly on CUDA because of the self.aot_schedule path (which is CUDA-only). The chunk misalignment causes zero accuracy in test_context_parallel.py on ROCm with dcp_size=4 using TRITON_MLA. This PR resolves the issue by shrinking max_context_chunk to the nearest size that divides evenly acro…

• efb5acf #46382 — [Bugfix] fix: stream Mimimax m2 tool call string arguments (#46382)
  • 作者: Chauncey | +33/-2 | 1 个文件

  Follow up https://github.com/vllm-project/vllm/pull/46351 [Bugfix] fix: stream Mimimax m2 tool call string arguments ## Test Result —

• b698160 #46595 — [Bugfix][MooncakeStore] track resumed requests via scheduler’s resumed_req_ids (#46595)
  • 作者: Yifan Qiao | +77/-11 | 3 个文件

  The MooncakeStore connector tracks block tables per request so it knows whether a step’s new_block_ids should be appended (normal decode) or replace the existing table (resumed from preemption). It was deciding this from a connector-local _preempted_req_ids set, which drifts from what the scheduler actually does. The scheduler already tells us the answer: CachedRequestData.resumed_req_ids …

• e06a834 #46101 — [Bugfix] Normalize slashes in Helion GPU names (#46101)
  • 作者: cyq | +4/-4 | 2 个文件

  Fixes #46084. Helion uses canonicalize_gpu_name() to build stable GPU identifiers. ROCm can report composite GPU names such as AMD Instinct MI250X / MI250; the slash currently survives normalization, which makes the identifier unsuitable for config lookup or path-like keys. This normalizes separator runs made of whitespace, hyphens, or / to a single underscore while preserving the existing alias l…

• d7ab9be #46408 — [Bugfix] Support -1 (invalid/non-local) slots in topk_ids for Triton MoE (#46408)
  • 作者: Woosuk Kwon | +88/-7 | 1 个文件

  Under expert parallelism, dispatch hands the experts a topk_ids tensor whose entries can be -1, marking invalid / non-local slots (a token’s routed expert lives on another rank). The Triton unfused MoE path (gpt_oss_triton_kernels_moe.py) mishandled these in two places: 1. expert_map[topk_ids] (global→local remap) indexed the map with -1, which wraps to expert_map[-1] — a valid local id — and …

• 6a15707 #46406 — [Bugfix] Support non-power-of-2 top_k in legacy triton_kernels routing (#46406)
  • 作者: Woosuk Kwon | +238/-0 | 1 个文件

  The bundled (v3.5.1) triton_kernels legacy routing path fails to compile for non-power-of-2 top_k. _routing_compute_indx does tl.arange(0, N_EXPTS_ACT * BLOCK_M), and tl.arange requires a power-of-2 length — so e.g. DeepSeek-V4 with top_k=6 (6 * 32 = 192) raises a compile error and the model can’t run on the Triton MoE backend. This installs a pow2-safe variant of the legacy routing kernel (_routi…

• 3c43237 #46560 — [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560)
  • 作者: JessieWei | +90/-16 | 10 个文件

  DFlash (and any speculative method that drafts many tokens) crashes the V1 engine at startup on the Model Runner V2 path: Minimal repro (current main, single H200): Root cause. The MRV2 sampler and rejection-sampling Triton kernels address the logits tensor with a per-row offset row * vocab_size, computed in int32 (tl.program_id and loaded indices are int32, so int32 * int32 -> int32). Spe…

• 24d5186 #46339 — [Bugfix] Re-enable FP8 MoE on NVIDIA Thor (#46339)
  • 作者: Cyrus Leung | +3/-3 | 2 个文件

  Partially revert a change in https://github.com/vllm-project/vllm/pull/45277 which broke Qwen/Qwen3.5-35B-A3B-FP8 inference on NVIDIA Thor (SM101 for CUDA 12 and SM110 for CUDA 13). This parallels how cutlass_3x_gemm_sm100_fp8 is also enabled for this architecture. ## Test Result —

• 93ec645 #44483 — [Bugfix] Fix illegal memory access from a forward during a partial wake_up (#44483)
  • 作者: meihanc | +12/-4 | 1 个文件

  Fixes #44395. Resume scheduling only once the executor is fully resident. model_executor.is_sleeping stays True until all sleep tags are awake (Executor.wake_up clears it only when sleeping_tags is empty), so a weights-only wake keeps the scheduler paused; it resumes at the final/full wake. No behavior change on the full-wake path. ### Reproduction ### Test plan & results Validated on **4× A100-80…

• f1a6703 #46220 — [Bugfix][Config] Keep pydantic validation for fields with a TYPE_CHECKING Literal alias (#46220)
  • 作者: Ting SUN | +6/-4 | 3 个文件

  Follow-up to #45196, addressing @hmellor’s review on load_format. A field annotated str | SomeAlias, where SomeAlias is a Literal imported only under TYPE_CHECKING and falling back to = Any at runtime, collapses to Any at runtime, so pydantic skips validation. #45196 worked around this by retyping load_format to plain str (gaining validation but dropping the LoadFormats Literal); @hmellor asked to…

🖥️ Kernel

• 1273a8f #36559 — [Kernel] Add swap AB optimization to fused_moe_kernel (#36559)
  • 作者: Xin Yang | +59/-14 | 2 个文件

  This PR add swap AB optimization to fused_moe_kernel to make better use of WGMMA. Similar as #20396. For small BLOCK_SIZE_M cases (BLOCK_SIZE_M < 64), change the gemm from a[M, K] @ b[K, N]-> c[M, N] to b[N, K] @ a[K, M] -> c[N, M]. 1% to 4% e2e output token throughput improvement. ## Test Result Unit tests passed. ## Micro bench Micro bench on H200 shows improvements for BLOCK_SIZE_M 16, and no d…

• 23aed9b #46508 — [Kernel] Enable PDL for per_token_group_quant_8bit_kernel (#46508)
  • 作者: Jee Jee Li | +62/-42 | 1 个文件

  follow up on https://github.com/vllm-project/vllm/pull/42996 ## Test Result —

• fc7fc42 #46518 — [Kernel][MoE] Allow FlashInfer MXINT4 MoE for gated SiLU (#46518)
  • 作者: Kaihang Jiang | +16/-2 | 2 个文件

  TrtLlmMxint4ExpertsMonolithic currently rejects MoEActivation.SILU, so vLLM does not select the FlashInfer TRT-LLM MXINT4 MoE backend for Kimi-K2.5-style gated-SiLU MoE layers. The underlying FlashInfer trtllm_mxint4_block_scale_moe path supports the standard gated SiLU computation. This PR updates the Python-side activation support predicate to accept MoEActivation.SILU in addition to the existin…

🦀 Rust Frontend

• cd34729 #46314 — [Frontend] Port seed_oss to the streaming parser engine as a Qwen3 subclass (#46314)
  • 作者: Maxwill Lin | +338/-1435 | 13 个文件

  seed_oss’s tool-call and reasoning grammar is identical to Qwen3 except for four wrapper tokens — seed:tool_call/</seed:tool_call> and seed:think/</seed:think>. <function=…> and <parameter=…> are byte-identical, and the streaming parser engine (#45413) already provides argument type-coercion (_fix_arg_types), so no custom converter is needed. This replaces the hand-rolled seed_oss tool and rea…

• 84c2f9f #46344 — [Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344)
  • 作者: Chauncey | +406/-178 | 11 个文件

  [Frontend] Fix Kimi K2 tool call IDs for required tool choice ## Test Result POST /v1/messages?beta=true HTTP/1.1 Proxy-Connection: Keep-Alive Accept: application/json Authorization: Bearer sk-W3Pywq7yIpIuf5s105C6041c0a424953924a729b10Fe9b5e Content-Type: application/json User-Agent: claude-cli/2.1.186 (external, cli) X-Claude-Code-Session-Id: 5c745fc5-7121-47fe-8dc7-ce85b18220ef X-Stainless-Arch:…

✨ New Feature

• 49f2104 #44044 — [Feature] Support DCP with FP8 KV cache in MLA decode path (#44044)
  • 作者: shivampr | +359/-15 | 4 个文件

  Fix DCP + FP8 KV cache support in MLA. Previously, MLA decode rejected this config with: DCP not support fp8 kvcache now. This PR allows MLA decode to all-gather the already-quantized FP8 query tensor across DCP ranks when FP8 query input is supported. Non-FP8 behavior is unchanged. This also updates FlashMLA FP8 decode metadata to account for gathered DCP query heads, and fixes the DCP + standard…

• 70749fd #40835 — [Feature] Triton INT4 per-token-head KV cache quantization (#40835)
  • 作者: JartX | +1463/-112 | 10 个文件

  Based on PR: Changed to only INT4_PER_TOKEN_HEAD INT4_PER_TOKEN_HEAD |Tasks|Version| Filter |n-shot| Metric | |Value| |Stderr| |—–|——:|—————-|—–:|———–|—|—-:|—|—–:| |gsm8k| 3|flexible-extract| 5|exact_match|↑ | 0.912|± |0.0273| | | |strict-match | 5|exact_match|↑ | 0.91|± |0.0273|

📖 Documentation

• 7dc0360 #44720 — [Doc] Document Qwen3.6 (dense + MoE) ViT CUDA graph support (#44720)
  • 作者: Nemani Harsha Vardhan | +2/-1 | 1 个文件

  Documents that Qwen3.6 (dense Qwen/Qwen3.6-27B and MoE Qwen/Qwen3.6-35B-A3B) is covered by the ViT-encoder CUDA-graph path. Per review feedback (@Isotr0py, @shen-shanshan): Qwen3.6 reuses the existing Qwen3.5 architecture — its config.json reports model_type: qwen3_5, so it is served by Qwen3_5ForConditionalGeneration / Qwen3_5MoeForConditionalGeneration, which already implement SupportsEncode…

🔩 Misc

• a2cb08b #46473 — [Misc][PD] Disable bidirectional xfer mode for NixlPushConnector (#46473)
  • 作者: Nicolò Lucchesi | +4/-0 | 1 个文件

  Bidirectional kv transfer with the newly added Push model is not supported, but there’s no validation right now