共 23 个 commit，涉及 94 个文件，+4636/-754 行变动。

概要

统计项 数值
Commit 数 23
变更文件 94
新增行数 +4636
删除行数 -754

Commit 列表

📦 Other

  • 4b643c4 #46961 — [GLM5] Fix minor typo (#46961)
    • 作者: Woosuk Kwon | +1/-1 | 1 个文件

    Fixes a minor attribute name error in #46876

  • 89876b0 #46876 — [GLM5] Implement op fusion for GLM5/DSV3.2 (#46876)
    • 作者: Woosuk Kwon | +1530/-91 | 9 个文件

    This PR ports #38595 to the new model definitions for GLM-5 and DeepSeek V3.2. On 8x GB200 GPUs across 2 nodes with TP8, this PR reaches ~290 tok/s using nvidia/GLM-5.2-NVFP4 with MTP=3. The current op fusion work includes: * Two large fused kernels for miscellaneous RoPE and normalization ops in attention and the sparse indexer * AllReduce + RMSNorm fusion after both attention and MoE * Small fus…

  • 5c91039 #46635 — [GLM5.2 Perf] Replace MOE all-reduce with reduce-scatter, 3.1%~3.2 E2E Throughput improvement (#46635)
    • 作者: Wentao Ye | +86/-12 | 1 个文件

    Origin: full -> all-reduce -> full on every rank -> chunk -> local MoE -> all-gather Now: full -> reduce-scatter -> local shard -> local MoE -> all-gather if needed Note that this works for deepseek series as well. vllm serve zai-org/GLM-5.2-FP8 –kv-cache-dtype fp8_e4m3 –enable-expert-parallel –tensor-parallel-size 8 –tool-call-parser glm47 –enable-auto-tool-choice –reasoning-parser glm45 –…

  • 09841ae #46846 — [Render][Speculator] Add return_loss_mask to render endpoint for training data generation (#46846)
    • 作者: Ranran | +288/-20 | 7 个文件

    Adds a return_loss_mask field to ChatCompletionRequest and a loss_mask field to GenerateRequest, so POST /v1/chat/completions/render can return a per-token assistant/trainable mask alongside token_ids. This is needed by downstream training frameworks (e.g. speculators) that use the render endpoint to delegate tokenization to vLLM instead of…

  • c6741b2 #46564 — [Model] Support Unlimited OCR (#46564)
    • 作者: Tianyu Guo | +1083/-48 | 35 个文件

    Support https://huggingface.co/baidu/Unlimited-OCR OmniDocBench | Metric | FA4 | FlexAttention | Paper v1.6 | |—|—|—|—| | Overall ↑ | 92.12 | 92.38 | 93.92 | | Text Edit ↓ | 0.089 | 0.087 | 0.042 | | Formula CDM ↑ | 95.34 | 95.53 | 95.79 | | Formula Edit ↓ | 0.108 | 0.105 | — | | Table TEDS ↑ | 89.90 | 90.33 | 90.16 | | Table TEDS-S ↑ | 93.27 | 93.60 | 93.32 | | Read-order Edit ↓ | 0.143 |…

  • 11a1230 #46786 — [Model Runner V2][Spec Decode] Handle tuple hidden states from MTP draft models (#46786)
    • 作者: Chauncey | +85/-22 | 4 个文件

    Some MTP models declare a single-tensor contract but return (logits_hidden, feedback_hidden) for final-norm correctness. Fix https://github.com/vllm-project/vllm/issues/46798 [Spec Decode] Handle tuple hidden states from MTP draft models this pr: —

  • 798185d #46888 — [KV-Offloading] Fix tensors_per_block stride (#46888)
    • 作者: Varun Sundar Rabindranath | +26/-2 | 2 个文件

    On main Offloading connector fails for Qwen/Qwen3.6-35B-A3B & nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 with the assert Why: PR https://github.com/vllm-project/vllm/pull/44577 introduced cross-layer layouts for DSV4. The PR introduced a change in the offloading/worker.py::register_kv_caches to set the 2D tensors_per_block stride directly from the layer_kv_cache. This is incorrect when the KV ca…

  • b6caeb5 #46878 — [Model Runner V2][Spec Decode] Use fp32 uniform threshold for acceptance (#46878)
    • 作者: Giancarlo Delfin | +12/-5 | 2 个文件

    Context Currently in MRV2 we do an FP64 uniform random draw in the rejection sample kernel. This precision level is higher than necessary. # This PR Switch to FP32 uniform random draw, similar to what is done in gumbel sampling. # Benchmarks ## Llama3 8B + Eagle + 4 spec tokens Metric | main | #46878 | Δ – | – | – | – Acceptance rate (%) | 24.83 | 25.03 | +0.20 Acceptance length | 1.99 | 2.0…

  • 8bf064f #46782 — Fixed chunked embedding aggregation with request-id metadata (#46782)
    • 作者: Taneem Ibrahim | +155/-60 | 3 个文件

    Chunked embedding aggregation currently recovers prompt/chunk identity by parsing generated request IDs such as …-prompt-{i}-chunk-{j}. That makes aggregation depend on request ID string structure, even though callers can influence the base request ID through X-Request-Id. If a user-provided request ID contains marker words like prompt or chunk, the parser can recover the wrong prompt index and …

  • 56aa067 #46927 — [CI Bug] Fix h100 AssertionError: Cold-start child failed (#46927)
    • 作者: Wentao Ye | +3/-3 | 1 个文件

    Introduced by https://github.com/vllm-project/vllm/pull/46862 Now 9 is the correct number pytest tests/compile/h100/test_startup.py::test_model_startup[deepseek_v3.2] Now

🐛 Bug Fix

  • 7544286 #46552 — [Bugfix] Transformers backend: recompute mm_token_type_ids per request for M-RoPE (#46552)
    • 作者: Gonzague de Carpentier | +13/-18 | 1 个文件

    The Transformers backend (model_impl=“transformers”) passes mm_token_type_ids from the cached multimodal features into model.get_rope_index(…). That tensor is full-sequence (one entry per prompt token), but it is cached per multimodal item, keyed by mm_hash (image content) — not by the request. So when two requests share an identical image (same mm_hash → multimodal-processor-cache hit) …

  • 6eb63a1 #46600 — [Bugfix][DSv3.2] Skip indexer weights for index-cache-skipped layers (#46600)
    • 作者: frida-andersson | +10/-0 | 1 个文件

    Follow-up to #45895. With index_topk_freq>1 (or an index_topk_pattern), only some layers build a sparse indexer; the rest are skipped at init via _skip_topk. But the checkpoint still ships indexer.* weights for every layer, so load_weights tries to assign them to params that don’t exist on the skipped layers and the load fails. Hit on DeepSeek-V3.2 with –hf_overrides ‘{“use_index_cache”: true, “i…

  • 35e6c86 #46034 — [Bugfix][MM][CG] Enable dual-path ViT CUDA graph for Step3-VL (#46034)
    • 作者: Shanshan Shen | +87/-125 | 2 个文件

    This PR enabled dual-path ViT CUDA graph for Step3-VL as default, leading to better performance and better stability. ## Test Result Mean Latency: | Input Size | Tiled | Single-Path ViT Graph (Main) | Dual-Path ViT Graph (This PR) | | :——–: | :—: | :——————-: | :—————————: | | (448, 448) | ❌ | 109.31ms | 103.54ms (5.28% ↓) | | (1024, 448) | ✅ | RuntimeError | 22…

  • 35e3850 #46915 — [Bugfix][Test] Fix test_flashinfer_cutlass_mxfp4_fused_moe on sm90 (stale weight/scale interleave) (#46915)
    • 作者: xiaolinchen | +14/-15 | 1 个文件

    Fixes the stale weight/scale preparation in test_flashinfer_cutlass_mxfp4_fused_moe, which fails on Hopper (sm90) with ~89% mismatch (vs the 0.2 threshold). Closes #46585. ## Root cause The test prepared inputs for the SM90 mixed-input cutlass MoE GEMM (cutlass_fused_moe(use_w4_group_scaling=True)) with: - a hand-rolled _interleave_scales_lastdim_by4 for scales, and - no interleave at all for …

⚡ Performance

  • 5ecae32 #45033 — [ROCm][Perf][MLA] Add AITER FlashAttention MLA prefill backend (ROCM_AITER_FA) (#45033)
    • 作者: xaguilar-amd | +269/-0 | 5 个文件

    Adds an AITER-backed MLA prefill backend that calls aiter.flash_attn_varlen_func directly instead of falling back to the upstream CK-tile FlashAttention path on ROCm. It dispatches the fast aiter::fmha_fwd_* kernel on gfx950 and, unlike the CK fallback, needs no V padding (native qk=192 / v=128 head dims) and returns softmax_lse already shaped (nheads, total_q) (no LSE transpose). This is …

  • c7ca0bc #44313 — [ROCm][Perf] Add Fused Shared Expert (FSE) support for GLM-4.5/6/7 (#44313)
    • 作者: Olga Miroshnichenko | +254/-105 | 2 个文件

    Extend the AITER Fused Shared Expert (FSE) path — originally added for DeepSeek-V2/V3 (#28540) and Qwen3-Next (#39280) — to the GLM-4 MoE family (GLM-4.5, GLM-4.6, GLM-4.7). When VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1, the shared expert is folded into the AITER FusedMoE kernel as n_shared_experts extra expert slots, eliminating the separate shared-expert MLP forward pass at low/medium concurr…

  • 51a9956 #46474 — [ROCm][Perf] Fused shared expert for Minimax M3 (#46474)
    • 作者: Fangzhou Ai | +46/-13 | 1 个文件

    Fuse MiniMax-M3 shared expert into the routed MoE call, a follow-up PR for https://github.com/vllm-project/vllm/pull/46419 vLLM server start: VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 VLLM_USE_BREAKABLE_CUDAGRAPH=0 .venv/bin/python -m vllm.entrypoints.cli.main serve amd/MiniMax-M3-MXFP4 –block-size 128 -tp 4 –max-model-len 9472 –attention-backen…

  • 867fd5e #46184 — [ROCm][Perf] Use flydsl moe with Minimax-M3 mxfp8 weights on gfx950 and implemented moe-backend selection (#46184)
    • 作者: Hongxia Yang | +374/-4 | 5 个文件

    [ROCm][Perf] Use flydsl moe with Minimax-M3 mxfp8 weights on gfx950 aiter’s flydls moe has shown perf improvement on various scenrios on mxfp8 serving on gfx950. This PR is to integrate the support, and also have the capability to fall back to triton mxfp8 dot-scaled implementation. Implementation and usability decisions: Reused moe-backend aiter and triton, and refactored the moe selection logic….

🧪 CI/Tests

  • a2a92cb #46930 — [Hardware][AMD][CI] Tweak mirrored tests; improve CI base dependency change detection (#46930)
    • 作者: Matt | +30/-35 | 6 个文件

    This PR makes the following changes in order to improve AMD CI stability: 1. Unmirrors the Distributed Tests (2xH100-2xMI300) test group, which was originally promoted in https://github.com/vllm-project/vllm/pull/45998. This test group was never passing but was believed to be due to a bug (that was recently fixed by https://github.com/vllm-project/ci-infra/pull/395). 2. Mirrors the Examples test g…

  • a65f93f #46886 — [ROCm][CI] Add ci_base metadata for external cache orchestration (#46886)
    • 作者: Andreas Karatzas | +194/-21 | 2 个文件

    The ci_base build now publishes richer metadata as both image labels and manifest annotations: - ci_base content hash and metadata version - content files and Dockerfile stages used to compute the hash - primary, content, commit, and stable image refs - Buildkite commit/build identity - ROCm base image digest, arch list, NIC/AINIC settings - RIXL, UCX, ROCShmem, and DeepEP repos, pinned commits, a…

  • 9036c89 #46928 — [Hardware][AMD][CI] Patch Whisper multi LoRA test to use TRITON_ATTN for now (#46928)
    • 作者: Matt | +10/-2 | 1 个文件

    This PR temporarily patches tests/lora/test_whisper.py::test_whisper_multi_lora to use TRITON_ATTN, as this test is currently failing on main and blocking CI. This test started failing after https://github.com/vllm-project/vllm/pull/46780, which had the effect of switching the test from using ROCM_AITER_UNIFIED_ATTN to ROCM_ATTN (because the former currently does not support FP16 dtypes after the …

  • 9fd00ee #46905 — [ROCm][CI] Move remaining mi250_2 tests out of the MI250 queue (#46905)
    • 作者: Andreas Karatzas | +64/-150 | 1 个文件

    The mi250_2 queue is being cleared out. This PR moves mi250_2 tests into gfx942. Migrations include: - Distributed Model Tests (2 GPUs) now runs on mi300_2 under mi300 - basic_correctness. - Distributed Comm Ops now runs on mi300_2 under mi300 - distributed. - Plugin Tests (2 GPUs) now runs on mi300_2 under mi300 - plugins. - Distributed DP Tests (2 GPUs) and V1 e2e (2 GPUs) now run on mi300_2 und…

🔩 Misc

  • ea2ead1 #46818 — [Misc] Fix incorrect layer type annotation in Fp8LinearMethod (#46818)
    • 作者: jj shao | +2/-2 | 1 个文件

    Fp8LinearMethod.create_weights and process_weights_after_loading annotate their layer parameter as RoutedExperts, but these methods only ever receive a LinearBase module. Fp8Config.get_quant_method routes LinearBase layers to Fp8LinearMethod and RoutedExperts layers to Fp8MoEMethod, so a RoutedExperts instance never reaches Fp8LinearMethod. The annotation is a copy-paste leftover from the MoE meth…