共 39 个 commit，涉及 248 个文件，+14100/-3412 行变动。

概要

统计项 数值
Commit 数 39
变更文件 248
新增行数 +14100
删除行数 -3412

Commit 列表

🐛 Bug Fix

  • 379acd4 #46860 — [Bugfix][Quantization] Fix W8A8 int-quantized scheme selection regression (#46860)
    • 作者: HDCharles | +197/-1 | 2 个文件

    (is_static_int8_in and is_static_int8_out) to (is_static_int8_in or is_static_int8_out) accidentally this meant that static_int8_in without static_int8_out would go into the wrong pathway and cause issues. I added unit tests to verify the configurations in general ## Testing vllm serve nm-testing/w8a8_static_asym-e2e (working) Added comprehensive parametrized test test_scheme_selection with 13 tes…

  • 36bbecd #46958 — [BugFix] Revert “[KV Offload] Use background thread for mmap / cpu_tensors pinning” (#46958)
    • 作者: Varun Sundar Rabindranath | +38/-89 | 1 个文件

    Fixes https://github.com/vllm-project/vllm/issues/46933 ## Test Result

  • 0ca39c4 #46973 — [Bugfix] Capture final-layer aux hidden state in deepseek_v2 backbone (#46973)
    • 作者: Michael Goin | +3/-0 | 1 个文件

    The deepseek_v2 backbone captures EAGLE3 / extract_hidden_states aux hidden states only at decoder-layer inputs (ids 0..num_hidden_layers-1), so an aux id equal to num_hidden_layers — the final layer’s output — is silently dropped, returning one fewer hidden state than configured. This makes deepseek_v2-family models (DeepSeek V2/V3, GLM-MoE, Kimi, …) unusable as aux-hidden-state targets wheneve…

  • e186107 #45961 — [Bugfix] Use native SiLU activation in CPU fused MoE (#45961)
    • 作者: Alden Lobo | +1/-1 | 1 个文件

    This PR fixes a startup failure in the CPU fused MoE path for certain Mixture-of-Experts (MoE) models. While starting the vLLM inference engine (version 0.22.0) on a CPU-only system, the initialization sequence can fail with an AssertionError during the warmup/profiling stage. ### Root Cause During the warmup phase on a CPU-only system, the current CPU fused MoE implementation triggers the SiLU pa…

  • 0e207da #46835 — [Bugfix] Transformers backend: apply learned lm_head.bias for tied-embedding models (#46835)
    • 作者: John | +26/-15 | 3 个文件

    The bug vLLM’s Transformers backend (CausalMixin, vllm/model_executor/models/transformers/causal.py) silently drops a learned lm_head.bias for tied-embedding models, producing garbage greedy output. Three independent failure points, all reached when tie_word_embeddings=True: - (A) self.skip_prefixes.append(“lm_head.”) — AutoWeightsLoader._can_skip matches by startswith, so this drops lm_hea…

  • 9552852 #46855 — [Bugfix][Mooncake] Fix Mooncake lookup prefixes with DCP > 1 (#46855)
    • 作者: Wei Zhao | +86/-10 | 3 个文件

    Current Mooncake store lookup only iterates over TP and PP ranks and fails to consider DCP and PCP ranks. This results in lookup to return True despite keys for certain ranks are missing. This leads to failure to get key error: Added lookup prefix unit tests to tests/v1/kv_connector/unit/test_mooncake_store_worker.py for DCP and PCP cases. ## Test Result —

  • 7544286 #46552 — [Bugfix] Transformers backend: recompute mm_token_type_ids per request for M-RoPE (#46552)
    • 作者: Gonzague de Carpentier | +13/-18 | 1 个文件

    The Transformers backend (model_impl=“transformers”) passes mm_token_type_ids from the cached multimodal features into model.get_rope_index(…). That tensor is full-sequence (one entry per prompt token), but it is cached per multimodal item, keyed by mm_hash (image content) — not by the request. So when two requests share an identical image (same mm_hash → multimodal-processor-cache hit) …

  • 6eb63a1 #46600 — [Bugfix][DSv3.2] Skip indexer weights for index-cache-skipped layers (#46600)
    • 作者: frida-andersson | +10/-0 | 1 个文件

    Follow-up to #45895. With index_topk_freq>1 (or an index_topk_pattern), only some layers build a sparse indexer; the rest are skipped at init via _skip_topk. But the checkpoint still ships indexer.* weights for every layer, so load_weights tries to assign them to params that don’t exist on the skipped layers and the load fails. Hit on DeepSeek-V3.2 with –hf_overrides ‘{“use_index_cache”: true, “i…

  • 35e6c86 #46034 — [Bugfix][MM][CG] Enable dual-path ViT CUDA graph for Step3-VL (#46034)
    • 作者: Shanshan Shen | +87/-125 | 2 个文件

    This PR enabled dual-path ViT CUDA graph for Step3-VL as default, leading to better performance and better stability. ## Test Result Mean Latency: | Input Size | Tiled | Single-Path ViT Graph (Main) | Dual-Path ViT Graph (This PR) | | :——–: | :—: | :——————-: | :—————————: | | (448, 448) | ❌ | 109.31ms | 103.54ms (5.28% ↓) | | (1024, 448) | ✅ | RuntimeError | 22…

📦 Other

  • 07d33e5 #44657 — [MyPy] Fix mypy incompatible assignment errors in LRUCacheLoRAModelManager (#44657)
    • 作者: Martin Hickey | +7/-39 | 1 个文件

    mypy raises two [assignment] errors in LRUCacheLoRAModelManager.init: - vllm/lora/model_manager.py: error: Incompatible types in assignment (expression has type “LoRALRUCache”, base class “LoRAModelManager” defined the type as “dict[int, LoRAModel]”) [assignment] - vllm/lora/model_manager.py: error: Incompatible types in assignment (expression has type “LoRALRUCache”, base class “LoRAModelMana…

  • 6185d73 #46827 — [Rust Frontend] Keep literal “null” string for string-typed tool params (#46827)
    • 作者: Blas Rodriguez Irizar | +48/-17 | 2 个文件

    A tool argument whose schema type is string and whose value is the literal text “null” is coerced to JSON null, so the real string value is lost (e.g. location=“null” becomes null). This affects the XML-style parsers that route through convert_with_optional_schema (qwen_coder, glm_xml, deepseek_dsml, minimax_m2/m3, hy_v3). This diverges from the Python coerce_to_schema_type, which keeps “null” as …

  • bc8481a #43373 — [MoE Refactor] Standardize Humming MoE experts + utilities (#43373)
    • 作者: bnellnm | +1178/-392 | 8 个文件

    Make Humming MoE + Quantization more standard. - Add proper config support check to Humming experts. - Have a single conversion function for preparing a layer for humming MoE kernels convert_to_humming_moe_kernel_format - Make humming experts use the standard apply signature and workspace management. Note: QuantKey conversion utilities generated with claude. cc @jinzhen-lin Tested the mxfp4 quanti…

  • 59575da #47008 — [XPU] exclude unsupported models for test_tensor_sechma.py (#47008)
    • 作者: Yan Ma | +15/-3 | 2 个文件

    Some multi-modality models(mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4, baidu/UnlimitedOCR, etc) are not supported on XPU platform and should excluded in tests like test_tensor_schema.py. pytest -s -v tests/models/multimodal/processing/test_tensor_schema.py ## Test Result —

  • a4e3cb4 #47018 — [mypy] Enable mypy for tests directory (#47018)
    • 作者: Martin Hickey | +76/-1 | 1 个文件

    MyPy was disabled for tests/ because it only checked for files in the vllm/ directory. This commit enables it and adds all relevant directories to the list to be fixed SEPARATE_GROUPS. This is next step of #26533 to enable MyPy for the project. Note: This PR not fix the MyPy errors for the tests/ directory. This will be done in a step by step basis in subsequent PRs.

  • ab132ee #46567 — Fix model info cache for package models (#46567)
    • 作者: soaringk | +37/-2 | 2 个文件

    Fix model-info cache invalidation for package-style model registrations. Some registered model entries point to a package entry point that re-exports classes implemented in submodules. If the cache key only covers the package init.py, changes in those implementation submodules may not invalidate cached model capability metadata. This was found while reviewing the MiniMax-M3 PP work in #45810, …

  • 9e86352 #47011 — [CI Failure] Add transformers version check for openai/privacy-filter (#47011)
    • 作者: wang.yuqi | +4/-0 | 1 个文件

    Ci is currently using transformers==5.5.3, but openai/privacy-filter requires 5.6.0. ## Test Result —

  • 5051698 #44589 — Remove unnecessary load_weights methods (#44589)
    • 作者: Harry Mellor | +821/-1975 | 54 个文件

    This PR adds missing functionality to AutoWeightsLoader which allows us to delete the load_weights method boilerplate from 41 architectures in vLLM. Every one of these architectures can automatically load: - GPTQ checkpoints with correct bias skipping - FP8 checkpoints with various scale formats - Checkpoints with fused or sharded qkv_proj/gate_up_proj weights — The specific changes are: - Enabl…

  • f6bb868 #47009 — Fix docs on main (#47009)
    • 作者: Harry Mellor | +1/-1 | 1 个文件
  • 4559c43 #43591 — [MM][CG] Gemma3 Encoder CUDA Graph (#43591)
    • 作者: Soyaazz | +151/-1 | 3 个文件

    Partial impl of #38175 ## Progress Engine serve command: Bench command: ## Test Result ### No Encoder CUDA Graph ### Encoder CUDA Graph Enabled ### Comparison | Metric | No CUDA Graph | CUDA Graph Enabled | Change | |——–|:————:|:——————:|:——:| | Mean TTFT (ms) | 618.10 | 556.59 | -9.9% | | Median TTFT (ms) | 514.64 | 480.83 | -6.6% | | P99 TTFT (ms) | 1607.50 | …

  • 5274c11 #46800 — [Rust Frontend] Add Harmony Renderer for GPT-OSS (#46800)
    • 作者: Bugen Zhao | +1033/-32 | 27 个文件

    Add a native Harmony chat renderer for GPT-OSS in the Rust frontend. With this change, RendererSelection::Auto chooses the new Harmony renderer when model_type == “gpt_oss”. The renderer uses oss-harmony to render the Chat Completions input directly into Prompt::TokenIds, matching the existing Harmony output processor path and avoiding the generic Jinja/HF rendering path for GPT-OSS. The renderer …

  • 58d6a6e #42920 — [CPU] Support cpu compressed-tensor w8a8 int8 moe (#42920)
    • 作者: Yuwen Zhou | +410/-9 | 6 个文件

    Support cpu compressed-tensor w8a8 int8 moe python -m pytest tests/kernels/moe/test_cpu_quant_fused_moe.py -v ## Test Result python examples/basic/offline_inference/generate.py –model RedHatAI/Qwen3-30B-A3B-Instruct-2507-quantized.w8a88 –max-model-len 512 –enforce-eager —

  • a2abce6 #38128 — [EPLB] Mask padding in EPLB load recording (#38128)
    • 作者: Ilya Markov | +289/-14 | 17 个文件

    When CUDA graphs or DP padding inflate the batch size, padded tokens flow through the MoE router and get counted in eplb_map_to_physical_and_record, polluting the expert load statistics used for EPLB rebalancing. This PR adds a num_unpadded_tokens and num_unpadded_tokens_tensor field to EplbModelState so that the EPLB router can distinguish real tokens from padding. Now runners need to fill the nu…

  • 311ad68 #46956 — Remove boilerplate missed by #46820 (#46956)
    • 作者: Harry Mellor | +1/-35 | 6 个文件

    This PR removes the remaining references to self.expert_weights in all modelling code. Fixes https://github.com/vllm-project/vllm/issues/46948

  • 0472436 #46968 — [Spec Decode] Avoid redundant hidden-states gather in draft prefill (#46968)
    • 作者: Woosuk Kwon | +4/-1 | 1 个文件

    Some EAGLE and MTP draft models reuse the same hidden states for both sampling and the next-step input. In these cases, we do not need to gather the last hidden states twice. This PR adds a small optimization to skip the redundant torch.gather, avoiding unnecessary overhead in the draft-model path.

  • 4dfbf15 #41026 — [Model] Add support for openai/privacy-filter (#41026)
    • 作者: Fabian Joswig | +225/-5 | 8 个文件

    Adds support for openai/privacy-filter (OpenAIPrivacyFilterForTokenClassification): a gpt-oss-style MoE encoder (GQA 14/2, YaRN RoPE, 128 experts top-4, attention sinks) repurposed as a bidirectional token classifier for PII detection. Every layer uses non-causal attention with a banded ±sliding_window mask. The LM head is replaced with a 33-class BIOES score head. The new model reuses gpt-oss by …

  • 03c6d01 #46629 — [OCP MX ] Add back emulation to available OCP MX backends list (#46629)
    • 作者: fxmarty-amd | +1/-11 | 3 个文件

    Addresses https://github.com/vllm-project/vllm/pull/46142#discussion_r3454573115. We have a single source of truth regarding which backend supports MXFP4/OCP MX - this PR reverts a previous change the made it compulsory to use –moe-backend emulation in order to dispatch on OCP MXFP4/MXFP6 emulation backend (https://github.com/vllm-project/vllm/pull/41436/changes#r3237495909). This revert aligns t…

  • 4b643c4 #46961 — [GLM5] Fix minor typo (#46961)
    • 作者: Woosuk Kwon | +1/-1 | 1 个文件

    Fixes a minor attribute name error in #46876

  • 89876b0 #46876 — [GLM5] Implement op fusion for GLM5/DSV3.2 (#46876)
    • 作者: Woosuk Kwon | +1530/-91 | 9 个文件

    This PR ports #38595 to the new model definitions for GLM-5 and DeepSeek V3.2. On 8x GB200 GPUs across 2 nodes with TP8, this PR reaches ~290 tok/s using nvidia/GLM-5.2-NVFP4 with MTP=3. The current op fusion work includes: * Two large fused kernels for miscellaneous RoPE and normalization ops in attention and the sparse indexer * AllReduce + RMSNorm fusion after both attention and MoE * Small fus…

  • 5c91039 #46635 — [GLM5.2 Perf] Replace MOE all-reduce with reduce-scatter, 3.1%~3.2 E2E Throughput improvement (#46635)
    • 作者: Wentao Ye | +86/-12 | 1 个文件

    Origin: full -> all-reduce -> full on every rank -> chunk -> local MoE -> all-gather Now: full -> reduce-scatter -> local shard -> local MoE -> all-gather if needed Note that this works for deepseek series as well. vllm serve zai-org/GLM-5.2-FP8 –kv-cache-dtype fp8_e4m3 –enable-expert-parallel –tensor-parallel-size 8 –tool-call-parser glm47 –enable-auto-tool-choice –reasoning-parser glm45 –…

  • 09841ae #46846 — [Render][Speculator] Add return_loss_mask to render endpoint for training data generation (#46846)
    • 作者: Ranran | +288/-20 | 7 个文件

    Adds a return_loss_mask field to ChatCompletionRequest and a loss_mask field to GenerateRequest, so POST /v1/chat/completions/render can return a per-token assistant/trainable mask alongside token_ids. This is needed by downstream training frameworks (e.g. speculators) that use the render endpoint to delegate tokenization to vLLM instead of…

🖥️ Kernel

  • 6149187 #46819 — [Kernel] Triton MLA logits workspace (#46819)
    • 作者: Nicolò Lucchesi | +64/-29 | 1 个文件

    TritonMLAImpl.forward_mqa allocates a fresh float32 scratch tensor attn_logits on every decode forward call. This PR addresses the TODO https://github.com/vllm-project/vllm/blob/bf292b5f6b537d154fc09a3b232f89cbc66827f5/vllm/v1/attention/backends/mla/triton_mla.py#L185-L196 by allocating a logits workspace ahead of time, using a single buffer that layer can share. This makes it so that we don’t pay…

  • 49e28e8 #44010 — [Kernel][Helion][1/N] Add Helion kernel for fused_qk_norm_rope (#44010)
    • 作者: Xiaohong (Sean) Chen | +5911/-0 | 4 个文件

    This PR is to add Helion kernel for operation. This is a subtask for https://github.com/vllm-project/vllm/issues/32962. ### Kernel level benchmark Environment Python: 3.12.12 Pytorch: 2.11.0 Cuda: 13.0 Helion: 1.0.0 Benchmark Setup Latency measure: Baseline: and Autotuning Setup Default “full” autotuning effort. i.e. LFBOTreeSearch with initial_population=FROM_RANDOM, copies=5, max_gen…

🦀 Rust Frontend

  • 3483240 #44512 — [Frontend] Consolidate scale out entrypoints (#44512)
    • 作者: wang.yuqi | +422/-289 | 45 个文件

    Following #22817 #41907 #44285 Consolidate scale out entrypoints - Tokens IN <> Tokens OUT APIs - Renderer APIs - Derenderer APIs 1. The name “Disaggregated Everything” (#22817) is great, but it is often confused with Disaggregated P/D (Prefill/Decoding) Inference. 2. split ServingRender & ServingDerender keep ci green ## Test Result —

⚡ Performance

  • eddfd4c #46750 — [Perf][2/N] Expand Triton kernel warmup coverage, Qwen (#46750)
    • 作者: Roberto L. Castro | +389/-0 | 2 个文件
  • 5ecae32 #45033 — [ROCm][Perf][MLA] Add AITER FlashAttention MLA prefill backend (ROCM_AITER_FA) (#45033)
    • 作者: xaguilar-amd | +269/-0 | 5 个文件

    Adds an AITER-backed MLA prefill backend that calls aiter.flash_attn_varlen_func directly instead of falling back to the upstream CK-tile FlashAttention path on ROCm. It dispatches the fast aiter::fmha_fwd_* kernel on gfx950 and, unlike the CK fallback, needs no V padding (native qk=192 / v=128 head dims) and returns softmax_lse already shaped (nheads, total_q) (no LSE transpose). This is …

  • c7ca0bc #44313 — [ROCm][Perf] Add Fused Shared Expert (FSE) support for GLM-4.5/6/7 (#44313)
    • 作者: Olga Miroshnichenko | +254/-105 | 2 个文件

    Extend the AITER Fused Shared Expert (FSE) path — originally added for DeepSeek-V2/V3 (#28540) and Qwen3-Next (#39280) — to the GLM-4 MoE family (GLM-4.5, GLM-4.6, GLM-4.7). When VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1, the shared expert is folded into the AITER FusedMoE kernel as n_shared_experts extra expert slots, eliminating the separate shared-expert MLP forward pass at low/medium concurr…

🧪 CI/Tests

  • db28ae2 #46999 — [ROCm][CI] Explicitly tear down multimodal offline LLMs (#46999)
    • 作者: Andreas Karatzas | +91/-29 | 6 个文件

    AMD CI hit a failure where test_chat_multi_image passed, but the next multimodal LLaVA test failed during engine startup because only about 15 GiB was free on a 192 GiB GPU. That points at the previous test not finishing teardown before the next engine starts. These tests construct large offline LLM instances directly, but teardown was relying on del llm and the global cleanup fixture. The old wea…

  • c2127a2 #46895 — [ROCm][CI] Fix rlhf_async_new_apis Example On ROCm (#46895)
    • 作者: Micah Williamson | +3/-4 | 1 个文件

    Workaround for a RCCL bug (see https://github.com/ROCm/rocm-systems/issues/5756). The weight-transfer NCCL group fails to initialize (P2P/IPC to the peer GPU) when Ray masks GPUs via ROCR_VISIBLE_DEVICES. Tell Ray to leave GPU visibility untouched so every actor can see all GPUs. Also, I’ve removed RAY_EXPERIMENTAL_NOSET_CUDA_ENV_VAR=1 from this test, which was added back in https://github.com/vll…

  • a2a92cb #46930 — [Hardware][AMD][CI] Tweak mirrored tests; improve CI base dependency change detection (#46930)
    • 作者: Matt | +30/-35 | 6 个文件

    This PR makes the following changes in order to improve AMD CI stability: 1. Unmirrors the Distributed Tests (2xH100-2xMI300) test group, which was originally promoted in https://github.com/vllm-project/vllm/pull/45998. This test group was never passing but was believed to be due to a bug (that was recently fixed by https://github.com/vllm-project/ci-infra/pull/395). 2. Mirrors the Examples test g…