共 43 个 commit，涉及 183 个文件，+4437/-3772 行变动。

概要

统计项 数值
Commit 数 43
变更文件 183
新增行数 +4437
删除行数 -3772

Commit 列表

📦 Other

  • b9a7cd4 #45415 — [12/n] final _C library kernel migration (#45415)
    • 作者: Chris Leonard | +239/-212 | 17 个文件

    This PR continues the libtorch stable ABI migration (see https://github.com/vllm-project/vllm/issues/26946) for vLLM and is the final _C library kernels to move to the _C_stable_libtorch library. The PR moves csrc/quantization/activation_kernels.cu to csrc/libtorch_stable/quantization/activation_kernels.cu, along with the weak_ref_tensor (defined in ops.h), silu_and_mul_quant, and persistent_maske…

  • ec67d7a #40367 — [xpu] bump up vllm-xpu-kernels v0.1.10 and upgrade 2618 umd (#40367)
    • 作者: Kunshang Ji | +10/-9 | 4 个文件

    https://github.com/intel/compute-runtime/releases/tag/26.14.37833.4 ## Test Result —

  • c9135db #45762 — [Docs] Update stale LMCache examples (#45762)
    • 作者: Samuel Shen | +84/-197 | 8 个文件

    LMCache has restructured its documentation around multi-process (MP) mode and removed the legacy in-process experimental path (see LMCache#3665). This PR brings the vLLM-side LMCache examples back in sync. Changes (all under examples/disaggregated/lmcache/): - Drop LMCACHE_USE_EXPERIMENTAL everywhere — it has been removed from LMCache (no references remain in the LMCache source) and the cu…

  • 2a6c6b9 #46001 — [DeepSeek-V4] Support TEP=16 for the block-FP8 shared expert (#46001)
    • 作者: Jeff (Junze) Ma | +44/-0 | 1 个文件

    Currently TEP=16 is not supported for DeepSeek-V4. The shared expert is a block-FP8 MLP (weight_block_size=[128,128], moe_intermediate_size=3072). A per-rank TP shard must be a whole number of 128-blocks (a block’s FP8 scale can’t be split). At TP=16, 3072/16 = 192 is not a multiple of 128, so the standard linears fail to build: Weight input_size_per_partition = 192 is not divisible by block_k = 1…

  • 560fb8b #46099 — [Cohere] Remove dead prepare_structured_tag override in Cohere parser (#46099)
    • 作者: Flora Feng | +0/-10 | 1 个文件

    BaseCohereCommandReasoningParser overrode prepare_structured_tag solely to return original_tag, with a comment claiming the base default returns None and would otherwise clear the structural tag produced in adjust_request. That comment is now stale. #45396 (3b8fc3fe6d) changed the base ReasoningParser.prepare_structured_tag from return None to return original_tag. The Cohere override is therefore …

  • 675cd5d #46095 — [Model Runner V2] Fix MRv2 memory leak test (#46095)
    • 作者: Wentao Ye | +1/-1 | 1 个文件

    MRv2 will allocate additional 2MB in second round warm up, then keep stable, which doesn’t have memory leak at all. This PR fixes the test. VLLM_USE_V2_MODEL_RUNNER=1 pytest tests/models/multimodal/generation/test_memory_leak.py::test_no_memory_leak[image_urls0] -xvs Originally Now

  • 35e4dd4 #45659 — [KV Connector][Mooncake] Async lookup to reduce scheduler overhead (#45659)
    • 作者: Yifan Qiao | +197/-11 | 6 个文件

    Looking up keys in Mooncake currently happens synchronously inside get_num_new_matched_tokens, on the scheduler’s critical path. Each lookup costs ~1–2 ms per request on average, and that latency is paid serially during scheduling. This PR adds an optional async lookup mode that offloads the lookup to a background thread so its latency overlaps with the current scheduling step; results are con…

  • 4ce2d01 #46025 — fix(anthropic): auto-detect template support for mid-conversation system messages (#46025)
    • 作者: MrFan | +120/-6 | 2 个文件

    Problem #44602 preserves inline role: system messages at their original position for prefix caching. However, some models have chat templates that reject system messages appearing after the first position (e.g., Qwen3.5/3.6, #41114). Without mitigation, these models would return 400 errors for valid Anthropic requests. ## Fix Auto-detect whether the chat template supports mid-conversation syste…

  • 16908e1 #45996 — [MRV2] Make FP32 Gumbel sampling more accurate (#45996)
    • 作者: Woosuk Kwon | +240/-8 | 2 个文件

    The current FP32 Gumbel sampling kernel is not accurate enough. This PR improves the accuracy with a simple trick: use log1p(-u) instead of log(u), enabling higher resolution in determining the argmax winner.

  • 225936a #46070 — [CI Bug] Revert #42379 to fix CI Multi-Modal Models (Extended Generation 1) (#46070)
    • 作者: Wentao Ye | +24/-12 | 2 个文件

    Fixes https://buildkite.com/vllm/ci/builds/72702#019ed6b9-5522-4391-9f85-3b8699e878f9 pytest tests/models/multimodal/generation/test_audioflamingo3.py::test_single_and_batched_generation_match -xvs Originally, error like CI Now

  • f6ba720 #45675 — (security) Upgrade Starlette to >= 1.0.1 to fix CVE-2026-48710 (#45675)
    • 作者: Juan Pérez de Algaba | +49/-98 | 4 个文件

    Starlette < 1.0.1 does not validate the HTTP Host header before reconstructing request.url, allowing attackers to poison request.url.path and bypass path-based security middleware (Advisory X41-2026-002). Upgrading was blocked by prometheus-fastapi-instrumentator 7.x pinning starlette < 1.0.0. Bump it to >= 8.0.0 (released 2026-05-29) which requires starlette >= 1.0.0, and raise the FastAPI floor …

  • b53b1c7 #44446 — [Model Runner V2] Migration to support quantized model by default [5/N] (#44446)
    • 作者: Wentao Ye | +1/-4 | 2 个文件

    Part of https://github.com/vllm-project/vllm/issues/41286 Tests coverd by CI

  • ea6078f #46044 — [KV Connector][Offloading] Disable parallel-agnostic fs-tier cache on V2 model runner (#46044)
    • 作者: Itay Etelis | +18/-0 | 3 个文件

    Disable the parallel-agnostic fs-tier KV cache (#44733) when the V2 model runner is active. The parallelism-invariant assumption it relies on does not hold under V2.

  • a0df04e #45708 — [Tests] Add Qwen3 streaming parser delta boundary cases (#45708)
    • 作者: Palaiologos1453 | +67/-0 | 1 个文件

    Adds focused regression coverage for Qwen3 streaming tool-call delta boundaries, originally opened against bbrowning/vllm#1 and redirected upstream by @bbrowning. Covered cases: - a complete <tool_call>…</tool_call> arriving in a single streaming delta - a delta that closes one tool call and starts the next tool call Related to vllm-project/vllm#45439 and vllm-project/vllm#45440. The parser beha…

  • e2352c2 #45706 — [ROCm][Spec Decode] Fix probabilistic draft probs test attention backend (#45706)
    • 作者: stefankoncarevic | +12/-2 | 2 个文件

    test_propose_stores_probabilistic_draft_probs hardcodes the FLASH_ATTN attention backend, which builds FlashAttentionMetadata. On ROCm, the speculative-decoding proposer only accepts Triton/Rocm/AITER metadata (allowed_attn_types is built under current_platform.is_rocm() in vllm/v1/spec_decode/llm_base_proposer.py), so the test fails on AMD MI architectures (gfx942 / MI325, gfx950 / MI355) with: T…

  • bf2a393 #46053 — Temporarily remove @markmc from CODEOWNERS (#46053)
    • 作者: Mark McLoughlin | +0/-10 | 1 个文件

    Removing myself temporarily while I’m on leave. For reference, much of the current observability review backlog is tracked by: - https://github.com/orgs/vllm-project/projects/44 - Prometheus metrics - https://github.com/orgs/vllm-project/projects/55 - OpenTelemetry tracing

  • d682968 #45990 — [Model] Remove BambaForCausalLM (#45990)
    • 作者: Tiezhen WANG | +6/-529 | 5 个文件
    • Remove BambaForCausalLM (IBM Bamba-9B, hybrid Mamba-2 + attention model) - The Bamba architecture served as the basis for nemotron_h.py (NemotronHForCausalLM) which remains fully supported and is the actively maintained successor - Add BambaForCausalLM to _PREVIOUSLY_SUPPORTED_MODELS pointing to vLLM v0.23.0 ## Files changed - vllm/model_executor/models/bamba.py — deleted - vllm/model_executor/m…
  • 021cdf7 #43179 — Fix _riscv_supports_rvv_vlen128() to detect RVV on hardware without zvl flags (#43179)
    • 作者: lyd1992 | +30/-3 | 3 个文件

    Fix _riscv_supports_rvv_vlen128() so that RISC-V hardware without zvlb flags in /proc/cpuinfo can still correctly detect RVV support. On some RISC-V hardware (e.g. SG2044), /proc/cpuinfo advertises the V extension (rv64imafdcv) and zve* / zvfh flags but does not report any zvlb VLEN hint. The existing Python-only check required zvl128b to be present: This caused _get_attn_isa() to fall b…

  • 4cb5e74 #44801 — [Rust Frontend]: Add /get_world_size route with static parallel size (#44801)
    • 作者: Ashar | +258/-2 | 11 个文件

    Add parallel config for world size in EngineCoreReadyResponse that is captured and persisted during startup and served directly. This makes the implementation fast but there are cases such as Elastic DP where ranks can change after startup. For those cases. We have two options: - Use a utility function that provides current rank each time a request arrives. - Add a new Output Stream message like c…

  • afdcbd5 #45681 — [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
    • 作者: Tuukka Sarvi | +545/-52 | 8 个文件

    This PR fixes DeepSeek V4 / DeepSeek V4 Flash functional issues on ROCm gfx942 (MI300X), while preserving the gfx950 path. The main fixes are: - Avoid unsupported FP8 arithmetic when converting UE8M0 block scales. - Make the fused DeepSeek V4 qnorm/RoPE/KV-insert path emit the correct FP8 bytes on ROCm: - gfx942 uses FNUZ encoding. - gfx950 uses OCP encoding. - FNUZ uses max value 224.0. - Propaga…

  • 8d4f549 #42727 — fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config (#42727)
    • 作者: AlexHuang | +579/-428 | 20 个文件

    This PR fixes the AWQ dequantize issue on Intel XPU (#41469) and refactors the AutoAWQ config for better XPU performance support. The refactoring approach is inspired by #38288 (consolidate GPTQ). Key changes: 1. Fix AWQ dequantize on XPU: Added AutoAWQXPULinearMethod using Intel’s oneDNN int4 GEMM kernel for proper XPU support 2. Unified AutoAWQConfig: Merged awq_marlin.py and awq.py into…

  • 351c72d #44991 — [CPU] Skip Triton kernel monkey-patches when Triton-CPU is available (#44991)
    • 作者: Jonathan Mamou | +26/-6 | 3 个文件

    When Triton-CPU is installed and active (the default in x86 Docker images since 0.22.0), the @triton.jit kernels in rejection_sampler.py, block_table.py, and llm_base_proposer.py can run natively on CPU — but they fail to compile due to int32/int64 type mismatches that Triton-CPU’s stricter compiler catches. This PR: 1. Fixes the type mismatches in 5 Triton kernels by replacing 0 (int32) with …

  • 7299e65 #45950 — [Rust Frontend] Return model metadata fields in /v1/models (#45950)
    • 作者: Tahsin Tunan | +153/-26 | 8 个文件

    The Rust frontend’s GET /v1/models returned only id, object, created, and owned_by, dropping the vLLM extension fields the Python frontend includes on each model card. Most visibly, it omitted max_model_len. This brings the Rust /v1/models response into parity with the Python frontend so it stays a drop-in replacement. This was an unlisted gap (not on the #44280 roadmap); /v1/models itself was alr…

  • 0898535 #45093 — Fix Stale Encoder Cache After Weight Update (#45093)
    • 作者: littlecircle0730 | +12/-0 | 2 个文件

    Fixes #44910. After a native weight update (update_weights() / finish_weight_update()), the V1 multimodal encoder cache could continue serving stale encoder outputs. Encoder cache entries are keyed only by mm_hash. As a result, repeated multimodal inputs could still hit existing cache entries and reuse encoder outputs generated before the weight update. The cache remained stale until reset_encoder…

  • 5fd3b27 #45444 — [Mooncake] Skip KV lookup for non-reachable SWA blocks (#45444)
    • 作者: Wei Zhao | +120/-18 | 4 个文件

    This PR adds some optimizations for reducing overhead in Mooncake KV offloading. - In lookup, skip SWA blocks that are not eligible to be considered cache hit, using kv cache group’s reachable_block_mask. - Use None as all-True mask for store_mask, saving list construction overhead for full cache: masks.append([True] * num_chunks if mask is None else mask). ## Performance benchmark: DeepSeek v4 TP…

  • 1e9f04d #44602 — fix(anthropic): preserve inline system message position for prefix caching (#44602)
    • 作者: MrFan | +103/-28 | 2 个文件

    Problem PR #44283 merged all inline role: system messages from the messages array into a single leading system message. This changes the conversation prefix, breaking KV-cache hits in multi-turn dialogues. #44048 (currently open) moves the same merge logic to the protocol layer but retains the same prefix-breaking behavior. ### Example of the problem ## Fix - Remove inline system message extrac…

🐛 Bug Fix

  • 69bdd34 #46038 — [Bugfix] Fall back to Pydantic loc for param in validation errors (#46038)
    • 作者: Muhammad Fawaz | +71/-0 | 2 个文件

    Previously, param was only populated for errors carrying a custom VLLMValidationError in their Pydantic ctx, leaving it None for ordinary missing-field/wrong-type validation errors even though the field name was available from error[’loc’]. Fixes the param field in validation error responses being silently None for ordinary Pydantic validation failures (missing required fields, wrong types), even …

  • ab66606 #45895 — [bugfix]Indexer init skip and MTP TopK share for iteration (#45895)
    • 作者: Jared Wen | +69/-30 | 9 个文件

    fix GLM-5.2 BF16 init Indexer in skip_topk layer fix GLM-5.2 MTP Port-Norm cycle ## Test Result MTP Mean Acceptance Length raise form 3~ to 4~, while Mean Acceptance Rate raise to 60% IFBench remain 74.62 —

  • 9ea3a40 #42120 — [Bugfix] Fix corrupt outputs in MoE FP8 LoRA responses and MoE base model responses when LoRAs are loaded (#42120)
    • 作者: nv-nedelman-1 | +196/-4 | 5 个文件

    The bugs below were originally discovered/introduced in vLLM 0.20.0. The recent/ongoing MoE refactor slightly changed the bug’s preconditions and manifestations, but the root causes remain and are fixed by this PR. Fixes two bugs: - #42007 - #42008 ## Implementation Details/Context - uses no_lora_flag_cpu to short-circuit the moe lora path; precedence in lora_shrink_op.py, lora_expand_op.py etc - …

  • 7f616c3 #46091 — [Bugfix] [Parser] Fix empty tool block silently dropping subsequent content (#46091)
    • 作者: Ben Browning | +26/-3 | 4 个文件

    I regularly run reviews of vLLM code through a variety coding assistants point to self-hosted models. One of my sessions (Codex CLI running w/ Qwen 3.6-27B) picked up a theoretical bug where an empty tool block (ie no parsed function, just open and close tool markers with empty or nonsense content in the middle) would cause all subsequent content to get dropped. So, I had a patch sitting on the si…

  • 41dcf49 #45371 — [Bugfix][KV Connector] Disable Mooncake TP put-striding when DCP > 1 (#45371)
    • 作者: Yifan Qiao | +87/-8 | 2 个文件

    Fix Mooncake store offloading under decode context parallelism (DCP). Running DCP + Mooncake offload produces Mooncake OBJECT_NOT_FOUND (-704) errors on load (Failed to get N Mooncake keys from sub-batch) at a rate of exactly 1 - 1/put_step (e.g. 75% of keys at TP=DCP=4 with MLA). Root cause: when num_kv_head < tp_size, TP ranks holding the same KV heads deduplicate stores by striping PUTs across …

  • 79ca54d #45040 — [Bugfix][Quantization] Don’t reject fp8_e5m2 KV cache for non-fp8 quantized checkpoints (#45040)
    • 作者: Ting SUN | +16/-2 | 1 个文件

    Closes #39137. _init_kv_cache_quant rejects –kv-cache-dtype fp8_e5m2 for every quantized checkpoint with “fp8_e5m2 kv-cache is not supported with fp8 checkpoints”. But weight-only checkpoints (INT4/INT8 compressed-tensors, AWQ, GPTQ) carry no fp8 KV scales and are not fp8. On Ampere, where fp8_e5m2 is the only fp8 KV cache dtype that runs at all (fp8_e4m3 is unsupported by the hardware), this mak…

  • 09f3cd5 #46047 — [Bugfix] [Parser] Fix Qwen3 latent bug in partial params dropping values containing < (#46047)
    • 作者: Ben Browning | +19/-1 | 2 个文件

    In the new Qwen3 parser, _PARTIAL_PARAM_RE used [^<]*$ to capture in-progress parameter content. When a value contained a literal < (HTML, code, math like x<5), the regex failed to match entirely. In practice this appears to be handled gracefully by the holdback mechanism. But, it is a latent bug even though I have not been able to trigger that in the real world so far outside of the unit tests he…

  • 4583630 #45466 — [Bugfix][Kernel] Check output alignment in vectorize_with_alignment (fixes misaligned-address crash for non-multiple-of-8 head sizes) (#45466)
    • 作者: Humphrey | +61/-6 | 2 个文件

    Fixes #41257. FLEX_ATTENTION with head_size=46 (e.g. the inferno-project/vllm-mixtral-2 fuzz model) crashes with CUDA error: misaligned address. Root-cause analysis (details in the issue comment): the crash is not in FlexAttention, CUDA graphs, or Ada drivers — it is reshape_and_cache_flash_kernel issuing vectorized stores to misaligned KV-cache rows. vectorize_with_alignment (csrc/libtorch_stable…

  • 5099474 #45747 — [Bugfix][ROCm] Fix rocm_aiter_per_tensor_quant custom op aliasing (#45747)
    • 作者: Rohan Potdar | +166/-13 | 2 个文件

    Fixes the rocm_aiter_per_tensor_quant custom op aliasing crash on ROCm FP8 per-tensor models (e.g. amd/Llama-3.1-8B-Instruct-FP8-KV) with torch 2.12: In the static-scale path, per_tensor_quant_hip returns the input scale tensor as its second output, which aliases an op input and violates the custom op’s non-aliasing contract. This is latent but only raised under torch.compile once the aliasing che…

  • 058cc0a #45656 — [Bugfix] Restore is_sym guard for zp in GPTQ/CT MoE to fix symmetric quant regression (#45656)
    • 作者: Yuwen Zhou | +10/-4 | 2 个文件

    Fix a regression introduced by https://github.com/vllm-project/vllm/pull/43409#issuecomment-4700406550 that broke symmetric GPTQ MoE quants (e.g. Autoround) on NVIDIA GPUs. #43409 removed the if not self.quant_config.is_sym else None guard in AutoGPTQMoEMethod.get_fused_moe_quant_config so that CPU could pass synthesized zero points for symmetric models. However, this caused symmetric GPU quants (…

  • 837db76 #43984 — [Bugfix][Tool Parser] Handle non-finite numbers in coerce_to_schema_type (#43984)
    • 作者: Ashish Patel | +102/-3 | 2 个文件

    vllm/tool_parsers/utils.py::coerce_to_schema_type() coerces model-emitted tool-call argument strings to their JSON Schema type. It is a shared helper used by a growing set of tool parsers (Qwen3 Coder, MiniMax-M2, Step3, DeepSeek V3.2, Seed-OSS), so a defect here affects all of them. This PR fixes two issues with non-finite numeric values: 1. Uncaught OverflowError (crash). For a number-typed …

🧪 CI/Tests

  • ecf9d83 #45509 — [AMD][CI] Fix Language Models Test (Extended Generation) failures (#45509)
    • 作者: Oxana Korzh | +5/-1 | 1 个文件

    Two unrelated failures in tests/models/language/generation/test_common.py::test_models: - openbmb/MiniCPM4.1-8B: its remote modeling_minicpm.py imports is_torch_fx_available from transformers.utils.import_utils, which was removed in Transformers v5, so the HF reference run fails to load with an ImportError. Cap the HF run at transformers 4.57 via max_transformers_version so the comparison is skipp…

  • 25faa1f #43802 — [CI]Enable mxfp4 lora test for ROCm platform (#43802)
    • 作者: qli88 | +25/-11 | 1 个文件

    Enable mxfp4 lora test for ROCm platform

  • 21da47d #45970 — [ROCm][CI] move lora%N test to mi300 and gate (#45970)
    • 作者: Divakar Verma | +25/-16 | 2 个文件

    This PR moves the lora test to mi300 for numerical stability, since some of the tests depend on exact output matching. (Notes: “lora/test_whisper.py::test_whisper_multi_lora” is unstable on mi250)

⚡ Performance

  • c3c6d72 #45988 — [Perf] Remove unused loggers in reasoning/ (#45988)
    • 作者: Ivy Xu | +0/-27 | 9 个文件

    Remove unused loggers in reasoning/. This makes parsers faster at startup. ## Test Result Tests passed.

🦀 Rust Frontend

  • 6c379b9 #45915 — [Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser (#45915)
    • 作者: Chauncey | +534/-1948 | 11 个文件

    [Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser I tested it locally with both enable_thinking=True and enable_thinking=False, as well as with stream=True and stream=False. In all cases, the output was parsed correctly. ## Test Result —

🖥️ Kernel

  • 22cc891 #46006 — [Kernel] Add PDL support for DeepGEMM kernel (#46006)
    • 作者: Jee Jee Li | +87/-23 | 3 个文件

    Test Result —