共 45 个 commit，涉及 178 个文件，+5967/-1299 行变动。

概要

统计项 数值
Commit 数 45
变更文件 178
新增行数 +5967
删除行数 -1299

Commit 列表

⚡ Performance

  • 51a9956 #46474 — [ROCm][Perf] Fused shared expert for Minimax M3 (#46474)
    • 作者: Fangzhou Ai | +46/-13 | 1 个文件

    Fuse MiniMax-M3 shared expert into the routed MoE call, a follow-up PR for https://github.com/vllm-project/vllm/pull/46419 vLLM server start: VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 VLLM_USE_BREAKABLE_CUDAGRAPH=0 .venv/bin/python -m vllm.entrypoints.cli.main serve amd/MiniMax-M3-MXFP4 –block-size 128 -tp 4 –max-model-len 9472 –attention-backen…

  • 867fd5e #46184 — [ROCm][Perf] Use flydsl moe with Minimax-M3 mxfp8 weights on gfx950 and implemented moe-backend selection (#46184)
    • 作者: Hongxia Yang | +374/-4 | 5 个文件

    [ROCm][Perf] Use flydsl moe with Minimax-M3 mxfp8 weights on gfx950 aiter’s flydls moe has shown perf improvement on various scenrios on mxfp8 serving on gfx950. This PR is to integrate the support, and also have the capability to fall back to triton mxfp8 dot-scaled implementation. Implementation and usability decisions: Reused moe-backend aiter and triton, and refactored the moe selection logic….

  • c2507fb #46545 — [ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model (#46545)
    • 作者: Hongxia Yang | +110/-20 | 5 个文件

    Perf optimization MiniMax-M3 runs its single shared expert as a separate dense MLP every MoE layer (a gate_up GEMM + activation + down GEMM on a side stream, ×60 layers). Folding it into the routed grouped GEMM removes those per-layer launches, which is the dominant cost at low/medium concurrency (decode is launch-bound). The fusion is numerically equivalent to the separate-MLP path. - **r…

🧪 CI/Tests

  • 9fd00ee #46905 — [ROCm][CI] Move remaining mi250_2 tests out of the MI250 queue (#46905)
    • 作者: Andreas Karatzas | +64/-150 | 1 个文件

    The mi250_2 queue is being cleared out. This PR moves mi250_2 tests into gfx942. Migrations include: - Distributed Model Tests (2 GPUs) now runs on mi300_2 under mi300 - basic_correctness. - Distributed Comm Ops now runs on mi300_2 under mi300 - distributed. - Plugin Tests (2 GPUs) now runs on mi300_2 under mi300 - plugins. - Distributed DP Tests (2 GPUs) and V1 e2e (2 GPUs) now run on mi300_2 und…

  • 091d139 #46891 — [ROCm][CI] Add TRITON_ATTN score absolute tolerance floor (#46891)
    • 作者: peizhang56 | +8/-3 | 1 个文件

    The text_vs_text score (~0.10) in test_cross_encoder_online_vision.py drifts ~0.008 absolute on gfx942/ROCm 7.2 under the TRITON_ATTN backend. Because the value is near zero, this small fixed absolute drift reads as ~7.9% relative error and exceeds the tight 0.045 relative tolerance ~@~T even though the larger scores (0.53, 0.74) stay well within it. Relative tolerance is the wrong metric for a pr…

  • 68ee830 #46409 — [ROCm][CI]Fix test_concat_and_cache_mla_rope_fused on ROCm (#46409)
    • 作者: Divakar Verma | +39/-1 | 1 个文件

    This test was failing in CI on ROCm due to fp16 numerical mismatches between the reference and the fused kernel. The root cause is that the torch-native reference (forward_native) implicitly upcasts fp16×fp16 to fp32, while the fused CUDA kernel runs in native fp16. (Also refer to the in-depth analysis by @mawong-amd - https://github.com/vllm-project/vllm/pull/32408) Changes: - Use the AITER path …

  • 17a71d8 #46658 — [ROCm][CI] Relax fused layernorm quant test tolerances for one-ULP outliers (#46658)
    • 作者: Divakar Verma | +90/-55 | 4 个文件

    The fused RMSNorm+quant kernels compute the normalized intermediate and quantization in one pass, while the reference path runs a native fp32 RMS norm followed by a separate quant kernel. The differing fp32 reduction order (and FMA contraction in the x * s_variance * w chain on HIP) occasionally flips a single group’s bf16 abs-max by one ULP, or pushes an element across an E4M3 tie boundary into t…

  • 3f67477 #46854 — [CI] Don’t try and download files that we already know don’t exist (#46854)
    • 作者: Harry Mellor | +55/-9 | 3 个文件

    huggingface_hub keeps track of files which were requested but do not exist. This information persists in the HF cache. It does this so that if you request this file again, huggingface_hub doesn’t have to make a request because it already knows this file doesn’t exist. The documentation for this is here: https://huggingface.co/docs/hub/en/local-cache#noexist--non-existence-cache Before this PR vLLM…

  • 75fdcc8 #46873 — [CI] Add @ivanium to CODEOWNERS for KV-cache/offload areas (#46873)
    • 作者: Yifan Qiao | +10/-8 | 1 个文件

    Add @ivanium as a reviewer/code owner for the KV-cache and KV-offload areas of the codebase, and register ownership for the recently merged simple_kv_offload connector (both source and tests). Specifically: - Adds @ivanium to existing KV-related entries: kv_transfer, config/cache.py, v1/core, v1/kv_cache_interface.py, kv_connector_model_runner_mixin.py, tests/v1/core, tests/v1/kv_connector. - Adds…

  • 6e2fb02 #46851 — [ROCm][CI] Fix rlhf_nccl.py on ROCm (#46851)
    • 作者: Charlie Fu | +16/-1 | 1 个文件

    There is a runtime bug in RCCL handling the envs HIP_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES causing crash of rlhf_nccl.py. This PR tries to post a temporary workaround for this.

  • 274325d #46867 — [ROCm][CI] Remove V1 Sample + Logits from mi250 Queue (#46867)
    • 作者: Micah Williamson | +0/-20 | 1 个文件

    Remove V1 Sample + Logits from mi250 in AMD CI as it is no longer needed

  • 95e6442 #46859 — [Hardware][AMD][CI] Fix Kernels Quantization test timeout (#46859)
    • 作者: Matt | +29/-28 | 1 个文件

    This PR fixes the Kernels Quantization test group on AMD CI, which is currently hitting a hang in tests/kernels/quantization/test_nvfp4_emulation.py::test_nvfp4_moe_correctness and causing 6h timeouts. The issue arises because individual HF file downloads of model-00002-of-00060.safetensors from nvidia/Kimi-K2.6-NVFP4 are timing out for unclear network reasons. Thus, this model is temporarily remo…

  • 37ce349 #46735 — [CI] Fix failing CUDA graph capture in Triton MOE (#46735)
    • 作者: fxmarty-amd | +6/-1 | 2 个文件

    Following a change in https://github.com/vllm-project/vllm/pull/46254 that used self.a1_scale or self.a1_gscale as moe_kernel_quantize_input argument in Triton MOE, graph capture started failing with: resulting in the test CUDA_VISIBLE_DEVICES=0 pytest tests/models/quantization/test_gpt_oss.py -s -vvvvv -k “test_gpt_oss_attention_quantization and amd/gpt-oss-20b-MoE-Quant-W-MXFP4-A-FP8-KV-FP8-0.89…

  • 302954e #46823 — [ROCm] [CI] fix transcription flakiness AMD: Entrypoints Integration (API Server OpenAI - Part 1) (mi325_1) (#46823)
    • 作者: TJian | +1/-0 | 1 个文件

    Fix AMD: Entrypoints Integration (API Server OpenAI - Part 1) (mi325_1) https://buildkite.com/vllm/ci/builds/74553/list?sid=019f026d-b093-4a93-bacc-e23d26bcb2db&tab=output The language autodetection might have failed. So, we tell the whisper that it is english audio explicitly. ## Test Result —

📦 Other

  • b588f66 #46862 — [GLM5.2 Perf] fused_indexer_q_rope_quant triton kernel, 1.9% ~ 3.3% E2E Throughput improvement. (#46862)
    • 作者: Wentao Ye | +167/-0 | 2 个文件

    We have similar fused kernel in DSv4, this path works for GLM as wel Originally: Q RoPE -> cat -> FP8 quant -> q_scale fold into weights Now: Q RoPE + FP8 quant + q_scale fold into weights in one triton kernel. vllm serve zai-org/GLM-5.2-FP8 –kv-cache-dtype fp8_e4m3 –enable-expert-parallel –tensor-parallel-size 8 –tool-call-parser glm47 –enable-auto-tool-choice –reasoning-parser glm45 –port…

  • 455f25a #46775 — [CLI] Add flag to print TTFT and TPS in vllm chat (#46775)
    • 作者: Benjamin Chislett | +67/-9 | 2 个文件

    An opt-in mode to print the speed timings (TTFT and TPS, where TPS includes TTFT in the average, number of generated tokens and e2e latency) after each turn in vllm chat and vllm complete. Opt-in to maintain current UX, but it’s super handy for demos and quick sanity checks that correctness is maintained and ballpark perf is good (especially with spec decoding where speedup is domain dependent) I …

  • d706dec #44551 — fix: Correct reasoning-end detection for prompt history (#44551)
    • 作者: JasonCohere | +645/-1 | 4 个文件

    This PR fixes a bug in the way our reasoning parser determines whether or not a collection of tokens is considered a concluded reasoning block via the is_reasoning_end method. The original implementation simply scanned for the presence of any <|END_THINKING|> token, and could be called on the entire input prompt (i.e. on the initial turn) or the intermediate deltas. This meant that it would treat …

  • ddd3855 #45924 — [MoE Backend] add HPC-Ops MoE backend (#45924)
    • 作者: Cheng Jiang | +453/-2 | 5 个文件

    Support hpc-ops moe backend. HPC-Ops is a production-grade, high-performance, and easy-to-use operator library for LLM inference, developed by the Tencent Hunyuan AI Infra team. You can enable hpc-ops moe backend by specify –moe_backend hpc. Unit test – skip when hpc-ops not install: Benchmark： ## Test Result Unit test result: Benchmark Result： —

  • 00e045b #46758 — [ROCm][CI TG] refactor and fix deepep_moe test group (#46758)
    • 作者: Divakar Verma | +84/-58 | 3 个文件

    Fix test_deepep_moe.py on ROCm and consolidate MoE test accuracy checks 1. Buffer reuse (low-latency DeepEP): The modular kernel and its DeepEP all2all buffer are now built once and reused across all token chunks instead of being re-created per chunk. Re-creating it per chunk re-initializes rocSHMEM, which only supports a single allocation per process on ROCm and aborts with “Unknown allocator…

  • 2e05885 #44984 — fix(docker): eliminate race conditions in shared buildkit cache mounts (#44984)
    • 作者: weizhoublue | +8/-6 | 3 个文件

    Root Cause: Omission of sharing=locked on –mount=type=cache defaults BuildKit to sharing=shared . Multiple containers read/write same host directory concurrently, causing cargo write collisions. Remedy: Add sharing=locked to cargo cache mounts in Dockerfile , matching configuration in Dockerfile.cpu and Dockerfile.rocm ## Test Result —

  • 1a92dfc #35232 — [Build] Show error message when using ROCm with LTO and different compilers (#35232)
    • 作者: Dāvis | +10/-0 | 1 个文件

    Currently when default compiler /usr/bin/c++ is GCC and when building with LTO (-flto=auto) when ROCm is using Clang then created shared libraries _C.abi3.so _moe_C.abi3.so, _rocm_C.abi3.so will be missing symbols like PyInit__C. The issue is that .o files contain GCC LTO IR which are ignored when linking with Clang. 1. Inspect symbols for _C.abi3.so _moe_C.abi3.so, _rocm_C.abi3.so 2. Ensure Pytho…

  • d0f8008 #46644 — [Build] Update vllm to point to vllm-project/flash-attention commit that builds FA3 with torch stable API. (#46644)
    • 作者: Chris Leonard | +1/-1 | 1 个文件

    As a part of https://github.com/vllm-project/vllm/issues/26946. Updated vllm_flash_attn.cmake to use FA fork that builds a libtorch ABI stable FA3 library. Currently, this is pointing towards my flash-attention fork but if this passes the CI, we will merge that fork with vllm-project/flash-attention, then we will point to the new tag. see https://github.com/vllm-project/flash-attention/pull/152 to…

  • c6dd32a #46762 — [ModelRunner V2] Support realtime embeddings (#46762)
    • 作者: Nick Hill | +52/-36 | 6 个文件

    Realtime models like voxtral require embeddings for decode steps too.

  • af16446 #44465 — Vram semaphore infra (#44465)
    • 作者: Brandon Pelfrey | +1087/-15 | 11 个文件

    Continuing the work described in RFC #30839, this change introduces two new things: 1. A new video decoding backend based on pynvvideocodec which is a lightweight dependency enabling HW video decoding on all NVIDIA GPUs. Video decoding always takes place using GPU index 0. 2. A VRAM Semaphore. The semaphore enables the GPU video decoding backend to logically allocate N bytes from a pool, o…

  • 1d41009 #46753 — [ModelRunner V2] Fix cross-attention block table sizing (#46753)
    • 作者: Nick Hill | +3/-2 | 1 个文件

    Without this there can be OOB errors with e.g. Nemotron-Parse

  • b94f212 #46776 — [ModelRunner V2] Deduplicate ModelState init logic (#46776)
    • 作者: Nick Hill | +26/-77 | 4 个文件

    Just simplification, no functional changes.

  • d8eb734 #46820 — Fix Transformers backend FP8 MoE and remove some boilerplate (#46820)
    • 作者: Harry Mellor | +80/-156 | 29 个文件

    The Transformers backend was eagerly storing expert_weights before quantization. This meant that when using FP8 models a stale reference to the unquantized expert weights was kept and the memory usage is higher than expected. This PR fixes the issue by removing the eager population of self.expert_weights and removing the set_eplb_state method now that an implementation has been added to MixtureOfE…

  • c40d307 #36701 — [Core] Remove FlashAttention block size restriction for hybrid models (#36701)
    • 作者: Thomas Parnell | +0/-17 | 1 个文件
    • Remove the block size restriction in FlashAttentionBackend.get_supported_kernel_block_sizes() that limited hybrid models with float32 Mamba cache to block sizes [16, 32, 64] - This restriction was introduced in #27753 to work around NaN propagation from stale fp32 Mamba data in reused KV cache blocks - #35219 has since solved the root cause by zeroing freshly allocated KV cache blocks via KVBloc…
  • 65e655d #46808 — [GLM-5] Add DSV3.2/GLM5 to vllm/models/ (#46808)
    • 作者: Woosuk Kwon | +1170/-0 | 5 个文件

    This PR adds a new, compile-free, hardware-specific implementation of DeepSeek V3.2 and GLM-5 under vllm/models/deepseek_v32/. This PR does NOT remove or replace the existing (torch-compilable) implementation though. The initial implementation is mostly hardware-neutral, but is put under deepseek_v32/nvidia, as it’ll be soon customized for NVIDIA GPUs. AMD and XPU support will follow as well s…

  • c6554f3 #46769 — [CPU] Fix macOS/Apple Silicon hang by enabling OpenMP in the build (#46769)
    • 作者: Michael Goin | +49/-15 | 10 个文件

    macOS CPU builds never passed -fopenmp (regression from #16086, which added omp.h but not the flag), so the kernels’ #pragma omp parallel regions were compiled out and ran serially while omp_get_max_threads() still reported all cores. The attention split-KV path then spins on an in-region barrier (guard_counter != thread_num) waiting for a thread team that never starts → deadlock on any prompt lon…

  • 3d3b964 #46705 — Migrate Voxtral to mistral-common 1.11.5 audio API (#46705)
    • 作者: Julien Denize | +22/-29 | 12 个文件

    This PR bumps mistral-common and update relevant files to prevent deprecation warnings. ## Test Result —

  • 658b54e #46771 — [ModelRunner V2] Update scheduler tests to cover MRV2 paths (#46771)
    • 作者: Nick Hill | +53/-18 | 2 个文件
  • abc7154 #46831 — [CI/Build][CPU] Add test image cache clean-up (#46831)
    • 作者: Li, Jiang | +38/-0 | 1 个文件
    • Add image build cache clean-up to reduce issue. ## Test Result —
  • 4e07ca2 #44800 — [Core] Add VLLM_GPU_SYNC_CHECK env var (#44800)
    • 作者: Nick Hill | +279/-0 | 5 个文件

    vLLM now uses asynchronous scheduling by default and in the majority of cases. Performance relies on the absence of any gpu<->cpu synchronizations on the main cuda stream, but such syncs can be opaque and it is easy for them to creep in accidentally. This change adds a VLLM_GPU_SYNC_CHECK env var which enables torch.cuda.set_sync_debug_mode for the model forward pass and sampler, so that we can ea…

  • e71bc6d #46799 — [Rust Frontend] Use oss-harmony for Harmony output processing (#46799)
    • 作者: Bugen Zhao | +51/-427 | 2 个文件

    Switch the library used for Harmony output processing from Inferact/openai-harmony (introduced in #46696) to oss-harmony, a more actively maintained fork of the original library. This gives the Rust frontend two practical benefits: - picks up upstream Harmony bug fixes and improvements - embeds the tokenizer vocabularies in the library itself, removing the runtime download path and the related req…

  • 8921c4b #46122 — [ROCm] [Performance] Optimize aiter moe for DeepSeekV4 (#46122)
    • 作者: TJian | +28/-15 | 1 个文件

    NOTE: This feature is validated with aiter v0.1.15.post1, we require the AITER on upstream to be updated. Mxfp4MoeBackend.AITER_MXFP4_BF16 is introduced only for DeepSeek V4 original weights. It follows the speed of light reference (ATOM repo) https://github.com/ROCm/ATOM/blob/d7964d50be17a3910dec1d22cf1d4f6205764cb4/recipes/mesh/multi-node-atom.md?plain=1#L74 in preparing the weights. 1. lmeval b…

  • 8e39424 #46419 — [ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4 (#46419)
    • 作者: qli88 | +27/-8 | 4 个文件

    [ROCm][feature] Enable MiniMax-M3-MXFP4 with AITER MoE This feature requires AITER version bump (latest version). Accuracy test: 1. vLLM server start: VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 VLLM_USE_BREAKABLE_CUDAGRAPH=0 vllm serve /data/amd-MiniMax-M3-MXFP4/ –block-size 128 -tp 8 –attention-backend TRITON_ATTN –tool-call-parser minimax_m3 –enable-auto-tool-choice –reasoning-pars…

  • 950ee4c #44226 — [API] Add token offsets to render endpoints (/v1/…/render) (#44226)
    • 作者: Hyunkyun Moon | +516/-20 | 13 个文件

    vLLM’s render endpoints (/v1/completions/render and /v1/chat/completions/render) are GPU-less routes that external serving systems layered on top of vLLM use to fetch preprocessing results (rendered token IDs, chat-template-applied prompts, etc.). Typical consumers include inference orchestration / scheduling layers like llm-d that need to make prefix-cache-aware routing or load-balancing decision…

  • d980a3c #46780 — [ROCm] Fix AITER_UNIFIED_ATTN Dispatching After AITER Bump (#46780)
    • 作者: Micah Williamson | +30/-3 | 6 个文件

    After bumping AITER to v0.1.16.post2, we are seeing failures such as the following: Here we update the AITER_UNIFIED_ATTN backend to specify its compatible dtypes.

  • bf292b5 #46071 — [Docs] Remove BambaForCausalLM from supported hybrid models list (#46071)
    • 作者: AgenticSpark | +1/-1 | 1 个文件

    #45990 removed BambaForCausalLM: vllm/model_executor/models/bamba.py was deleted and the model moved into _PREVIOUSLY_SUPPORTED_MODELS in registry.py. docs/usage/v1_guide.md still lists BambaForCausalLM among the supported hybrid models, so this drops it from that list. This PR originally also repointed a stale BambaForCausalLM reference link in docs/contributing/model/basic.md, but that line was …

🐛 Bug Fix

  • 2ff76a5 #46760 — [ROCm][Bugfix] Pass num_kv_splits to aiter mla_reduce_v1 (#46760)
    • 作者: Rohan Potdar | +3/-0 | 1 个文件

    #46692 bumped AITER to v0.1.16.post2, which includes ROCm/aiter#3391. That aiter PR added a required num_kv_splits argument to mla_reduce_v1 (now position 7 of the registered op schema). The FP8 ASM prefill reduce call in the AITER dense MLA backend (rocm_aiter_mla.py) was not updated, so on main today the out_3d Tensor is bound to the num_kv_splits (int) slot and **every MLA model crashes at …

  • 701a23d #45719 — [Bugfix][Model] Support tensor parallelism for DiffusionGemma (#45719) (#46177)
    • 作者: Carlos Alvarado Martinez | +69/-3 | 4 个文件
  • dccb412 #46843 — [Bugfix][Parser] Pass token IDs to parser.parse() in Responses API and batch serving (#46843)
    • 作者: Ben Browning | +3/-0 | 3 个文件

    Chat completion serving already passes model_output_token_ids to parser.parse(), but Responses API and batch serving did not. Wire those call sites to pass token IDs consistently, so that all non-streaming calls to parser.parse() provide token ids and we get consistent parsing logic across all based on those token ids. The parser engine based parsers don’t really use those token ids today, but thi…

🦀 Rust Frontend

  • 77f8796 #46437 — [Frontend][Gpt-oss] Use process_eos() to flush Harmony Parser outputs. (#46437)
    • 作者: yzong-rh | +91/-61 | 2 个文件

    Add a flush() function to HarmonyParser that calls process_eos() to flush partial content (used to be directly read off the parser state) and reset parser after parse() and parse_delta(finish=True). This 1. paves the way for fixing https://github.com/vllm-project/vllm/issues/45736 (full fix requires updating serving layer code as well, deferred) 2. is the first step of <https://github.com/vllm-p…

🔩 Misc

  • 5e3dad0 #46783 — [Misc] Move the legacy api_server.py to the examples directory. (#46783)
    • 作者: wang.yuqi | +6/-6 | 5 个文件

    Move the legacy api_server.py to the examples directory. Eliminate the confusion caused by vllm/entrypoints/api_server.py and vllm/entrypoints/openai/api_server.py. This example can still works. ## Test Result —