共 78 个 commit，涉及 531 个文件，+20249/-6698 行变动。

概要

统计项	数值
Commit 数	78
变更文件	531
新增行数	+20249
删除行数	-6698

Commit 列表

📦 Other

• 10ad649 #50412 — Update torch version to 2.13.0+cpu (#50412)
  • 作者: ylangtsou | +1/-1 | 1 个文件

  Update the torch version from 2.11.0+cpu to 2.13.0+cpu in requirements/build/tpu.txt to fix the vllm-tpu wheel build failure caused by missing dependency torch==2.13.0 reference PR: https://github.com/vllm-project/vllm/pull/48155 1. Build vllm-tpu package locally 2. Verify the built wheel by launching the server ## Test Result vllm-tpu built successfully, and the vLLM server started without issues…

• df71917 #49236 — [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
  • 作者: Wentao Ye | +354/-30 | 15 个文件

  Part of https://github.com/vllm-project/vllm/issues/45861 Optimize workspace reuse for eager break vllm serve deepseek-ai/DeepSeek-V4-Flash –tensor-parallel-size 4 –enable-expert-parallel –attention-backend FLASHMLA_SPARSE_DSV4 –attention-config ‘{“use_fp4_indexer_cache”:true}’ –kv-cache-dtype fp8 –tokenizer-mode deepseek_v4 –all2all-backend allgather_reducescatter –port 8003 ### Acc ### P…

• 82ae416 #48770 — [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770)
  • 作者: Matthew Bonanni | +855/-26 | 11 个文件

  Depends on https://github.com/vllm-project/flash-attention/pull/155 For pure prefills, there is a threshold of sequence length below which a masked MHA pathway is faster than sparse MQA. The DeepSeek V3.2 paper mentions this: > Note that for short-sequence prefilling, we specially implement a masked MHA mode to simulate DSA, which can achieve higher efficiency under short-context conditions. ## Te…

• f5ffc59 #50408 — [Renderer] Warm up the renderer properly. (#50408)
  • 作者: wang.yuqi | +50/-46 | 6 个文件

  Warm up the renderer properly. get_hf_processor takes about 22 seconds to run for the first time on my machine. main 1 22.131629880001128 2 0.14735023300090688 this pr 1 0.19055349900008878 2 0.1441873680014396 ## Test Result —

• 2c4d348 #48408 — [KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408)
  • 作者: Itay Etelis | +1016/-7 | 6 个文件

  Records on CanonicalKVCacheRef, per layer, how the worker’s KV page maps into a canonical (parallelism-free) page: the full offloaded block, all KV heads, all block_size * dcp * pcp tokens. A cache offloaded under one parallel config can later be rearranged or consumed under another. All parallelism reasoning lives in vllm/v1/kv_offload/sharding.py; the connector and everything downstream consume …

• b2fb83e #50148 — [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148)
  • 作者: Martin Hickey | +34/-33 | 14 个文件

  AttentionMetadataBuilder.init declares kv_cache_spec: AttentionSpec but the mamba backends are handed a MambaSpec. Those two are siblings under KVCacheSpec not subtypes, so the hint on the base is wrong for every SSM backend. The base also does self.kv_cache_spec = kv_cache_spec which pins the attribute’s type for the whole hierarchy. That’s why gdn_attn, mamba_attn and linear_attn each carry …

• 92643d6 #50242 — K3 DSpark AR fusion (#50242)
  • 作者: Jee Jee Li | +24/-8 | 5 个文件

  This PR depends on #50000 and should be merged after it. ### GSM8K - main branch - This PR ## Test Result —

• 6e311c6 #44941 — [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941)
  • 作者: bnellnm | +243/-234 | 80 个文件

  #41184 deleted the FusedMoE class and replaced it with a function that constructed a MoERunner and related MoE classes. This PR renames FusedMoE -> FusedMoEFactory to better reflect the purpose of the function. CI/Lint ## Test Result —

• 0e9b500 #50116 — [chore] clean-up weight prepack for INT8 MoE (#50116)
  • 作者: Fadi Arafeh | +7/-26 | 4 个文件

  This aims to simplify / fix the weight prepack path for INT8 MoE. Currently, weight prepack is owned by the MoE Backend (not by the MoE kernel), this is problematic because: - in CPU, we have a single MoE backend, but multiple kernels (for AArch64, x86). As a result, choosing the correct packing function requires: (1) selecting CPU packing function for the CPU MoE backend (2) inside the CPU packin…

• 88bc8fb #50219 — [CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219)
  • 作者: Rehan Khan | +182/-27 | 6 个文件

  Enable s390x specific CPU optimizations: - Wire DEFINE_FAST_EXP for attention softmax via VXE minimax polynomial - Add vectorized reduce_sum/reduce_max using vec_add/vec_max/vec_sld - Add INT8Vec16 for FP32→INT8 packing (vec_cts/vec_packs) - Add FP32Vec16 exp/abs/min/clamp/reduce_min for dnnl_kernels support - Enable oneDNN build and op registration for s390x (INT8 W8A8 GEMM) Run INT 8 model and r…

• 482cfc2 #46981 — [XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation (#46981)
  • 作者: Chaojun Zhang | +19/-74 | 7 个文件

  Previously, vllm_c RMSNorm kernels were only enabled for CUDA_ALIKE; XPU used a duplicate xpu_kernels provider. This PR switches XPU to the shared vllm_c implementation and sets IR priority to: vllm_c > native. No duplicate work: This is a small refactor to remove redundant XPU dispatch code and reuse vllm_c for RMSNorm on XPU.

• 5d7647a #50530 — [UT] add skipif for rocm aiter sampler UT (#50530)
  • 作者: Qiming Zhang | +4/-0 | 1 个文件

  This case use MockPlatform to replace current_platform, which causes error on other platform like CUDA and XPU. On CUDA: On XPU: Add skipif to make this case run only on ROCM.

• 3ee2bd1 #49691 — Fix duplicate HunyuanVL image boundary tokens (#49691)
  • 作者: Mi Jiazhi | +10/-3 | 1 个文件

  The prompt replacement matched only the image token while adding a full start/image/end sequence, resulting in duplicated image boundary tokens and XD-RoPE indexing errors. Match the full start/image/end placeholder and use the same format for dummy inputs.

• 1180b60 #49686 — [Multimodal] Expose mm hash algothrim selection to cli args (#49686)
  • 作者: Isotr0py | +234/-61 | 17 个文件

  • Currently, mm hash algos can only be configured through env var VLLM_MM_HASHER_ALGORITHM - This PR exposes it to cli args so that user can specify it through –mm-hasher-algorithm ## Test Result Tests should pass —

• 0f17394 #50293 — [Model Runner V2] Enable encoder token classification (#50293)
  • 作者: Taneem Ibrahim | +91/-19 | 4 个文件

  Model Runner V2 supports sequence embedding and classification after #48791, but rejects token_classify even for encoder-only models. Encoder-only attention support from #49331 provides the required single-step prefill path. This PR enables executable encoder-only token classification in MRV2, including task discovery, validation, output handling, and Hugging Face accuracy coverage. ## Reproducer …

• b49eaf2 #48047 — [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047)
  • 作者: Jeff (Junze) Ma | +16/-19 | 1 个文件

  Remove the query-head padding in the DeepSeek V4 FlashInfer sparse-MLA attention layers. flashinfer-ai/flashinfer#3545 relaxes the trtllm_batch_decode_sparse_mla_dsv4 guard to accept h_q ∈ {8, 16, 32, 64, 128} on both the SM100 and SM120 decode paths (previously only {64, 128}). Before this, vLLM padded the per-rank head count up to the nearest kernel-supported value — SM100 (DeepseekV4FlashIn…

• 541128b #50301 — [KV Offload] Enable single-copy MLA layout for CPUOffloadingSpec (#50301)
  • 作者: Chang Guo | +119/-17 | 3 个文件

  Enable the single-copy (replicated) MLA layout for the default KV-offload backend CPUOffloadingSpec, closing out the plan in #47929. After #50094 moved the default backend’s worker CPU buffer onto the shared SharedOffloadRegion (on CUDA/ROCm), the default backend can now deduplicate the per-TP-rank-replicated MLA KV the same way TieringOffloadingSpec already does — but only on deployments that act…

• ef0d084 #50349 — [XPU] Fix FP8 block scale layout for MLA compatibility (#50349)
  • 作者: Ma Jian | +34/-25 | 1 个文件

  Store block scale as [n_blocks, k_blocks] view (matching checkpoint shape) instead of contiguous [k_blocks, n_blocks]. This ensures: - MLA’s scaled_dequantize sees the expected shape for dequantization during process_weights_after_loading - apply_block_scaled_mm recovers the contiguous [k_blocks, n_blocks] buffer that oneDNN expects via .t() - test_can_initialize_large_subset[DeepseekV3ForCausalLM…

• 6724051 #50328 — [CI/Build][AMD] Install triton_kernels via CMake (#50328)
  • 作者: Ryan Rock | +20/-30 | 3 个文件

  Starting with ROCm 7.14 we will install dependencies such as Triton from a package index instead of building from source. However, the package index does not include triton_kernels. Running gpt-oss-120b on gfx950 without a ROCm-specific triton_kernels results in the below error due to using the triton_kernels that ships with vLLM by default. > triton.runtime.errors.OutOfResources: out of resource:…

• 4689c7d #50006 — [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI325X (#50006)
  • 作者: vanshbhatia-amd | +51/-0 | 1 个文件

  Adds a tuned selective_state_update (Mamba SSU decode kernel) launch config for the AMD Instinct MI325X for the cache_dtype=float16 variant, mirroring how the MI300X (#47945), MI355 (#47767, #48372) and MI350 (#48159) configs were done. vLLM bundles per-device SSU configs for NVIDIA parts (B200, GB200, H100, H200, RTX PRO 6000) and, more recently, MI300X / MI350 / MI355. On MI325X get_ssm_conf…

• 4f1da84 #46516 — Enable gfx1250 ROCm architecture (#46516)
  • 作者: Juan Villamizar | +1850/-104 | 33 个文件

  Initial enablement of gfx1250, currently working through known models and UTs. Testing key models and expanding as hardware becomes available ## Test Result testing so far: | Model | Backend | Decoding | flexible | strict | |—|—|—|—:|—:| | dsv4f | AITER_MXFP4_BF16 | greedy t=0 | 0.96 ±0.0197 | 0.96 ±0.0197 | | dsv4f | AITER_MXFP4_BF16 | sampling t=1.0 | 0.97 ±0.0171 | 0.97 ±0.0171 | | gp…

• 3333d7c #49361 — [ROCm]: bump AITER to 0.1.19 (#49361)
  • 作者: Rohan Potdar | +1/-1 | 1 个文件

  Test Result —

• 0bff0ce #50458 — [Kimi K3 Bug] Fix deepgemm support for kimi k3 (#50458)
  • 作者: Wentao Ye | +4/-1 | 2 个文件

  VLLM_USE_RUST_FRONTEND=0 vllm serve moonshotai/Kimi-K3 –moe-backend auto –gpu-memory-utilization 0.95 –tensor-parallel-size 8 –load-format fastsafetensors –no-enable-flashinfer-autotune –max-model-len 1048576 –kv-cache-dtype fp8 –attention-config ‘{“use_prefill_query_quantization”:true,“mla_prefill_backend”:“TRTLLM_RAGGED”}’ –enable-prefix-caching –enable-auto-tool-choice –tool-call-par…

• dec13a3 #48892 — [Model Runner V2][Spec Decode] Add multi-layer MTP speculator (#48892)
  • 作者: Giancarlo Delfin | +1227/-32 | 9 个文件

  Context Multi-Module MTP is a variant of MTP that has a separate model layer for each speculative step (i.e. MTP module #i drafts the ith token). Currently the Inkling model’s MTP model supports this, with up to 8 specualtive tokens. This PR adds a new speculator class (MultiModuleMTPSpeculator) to support this functionality. Currently, only single-module MTP is supported in Model Runner V2, whi…

• e9096fc #50406 — [Rust Frontend] Improve startup failure and readiness logs (#50406)
  • 作者: Bugen Zhao | +39/-12 | 2 个文件

  • report top-level Rust frontend failures through the configured tracing formatter, including the full error cause chain - distinguish server startup from HTTP and gRPC readiness - include the served model and endpoint details in readiness logs ## Why Rust frontend startup errors previously escaped through main’s Result, so their final presentation did not use the frontend’s normal log format. The…

• 0eec856 #50468 — Add Humming indexed-MoE regression test (#50468)
  • 作者: Michael Goin | +89/-17 | 1 个文件

  Follow up to #50338 to add a regression unit test ## Test Result —

• 70bd109 #50273 — [Quantization] Honor --linear-backend for ModelOpt W4A16 (#50273)
  • 作者: Netanel Haber | +42/-21 | 3 个文件

  ModelOptNvFp4W4A16LinearMethod currently hardcodes usage of the Marlin kernel, ignoring –linear-backend. #### PR * –linear-backend=auto: Marlin remains the default * –linear-backend=humming: Explicitly selected compatible backend is now honored. Related but not duplicate: #49382 adds a FlashInfer W4A16 kernel and changes the auto selection on SM121. This PR adds no kernel and does not change th…

• c27b080 #50444 — [compile] Fix fake kernel return dtype (#50444)
  • 作者: Richard Zou | +34/-2 | 3 个文件

  Align the runtime and fake-kernel metadata for unified KV-cache update operators. This prevents Inductor metadata assertions when quantized KV caches use uint8 while attention activations use bfloat16 or float16. In PyTorch 2.14, we have increased metadata assertions in Inductor that require these to be more exact. Add regression coverage for both regular and fused RoPE update paths. ## Test Resul…

• 3f90c7e #50378 — [ROCm] Pass pointers to FlyDSL MoE kernels (#50378)
  • 作者: Andreas Karatzas | +46/-69 | 1 个文件

  • Fix FlyDSL MoE compilation failures across all five MI355 shards in AMD CI build 11459: 1, 2, 3, 4, and 5. - Convert tensor-backed stage-one and stage-two buffers to fx.Pointer values, including null pointers for optional scales, to match AITER’s updated FlyDSL ABI. - Reuse the normalized launch arguments and avoid launching a compiled kernel twice on the initial cache miss.

• 12a34a6 #46720 — [ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)
  • 作者: Mehmet Cagri | +187/-9 | 5 个文件

  Speed up the DeepSeek-V4 fp8 attention projections fused_wqa_wkv (fused Q/KV down-proj) and wo_b (output proj) on ROCm (gfx950) by routing them through AITER’s weight-preshuffled block-scale GEMM instead of the default block-scale GEMM. ## What it does - Load time: B-preshuffle the two projections’ fp8 weights via the existing shuffle_weight((16, 16)) and cache the block scales (prepare_attn_p…

• 5f8f728 #50243 — [Build] Fix CUDA release wheel builds (#50243)
  • 作者: Kevin H. Luu | +90/-50 | 3 个文件

  Fix two independent release-v2 wheel-build failures: - Pin CUDA 12’s nvidia-nccl-cu12 to NCCL 2.30.7 in both the DeepEP wheel-build stage and final runtime image stage. - Guard Kimi K3’s BF16 conversion paths for CUDA host and pre-Ampere compilation, and reject BF16 Kimi fused-MLA launches below SM80 rather than silently skipping work. ## Root causes DeepEP d4f41e4e93 uses NCCL GIN APIs that requi…

• 837eae6 #50298 — [DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298)
  • 作者: Wentao Ye | +44/-23 | 3 个文件

  Part of https://github.com/vllm-project/vllm/issues/45861 Passing an out tensor to avoid additional torch.full kernel call Covered in unit tests Perf can be seen in this AI generated script And we get

• 904fae8 #50312 — [DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved (#50312)
  • 作者: Wentao Ye | +24/-28 | 3 个文件

  We do torch.empty and copy in pp last rank even if there is no mtp enabled, this PR fixes the issue, Perf gain can be seen in this AI generated script And we get

• 5b95890 #50339 — [FlexAttention] Avoid encoder block-mask compile explosion (#50339)
  • 作者: Andreas Karatzas | +66/-13 | 4 个文件

  • Default encoder-only FlexAttention masks to 128-token Q/KV blocks, while keeping the existing small-block defaults for paged KV attention and honoring explicit block-size overrides. The long-text fixture also uses a local seeded generator so token counts and compiler shapes are reproducible. - On MI355, the exact cold-cache test improved from 300.31 seconds to 37.99 seconds; the full long-text m…

• 30b4e7f #48981 — [rl] Stateful Trainer Send: IPC [2/N] (#48981)
  • 作者: Aaron Hao | +392/-422 | 8 个文件

  Context Second of the planned three-PR split of the trainer-side weight-transfer rework. - PR 1 (merged): introduced the new trainer-side abstractions (WeightSource / ModuleSource, VLLMWeightSyncClient, TrainerWeightTransferEngine, WeightTransferTrainerFactory). Purely additive; no backend migrated. https://github.com/vllm-project/vllm/pull/48042 - PR 2 (this one): migrate the IPC b…

• 59e831c #48757 — [Compilation]Fuse Transformers Residual Add + RMSNorm (#48757)
  • 作者: BadrBasowid | +298/-2 | 4 个文件

  Transformers models emit residual add and RMSNorm separately, preventing the use of FusedAddRMSNorm kernels. This change canonicalizes them into fused_add_rms_norm and handles the intervening reshape. Furthermore, the separate residual add and reshape operations prevented some patterns, like AR + rms, from being discovered and fused. Since this PR hides those offending ops, the fusions are fixed a…

• 1a20d23 #48947 — [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947)
  • 作者: Julien Denize | +2931/-1492 | 30 个文件

  This PR aims to discard previously separated tool call and reasoning parsers to a unified one. It should supports all mistral format tokenizers including: - pre v11 version (old tool calling format and no reasoning) - post v11 version (new tool calling format and potentially reasoning) - text or special token based reasoning - grammar from mistral-common guidance - empty tool-call names are now va…

• e2efe79 #47207 — [ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ (#47207)
  • 作者: sroberts-amd | +1031/-70 | 12 个文件

  This PR migrates deepseek_v32 to /vllm/models/deepseek_v32 This is an effort done with the help of Claude and is cleaned up through human review. Migrate the AMD specific needs for deepseekv32 to vllm/models/deepseek_v32/ Following patterns of other models, moved common elements into common, updated nvidia dependencies to reflect move. Validate DeepseekV3 2 > VLLM_ROCM_USE_AITER=1 vllm serve deeps…

• 38a267c #49570 — [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570)
  • 作者: Martin Hickey | +166/-101 | 30 个文件

  Part 1 of enabling mypy for tests directory. Implements PR0 and PR1 of the plan defined in feature #49569. MyPy checks tests/** with –follow-imports skip which hides real type errors. group_files() also claims every tests/** file for the “tests” entry in SEPARATE_GROUPS regardless of more specific entries also being present. Therefore removing a directory from SEPARATE_GROUPS alone does not enfor…

• 89d97d9 #50397 — docs(security): document Ray cluster trust model and env var propagation (#50397)
  • 作者: Juan Pérez de Algaba | +98/-0 | 1 个文件

  Adds a new section to the security guide explaining that vLLM treats the entire Ray cluster as a single trust domain and that RayExecutorV2 propagates driver environment variables to workers by default. Includes hardening recommendations for operators who scope credentials to the driver alone.

• 072a472 #50387 — [CPU] Bump up CPU kernels to latest version (#50387)
  • 作者: Li, Jiang | +2183/-1292 | 15 个文件

  Sync csrc/cpu/sgl-kernels/ from sglang (sgl-kernel/csrc/cpu) to latest upstream main, reconciling vLLM-specific patches layered on since the last bump (#41924): MXFP4 W4A16 MoE, AMX GDN dispatch, ISA-portability BLAS fallback, RISC-V scalar/RVV support, chunked-prefill has_initial_state fix, DFlash speculative decoding, and batched spec-decode conv-state updates. Also rejects one upstream regressi…

• aeeb36b #50000 — [New model] Kimi K3 (#50000)
  • 作者: Jiangyun Zhu | +2931/-1349 | 82 个文件

  7.30 Note: after merging, it still needs to install https://github.com/flashinfer-ai/flashinfer/releases/tag/v0.6.16rc5 to run K3 add moonshotai/Kimi-K3 model support Please use the docker image vllm/vllm-openai:kimi-k3 to run it, see vllm recipes https://recipes.vllm.ai/moonshotai/Kimi-K3. This branch has some extra dependencies: - SITU trtllmgen MOE: https://github.com/flashinfer-ai/flashinf…

📖 Documentation

• c036cb2 #50571 — [Doc] Add BgeM3EmbeddingModel to embedding supported models (#50571)
  • 作者: LG | +1/-0 | 1 个文件

  BgeM3EmbeddingModel is implemented and registered as an embedding architecture in vLLM, with BAAI/bge-m3 included in the embedding model registry tests. The BGE-M3 specific-model documentation also instructs users to override the model architecture with BgeM3EmbeddingModel in order to load its additional sparse and ColBERT embedding weights. However, BgeM3EmbeddingModel is currently missing from t…

🐛 Bug Fix

• a0cd2b6 #50302 — [Bugfix] Universally align block table width to 128 tokens (#50302)
  • 作者: Matthew Bonanni | +90/-30 | 8 个文件

  Share one block-table width calculation between MRV1, MRV2, and the DSA indexer. It applies 128-token alignment and kernel-block splitting, preventing indexer buffer mismatches under alignment and DCP. Fixes #46074. Supersedes #48404, #50050, and the alignment portion of #43970. ## Testing CUDA_VISIBLE_DEVICES=’’ .venv/bin/python -m pytest tests/v1/worker/test_gpu_model_runner.py -k get_block_tabl…

• 94e9ef0 #50137 — [Bugfix] Don’t transpose fused MoE quantization scales in RoutedExperts.load_weights (#50137)
  • 作者: Harry Mellor | +77/-8 | 2 个文件

  Fixes a regression from #47058 that breaks loading of fused, per-channel-quantized MoE checkpoints with TP > 1. RoutedExperts.load_weights normalises the orientation of fused expert tensors so that checkpoints storing them either way (e.g. Qwen3 VL MoE) load correctly. It identified fused tensors by rank alone and then transposed by testing for the absence of the hidden dim: Any 3D tensor with n…

• 0b5b49d #50491 — [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491)
  • 作者: Recoordinate | +13/-7 | 3 个文件

  Closes #50253. After the VLLMError hierarchy migration in #49665, seven user-facing errors in vllm/entrypoints/chat_utils.py still raised a bare ValueError, so they bypassed VLLMValidationError and the structured request-error handling (the API param field came back null). Convert all seven to VLLMValidationError, adding parameter= where the failing request field is unambiguous: - validate_chat_te…

• f727951 #50305 — [Bugfix] Re-land MiniMax M3 default video processor (#50305)
  • 作者: Taneem Ibrahim | +42/-12 | 3 个文件

  This forward-fix re-lands the MiniMax M3 default video-processor support from #50092 after its revert in #50313 : - Keeps MiniMaxM3VideoBackend in the model-specific MiniMax module. - Registers MiniMaxM3VLVideoProcessor with that backend. - Preserves Hugging Face-aligned frame sampling. - Treats a missing optional fused-allreduce operation as unavailable when it raises either ImportError or Attrib…

• 10e6b40 #50437 — [CPU][BugFix] Remove redundant kv cache write (#50437)
  • 作者: Fadi Arafeh | +45/-12 | 3 个文件

  While profiling some LLMs, I noticed that cpu_attn_reshape_and_cache gets called twice for each invocation of the attention kernel for causal and cross attention. Since https://github.com/vllm-project/vllm/pull/40470 - the kv write, for causal and cross attention is done by the caller (CPUAttentionBackend.do_kv_cache_update) - forward_includes_kv_cache_update is set to false We now only run cpu_at…

• bebf918 #50352 — [Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error (fixes #50337) (#50352)
  • 作者: Sebastian Woo | +96/-0 | 2 个文件

  Fixes #50337. jinaai/jina-embeddings-v5-text-nano fails at weight loading with 24 uninitialized weights: That message does not point at the actual problem. This adds a config-time check that rejects the unsupported variant with an explanation instead. ## Root cause The V5 family ships two different backbones under one architectures entry. Both -nano and -small declare architectures: [“JinaEmbe…

• 1d8be5c #50434 — [XPU] [BugFix] Add deepseek_v4_fp8 to xpu supported_quantization list (#50434)
  • 作者: Xiaochang Wu | +1/-0 | 1 个文件

  Fix related PR: https://github.com/vllm-project/vllm/pull/44513 ## Test Result —

• d91f7af #50467 — [Hardware][AMD][Kernel][CI][Bugfix] Fix ROCm DeepEP FP8 max (#50467)
  • 作者: Matt | +6/-11 | 2 个文件

  Fixes two FP8-related issues in the current ROCm DeepEP pin: 1. On FP8 e4m3fn-based platforms (gfx950) on the current pin, the low-latency FP8 kernels were using an incorrect FP8 e4m3fnuz max of 240.0 instead of 448.0 because of a missing macro guard. 2. On FP8 e4m3fnuz-based platforms (gfx942), vLLM uses an adjusted max of 224.0 for easy parity with e4m3fn math (with a x2 scale and flushing negat…

• ab98034 #50420 — [Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness (#50420)
  • 作者: Bugen Zhao | +55/-53 | 5 个文件

  Fixes #50295. Kimi-K3’s XTML index is local to one assistant message. The Python and Rust parsers previously derived northbound tool-call IDs from that index, so the same tool called in separate turns could receive the same ID. Downstream agents that key calls and results across the conversation can then associate a result with the wrong call. This PR: - lets Python’s ToolCall default factory gene…

• f1899b2 #45227 — [Bugfix][ROCm] AITER MLA: size MTP verification decode metadata for real qlen/dtype (#45227)
  • 作者: chaeminlim-mb | +555/-36 | 2 个文件

  On ROCm AITER MLA, MTP verification decode (qlen = num_speculative_tokens + 1, so 4 for MTP3) was sized as ordinary qlen=1 decode: get_mla_metadata_info_v1 received qo_len=1 and the kernel recomputed metadata internally for qlen>1. The verification logits come out wrong, the accept/reject step drifts from the target distribution, and accuracy drops. Passing q/kv dtypes (#46997) does not fix this o…

• d6938b7 #50200 — [Bugfix][Rust Frontend] Select earliest-completing stop string (#50200)
  • 作者: Samuel Laferriere | +50/-3 | 1 个文件

  matches_stop_string in the Rust frontend selects the first stop string in list order that matches in the search window. Python selects the one that completes earliest in the text, ties broken by list order — fixed deliberately in #49391 and pinned by tests/detokenizer/test_check_stop_strings.py. The Rust frontend still has the pre-#49391 logic, so the two frontends disagree on the same req…

• 68fb303 #48438 — [Bugfix] Preserve Marlin runtime tensor storage across weight reload (#48438)
  • 作者: Ryan Clark | +330/-17 | 6 个文件

  The fix is verified at three levels: red/green CPU regression tests (in this PR), live-engine GPU validation on an RTX 4090 (pointer identity across reload, a graph-coupling flip test, and a livelock demonstration on unfixed main), and live bug-side/fixed-side validation of every sibling Marlin path. Method, results, and the validation script for the primary fix are in this description; the per-si…

• 30e333c #49840 — [Bugfix] Shut down private Tensorizer engines (#49840)
  • 作者: Andreas Karatzas | +96/-10 | 2 个文件

  • Explicitly close the renderer and EngineCore owned by tensorize_vllm_model after serialization. - Preserve the primary serialization error while attempting both cleanup paths and give nested workers a bounded reap grace. - Use the existing runner context in the direct serialization test. The private engine returned after serialization while roughly 23.8 GiB remained resident, so the next in-proc…

• 61cacd2 #50338 — [Bugfix][MoE] Write Humming results to the supplied output buffer (#50338)
  • 作者: Netanel Haber | +3/-12 | 1 个文件

  Purpose Kimi K3 PR #50089 enabled the generic CUDA modular-MoE output alias when the caller-provided output matches the fused-experts result. Humming ignored the supplied output argument and wrote their final results to views of the internal workspaces. Once the alias was enabled, vLLM could return the caller-provided buffer even though Humming had written to a different buffer, causing corrup…

🧪 CI/Tests

• 5233368 #50516 — [ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 (#50516)
  • 作者: Andreas Karatzas | +34/-5 | 4 个文件

  • git bisect identified aeeb36b1f171, vLLM #50000, as the first revision with both Kimi K3 failures. - Kimi K3’s default MXFP4 RoutedExperts path only advertises AITER on ROCm, but the native backend rejects SiTU on MI300/gfx942 and leaves the model without a selectable backend. Motivation: - AMD CI build 11504 — Kimi K3 Model Initialization - AMD CI build 11504 — multimodal tensor schema PR #5000…

• 864a87f #50109 — [Kernel][CI] --jit-monitor-mode error e2e tests for kernel warmup infra (#50109)
  • 作者: Nicolò Lucchesi | +716/-539 | 10 个文件

  This PR actions this comment https://github.com/vllm-project/vllm/issues/47456#issuecomment-5068299757 (https://github.com/vllm-project/vllm/issues/47456) as part of the broader effort of getting JITs under control with a proper warmup framework. It does 3 things: - centralizes existing jit tests into a separate directory - separates tests into a new buildkite job for better parallelization as wel…

• 7fdc1ab #50560 — [CI] Remove default_torch_num_threads workaround from llava-onevision-transformers test (#50560)
  • 作者: Oğuzhan KIR | +0/-1 | 1 个文件

  Removes the “default_torch_num_threads”: 1 workaround from the llava-onevision-transformers entry in tests/models/multimodal/generation/test_common.py. Fixes #50130 (or reopens it with a real reproduction — see below). The line was added in #25307 to avoid a hang reportedly occurring at the third request. I could not reproduce that hang locally, and I found a reason it may no longer bite: in my en…

• c911120 #50517 — [ROCm][CI] Update Transformers AR+RMS fusion expectation (#50517)
  • 作者: Andreas Karatzas | +3/-3 | 1 个文件

  • git bisect identified 59e831c09a22, vLLM #48757, as the first revision with this failure. - The PR intentionally canonicalizes separate residual-add and RMSNorm operations before the AITER AR+RMS fusion pass. That exposes all nine valid fusion sites instead of the one visible before the change. Motivation: - AMD CI build 11504 — Distributed Compile Unit Tests PR #48757 enables AddRMSNormFusionPa…

• 34bb795 #49143 — [CI] Add M3 MSA tests to CI (#49143)
  • 作者: Thien Tran | +34/-23 | 3 个文件

  Follow up from #47442. Turns out some SM100 kernel tests did not run in CI because they were not included in the buildkite config. - Inkling TML FA4: Fixed by #48988 - Edit: The tests are added by #49325. This PR now only cover M3 MSA - Minimax M3 MSA: To be fixed by https://github.com/vllm-project/MSA/pull/8 and #49016 cc @arpera Edit: ~~This PR also makes RNG inputs for test_gdn_prefill_cutedsl….

• 0351e9a #50373 — [XPU][CI]Adjust source_file_dependencies for NixlConnector PD accuracy (4 GPUs) (#50373)
  • 作者: xiangdong | +1/-1 | 1 个文件

  Adjust source_file_dependencies for NixlConnector PD accuracy (4 GPUs) ## Test Result —

• 2773ec3 #49309 — [ROCm][CI] Use explicit wvSplitKrc skinny-GEMM test tolerance for bf16 (gfx950) (#49309)
  • 作者: stefankoncarevic | +8/-3 | 1 个文件

  tests/kernels/quantization/test_rocm_skinny_gemms.py::test_rocm_wvsplitkrc_kernel compares the ROCm wvSplitKrc skinny-GEMM kernel against a torch F.linear reference. On gfx950 (MI355X) the xnorm=True + bfloat16 parametrizations that use a bias fail the comparison at the old atol=1e-3, rtol=1e-8. This is bounded bf16 rounding, not a kernel bug: - With xnorm=True the A/B inputs are Xavie…

• 5d5f22e #50450 — [ROCm][CI] Use larger atol value for INT3 in test_quick_all_reduce.py (#50450)
  • 作者: music-dino | +14/-4 | 1 个文件

  The test mi355_2: Distributed Tests (2xH100-2xMI355) has been consistently failing on AMD CI, the most recent one - https://buildkite.com/vllm/amd-ci/builds/11474/list?sid=019fb241-807d-4bc1-b35e-59e981101afc&tab=output The test was failing due to the fact that the atol value was not large enough for use with int3, the PR sets the atol value to the theoretical maximum error for the utilised range …

• 60399d4 #50481 — [CI] Retry Buildkite API rate limits (#50481)
  • 作者: Kevin H. Luu | +178/-13 | 2 个文件

  Why The first attempt of Run CI from PR comment #30572134207 failed while listing Buildkite builds: A later manual rerun succeeded. A live quota query also showed the vLLM organization REST bucket at its configured limit of 400/400 requests per minute. ## What changed - Retry HTTP 429 responses up to three times after the initial request, for four maximum attempts. - Select RateLimit-Reset or R…

• 553fcb8 #49908 — [CI] Retry Hugging Face processor loading (#49908)
  • 作者: Andreas Karatzas | +10/-3 | 1 个文件

  • Retry AutoProcessor loading when a concurrent refresh briefly hides files in the shared Hugging Face cache. - Reuse the existing bounded repository retry helper.

• 8700f86 #50451 — [CI] Fix tests/entrypoints/multimodal/openai/chat_completion/test_audio.py::test_chat_streaming_audio (#50451)
  • 作者: Nicolò Lucchesi | +1/-6 | 1 个文件

  Fix https://github.com/vllm-project/vllm/issues/50385 I think prefix caching hit might be messing numerics slightly, though we should probably relax token-per-token matching here

• 7fe5312 #50475 — [CI] Retire the v1 PR label rule, add mrv2 (#50475)
  • 作者: jcotant-inferact | +7/-5 | 1 个文件

  Two related changes to PR auto-labeling, per discussion with @njhill: Retire label-v1. Now that V1 is the only engine, ^vllm/v1/ or ^tests/v1/ matches ~28% of PRs (55 of the last 200 commits). The label no longer partitions anything, so it reads as noise. This removes the rule only — the v1 label itself is untouched and stays intact on every PR and issue that already carries it. **Add label-…

• bdc98bf #50377 — [CI] Initialize fused gated RMSNorm weights (#50377)
  • 作者: Andreas Karatzas | +7/-0 | 1 个文件

  • Fix the two BF16 affine failures from AMD CI build 11459, where both the eager and compiled paths produced NaNs. - Initialize FusedRMSNormGated.weight in both standalone test paths when affine mode is enabled so the result no longer depends on allocator contents. - Leave the production torch.empty initialization unchanged because model checkpoints overwrite that parameter.

• f388dd6 #50447 — [XPU][CI] skip kimi-k3 test (#50447)
  • 作者: Kunshang Ji | +1/-0 | 1 个文件

  fix CI by skip unsupported kimi-k3 model ## Test Result —

• 0c64be8 #49839 — [Test][ROCm] Account for gfx950 FP8 RMSNorm rounding (#49839)
  • 作者: Andreas Karatzas | +35/-10 | 2 个文件

  • Use the shared FP8 allclose contract only for the measured gfx950 fused RMSNorm cases. - Bound every accepted difference to one FP8 ULP while leaving scales, residuals, other dtypes, other architectures, and other ROCm checks unchanged. E4M3 adjacent codes can differ by 12.5%, and its minimum subnormal step is approximately 0.002. Fused and unfused gfx950 reductions selected adjacent codes at th…

• 61c1d09 #50284 — [CI] Stabilize speculator memory teardown (#50284)
  • 作者: Andreas Karatzas | +37/-29 | 1 个文件

  Use managed runners for the speculator and reference engines so ROCm teardown completes before the next engine starts. Motivation: - https://buildkite.com/vllm/amd-ci/builds/11417/list?sid=019fad1a-516e-49db-9547-c075d31390bb&tab=output

• 165ed33 #50340 — [CI][ROCm] Stabilize LLM GC teardown check (#50340)
  • 作者: Andreas Karatzas | +9/-0 | 1 个文件

  Retry only the intermittent weakref assertion on ROCm after fixture cleanup without forcing cyclic collection and weakening cycle detection.

• 48a077e #50414 — [CI] Improve comment-triggered authorization and retries (#50414)
  • 作者: Kevin H. Luu | +363/-17 | 4 个文件

  Improve the /ci run and /ci retry workflow so CI access is explicit, reviewable, and useful before an entire Buildkite build finishes. ### Authorization - Users with repository write, maintain, or admin permission can always run CI on an open PR, regardless of authorship. - Configured CI_TRUSTED_USERS can also always run CI. - Other contributors can run CI only on their own non-draft PR after eith…

🔩 Misc

• 17beffd #50141 — [Misc] Clarify mono audio requirement (#50141)
  • 作者: Nicolò Lucchesi | +31/-5 | 4 个文件

  Mono is unambiguously the standard for audio processing with LLM and it’s enforced at various levels of the stack already (hf reference impls, preprocessing libraries etc.). This PR only makes it clearer and asserts split_audio always receives a 1D signal, since the util is also meant to use with vLLM offline mode, which doesn’t otherwise enforce mono audio directly. cc @DarkLight1337

🖥️ Kernel

• 45b60e3 #50345 — [Kernel][Helion] Disable unsafe B200 RMS reduction warp specialization (#50345)
  • 作者: Shangdi Yu | +19/-6 | 2 个文件

  Prevent Triton compiler failures when tuned RMS configs are reused for shapes requiring multiple hidden-size reduction tiles. Same as the fix in https://github.com/vllm-project/vllm/pull/48797. While these configs work for the in-config shapes, warp-specialization can fail to compile for out-of-config shapes.

🦀 Rust Frontend

• 629a938 #50403 — [Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403)
  • 作者: Bugen Zhao | +144/-18 | 4 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Without tools or reasoning, Inkling may directly output plain text directly after the starter <|message_model|>, without opening a channel with a content-kind marker (like <|content_text|>). This PR updates both Python and Rust parsers to buffer the text while the parser is waiting for an Inkling content-kind marker, then - emit the buffered text as assi…