共 57 个 commit，涉及 373 个文件，+17931/-4209 行变动。

概要

统计项	数值
Commit 数	57
变更文件	373
新增行数	+17931
删除行数	-4209

Commit 列表

📦 Other

• 03c782e #50673 — [model registry] some simple typos (#50673)
  • 作者: Ning Xie | +26/-26 | 2 个文件

  1. the first return value for inspect_model_cls is model info instead of model class. Make it more understandable. 2. typo fix. NA ## Test Result NA —

• 39f55ff #49608 — [Core] Offload raw-prompt preprocessing to renderer thread pool in AsyncLLM (#49608)
  • 作者: Almog Tavor | +37/-12 | 2 个文件

  Fixes #49317. AsyncLLM.add_request() calls InputProcessor.process_inputs() synchronously on the asyncio event loop thread. For raw prompts this includes tokenization and HF multimodal preprocessing (image resize/patchify into pixel_values), which holds the event loop for the whole duration. Under concurrent multimodal requests everything serializes before reaching the EngineCore scheduler, so max_…

• 63e78ce #49611 — [Benchmark] Add probe requests to vllm bench serve (#49611)
  • 作者: Guan-Ming Chiu | +106/-0 | 3 个文件

  • Addresses part of #16353: no benchmark for how concurrent MM requests affect each other - Per review, implemented in vllm bench serve instead of a separate mm-processor mode - –probe-request-rate sends single-token text-only probes alongside the main workload (bypassing –max-concurrency) and reports their latency separately - Server: vllm serve Qwen/Qwen2.5-VL-3B-Instruct –max-model-len 8192 …

• 652ba59 #50574 — [Model Runner V2] Enable encoder token embedding (#50574)
  • 作者: Taneem Ibrahim | +106/-7 | 4 个文件

  This PR enables the token_embed pooling task on Model Runner V2 for encoder-only models, and wires the existing LateInteractionRunner into the V2 pooling path so ColBERT-style scoring works under MRV2. This is the third task in the MRV2 pooling enablement: | Task | Status | | — | — | | embed, classify | #48791 (merged) | | token_classify | #50293 (merged) | | token_embed | this PR | ## Test Pl…

• 3986b96 #50498 — (feat): optionally disable lookup on PD decode (#50498)
  • 作者: Jeff (Junze) Ma | +67/-3 | 5 个文件

  The original implementation is done by @ivanium . This PR introduces the enable_lookup flag for MooncakeStoreConnector, to be added in kv_connector_extra_config with kv_consumer role. This flag is for a PD setup where decode instance doesn’t read from or write to the Mooncake, and only contributes its segment for extending prefill worker KV cache capacity. e.g. - This avoid the potential overhead …

• 77469c9 #50476 — [ROCm][MLA] Mask the AITER MLA small-head verify flatten causally (#50476)
  • 作者: Matt Qin | +284/-10 | 2 个文件

  [ROCm][MLA] Mask the AITER MLA small-head verify flatten causally On ROCM_AITER_MLA, a multi-token speculative verify block with fewer than 16 query heads per rank is flattened into single-token Gluon decodes with no causality check (vllm/v1/attention/backends/mla/rocm_aiter_mla.py:1006-1009), and every resulting row is handed its request’s entire paged-KV range, per_req_len[row_req] (rocm_a…

• 4ee9702 #50600 — Add Understanding the Latency Metrics docs (#50600)
  • 作者: Michael Goin | +193/-0 | 3 个文件

  Test Result —

• 124154a #50655 — Add @shen-shanshan to CODEOWNERS (#50655)
  • 作者: Shanshan Shen | +6/-3 | 1 个文件

  Thank you all for the opportunity to serve as a vLLM committer! I’m excited to help maintain and shepherd works across hardware portability and upstream integration across Ascend, AMD ROCm, and multimodal execution. I’m deeply grateful to @simon-mo @ywang96 @DarkLight1337 @Isotr0py @tjtanaa @AndreasKaratzas and the vLLM community — for every thoughtful review, every fun interaction, and every mome…

• eb6453d #50474 — [Build] Update pin to build ABI stable FA2 (#50474)
  • 作者: Jane (Yuan) Xu | +1/-2 | 2 个文件

  As a part of #26946, completes making FA2 ABI stable (predecessor #46640). For now, this PR tests the new pin at https://github.com/vllm-project/flash-attention/pull/175 CI should be green. the .so has no unstable APIs: Locally: pytest tests/kernels/attention/test_flash_attn.py -v -k “2-” and the sparse test script in #46640. ## Test Result All FA2 tests pass, failures are all fa3. Sparse script p…

• e3be896 #50019 — Enable ModelOpt FP8 emulation on SM80 (#50019)
  • 作者: Mike G | +1/-1 | 1 个文件

  ModelOpt FP8 checkpoints are currently rejected on NVIDIA A100 GPUs before vLLM can select its existing weight-only FP8 execution path: This PR lowers ModelOptFp8Config’s minimum CUDA capability from SM89 to SM80. On GPUs with capability SM80 through SM88, it also emits one model-level warning explaining that FP8 is not executed natively, that vLLM will use emulation for the model, and that some m…

• 454ea5b #44570 — [MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod (#44570)
  • 作者: bnellnm | +545/-855 | 14 个文件

  Now that the int_wna16.py handles backend selection, the CompressedTensorsWNA16MarlinMoEMethod can be combined with/replaced by CompressedTensorsWNA16MoEMethod. Based on https://github.com/vllm-project/vllm/pull/44120 cc @bedeks , @yzong-rh, @robertgshaw2-redhat CI Ran Kimi + Qwen1.5 tests locally. Made sure Marlin backend was still selected for the appropriate layers. ## Test Result pytest tests …

• 0c4dc7c #49668 — [graceful shutdown] fix http server start firstly before app signal handler register (#49668)
  • 作者: Ning Xie | +12/-0 | 1 个文件

  For now, server.serve is run concurrently with the app signal SIGINT and SIGTERM handlers. There is a race condition. 1. server.serve will first call capture_signals. capture_signals will overwrite the signal SIGINT and SIGTERM handler before invoke server._serve which will chain invoke server.startup and start the server If the coroutine server.serve is delayed after app signal SIGINT and SIGTERM…

• 9032991 #50590 — [UX] Reduce startup log noise (#50590)
  • 作者: Michael Goin | +36/-50 | 9 个文件

  • remove the default asynchronous-scheduling status log - emit repeated configuration and warmup messages once per process - move routine backend-selection, registration, and no-op details to DEBUG - preserve warnings whose content may differ by rank This only changes logging; serving behavior and model outputs are unchanged. ## Duplicate-work check Searched open PRs for startup logging, log noise…

• 4fdd3e0 #40289 — [ROCm][ViT] Detect Triton-AMD kernels at their new aiter location (#40289)
  • 作者: Adrian | +15/-3 | 1 个文件

  flash_attn_triton_available() in vllm/platforms/rocm.py gates whether AttentionBackendEnum.FLASH_ATTN is offered for the ViT encoder on ROCm. The check looks for flash_attn.flash_attn_triton_amd, but ROCm/flash-attention commit 3f94643 (2026-03-12, "[AMD] Migrate to Triton Backend to Aiter") removed that subpackage and moved the kernels into aiter at aiter.ops.triton._triton_kernels.flash_attn…

• 9a7ae4b #49437 — [chore] log process manager shutdown with more details (#49437)
  • 作者: Ning Xie | +17/-6 | 1 个文件

  process manager shutdown function is used in many places. There are some reasons to add more context to the log: 1. Without a process name or process kind, it is hard to diagnose which process manager is trigging the shutdown. The process name is not that long and this shutdown procedure is only called once, so the added name info to the log record should be ok. 2. details more process meta info, …

• 726ef43 #49424 — [chore] delete useless code (#49424)
  • 作者: Ning Xie | +6/-5 | 3 个文件

  num_api_servers is useless for now in launch_core_engines, so delete it. NA ## Test Result NA —

• 963a658 #50307 — Bump Helion to 1.4.0 (#50307)
  • 作者: Shangdi Yu | +3/-3 | 3 个文件

  Update Helion commit to 1.4.0. This includes fix https://github.com/pytorch/helion/pull/3081, which fixes numerics issue in the fused_qk_norm_rope Helion kernel

• e67a2e0 #48949 — [ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend (#48949)
  • 作者: fxmarty-amd | +825/-91 | 17 个文件

  Co-authored-by: Claude Refactor QuarkOCP_MX (the Quark OCP MX4/MX6 linear scheme) to use the shared vllm/model_executor/kernels/linear kernel abstraction for its emulation backend, similar to the refactor done in https://github.com/vllm-project/vllm/pull/49348 for the aiter backend. Previously QuarkOCP_MX hard-coded its own self.emulate boolean and directly registered/dispatched the AITER gemm_wit…

• d87d2ca #50500 — [Compressed-Tensors] Support Kimi-K3 quantized models (#50500)
  • 作者: Kyle Sayers | +8/-1 | 1 个文件

  Purpose ## * Support CT configs for Kimi-K3 ## Changes ## * Add SupportsQuant and packed_modules_mapping to KimiLinearModel. SupportsQuant reads the packed_modules_mapping and updates the quantization config accordingly so that fused modules (fused gate up modules of shared expert) are properly ignored. ## Testing ## * Validated serve and eval against RedHatAI/Kimi-K3-NVFP4 * Validated serve an…

• 7c08664 #50522 — Upgrade tpu-inference to v0.26.0 (#50522)
  • 作者: Dennis Yeh | +1/-1 | 1 个文件

  Upgrade tpu-inference to latest stable release v0.26.0 Verified on tpu-inference CI. ## Test Result Success. — [ ] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”. [ ] The test plan, such as providing test command. [ ] The test results, such as pasting the results comparison before and after, or e2e results [ ] (Optional) The necessary documentation up…

• 10ad649 #50412 — Update torch version to 2.13.0+cpu (#50412)
  • 作者: ylangtsou | +1/-1 | 1 个文件

  Update the torch version from 2.11.0+cpu to 2.13.0+cpu in requirements/build/tpu.txt to fix the vllm-tpu wheel build failure caused by missing dependency torch==2.13.0 reference PR: https://github.com/vllm-project/vllm/pull/48155 1. Build vllm-tpu package locally 2. Verify the built wheel by launching the server ## Test Result vllm-tpu built successfully, and the vLLM server started without issues…

• df71917 #49236 — [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
  • 作者: Wentao Ye | +354/-30 | 15 个文件

  Part of https://github.com/vllm-project/vllm/issues/45861 Optimize workspace reuse for eager break vllm serve deepseek-ai/DeepSeek-V4-Flash –tensor-parallel-size 4 –enable-expert-parallel –attention-backend FLASHMLA_SPARSE_DSV4 –attention-config ‘{“use_fp4_indexer_cache”:true}’ –kv-cache-dtype fp8 –tokenizer-mode deepseek_v4 –all2all-backend allgather_reducescatter –port 8003 ### Acc ### P…

• 82ae416 #48770 — [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770)
  • 作者: Matthew Bonanni | +855/-26 | 11 个文件

  Depends on https://github.com/vllm-project/flash-attention/pull/155 For pure prefills, there is a threshold of sequence length below which a masked MHA pathway is faster than sparse MQA. The DeepSeek V3.2 paper mentions this: > Note that for short-sequence prefilling, we specially implement a masked MHA mode to simulate DSA, which can achieve higher efficiency under short-context conditions. ## Te…

• f5ffc59 #50408 — [Renderer] Warm up the renderer properly. (#50408)
  • 作者: wang.yuqi | +50/-46 | 6 个文件

  Warm up the renderer properly. get_hf_processor takes about 22 seconds to run for the first time on my machine. main 1 22.131629880001128 2 0.14735023300090688 this pr 1 0.19055349900008878 2 0.1441873680014396 ## Test Result —

• 2c4d348 #48408 — [KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408)
  • 作者: Itay Etelis | +1016/-7 | 6 个文件

  Records on CanonicalKVCacheRef, per layer, how the worker’s KV page maps into a canonical (parallelism-free) page: the full offloaded block, all KV heads, all block_size * dcp * pcp tokens. A cache offloaded under one parallel config can later be rearranged or consumed under another. All parallelism reasoning lives in vllm/v1/kv_offload/sharding.py; the connector and everything downstream consume …

• b2fb83e #50148 — [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148)
  • 作者: Martin Hickey | +34/-33 | 14 个文件

  AttentionMetadataBuilder.init declares kv_cache_spec: AttentionSpec but the mamba backends are handed a MambaSpec. Those two are siblings under KVCacheSpec not subtypes, so the hint on the base is wrong for every SSM backend. The base also does self.kv_cache_spec = kv_cache_spec which pins the attribute’s type for the whole hierarchy. That’s why gdn_attn, mamba_attn and linear_attn each carry …

• 92643d6 #50242 — K3 DSpark AR fusion (#50242)
  • 作者: Jee Jee Li | +24/-8 | 5 个文件

  This PR depends on #50000 and should be merged after it. ### GSM8K - main branch - This PR ## Test Result —

• 6e311c6 #44941 — [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941)
  • 作者: bnellnm | +243/-234 | 80 个文件

  #41184 deleted the FusedMoE class and replaced it with a function that constructed a MoERunner and related MoE classes. This PR renames FusedMoE -> FusedMoEFactory to better reflect the purpose of the function. CI/Lint ## Test Result —

• 0e9b500 #50116 — [chore] clean-up weight prepack for INT8 MoE (#50116)
  • 作者: Fadi Arafeh | +7/-26 | 4 个文件

  This aims to simplify / fix the weight prepack path for INT8 MoE. Currently, weight prepack is owned by the MoE Backend (not by the MoE kernel), this is problematic because: - in CPU, we have a single MoE backend, but multiple kernels (for AArch64, x86). As a result, choosing the correct packing function requires: (1) selecting CPU packing function for the CPU MoE backend (2) inside the CPU packin…

• 88bc8fb #50219 — [CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219)
  • 作者: Rehan Khan | +182/-27 | 6 个文件

  Enable s390x specific CPU optimizations: - Wire DEFINE_FAST_EXP for attention softmax via VXE minimax polynomial - Add vectorized reduce_sum/reduce_max using vec_add/vec_max/vec_sld - Add INT8Vec16 for FP32→INT8 packing (vec_cts/vec_packs) - Add FP32Vec16 exp/abs/min/clamp/reduce_min for dnnl_kernels support - Enable oneDNN build and op registration for s390x (INT8 W8A8 GEMM) Run INT 8 model and r…

• 482cfc2 #46981 — [XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation (#46981)
  • 作者: Chaojun Zhang | +19/-74 | 7 个文件

  Previously, vllm_c RMSNorm kernels were only enabled for CUDA_ALIKE; XPU used a duplicate xpu_kernels provider. This PR switches XPU to the shared vllm_c implementation and sets IR priority to: vllm_c > native. No duplicate work: This is a small refactor to remove redundant XPU dispatch code and reuse vllm_c for RMSNorm on XPU.

• 5d7647a #50530 — [UT] add skipif for rocm aiter sampler UT (#50530)
  • 作者: Qiming Zhang | +4/-0 | 1 个文件

  This case use MockPlatform to replace current_platform, which causes error on other platform like CUDA and XPU. On CUDA: On XPU: Add skipif to make this case run only on ROCM.

• 3ee2bd1 #49691 — Fix duplicate HunyuanVL image boundary tokens (#49691)
  • 作者: Mi Jiazhi | +10/-3 | 1 个文件

  The prompt replacement matched only the image token while adding a full start/image/end sequence, resulting in duplicated image boundary tokens and XD-RoPE indexing errors. Match the full start/image/end placeholder and use the same format for dummy inputs.

🐛 Bug Fix

• ab06486 #46805 — [Bugfix][Kernel] Fix dangling temporary in AWQ gemm torch::stable::sum dim arg (#46805)
  • 作者: xiaolinchen | +5/-1 | 1 个文件

  awq_gemm (in csrc/libtorch_stable/quantization/awq/gemm_kernels.cu) ends with: The scalar 0 is wrapped into a single-element HeaderOnlyArrayRef whose constructor only stores a pointer to the temporary. When this sum call is not inlined, that temporary’s stack slot is overwritten (by the adjacent CUDA launch-config setup) before the boxed dispatcher reads it → the serialized dim becomes gar…

• 6c91de3 #50642 — [Bugfix][Parser] Forward model_config to nested reasoning parsers (#50642)
  • 作者: Chauncey | +1/-1 | 1 个文件

  [Bugfix][Parser] Forward model_config to nested reasoning parsers ## Test Result —

• f7097a9 #50639 — [Bugfix][CI] Prevent common ops imports from initializing CUDA (#50639)
  • 作者: Andreas Karatzas | +3/-5 | 4 个文件

  The PyTorch Fullgraph job started failing with Cannot re-initialize CUDA in forked subprocess, while the Extra Initialization job failed when the Kimi initialization test unexpectedly ran a model profile. I ran git bisect between the failing commit fe33a3ed and the known-good commit 5d7647a1. It identified 92643d68 from #50242 as the first bad commit. That commit added a package-level re-export wh…

• c4a4a42 #50640 — [Bugfix][Test] Fix monolithic routing replay test buffer capacity (#50640)
  • 作者: Amir Samani | +3/-3 | 1 个文件

  The test fixtures allocate routing replay buffers for 8 tokens, but some cases use up to 16 tokens. Set max_num_tokens=16 in all three fixtures to match the largest test case and prevent false failures. ## Test Result —

• e8b358b #50334 — [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334)
  • 作者: yzong-rh | +521/-43 | 8 个文件

  Add a test for rendering parity between Chat Completions and Responses. Tests on - multi-turn conversation - tools (strict, defer_loading, extra fields) - tool_choice - reasoning_effort and enable_thinking - template kwargs Includes two bug fixes: - https://github.com/vllm-project/vllm/pull/49824: FunctionTool not rendered identically to ChatCompletionToolsParam - tool_choice=“none” on Responses A…

• aef85ae #50533 — [Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533)
  • 作者: Soila Kavulya | +32/-2 | 4 个文件

  Adds KV quant mode for turboquant KV cache. This fix prevents the KV cache dtype string being set to auto which causes the following failure in TurboQuant observed in https://github.com/vllm-project/vllm/pull/47609 https://github.com/vllm-project/vllm/pull/48177/ https://github.com/vllm-project/vllm/pull/48907 ## Test Result —

• a0cd2b6 #50302 — [Bugfix] Universally align block table width to 128 tokens (#50302)
  • 作者: Matthew Bonanni | +90/-30 | 8 个文件

  Share one block-table width calculation between MRV1, MRV2, and the DSA indexer. It applies 128-token alignment and kernel-block splitting, preventing indexer buffer mismatches under alignment and DCP. Fixes #46074. Supersedes #48404, #50050, and the alignment portion of #43970. ## Testing CUDA_VISIBLE_DEVICES=’’ .venv/bin/python -m pytest tests/v1/worker/test_gpu_model_runner.py -k get_block_tabl…

• 94e9ef0 #50137 — [Bugfix] Don’t transpose fused MoE quantization scales in RoutedExperts.load_weights (#50137)
  • 作者: Harry Mellor | +77/-8 | 2 个文件

  Fixes a regression from #47058 that breaks loading of fused, per-channel-quantized MoE checkpoints with TP > 1. RoutedExperts.load_weights normalises the orientation of fused expert tensors so that checkpoints storing them either way (e.g. Qwen3 VL MoE) load correctly. It identified fused tensors by rank alone and then transposed by testing for the absence of the hidden dim: Any 3D tensor with n…

• 0b5b49d #50491 — [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491)
  • 作者: Recoordinate | +13/-7 | 3 个文件

  Closes #50253. After the VLLMError hierarchy migration in #49665, seven user-facing errors in vllm/entrypoints/chat_utils.py still raised a bare ValueError, so they bypassed VLLMValidationError and the structured request-error handling (the API param field came back null). Convert all seven to VLLMValidationError, adding parameter= where the failing request field is unambiguous: - validate_chat_te…

• f727951 #50305 — [Bugfix] Re-land MiniMax M3 default video processor (#50305)
  • 作者: Taneem Ibrahim | +42/-12 | 3 个文件

  This forward-fix re-lands the MiniMax M3 default video-processor support from #50092 after its revert in #50313 : - Keeps MiniMaxM3VideoBackend in the model-specific MiniMax module. - Registers MiniMaxM3VLVideoProcessor with that backend. - Preserves Hugging Face-aligned frame sampling. - Treats a missing optional fused-allreduce operation as unavailable when it raises either ImportError or Attrib…

• 10e6b40 #50437 — [CPU][BugFix] Remove redundant kv cache write (#50437)
  • 作者: Fadi Arafeh | +45/-12 | 3 个文件

  While profiling some LLMs, I noticed that cpu_attn_reshape_and_cache gets called twice for each invocation of the attention kernel for causal and cross attention. Since https://github.com/vllm-project/vllm/pull/40470 - the kv write, for causal and cross attention is done by the caller (CPUAttentionBackend.do_kv_cache_update) - forward_includes_kv_cache_update is set to false We now only run cpu_at…

🦀 Rust Frontend

• 81a42d3 #49498 — [Frontend] Add cache_salt support to Anthropic Messages API (#49498)
  • 作者: Cao Qian | +35/-0 | 3 个文件

  Closes #46688. The OpenAI-compatible Chat Completions / Completions APIs accept a cache_salt for explicit prefix-cache isolation in multi-user deployments. The Anthropic Messages API (/v1/messages) did not expose it, so clients built on the Anthropic schema had to switch API formats just to get deterministic prefix-cache isolation. ## Change - Add cache_salt: str | None to AnthropicMessagesRequest…

• 9c110fa #47189 — [Frontend] Cohere chat v2 api support (#47189)
  • 作者: andrewbcohere | +8770/-20 | 31 个文件

  Implement the Cohere chat v2 api, see the RFC here: https://github.com/vllm-project/vllm/issues/43015 Unit & integration tests for new components E2E tests are also being run, will add these and more tests once the general approach has been reviewed ## Test Result Tests run via pytest pass — - [X] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”. - [X] …

• 62195e9 #50058 — [Rust][Benchmark] Prevent invalid token IDs in random benchmarks (#50058)
  • 作者: Reid | +80/-18 | 1 个文件

  Random benchmark generation treated every ID in 0..vocab_size as valid for built-in tiktoken encodings. That assumption is incorrect for sparse vocabularies: - o200k_base contains 275 unassigned IDs in the configured range. - cl100k_base contains 16 unassigned IDs in the configured range. When the random dataset sampled one of these IDs, decoding panicked. Release builds use abort-on-panic behavio…

🧪 CI/Tests

• fcdc7c2 #50330 — [CI] Organize speculative decoding E2E tests by coverage (#50330)
  • 作者: Michael Goin | +1941/-1863 | 39 个文件

  • Split the monolithic speculative-decoding E2E suite into feature directories that match CI ownership. - Make Buildkite run directory scopes instead of keyword/file allowlists, so tests added to an existing area are discovered by default. - Restore nightly DFlash and DSpark acceptance/accuracy coverage, using poolside/Laguna-XS-2.1-NVFP4 to keep Laguna practical on smaller GPUs. - Fix the Gemma4 …

• 8d8a4e0 #50515 — [ROCm][CI] Restore Mistral tool-parser compatibility after unification (#50515)
  • 作者: Andreas Karatzas | +21/-4 | 2 个文件

  • git bisect identified 1a20d23dab6e, vLLM #48947, as the first revision with this failure. - The unified parser moved bot_token and bot_token_id onto its nested engine and made legacy extraction assume a request object, breaking two behaviors preserved by the former MistralToolParser. Motivation: - AMD CI build 11504 — Language Models Test (Extended Generation) PR #48947 replaced the former stand…

• 5233368 #50516 — [ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 (#50516)
  • 作者: Andreas Karatzas | +34/-5 | 4 个文件

  • git bisect identified aeeb36b1f171, vLLM #50000, as the first revision with both Kimi K3 failures. - Kimi K3’s default MXFP4 RoutedExperts path only advertises AITER on ROCm, but the native backend rejects SiTU on MI300/gfx942 and leaves the model without a selectable backend. Motivation: - AMD CI build 11504 — Kimi K3 Model Initialization - AMD CI build 11504 — multimodal tensor schema PR #5000…

• 864a87f #50109 — [Kernel][CI] --jit-monitor-mode error e2e tests for kernel warmup infra (#50109)
  • 作者: Nicolò Lucchesi | +716/-539 | 10 个文件

  This PR actions this comment https://github.com/vllm-project/vllm/issues/47456#issuecomment-5068299757 (https://github.com/vllm-project/vllm/issues/47456) as part of the broader effort of getting JITs under control with a proper warmup framework. It does 3 things: - centralizes existing jit tests into a separate directory - separates tests into a new buildkite job for better parallelization as wel…

• 7fdc1ab #50560 — [CI] Remove default_torch_num_threads workaround from llava-onevision-transformers test (#50560)
  • 作者: Oğuzhan KIR | +0/-1 | 1 个文件

  Removes the “default_torch_num_threads”: 1 workaround from the llava-onevision-transformers entry in tests/models/multimodal/generation/test_common.py. Fixes #50130 (or reopens it with a real reproduction — see below). The line was added in #25307 to avoid a hang reportedly occurring at the third request. I could not reproduce that hang locally, and I found a reason it may no longer bite: in my en…

• c911120 #50517 — [ROCm][CI] Update Transformers AR+RMS fusion expectation (#50517)
  • 作者: Andreas Karatzas | +3/-3 | 1 个文件

  • git bisect identified 59e831c09a22, vLLM #48757, as the first revision with this failure. - The PR intentionally canonicalizes separate residual-add and RMSNorm operations before the AITER AR+RMS fusion pass. That exposes all nine valid fusion sites instead of the one visible before the change. Motivation: - AMD CI build 11504 — Distributed Compile Unit Tests PR #48757 enables AddRMSNormFusionPa…

• 34bb795 #49143 — [CI] Add M3 MSA tests to CI (#49143)
  • 作者: Thien Tran | +34/-23 | 3 个文件

  Follow up from #47442. Turns out some SM100 kernel tests did not run in CI because they were not included in the buildkite config. - Inkling TML FA4: Fixed by #48988 - Edit: The tests are added by #49325. This PR now only cover M3 MSA - Minimax M3 MSA: To be fixed by https://github.com/vllm-project/MSA/pull/8 and #49016 cc @arpera Edit: ~~This PR also makes RNG inputs for test_gdn_prefill_cutedsl….

🖥️ Kernel

• b40d859 #48968 — [Kernel][Helion] Add numerics checks to benchmark script (#48968)
  • 作者: Shangdi Yu | +339/-10 | 2 个文件

  • Run one numerics check for every shape before benchmarking it. Default to compare numerics with eager baseline. Can optionally compare with perf baseline with –numerics_with_perf_baseline - Compare both returned values and mutated arguments using the kernel’s configured tolerances, with the one-ULP allowance used by existing FP8 kernel tests. - Use independent input copies for correctness and t…

📖 Documentation

• c036cb2 #50571 — [Doc] Add BgeM3EmbeddingModel to embedding supported models (#50571)
  • 作者: LG | +1/-0 | 1 个文件

  BgeM3EmbeddingModel is implemented and registered as an embedding architecture in vLLM, with BAAI/bge-m3 included in the embedding model registry tests. The BGE-M3 specific-model documentation also instructs users to override the model architecture with BgeM3EmbeddingModel in order to load its additional sparse and ColBERT embedding weights. However, BgeM3EmbeddingModel is currently missing from t…

🔩 Misc

• 17beffd #50141 — [Misc] Clarify mono audio requirement (#50141)
  • 作者: Nicolò Lucchesi | +31/-5 | 4 个文件

  Mono is unambiguously the standard for audio processing with LLM and it’s enforced at various levels of the stack already (hf reference impls, preprocessing libraries etc.). This PR only makes it clearer and asserts split_audio always receives a 1D signal, since the util is also meant to use with vLLM offline mode, which doesn’t otherwise enforce mono audio directly. cc @DarkLight1337