共 56 个 commit，涉及 177 个文件，+5399/-3186 行变动。

概要

统计项	数值
Commit 数	56
变更文件	177
新增行数	+5399
删除行数	-3186

Commit 列表

🐛 Bug Fix

• 429f405 #47296 — [Bugfix] Guard CUDA-only rms_norm_per_block_quant in FUSED_OPS for non-CUDA builds (#47296)
  • 作者: Tsvika Shapira | +7/-12 | 1 个文件

  FUSED_OPS in vllm/compilation/passes/fusion/rms_quant_fusion.py dereferences torch.ops._C.rms_norm_per_block_quant.default at import time, but that is a CUDA-only custom op. On builds where it is not registered (e.g. VLLM_TARGET_DEVICE=tpu), merely importing this module raises AttributeError. Because the module is imported by pass_manager and several sibling fusion passes, this breaks importin…

• 85b3a72 #47381 — [Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren’t misclassified as prefills (#47381)
  • 作者: Woosuk Kwon | +86/-3 | 3 个文件

  Fixes #47239 (GLM-5.2 on Model Runner V2 + MTP: large TPOT fluctuations and low aa_lcr scores). split_decodes_and_prefills assumes the batch is ordered decode -> short_extend -> prefill (this contract is in its docstring; the V1 runner enforces it via reorder_batch_to_split_decodes_and_prefills). Model Runner V2 sorted requests purely by scheduled-token count. With spec decode (decode_query_len > …

• ae6170f #46694 — [P/D][Bugfix] Fix PD async KV load lookahead handling for MTP spec decode (#46694)
  • 作者: Chauncey | +32/-7 | 2 个文件

  Fix NIXL async KV load handling for speculative decoding by deferring lookahead slot allocation during remote KV receive. Previously, the P/D disaggregation path only disabled lookahead allocation for EAGLE during async KV load. MTP and other draft-model speculative decoding paths still allocated lookahead slots while receiving remote KV blocks, which could mismatch local and remote block usage an…

• 1cd75b3 #48085 — [Bugfix] Fix race condition in KVBlockZeroer (#48085)
  • 作者: Benjamin Chislett | +83/-20 | 4 个文件

  Causing a crash in MRV2 async scheduling + hybrid models + specdec ## Testing Included a test, and confirmed locally that it fixes the garbage outputs problem

• a07765c #42478 — [Bugfix] Fix Qwen3-ASR transcription streaming postprocessing (#42478)
  • 作者: JooHo Lee | +207/-17 | 4 个文件

  Fix Qwen3-ASR transcription streaming leaking raw ASR prefixes. Related to #35767. ### Why this is not a duplicate I checked the related open PRs and issues before opening this: - gh issue view 35767 –repo vllm-project/vllm –comments - Confirmed #35767 discusses the broader Qwen3-ASR realtime/raw-format leak problem and links adjacent work. - gh pr list –repo vllm-project/vllm –state open –se…

• 2c17d33 #47144 — [Bugfix][ROCm] Change AttentionCGSuppoort in TritonMLA to UNIFORM_SINGLE_TOKEN_DECODE (#47144)
  • 作者: music-dino | +3/-1 | 1 个文件

  TritonMLAMetadataBuilder advertised _cudagraph_support = AttentionCGSupport.UNIFORM_BATCH (added in #42885) while keeping the inherited query_len_support = SINGLE_ONLY (so reorder_batch_threshold = 1). With speculative decoding enabled, the UNIFORM_BATCH claim bypasses a downgrade in resolve_cudagraph_mode_and_sizes (which only triggers when min_cg_support < UNIFORM_BATCH). Full decode cudagraph c…

• 95d6d6f #48046 — [Bugfix] Use int8 workspace for FlashInfer MLA decode (#48046)
  • 作者: Nick Hill | +93/-47 | 3 个文件

  FlashInfer’s CuteDSL MLA-decode tactic requires an int8 workspace_buffer, but vLLM allocated it as uint8. The trtllm-gen path (used for normal inference) views the buffer as uint8, so the mismatch is invisible until the FlashInfer autotuner enumerates the CuteDSL tactic. Model Runner V2’s warmup autotunes trtllm_batch_decode_mla, so once V2 became the default runner for MoE models it hit workspace…

• 49abada #47894 — [ROCm][Bugfix] Fix empty-tensor .max() crash in AITER FA (#47894)
  • 作者: djramic | +5/-1 | 1 个文件

  On ROCm, the AITER FA backend crashes during warmup for speech-to-text models (Whisper, Voxtral) when num_chunks == 0: cu_seq_lens_cpu is empty and .max() raises RuntimeError: max(): … input.numel() == 0, killing EngineCore. Fix: use 0 when there are no chunks instead of calling .max() on an empty tensor. ## Test Result Before the fix: EngineCore fails to start during warmup with: RuntimeErr…

• d1f1d86 #47033 — [Bugfix] Re-enable benchmarking of librispeech dataset. (#47033)
  • 作者: almayne | +18/-7 | 2 个文件

  The PR https://github.com/vllm-project/vllm/pull/44587 broke a couple of things for benchmarking with the librispeech dataset. It assumes datasets with raw bytes don’t also have a path provided, which the librispeech data does. It also now requires us to disable decoding. Added another test to test_audio_dataset that would have caught the filepath issue. Ran: pytest -sv tests/benchmarks/test_audio…

• f05603f #47801 — [Bugfix][DCP] Cast LSE to fp32 in a2a combine to fix bf16 bitcast crash (#47801)
  • 作者: shawn | +4/-0 | 1 个文件

  [Bugfix][DCP] Cast LSE to fp32 in a2a combine to fix bf16 bitcast crash Fixes #47800. The DCP all-to-all combine backend (–dcp-comm-backend a2a) crashes at startup on MLA models whose decode backend returns a non-fp32 softmax LSE: dcp_a2a_lse_reduce (in vllm/v1/attention/ops/dcp_alltoall.py) and its pack/unpack Triton kernels treat the LSE as fp32 — the packed layout bit-casts a 32-bit LSE …

• cd0de48 #47728 — [Bugfix][V1] Free out-of-window blocks on the processed-token basis under async scheduling (#47728)
  • 作者: Saddss | +549/-143 | 14 个文件

  Attribution. Core fix is the main contribution of @ivanium (Yifan Qiao), from ivanium/vllm#50 (original commit authorship preserved). The problem/fix write-up below is transcribed from that PR. @Saddss added the empirical verification, the regression analysis, and additional test cases (second commit). ## Problem With async scheduling (now default) or PP, the scheduler runs ahead of the GPU:…

• 934eeae #47781 — [CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781)
  • 作者: rasmith | +24/-9 | 8 个文件

  On The Rock, the GPU name for MI250 will be AMD Instinct MI250X / MI250. The old output was: AMD_Instinct_MI250X_/_MI250 which causes pathing errors during the run of: pytest -sv kernels/mamba/test_mamba_ssm_configs.py The new output is: AMD_Instinct_MI250X_MI250 which does not cause any issues and tests pass successfully. Using sanitize_filename to prevent any possible future errors. pytest -sv k…

• a1ab51a #47797 — [Bugfix] Allocate HY V3 expert_bias in float32 to prevent silent downcasting (#47797)
  • 作者: aoright | +3/-1 | 1 个文件

  This PR fixes a bug in Hunyuan-V3 (hy_v3 / hy_v3_mtp) MoE model execution where the router selection bias expert_bias was allocated in the serving dtype instead of torch.float32 (closes #47777). HYV3MoEFused allocated expert_bias as: Under default torch dtypes like torch.bfloat16/torch.float16, this parameter was allocated in 16-bit precision. During checkpoint weight loading, the true float32 bia…

• d35eba3 #47028 — [Bugfix] Avoid leaking Pydantic repr in tool_choice error message (#47028)
  • 作者: Muhammad Fawaz | +13/-1 | 2 个文件

  Part of #31683 (Error Logging Redesign). When a named or forced tool_choice is sent to a vLLM server that has no –tool-call-parser configured, the client-facing error message leaks Pydantic’s default str repr of the internal ChatCompletionNamedToolChoiceParam / ChatCompletionNamedFunction objects. This affects both /v1/messages and /v1/chat/completions, since both route through the same logic…

📦 Other

• 753c503 #48056 — Pin PyNvVideoCodec to tested 2.0.4 wheel (#48056)
  • 作者: Brandon Pelfrey | +1/-1 | 1 个文件

  Pin PyNvVideoCodec to 2.0.4 in CUDA requirements. The CUDA dependency currently points at PyNvVideoCodec==2.1.0, which is blocking CI builds while the needed wheel availability catches up. 2.0.4 wheels are now published and we validated the vLLM PyNvVideoCodec video decode path with that version, so this adopts the tested wheel version to unblock CI while preserving the new video decode backend be…

• b83be00 #48100 — Migrate Olmo and Olmo2 to the Transformers modeling backend (#48100)
  • 作者: Harry Mellor | +22/-400 | 4 个文件

  Migrate support of Olmo and Olmo2 to use the Transformers modeling backend instead of having duplicated implementations in vLLM. These architectures are >1.5 years old. Olmo3 now takes full ownership of its native vLLM support.

• 412414d #48096 — Remove PersimmonForCausalLM and FuyuForCausalLM model architectures (#48096)
  • 作者: Tiezhen WANG | +118/-1093 | 14 个文件

  Removing the two as they’re low in usage ## Test Result —

• e875216 #46415 — Sanitize server file paths from validation error responses (#46415)
  • 作者: Muhammad Fawaz | +114/-8 | 4 个文件

  FastAPI’s ValidationException.str calls _format_endpoint_context(), which embeds the server’s absolute installed-package file path, line number, and function name into str(exc). validation_exception_handler was forwarding str(exc) to the client via the error response body, leaking server internals on every validation error across every endpoint (chat/completions, completions, tokenize, etc.). …

• 0206f10 #47880 — Add Intel XPU Docker release pipeline (#47880)
  • 作者: wenjun liu | +42/-0 | 2 个文件

  This PR adds Docker image release and nightly publishing support for Intel XPU. The main goal is to align XPU with the existing CUDA release flow where it makes sense, while publishing Intel XPU images to a dedicated DockerHub repository: - release images are published to vllm/vllm-openai-xpu - nightly images are also published to vllm/vllm-openai-xpu - XPU build/publish follows the same release b…

• ab7961a #47989 — Remove TeleChatForCausalLM (#47989)
  • 作者: Tiezhen WANG | +1/-6 | 3 个文件

  Remove TeleChatForCausalLM due to low usage (less than 10 hours in the past 3 months) ## Test Result —

• 1171467 #48073 — [CPU] Fix Qwen-Next SSM type for AMX GDN (#48073)
  • 作者: Li, Jiang | +8/-0 | 1 个文件

  AMX GDN requires SSM type ## Test Result —

• 529af88 #47849 — [KV Offloading] Add free block iterator for CPU offload scheduling (#47849)
  • 作者: Chauncey | +18/-17 | 2 个文件

  Add free block iterator for CPU offload scheduling SimpleCPUOffloadScheduler currently traverses the free KV block queue by directly accessing FreeKVCacheBlockQueue internals (fake_free_list_head, fake_free_list_tail, and next_free_block). This tightly couples the offload scheduler to the queue’s implementation details. I discovered this issue while trying to add a new PriorityAwareFreeKVCacheBloc…

• b8c7c86 #47944 — [XPU][LoRA] Fix torch.compile DEVICE_LOST by avoiding view-mutation in LoRA shrink (#47944)
  • 作者: Chaojun Zhang | +15/-2 | 1 个文件

  Problem: - add_shrink directly mutates output tensor views (y[0], y[1], …) via opaque custom ops. - auto_functionalize fails to track view lifetimes, causing SYCL kernels to access freed memory → UR_RESULT_ERROR_DEVICE_LOST. ## Changes: - punica_xpu.py: - add_shrink: allocate temporary buf per output slice, compute via bgmv_shrink into buf, then y[i].copy_(buf) to safely write back. - add_lor…

• 7802c20 #45880 — [KVConnector][NIXL] Support pipeline-parallel prefill in push mode (#45880)
  • 作者: qizixi | +506/-104 | 15 个文件

  Adds pipeline-parallel (PP) prefill support to the NIXL push connector (NixlPushConnector), so a PP-sharded prefiller (P) can push KV to a non-PP decoder (D) for prefill/decode disaggregation. In push mode each P pipeline stage writes only its own layers into the matching sub-range of D’s region list, and D only needs P’s agents for completion notifications — so the change is small and self-co…

• 56da398 #48010 — Fix embed scaling + CUDA graphs in Transformers modelling backend (#48010)
  • 作者: Harry Mellor | +26/-24 | 1 个文件

  This bug caused gibberish outputs on models which perform embed scaling inside the input embedding module on the Transformers side. i.e. Gemma. This PR creates a subclass of VocabParallelEmbedding which makes the scaling visible to CUDA graphs.

• 2683194 #47912 — [ROCm] Fix pooling startup workspace lock (#47912)
  • 作者: Andreas Karatzas | +40/-5 | 4 个文件

  Motivation: https://buildkite.com/vllm/ci/builds/76664/list?sid=019f3900-a887-45a4-bd39-f1e71a7beaee&tab=output Language Models Test (Extended Pooling) failed in test_openai_privacy_filter while starting openai/privacy-filter. This model exposes the startup bug because it combines token-classification pooling, a GptOss/FusedMoE backbone, ROCm modular MoE workspace allocation, and a post-capture wa…

• 6cf7b26 #48008 — [docs] Fix the docs build (#48008)
  • 作者: Martin Hickey | +6/-5 | 2 个文件

  Seems to be an issue with docs hence the latest version of the docs in main do not seem to be published on https://docs.vllm.ai/en/latest/. Here are the 5 warnings mkdocs build –strict produces:

• 0d2f4e7 #46661 — Allow FlashInfer A2A backends for TRTLLM FP8 MoE Modular (#46661)
  • 作者: Thien Tran | +2/-0 | 1 个文件

  Allow FlashInfer A2A backends for TRTLLM FP8 MoE Modular. Port of #46551 to main ## Test Result —

• a5d19cb #48014 — [Core] Move MRV1 late_interaction_runner.py out of MRV2 subtree (#48014)
  • 作者: Nick Hill | +2/-2 | 3 个文件

  This file was added in https://github.com/vllm-project/vllm/pull/36159 but inadvertently put into the MRV2 source subtree (v1/worker/gpu/).

• 8347c6e #47995 — updated flash_attn GIT_TAG to point to torch Stable ABI FA3 commit (#47995)
  • 作者: Chris Leonard | +1/-1 | 1 个文件

  Redoing the PR #46644 because the flash-attention GIT_TAG got changed to an older tag in #46182, getting rid of the torch Stable ABI FA3 build. cc @Harry-Chen @janeyx99 @LopezCastroRoberto build with new tag and utilize torch-abi-audit to make sure FA3 only uses stable symbols. The actual unit test where ran in PR #46644. ## Test Result —

• c2ecd0f #42642 — Fix FlashAttention MLA prefill V unpadding (#42642)
  • 作者: Martin Vit | +12/-4 | 3 个文件

  Fix a regression in the FlashAttention MLA prefill path introduced when the prefill implementations were split out in #32623. Before #32623, the FlashAttention MLA helper padded V when the selected FlashAttention implementation did not support different QK/V head dimensions. The padded output was kept through the context/suffix merge_attn_states path, and only then sliced back to v_head_dim in MLA…

• 0d12618 #47914 — [Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914)
  • 作者: Michael Goin | +79/-30 | 9 个文件

  Enables DFlash drafters that mix sliding-window and full-attention layers (e.g. z-lab/Qwen3.5-35B-A3B-DFlash) on the V2 model runner, by giving each KV cache group its own causal attention metadata so the sliding-window layers run causal and the full-attention layers non-causal. Also fixes lm_head sharing when the target is a *ForConditionalGeneration model: the head is nested under get_language_m…

• 68b4a1d #47892 — Fix NVML capability lookup for visible devices (#47892)
  • 作者: Tyler Michael Smith | +55/-2 | 3 个文件

  Fix for the following, when pairing * –data-parallel-multi-port-external-lb * –all2all-backend deepep_low_latency The unfortunate thing about removing the internal setting of CUDA_VISIBLE_DEVICES is that now we have 3 types of device ids instead of 2 * logical * visible * physical Switch get_device_capability to think in terms of visible device ids to match how PyTorch APIs behave

• 572b25b #47884 — [Bug] Fix Batched DeepGEMM (#47884)
  • 作者: Robert Shaw | +1/-1 | 1 个文件

  • recent PR that removed CUDA_VISIBLE_DEVICES internally seems to have intersected poorly ## Test Result no longer get a failure like: —

• 9f2b3b0 #46017 — Improvement of Docker image build for IBM Power using prebuilt wheels from IBM published devpi index (#46017)
  • 作者: Vivek Sharma | +355/-307 | 2 个文件

  Test Result —

• 2cae98d #47844 — [Rust Frontend] Handle continue_final_message with renderer sentinel (#47844)
  • 作者: Bugen Zhao | +200/-17 | 3 个文件

  Previously, we only passed the continue_final_message value to the chat template, but in practice, almost no Hugging Face chat template actually observes this value, so it was a no-op. This PR ports Transformers v5 semantics to the Rust frontend: it appends the last message with a sentinel and strips it from the rendered result to achieve this, without requiring the chat template itself to be awar…

• db39d60 #47943 — Add tuned selective_state_update float32 config for AMD Instinct MI355 (#47943)
  • 作者: vanshbhatia-amd | +35/-0 | 1 个文件

  Adds the float32 selective_state_update config for the AMD Instinct MI355 (MI355_OAM), the companion to the float16 config merged in #47767. Each NVIDIA device in configs/selective_state_update/ ships both cache_dtype=float16 and cache_dtype=float32; MI355 currently only has float16 (from #47752 / merged #47767). This brings MI355 to parity, so deployments that keep the SSM state cache in fp32…

• e7b3853 #47970 — Remove router weight upcast for DSv2-related models (#47970)
  • 作者: Thien Tran | +0/-2 | 1 个文件

  #47410 upcasts router weights to FP32, disallows BF16xBF16->FP32 GEMM codepath. This PR removes weight upcast, but retain FP32 logits output, so we can use the fast BF16xBF16->FP32 GEMM. Further discussion BF16xBF16->FP32 GEMM is still less accurate than FP32 SIMT GEMM. I believe this is due to accumulation across large K=6144. SIMT GEMM probably uses more accumulation buffers across threads b…

• eeaf231 #47947 — [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI300X (#47947)
  • 作者: vanshbhatia-amd | +51/-0 | 1 个文件

  Adds the float32 selective_state_update config for the AMD Instinct MI300X, the companion to the float16 config in #47945. Each NVIDIA device in configs/selective_state_update/ ships both cache_dtype=float16 and cache_dtype=float32, and MI355 recently gained both (#47767, #47943). This brings MI300X to parity so deployments that keep the SSM state cache in fp32 (the default mamba_ssm cache dty…

• 285c08c #47729 — [Model] Support MOSS-Transcribe-Diarize (#47729)
  • 作者: Canlin Guo | +925/-0 | 7 个文件

  MOSS-Transcribe-Diarize is an end-to-end speech-to-text model for long-form transcription with timestamped speaker labels. It has a Whisper-style audio encoder feeds projected audio embeddings into a Qwen3 causal decoder. ## Test Result —

• 1f4ad05 #47945 — [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI300X (#47945)
  • 作者: vanshbhatia-amd | +51/-0 | 1 个文件

  Adds a tuned selective_state_update (Mamba SSU decode kernel) launch config for the AMD Instinct MI300X. vLLM bundles per-device SSU configs for NVIDIA parts (B200, GB200, H100, H200, RTX PRO 6000) and, recently, MI355 (#47767). On MI300X get_ssm_configs() finds no matching file and falls back to the generic get_default_ssm_launch_config() heuristic, which is well off the optimal (BLOCK_SIZE…

• 4400025 #47962 — [XPU] [Fusion passes] Disable fuse_rope_kvcache_cat_mla & qk_norm_rope_ fusion on XPU (#47962)
  • 作者: Chaojun Zhang | +2/-0 | 1 个文件

  This PR temporarily disables two fusion passes on the XPU backend: - fuse_rope_kvcache_cat_mla – RoPE + KV cache + MLA fusion - enable_qk_norm_rope_fusion – QK Norm + RoPE fusion ## Background: These fusions are currently enabled by default on other backends while use_inductor_graph_partition=True, but on XPU they may introduce start issue due to some fusion kernels not implemented on XPU. (e.g. o…

• 7c67da9 #47969 — Remove unused _get_kv_cache_config_deepseek_v4 alias (#47969)
  • 作者: Nicolò Lucchesi | +0/-3 | 1 个文件

  Removes the unused alias _get_kv_cache_config_deepseek_v4 = _get_kv_cache_config_packed from vllm/v1/core/kv_cache_utils.py. The alias is defined but never referenced anywhere in the codebase. ## Why this is not duplicating an existing PR Searched open PRs for _get_kv_cache_config_deepseek_v4; existing PRs (#44432, #44454) refactor the function itself but none remove the dead alias. ## Test comman…

• d79855e #47044 — [Docs] kv_sharing_fast_prefill correction (#47044)
  • 作者: Nicolò Lucchesi | +2/-4 | 1 个文件

  Current docstrings seem to imply that this flag is not doing anything, while in fact this is used in gemma3n/4 to enable YOCO style prefill. This is a leftover from some initial version of the feature. @hmellor

• 51e5372 #47872 — [Model][HunyuanVL] Use native transformers processor and adapt to transformers 5.13 (#47872)
  • 作者: manayang | +21/-712 | 4 个文件

  hunyuan_vl: use native transformers processor + adapt to transformers 5.13 - Use transformers native HunYuanVLProcessor / HunYuanVLImageProcessor instead of vLLM’s bundled copies (vLLM only needs pixel_values and image_grid_thw; token expansion and xdrope positions are computed by vLLM itself). - get_hf_processor: select PIL backend via backend=‘pil’; drop deprecated use_fast. - _call_hf_proces…

• 7cc2e8e #47911 — fix: hash speculative draft model config (#47911)
  • 作者: Ace Eldeib | +3/-1 | 1 个文件

  AOT compilation cache keys include VllmConfig.compute_hash(), which delegates to SpeculativeConfig.compute_hash(). For DFlash and related aux-hidden-state methods, the speculative hash only distinguished whether aux states were used and which aux layer IDs were returned. That is insufficient when two deployments share an AOT cache root but use different draft model configs with the same aux layer …

• 2c64b4c #47158 — [ROCm] fixed aiter master flag and expert parallelism compatibility on minimax-m3-mxfp8 (#47158)
  • 作者: Hongxia Yang | +107/-12 | 2 个文件

  Fix the situation the accuracy is bad when –enable-expert-parallel is on and VLLM_ROCM_USE_AITER=1. Validation on gfx950) Kernel level: FlyDSL mxfp8 EP additivity test — sum of EP-shard outputs == full no-mask output, cosine 1.0, max-diff 0.0. Confirms the kernel + 0/1-mask contract are correct. Reproduced the bug + fix side-by-side: master-OFF (idx→convert) cos=1.0, master-ON FIXED (mask as-is) …

🧪 CI/Tests

• 299d2b5 #48101 — [CI] Annotate built Docker image tags on the Buildkite build page (#48101)
  • 作者: Kevin H. Luu | +124/-77 | 8 个文件

  On a Buildkite build page (e.g. https://buildkite.com/vllm/ci/builds/77272) there is currently no way to see which Docker image tags an image-build step produced — you have to open the job’s logs and grep for the tag. This makes it painful to find the exact public.ecr.aws/…/vllm-ci-test-repo: reference to pull for local repro. This PR surfaces the pushed image tags directly on the bu…

• bc44f9f #47874 — [ROCm][CI][MoE] Fix double-transpose of fused w3 expert weights (#47874)
  • 作者: stefankoncarevic | +10/-6 | 1 个文件

  Fixes a crash when loading fused MoE expert weights whose checkpoint stores the gate_up_proj tensor in a transposed layout (e.g. Qwen/Qwen3-VL-30B-A3B-Instruct-FP8). In RoutedExperts.load_weights, a single fused checkpoint tensor (gate_up_proj) is matched twice by the expert mapping — once for w1 and once for w3. The transpose reassigned the outer-loop loaded_weight, so the w3 iteration saw th…

• dcdd756 #46893 — [CI] GSM8K eval integration test for KV offloading (#46893)
  • 作者: Tyler Michael Smith | +330/-10 | 5 个文件

  • Adds a regression test that launches a vLLM server with OffloadingConnector (CPU KV offloading) and runs GSM8K eval against nvidia/Nemotron-H-8B-Base-8K (NemotronH — hybrid attention + Mamba) - Guards against stride computation bugs in the offloading worker (e.g. #46888) and silent KV cache data corruption during CPU offloading - Baseline accuracy measured at ~0.49 on GB200 (200 questions, 5-sho…

• b2cf70e #47980 — [CI] BugFix Eval Small Models Distributed test for DiffusionGemma (#47980)
  • 作者: Ilya Markov | +6/-0 | 2 个文件

  Fixes #47978 PR #45418 added _validate_diffusion which rejects temperature != 1.0 and seed is not None for diffusion models. The DiffusionGemma GSM8K eval config did not override these, so the eval harness sent the incompatible defaults (temperature=0.0, seed=42), causing all 1,319 requests to fail with 400 Bad Request and 0% accuracy. —

• 99a8561 #47946 — [Test] Skip DeepEP MoE layer tests without P2P access (#47946)
  • 作者: Tyler Michael Smith | +24/-0 | 1 个文件

  • Skip DeepEP low-latency and high-throughput backend tests when GPU peer-to-peer access is unavailable - Fixes CUDA error ‘peer access is not supported between these two devices’ failures in Kernels FP8 MoE Test (2xH100) CI (56 subtests) - Uses the existing gpu_p2p_access_check utility, cached per world size Extracted from https://github.com/vllm-project/vllm/pull/45321 which addresses the broade…

✨ New Feature

• 5f85975 #46718 — [Feat] Add runtime monitor for post-warmup TileLang compilation (#46718)
  • 作者: Roberto L. Castro | +367/-1 | 2 个文件

  Default: WARNING 06-25 10:59:52 [jit_monitor.py:134] TileLang JIT compilation during inference: mhc_pre_big_fuse_with_norm_tilelang. This causes a latency spike; consider extending warmup to cover this shape/config. Verbose (–jit-monitor-verbose): WARNING 06-25 10:53:22 [jit_monitor.py:131] TileLang JIT compilation during inference: mhc_pre_big_fuse_with_norm_tilelang (cache_key=(((‘hc_mult’, 4),…

⚡ Performance

• 089e412 #45182 — [Perf] Integrate TRTLLM BF16 MoE Modular Kernel (#45182)
  • 作者: Kaihang Jiang | +485/-55 | 5 个文件

  Add TRTLLM BF16 MoE Modular Kernel. The trtllm monolithic kernel has restrictions over the routing method of the model. This PR enables more models (e.g., glm5.1) to use the trtllm moe backend with all2all backend by adding the modular version. Tested with GLM-5.1 ## Test Result GSM8K result: - exact_match,strict-match = 0.949962 - exact_match,flexible-extract = 0.949204 —

🦀 Rust Frontend

• 04a703e #46793 — [Frontend] Support bad_words in the /v1/completions endpoint (#46793)
  • 作者: sungbin1015 | +39/-0 | 2 个文件

  The chat completions endpoint (/v1/chat/completions) exposes a bad_words field and forwards it to SamplingParams, but the legacy completions endpoint (/v1/completions) does not — even though SamplingParams fully supports bad_words. This is a feature-parity gap: users of the completions endpoint cannot suppress specific words. This PR adds the bad_words field to CompletionRequest and plumbs it thro…

🔩 Misc

• bd3bb4e #47608 — [Misc][Docs] Add human-readable integer support for more cli-args (#47608)
  • 作者: Nicolò Lucchesi | +66/-5 | 3 个文件

  Follow-up to align some other cli-args to human-readable format (e.g., 1K, 2M, 32K) - which I find handy :) cli-args recap: Supported Arguments | Argument | Category | Parser | Example Values | Status | |———-|———-|——–|—————-|——–| | –max-model-len | Model | human_readable_int_or_auto | 1K, 2M, 128K, auto | ✅ Existing | | –max-num-batched-tokens | Scheduler | human_…