共 87 个 commit，涉及 318 个文件，+7304/-2204 行变动。

概要

统计项	数值
Commit 数	87
变更文件	318
新增行数	+7304
删除行数	-2204

Commit 列表

🐛 Bug Fix

• f05603f #47801 — [Bugfix][DCP] Cast LSE to fp32 in a2a combine to fix bf16 bitcast crash (#47801)
  • 作者: shawn | +4/-0 | 1 个文件

  [Bugfix][DCP] Cast LSE to fp32 in a2a combine to fix bf16 bitcast crash Fixes #47800. The DCP all-to-all combine backend (–dcp-comm-backend a2a) crashes at startup on MLA models whose decode backend returns a non-fp32 softmax LSE: dcp_a2a_lse_reduce (in vllm/v1/attention/ops/dcp_alltoall.py) and its pack/unpack Triton kernels treat the LSE as fp32 — the packed layout bit-casts a 32-bit LSE …

• cd0de48 #47728 — [Bugfix][V1] Free out-of-window blocks on the processed-token basis under async scheduling (#47728)
  • 作者: Saddss | +549/-143 | 14 个文件

  Attribution. Core fix is the main contribution of @ivanium (Yifan Qiao), from ivanium/vllm#50 (original commit authorship preserved). The problem/fix write-up below is transcribed from that PR. @Saddss added the empirical verification, the regression analysis, and additional test cases (second commit). ## Problem With async scheduling (now default) or PP, the scheduler runs ahead of the GPU:…

• 934eeae #47781 — [CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781)
  • 作者: rasmith | +24/-9 | 8 个文件

  On The Rock, the GPU name for MI250 will be AMD Instinct MI250X / MI250. The old output was: AMD_Instinct_MI250X_/_MI250 which causes pathing errors during the run of: pytest -sv kernels/mamba/test_mamba_ssm_configs.py The new output is: AMD_Instinct_MI250X_MI250 which does not cause any issues and tests pass successfully. Using sanitize_filename to prevent any possible future errors. pytest -sv k…

• a1ab51a #47797 — [Bugfix] Allocate HY V3 expert_bias in float32 to prevent silent downcasting (#47797)
  • 作者: aoright | +3/-1 | 1 个文件

  This PR fixes a bug in Hunyuan-V3 (hy_v3 / hy_v3_mtp) MoE model execution where the router selection bias expert_bias was allocated in the serving dtype instead of torch.float32 (closes #47777). HYV3MoEFused allocated expert_bias as: Under default torch dtypes like torch.bfloat16/torch.float16, this parameter was allocated in 16-bit precision. During checkpoint weight loading, the true float32 bia…

• d35eba3 #47028 — [Bugfix] Avoid leaking Pydantic repr in tool_choice error message (#47028)
  • 作者: Muhammad Fawaz | +13/-1 | 2 个文件

  Part of #31683 (Error Logging Redesign). When a named or forced tool_choice is sent to a vLLM server that has no –tool-call-parser configured, the client-facing error message leaks Pydantic’s default str repr of the internal ChatCompletionNamedToolChoiceParam / ChatCompletionNamedFunction objects. This affects both /v1/messages and /v1/chat/completions, since both route through the same logic…

• c0e8e1f #45313 — [Bugfix] Register VLLM_BUILD_* and VLLM_IMAGE_TAG provenance env vars (#45313)
  • 作者: Nicklas Frahm | +10/-0 | 1 个文件

  #40653 added build provenance metadata (VLLM_BUILD_COMMIT, VLLM_BUILD_PIPELINE, VLLM_BUILD_URL, VLLM_IMAGE_TAG) as ENV vars baked into the official vllm-openai images, but did not register them in vllm/envs.py’s environment_variables dict. As a result, every container start of an official vllm-openai image logs: This PR registers the four vars as plain str getters with the same defaults used in do…

• 5d5fab0 #44303 — [Bugfix][Frontend] Fix http_requests_total metric recording some 4xx errors as 5xx (#44303)
  • 作者: Zach Zhu | +219/-1 | 2 个文件

  This PR fixes a bug where vllm’s Prometheus http_requests_total metric incorrectly records certain 4xx client errors as 5xx server errors. This bug was discovered while we were investigating noise in our production vllm monitoring. Detail: In Starlette’s middleware architecture, exception handlers registered for Exception are moved to ServerErrorMiddleware (the outermost layer), while handlers for…

• 80eb01e #47493 — [Bugfix] DSV4 TP16 garbage output (#47493)
  • 作者: Jeff (Junze) Ma | +61/-6 | 5 个文件

  Fixes DeepSeek-V4 garbage output (gsm8k ~0% #47783) with the FLASHINFER_MLA_SPARSE_DSV4 backend on the packed KV regression from #44577, coauthored by @LucasWilkinson. Remapping is adopted from #47895. ## Problem #44577 packs a block’s KV components (indexer + compressor + SWA) into one contiguous per-block allocation, so a layer’s per-block stride becomes the whole block size. The FlashInfer spar…

• 0303f37 #47772 — [Bugfix][Pooling] Align CrossEncoder token type ids after truncation (#47772)
  • 作者: Ting SUN | +85/-3 | 2 个文件

  CrossEncoder scoring compresses token_type_ids into a query/document split point before post-tokenization truncation runs. When a request uses truncate_prompt_tokens with truncation_side=“left”, the kept prompt tokens can all come from the document segment, but the stale split still marks them as query tokens. LLM.score(), /score, and /rerank can then return HTTP 200 with silently wrong scores for…

• 5e975ea #47888 — [Bugfix] Avoid blocking model launching when no system ffmpeg available for TorchCodec (#47888)
  • 作者: Isotr0py | +21/-2 | 2 个文件

  • import torchcodec will raise RuntimeError when FFMPEG unavailable, which will block model launching even if TorchCodec is not specified. - This PR catches it to only raise error at runtime to avoid blocking model launching. ## Test Result —

• f7efab5 #47848 — [CPU][Bugfix] Fix flaky ShortConv prefill test on ARM (uninitialized weights) (#47848)
  • 作者: Rahul Vishwakarma | +9/-0 | 1 个文件

  tests/kernels/mamba/test_cpu_short_conv.py::test_short_conv_forward_native_prefill failed on ARM CI while passing on x86. ## Root cause The test never initializes the ShortConv layer weights — it only calls dispatch_cpu_unquantized_gemm, which packs the existing weights. vLLM Linear layers allocate weights with torch.empty (uninitialized). - On x86, the heap returns dirty (nonzero) pages, so in_…

• 4aceabf #47766 — [ROCm][Bugfix] Key sparse-MLA persistent metadata on per-request context lengths (#47766)
  • 作者: Rohan Potdar | +12/-15 | 1 个文件

  Fixes the sparse-MLA persistent-metadata cache collision behind #47042 (GLM-4.6 / DeepSeek-V3.2-style DSA models on ROCm produce correct output up to ~20K tokens, then collapse into repetition/garbage at longer context). The ROCm sparse-MLA backend caches the persistent work-stealing metadata and recomputes it only when metadata_key changes. The key was keyed on min(seq_lens, topk), but the metada…

• 7d2ce57 #47910 — [Bugfix] Patch Hopper MXFP4 OOB scales reads leading to NaN (#47910)
  • 作者: yzong-rh | +6/-0 | 1 个文件

  Patch https://github.com/vllm-project/vllm/issues/47303 by padding the swizzled scales so that OOB reads won’t happen. Mirrors upstream fix in https://github.com/triton-lang/triton/pull/10800. Can be cleaned up when we upgrade our triton kernels to v.3.8.0+. TLDR: (E, M, K) = (…, 5760, 90) gets swizzled into (E, M / 32, K * 32) -> (…, 184, 2880), which is read in tiles of (8, 128), leading to …

• d99adce #47445 — [BugFix] Fix ModelOpt quantization inference for fused siblings (#47445)
  • 作者: Jason Li | +42/-0 | 2 个文件

  Fix ModelOpt mixed-precision dispatch so unlisted fused projections remain unquantized. This removes the generic parent-sibling fallback from ModelOptMixedPrecisionConfig._resolve_quant_algo and adds explicit packed_modules_mapping declarations to the NVIDIA MiniMax-M3 causal and conditional model classes. Explicit packed mappings preserve MiniMax qkv_proj / gate_up_proj dispatch without inferring…

• 55da232 #45207 — [Bugfix] Pad Mamba page size instead of scaling block_size in unify_kv_cache_spec_page_size (#45207)
  • 作者: Sahil Kadadekar | +101/-5 | 2 个文件

  Fixes #43626. unify_kv_cache_spec_page_size unifies differing KV page sizes by scaling block_size of the smaller-page layers. That is a no-op for MambaSpec, whose page_size_bytes is determined by its state shapes (and an optional page_size_padded override) and does not change with block_size — so the post-scaling assert new_spec.page_size_bytes == max_page_size fires as a bare AssertionError at en…

• 3dd910d #47908 — [Bugfix] Allow non-contiguous query in FlashInfer FP8 query quantization (#47908)
  • 作者: Matthew Bonanni | +1/-1 | 1 个文件

  Fix #47905 Fix an AssertionError in the FlashInfer attention backend when running models that slice their query from a fused QKV projection (e.g. Nemotron) with FP8 KV cache. #43232 moved the query→FP8 quantization inline into FlashInferImpl.maybe_quant_query, guarded by assert query.is_contiguous(). Models such as Nemotron produce their query via q, k, v = qkv.split(…), which returns a view…

• 7bd1543 #47698 — [Bugfix] Fix mamba+dflash for MRV2 (#47698)
  • 作者: Benjamin Chislett | +1/-4 | 1 个文件

  Running hybrid mamba models in MRV2, particularly with speculative decoding, is broken. This fixes the crash (I’m still debugging some separate correctness issues) ## Testing Crashed before. Launches now, and is mostly sensible

• d687519 #47356 — [Bugfix] Exclude kv_cache_memory_bytes from CacheConfig.compute_hash (#47356)
  • 作者: Nils Matteson | +14/-0 | 2 个文件

  Setting kv_cache_memory_bytes — the documented alternative to gpu_memory_utilization for sizing the KV cache, and the way to skip the memory-profiling measurement on boot — changes CacheConfig.compute_hash and therefore the torch.compile cache key. The result: using the fast-boot knob silently invalidates the compile cache and forces a full recompile. The field only sizes the KV-cache allocation; …

• 392d1b4 #47725 — [BugFix][LoRA] Refresh punica metadata when LoRA slots are reassigned under an unchanged mapping (#47725)
  • 作者: Ameen Patel | +56/-1 | 2 个文件

  FIX #47724 LoRAModelManager.set_adapter_mapping memoizes on self._last_mapping (the per-token LoRA ID mapping of the running batch) and skips the punica metadata refresh when it is unchanged. The memo is never invalidated when a GPU slot is reassigned: an out-of-band add_lora() on a live engine eagerly activates the new adapter (LRUCacheWorkerLoRAManager.add_adapter -> activate_adapter), which can…

• 65dcde1 #47466 — [Bugfix] Fix PD disagg + MTP correctness for Qwen3.5(GDN) (#47466)
  • 作者: Dakai An | +22/-22 | 6 个文件

  Problem Qwen3.5-0.8B with MTP=1 can lose GSM8K accuracy under PD disaggregation when concurrency is greater than 1 and default cudagraph is enabled. The issue was reproduced with both Mooncake and NIXL. Trigger pattern: - model: Qwen3.5-0.8B - hybrid/GDN state model - PD disaggregation - MTP/spec decode enabled, num_speculative_tokens=1 - TP=1 - concurrent requests, so decode can batch normal r…

• 8b74552 #43161 — [Bugfix] Fix UBatchWrapper CUDA graph key to sum all ubatches, not just first two (#43161)
  • 作者: Lanze Liu | +3/-3 | 1 个文件

  When ubatch_size > 2, _capture_ubatches() stored the captured graph under ubatch_metadata[0].num_tokens + ubatch_metadata[1].num_tokens (first two ubatches only), but call looked it up with sum(ubatch_slice.num_tokens for ubatch_slice in ubatch_slices) (all ubatches). The key mismatch caused the cached graph to never be found, triggering unnecessary re-captures on every call. Fixes #43145 Fix …

• 8b91cd5 #44726 — [Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path (#44726)
  • 作者: Ting SUN | +31/-2 | 2 个文件

  merge_async_iterators has a fast path for the single-iterator case that yields directly, without the try/finally cleanup the multi-iterator path uses: When the merged generator is closed (a cancelled or disconnected consumer), the underlying generator is never closed and keeps running. The multi-iterator path already aclose()s its iterators in a finally; the single-iterator path does not. For a si…

• 0a2965b #47318 — [BugFix] Fix ModelOpt mixed-precision quantization for sparse quantized_layers configs. (#47318)
  • 作者: danielafrimi | +66/-10 | 2 个文件

  Fix ModelOpt mixed-precision dispatch so missing sibling layers do not inherit quantization from unrelated siblings. The previous fallback could see one quantized layer under a parent, e.g. o_proj, and incorrectly apply the same quantization to a missing sibling like qkv_proj. This is unsafe for sparse mixed FP8/NVFP4 configs. Now we only infer quantization for known packed projections: qkv_proj <…

• ed051fa #45418 — [Bugfix] Reject sampling params unsupported by diffusion models (#45418)
  • 作者: Guan-Ming Chiu | +78/-0 | 5 个文件

  Diffusion models ignore per-request sampling params Related: #45163 pytest -v tests/test_sampling_params.py ## Test Result 12 passed —

• 3354dba #46972 — [Bugfix][KV offload] Store interior chunk-boundary blocks under MTP/Eagle (#46972)
  • 作者: drakosha | +131/-27 | 2 个文件

  Fixes #46971. With MTP/Eagle speculative decoding, OffloadingConnector stored KV to CPU but prefix reuse almost never hit: re-sending an evicted long prompt recomputed the whole prefill instead of loading it from CPU, and External prefix cache hit rate stayed at 0%. _build_store_jobs computes num_blocks twice. The key-collection loop applies the Eagle trailing-block exclusion (num_blocks - 1), but…

• c5b6623 #47464 — [Bugfix][Spec Decode] Skip uniform spec-decode padding for diffusion models (#47464)
  • 作者: Kyung Sub Lee (Daniel) | +37/-0 | 2 个文件

  Fix a deterministic engine-core crash for diffusion models (model_config.is_diffusion) caused by the scheduler’s uniform spec-decode padding. The waiting-path padding promotes any request needing exactly 1 token to 1 + num_spec_tokens so decode batches stay uniform for full CUDA graphs. For AR speculative decoding this is harmless — the placeholder drafts simply fail verification. For diffusion mo…

• ba50b97 #47586 — [Bugfix] Match the mapped filename in find_loaded_library (#47586)
  • 作者: Gabriel Wu | +17/-17 | 1 个文件

  Fix a nondeterministic library hijack in find_loaded_library(). The function picks the first /proc/self/maps line containing lib_name as a plain substring. Two problems compound: 1. Any co-loaded library whose filename merely contains the target matches — e.g. TileLang dlopens its bundled libcudart_stub.so (a linking stub exporting only ~39 cudart symbols) at runtime, and libcudart_stub both…

• b4cfbc2 #44490 — [Bugfix][Core] Fix host memory leak from undrained new_block_ids (#44490)
  • 作者: Ting SUN | +18/-15 | 3 个文件

  Fixes #44175 Under sustained max_tokens=1 load, host RSS grows linearly with no plateau. The reporter saw ~6.0 → 6.57 GiB over a 12h run with Gemma-3-1b and prefix caching. Who is affected: any V1 model with a full-attention or MLA KV-cache layer and no Mamba layers — i.e. essentially all standard attention models (Llama, Qwen, GPT/OPT, Gemma-2/3, DeepSeek-MLA). Mamba / hybrid-Mamba models…

📦 Other

• c2ecd0f #42642 — Fix FlashAttention MLA prefill V unpadding (#42642)
  • 作者: Martin Vit | +12/-4 | 3 个文件

  Fix a regression in the FlashAttention MLA prefill path introduced when the prefill implementations were split out in #32623. Before #32623, the FlashAttention MLA helper padded V when the selected FlashAttention implementation did not support different QK/V head dimensions. The padded output was kept through the context/suffix merge_attn_states path, and only then sliced back to v_head_dim in MLA…

• 0d12618 #47914 — [Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914)
  • 作者: Michael Goin | +79/-30 | 9 个文件

  Enables DFlash drafters that mix sliding-window and full-attention layers (e.g. z-lab/Qwen3.5-35B-A3B-DFlash) on the V2 model runner, by giving each KV cache group its own causal attention metadata so the sliding-window layers run causal and the full-attention layers non-causal. Also fixes lm_head sharing when the target is a *ForConditionalGeneration model: the head is nested under get_language_m…

• 68b4a1d #47892 — Fix NVML capability lookup for visible devices (#47892)
  • 作者: Tyler Michael Smith | +55/-2 | 3 个文件

  Fix for the following, when pairing * –data-parallel-multi-port-external-lb * –all2all-backend deepep_low_latency The unfortunate thing about removing the internal setting of CUDA_VISIBLE_DEVICES is that now we have 3 types of device ids instead of 2 * logical * visible * physical Switch get_device_capability to think in terms of visible device ids to match how PyTorch APIs behave

• 572b25b #47884 — [Bug] Fix Batched DeepGEMM (#47884)
  • 作者: Robert Shaw | +1/-1 | 1 个文件

  • recent PR that removed CUDA_VISIBLE_DEVICES internally seems to have intersected poorly ## Test Result no longer get a failure like: —

• 9f2b3b0 #46017 — Improvement of Docker image build for IBM Power using prebuilt wheels from IBM published devpi index (#46017)
  • 作者: Vivek Sharma | +355/-307 | 2 个文件

  Test Result —

• 2cae98d #47844 — [Rust Frontend] Handle continue_final_message with renderer sentinel (#47844)
  • 作者: Bugen Zhao | +200/-17 | 3 个文件

  Previously, we only passed the continue_final_message value to the chat template, but in practice, almost no Hugging Face chat template actually observes this value, so it was a no-op. This PR ports Transformers v5 semantics to the Rust frontend: it appends the last message with a sentinel and strips it from the rendered result to achieve this, without requiring the chat template itself to be awar…

• db39d60 #47943 — Add tuned selective_state_update float32 config for AMD Instinct MI355 (#47943)
  • 作者: vanshbhatia-amd | +35/-0 | 1 个文件

  Adds the float32 selective_state_update config for the AMD Instinct MI355 (MI355_OAM), the companion to the float16 config merged in #47767. Each NVIDIA device in configs/selective_state_update/ ships both cache_dtype=float16 and cache_dtype=float32; MI355 currently only has float16 (from #47752 / merged #47767). This brings MI355 to parity, so deployments that keep the SSM state cache in fp32…

• e7b3853 #47970 — Remove router weight upcast for DSv2-related models (#47970)
  • 作者: Thien Tran | +0/-2 | 1 个文件

  #47410 upcasts router weights to FP32, disallows BF16xBF16->FP32 GEMM codepath. This PR removes weight upcast, but retain FP32 logits output, so we can use the fast BF16xBF16->FP32 GEMM. Further discussion BF16xBF16->FP32 GEMM is still less accurate than FP32 SIMT GEMM. I believe this is due to accumulation across large K=6144. SIMT GEMM probably uses more accumulation buffers across threads b…

• eeaf231 #47947 — [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI300X (#47947)
  • 作者: vanshbhatia-amd | +51/-0 | 1 个文件

  Adds the float32 selective_state_update config for the AMD Instinct MI300X, the companion to the float16 config in #47945. Each NVIDIA device in configs/selective_state_update/ ships both cache_dtype=float16 and cache_dtype=float32, and MI355 recently gained both (#47767, #47943). This brings MI300X to parity so deployments that keep the SSM state cache in fp32 (the default mamba_ssm cache dty…

• 285c08c #47729 — [Model] Support MOSS-Transcribe-Diarize (#47729)
  • 作者: Canlin Guo | +925/-0 | 7 个文件

  MOSS-Transcribe-Diarize is an end-to-end speech-to-text model for long-form transcription with timestamped speaker labels. It has a Whisper-style audio encoder feeds projected audio embeddings into a Qwen3 causal decoder. ## Test Result —

• 1f4ad05 #47945 — [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI300X (#47945)
  • 作者: vanshbhatia-amd | +51/-0 | 1 个文件

  Adds a tuned selective_state_update (Mamba SSU decode kernel) launch config for the AMD Instinct MI300X. vLLM bundles per-device SSU configs for NVIDIA parts (B200, GB200, H100, H200, RTX PRO 6000) and, recently, MI355 (#47767). On MI300X get_ssm_configs() finds no matching file and falls back to the generic get_default_ssm_launch_config() heuristic, which is well off the optimal (BLOCK_SIZE…

• 4400025 #47962 — [XPU] [Fusion passes] Disable fuse_rope_kvcache_cat_mla & qk_norm_rope_ fusion on XPU (#47962)
  • 作者: Chaojun Zhang | +2/-0 | 1 个文件

  This PR temporarily disables two fusion passes on the XPU backend: - fuse_rope_kvcache_cat_mla – RoPE + KV cache + MLA fusion - enable_qk_norm_rope_fusion – QK Norm + RoPE fusion ## Background: These fusions are currently enabled by default on other backends while use_inductor_graph_partition=True, but on XPU they may introduce start issue due to some fusion kernels not implemented on XPU. (e.g. o…

• 7c67da9 #47969 — Remove unused _get_kv_cache_config_deepseek_v4 alias (#47969)
  • 作者: Nicolò Lucchesi | +0/-3 | 1 个文件

  Removes the unused alias _get_kv_cache_config_deepseek_v4 = _get_kv_cache_config_packed from vllm/v1/core/kv_cache_utils.py. The alias is defined but never referenced anywhere in the codebase. ## Why this is not duplicating an existing PR Searched open PRs for _get_kv_cache_config_deepseek_v4; existing PRs (#44432, #44454) refactor the function itself but none remove the dead alias. ## Test comman…

• d79855e #47044 — [Docs] kv_sharing_fast_prefill correction (#47044)
  • 作者: Nicolò Lucchesi | +2/-4 | 1 个文件

  Current docstrings seem to imply that this flag is not doing anything, while in fact this is used in gemma3n/4 to enable YOCO style prefill. This is a leftover from some initial version of the feature. @hmellor

• 51e5372 #47872 — [Model][HunyuanVL] Use native transformers processor and adapt to transformers 5.13 (#47872)
  • 作者: manayang | +21/-712 | 4 个文件

  hunyuan_vl: use native transformers processor + adapt to transformers 5.13 - Use transformers native HunYuanVLProcessor / HunYuanVLImageProcessor instead of vLLM’s bundled copies (vLLM only needs pixel_values and image_grid_thw; token expansion and xdrope positions are computed by vLLM itself). - get_hf_processor: select PIL backend via backend=‘pil’; drop deprecated use_fast. - _call_hf_proces…

• 7cc2e8e #47911 — fix: hash speculative draft model config (#47911)
  • 作者: Ace Eldeib | +3/-1 | 1 个文件

  AOT compilation cache keys include VllmConfig.compute_hash(), which delegates to SpeculativeConfig.compute_hash(). For DFlash and related aux-hidden-state methods, the speculative hash only distinguished whether aux states were used and which aux layer IDs were returned. That is insufficient when two deployments share an AOT cache root but use different draft model configs with the same aux layer …

• 2c64b4c #47158 — [ROCm] fixed aiter master flag and expert parallelism compatibility on minimax-m3-mxfp8 (#47158)
  • 作者: Hongxia Yang | +107/-12 | 2 个文件

  Fix the situation the accuracy is bad when –enable-expert-parallel is on and VLLM_ROCM_USE_AITER=1. Validation on gfx950) Kernel level: FlyDSL mxfp8 EP additivity test — sum of EP-shard outputs == full no-mask output, cosine 1.0, max-diff 0.0. Confirms the kernel + 0/1-mask contract are correct. Reproduced the bug + fix side-by-side: master-OFF (idx→convert) cos=1.0, master-ON FIXED (mask as-is) …

• 9021589 #47502 — [Minimax-M3] Using tok_sparse_select from MSA instead of triton kernels (#47502)
  • 作者: Yongye Zhu | +233/-72 | 9 个文件

  Replace the two per-path Triton top-k implementations in the MiniMax-M3 MSA (SM100/Blackwell) indexer with a single MSA sparse_topk_select over one unified score buffer shared by decode and prefill. Before, the MSA indexer ran two independent top-k paths: - decode → Triton fused minimax_m3_index_decode (block-score + split-K top-k) writing topk_indices_buffer[:, :nd] - prefill → fm…

• dd127d8 #47053 — [Core][Engine] only materialize tokens when thinking budget is in req (#47053)
  • 作者: Walter Beller-Morales | +29/-5 | 2 个文件

  Today when the –reasoning-config option is passed to vLLM serve that toggles logitsprocs_need_output_token_ids to be true and forces cpu materialization of token ids for all requests even when thinking_budget_tracks_reqs is false: This is probably unnecessarily conservative since the reasoning config is only used for thinking budget and thinking budget is only activated when included in the reque…

• 0ca6eee #47744 — [Core] Pass request context to CPU offload cache policy touch (#47744)
  • 作者: Posedge_Lin | +37/-8 | 5 个文件

  Follow-up to the discussion in #45405 (cc @orozery). OffloadingManager.touch() and the scheduler-side offload logic already carry per-request context, but CPUOffloadingManager.touch() currently drops it before calling CachePolicy.touch(). This PR forwards the existing ReqContext to the policy layer, enabling future request-aware cache policies without changing the behavior of the existing LRU/ARC …

• e97c3cb #47388 — [Core] Persist and reuse the memory-profiling result across boots (opt-in) (#47388)
  • 作者: Nils Matteson | +284/-0 | 4 个文件

  Part of the cold-start work discussed in RFC #34303’s orbit (Inferact-sponsored fellowship; measured work plan is with @simon-mo). For a fixed (model, config, hardware, library) combination, the memory profiler’s result is deterministic, yet every boot re-measures it, including the throwaway CUDA-graph capture pass used for the memory estimate. #27951 acknowledged the profiling interface’s problem…

• aad0fb7 #47897 — [CI/Build] Accept ready-run-all-tests label in pre-commit gate (#47897)
  • 作者: Ameen Patel | +3/-2 | 1 个文件

  The pre-commit workflow’s pre-run-check gate only recognizes the ready and verified labels. The newer ready-run-all-tests label is strictly stronger than ready (it triggers the full Buildkite suite), but the gate doesn’t know it exists — so when a maintainer replaces ready with ready-run-all-tests, the pre-commit check goes red with the label-gate error for any author with fewer than 4 merged PR…

• 675f429 #47845 — fix(security): bound completion prompt list to prevent unbounded engine fan-out (#47845)
  • 作者: Juan Pérez de Algaba | +177/-0 | 3 个文件

  The /v1/completions endpoint accepted prompt as a list of arbitrary length without any outer prompt-count limit. Each list element becomes a separate engine request, allowing an authenticated client to turn one HTTP request into an attacker-chosen number of backend subrequests, starving other tenants of CPU, memory, and scheduling resources. Add VLLM_MAX_COMPLETION_PROMPTS (default 1024) and rejec…

• c8c2f83 #47767 — Add tuned selective_state_update config for AMD Instinct MI355 (#47767)
  • 作者: vanshbhatia-amd | +35/-0 | 1 个文件

  Adds a tuned selective_state_update (Mamba SSU decode kernel) launch config for the AMD Instinct MI355 (MI355_OAM). vLLM already bundles per-device SSU configs, but only for NVIDIA parts (B200, GB200, H100, H200, RTX PRO 6000). On AMD GPUs get_ssm_configs() finds no matching file and falls back to the generic _get_default_ssm_launch_config() heuristic, which is well off the optimal (BLOCK_SIZE…

• 3f99883 #47902 — [CI Bug Fix] Temp fix for v3.2 accuracy (#47902)
  • 作者: Wentao Ye | +1/-0 | 1 个文件

  Temple fix for v3.2 accuracy I will look further about this and fix throughly vllm serve deepseek-ai/DeepSeek-V3.2 –trust-remote-code –kernel-config.enable_flashinfer_autotune=False –enable-expert-parallel –tensor-parallel-size 8 –tokenizer-mode deepseek_v32 –tool-call-parser deepseek_v32 –enable-auto-tool-choice –reasoning-parser deepseek_v3 –port 9256 –speculative-config ‘{“method”:“mt…

• 2f3f441 #45177 — fix: include topic frame in KV events replay response (#45177)
  • 作者: Rishabh Saini | +50/-27 | 4 个文件

  • Add self._topic_bytes to replay ROUTER responses so the wire format matches live PUB messages: [topic, seq, payload] - Switch test MockSubscriber replay socket from REQ to DEALER so all replayed events are received (REQ only receives one reply) - Add test_replay_includes_topic verifying replay format matches PUB - Tighten existing replay test assertions to exact counts and sequences ## Context T…

• abe41f2 #47835 — Upgrade tpu-inference to v0.24.0 (#47835)
  • 作者: Sting Lin | +1/-1 | 1 个文件

  Upgrade tpu-inference to latest stable release v0.24.0 Verified on tpu-inference CI. ## Test Result Success. —

• b93cbd7 #47858 — [XPU] Fix topk_sigmoid arg mismatch on XPU (#47858)
  • 作者: liuzhenwei | +12/-0 | 1 个文件

  Test Result —

• bdc6f3b #47755 — [Bug] Fix tmp directory for lm_eval (#47755)
  • 作者: Wentao Ye | +2/-1 | 1 个文件

  pytest tests/evals/gsm8k/test_gsm8k_correctness.py –config-list-file=tests/evals/gsm8k/configs/humming/config-act-fp8.txt and will raise This is becuase we hard code /tmp which some envs doesn’t have permission to write to Now

• 21b396a #47784 — AGENTS MD: Add suggestion on how to incorporate tests (#47784)
  • 作者: Simon Mo | +30/-19 | 1 个文件

  I have seeing more PRs with newly created test files by agents.

• bdaf275 #47868 — [XPU] Fix Event init failure w/ blocking (#47868)
  • 作者: liuzhenwei | +7/-1 | 1 个文件

  Error msg: ## Test Result —

• beb4327 #47745 — Enable causal masking for SWA in vllm-project/speculators models (#47745)
  • 作者: Eldar Kurtić | +4/-0 | 1 个文件

  Right now, vllm-project/speculators models contain sliding_window_non_causal to indicate that a model was trained with causal masking and SWA. Without this PR, those models are deployed with bidirectional masking, creating a discrepancy between training and inference setup. This mismatch has a negative impact on results, and the PR address the issue. See improved results in the table below: | mode…

• c46ced1 #46544 — [kv_offload] Establish tier-owned KV event handling (#46544)
  • 作者: Chang Guo | +61/-45 | 11 个文件

  KV event responsibility should follow storage ownership. A tier is the only component that can distinguish an actual store from a no-op and report its own placement changes. Having a parent construct child-tier events would assume a single destination medium and make the take_events() contract harder to reason about. This PR establishes the ownership and aggregation contract. Actual secondary-tier…

• 65a7b46 #47063 — [KV-Offloading] Support workload identity for objectstore secondary tier (#47063)
  • 作者: Pierangelo Di Pilato | +76/-11 | 3 个文件

  Make access_key and secret_key optional in ObjStoreConfig so that when omitted, the NIXL OBJ plugin falls back to the AWS SDK default credential provider chain (IAM roles, environment variables, credential files, etc.). This enables workload-identity based authentication on Kubernetes (AWS IRSA, GCP Workload Identity, Azure Workload Identity) without requiring explicit credentials in the vLLM conf…

• 93e2ab7 #45963 — Disable dynamic speculative decoding when DP is enabled (#45963)
  • 作者: Tyler Michael Smith | +60/-1 | 4 个文件

  Disable num_speculative_tokens_per_batch_size when data parallelism is enabled. Dynamic speculative decoding currently picks K from each scheduler’s local scheduled request count. With DP, ranks can choose different K values, and the EAGLE/draft proposer has K-dependent DP collectives. This change warns and falls back to the static num_speculative_tokens value when data_parallel_size > 1. Some cod…

• 9204699 #38641 — [UX] Log worker exit code when process dies unexpectedly (#38641)
  • 作者: Nick Cao | +5/-2 | 1 个文件

  When a worker process dies, the MultiprocWorkerMonitor logs the process name but not the exit code. This makes it impossible to distinguish between an OOM kill (exit code -9 / SIGKILL), a segfault (-11 / SIGSEGV), a Python exception (1), or other causes without external tools like dmesg. Log proc.exitcode alongside the process name so users and developers can immediately identify the class of fail…

• dd94484 #41359 — Bump Transformers version to 5.10.4 (#41359)
  • 作者: Harry Mellor | +271/-234 | 51 个文件

  TL;DR which PRs do I need in a patch release of Transformers: - https://github.com/huggingface/transformers/pull/46456 - https://github.com/huggingface/transformers/pull/46500 - https://github.com/huggingface/transformers/pull/46524 - https://github.com/huggingface/transformers/pull/46525 - https://github.com/huggingface/transformers/pull/46605 - https://github.com/huggingface/transformers/pull/46…

• 7ff656c #47440 — fix: ensure no double load of lm head in nemotron mtp (#47440)
  • 作者: Shaun Kotek | +1/-5 | 1 个文件

  fix nemotron mtp head loading for quantized heads load nemotron 3.5 nano, v3 super and v3 ultra and see that it works. make sure AL is still good and no accuracy change ## Test Result All tests pass. — - [X] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”. - [X] The test plan, such as providing test command. - [X] The test results, such as pasting the …

• 48fcfc9 #47274 — [KV Offload] Add ParentManager ABC for secondary tier callbacks (#47274)
  • 作者: Ronen Schaffer | +169/-44 | 2 个文件

  • Add ParentManager ABC enabling secondary tiers (e.g. P2P) to call back into the TieringOffloadingManager for block lookup, pinning, and request lifecycle management - Implement _SecondaryTierFacingParent wrapper that auto-injects exclude_tier to prevent fan-out loops - Add serve_external_requests() hook for secondary tiers to process remote requests - Extract create_store_job() from duplicated i…

• b3e85be #47834 — fix: use configured max_logprobs instead of hardcoded 20 in derender validation (#47834)
  • 作者: Juan Pérez de Algaba | +13/-4 | 2 个文件

  • Replace the hardcoded top_logprobs limit of 20 in _validate_derender_bounds() with self.model_config.max_logprobs, so the derender endpoints respect the server’s –max-logprobs setting. - The default value (20) comes from the OpenAI Chat Completions API spec which defines top_logprobs as “an integer between 0 and 20”. - When max_logprobs is -1 (no cap, via –max-logprobs -1), the check is skippe…

• c85d720 #47321 — [HARDWARE][POWER] optimize math functions of VSX power (#47321)
  • 作者: Rukhaiya2004 | +154/-54 | 5 个文件

  Adds PowerPC VSX (Vector Scalar Extension) support to the vLLM CPU attention backend for optimized inference on POWER architecture systems.

• 066f02a #47427 — [MoE] FI autotuning: max bucket = max token count [e.g. DP_size*MNBT] (#47427)
  • 作者: Netanel Haber | +38/-2 | 4 个文件

  vLLM does not pass tune_max_num_tokens to FlashInfer’s BF16, FP8, and NVFP4 TRT-LLM MoE kernels. FlashInfer therefore ends autotuning at its default maximum bucket of 8192. This can be too low. The estimated maximum token count is: DP size * max_num_batched_tokens Larger inputs still run, but use a tactic tuned for a smaller bucket. This PR passes that estimate to FlashInfer while keeping 8192 as …

• 1e823dc #47830 — [docs update] Update usage of hf cli for cache list and removal (#47830)
  • 作者: Aritra Roy Gosthipaty | +7/-33 | 1 个文件

  This PR updates the docs with the correct (updated) usage of the HF CLI for cache list and removal. CC: @hmellor

🦀 Rust Frontend

• 04a703e #46793 — [Frontend] Support bad_words in the /v1/completions endpoint (#46793)
  • 作者: sungbin1015 | +39/-0 | 2 个文件

  The chat completions endpoint (/v1/chat/completions) exposes a bad_words field and forwards it to SamplingParams, but the legacy completions endpoint (/v1/completions) does not — even though SamplingParams fully supports bad_words. This is a feature-parity gap: users of the completions endpoint cannot suppress specific words. This PR adds the bad_words field to CompletionRequest and plumbs it thro…

• f7fc0ca #47454 — [Frontend] Add endpoint plugins framework (#47454)
  • 作者: Martin Hickey | +660/-1 | 12 个文件

  Adding a new admin/debug HTTP endpoint today means editing api_server.py directly. We already have vllm.general_plugins for adding engine side behavior out-of-tree but nothing equivalent for the HTTP side. This adds that missing half as one complete part: contract, loader/gating, wiring, docs, security and a worked example. - endpoint_plugin.py: new EndpointPlugin Protocol for HTTP surface only. T…

🔩 Misc

• bd3bb4e #47608 — [Misc][Docs] Add human-readable integer support for more cli-args (#47608)
  • 作者: Nicolò Lucchesi | +66/-5 | 3 个文件

  Follow-up to align some other cli-args to human-readable format (e.g., 1K, 2M, 32K) - which I find handy :) cli-args recap: Supported Arguments | Argument | Category | Parser | Example Values | Status | |———-|———-|——–|—————-|——–| | –max-model-len | Model | human_readable_int_or_auto | 1K, 2M, 128K, auto | ✅ Existing | | –max-num-batched-tokens | Scheduler | human_…

🧪 CI/Tests

• 99a8561 #47946 — [Test] Skip DeepEP MoE layer tests without P2P access (#47946)
  • 作者: Tyler Michael Smith | +24/-0 | 1 个文件

  • Skip DeepEP low-latency and high-throughput backend tests when GPU peer-to-peer access is unavailable - Fixes CUDA error ‘peer access is not supported between these two devices’ failures in Kernels FP8 MoE Test (2xH100) CI (56 subtests) - Uses the existing gpu_p2p_access_check utility, cached per world size Extracted from https://github.com/vllm-project/vllm/pull/45321 which addresses the broade…

• 6e35c5e #47731 — [ROCm][CI] Minimize comment in RocmAttention q_scale check (#47731)
  • 作者: stefankoncarevic | +2/-7 | 1 个文件

  Follow-up to #46148 addressing review feedback from @Rohan138: the explanatory comment above the fp8-query q_scale guard in RocmAttentionImpl.forward() was too long. This PR shortens it to a concise two lines. No functional change — only the comment is edited; the guard logic is untouched. Comment-only change; no new tests required. Behavior is unchanged and remains covered by the existing tes…

• 0ed05b6 #47832 — [CI] Fix Transformers modeling backend LoRA test (#47832)
  • 作者: Harry Mellor | +8/-18 | 3 个文件

  As identified in https://github.com/vllm-project/vllm/pull/47804, https://github.com/vllm-project/vllm/pull/47187 added a new attribute to QKVParallelLinear for the split operation that is added as part of Transformers backend fusion. This new attribute was not copied over by LoRA wrapping classes which caused a test failure. This PR removes the need for this new attribute as all the necessary inf…

• cbb5f04 #46904 — [ROCm][CI] Refresh ROCm base images when docker rocm_base changes (#46904)
  • 作者: Andreas Karatzas | +724/-52 | 14 个文件

  ROCm Dockerfile.rocm_base changes currently do not have a clean release path through CI. A PR can change the base image recipe while the rest of the ROCm image stack is still validated against older base/ci_base images. This makes base-image changes harder to reason about and easy to under-test. This PR adds an explicit base-refresh lane to the AMD image build: - Check whether docker/Dockerfile.ro…

• e55cc59 #47735 — [Rust Frontend][CI] Unblock more end-to-end test cases (#47735)
  • 作者: Bugen Zhao | +105/-40 | 14 个文件

  Unblock some more end-to-end test cases in CI for Rust frontend as we’re filling more gaps. This PR only enables the test cases that require no or minimal code and test changes. For those requiring a larger feature scope or fixes, separate follow-up PRs will be opened to unblock them. ## Test Result —

⚡ Performance

• 2afa3f7 #47631 — [Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631)
  • 作者: Wei Zhao | +65/-17 | 7 个文件

  This PR supports fusing Minimax M3’s MoE layers’ allreduce with next’s layer’s input norm. ## Performance TP2 with InfX benchmarking: 13406 tok/s -> 13709 tok/s (2.2% speedup) main: PR: - GSM8k Eval (TP, TEP, DEP) ## Test Result —

• d9e57ea #46117 — [ROCm][Perf] MXFP8 dense-linear + grouped-MoE GEMM optimizations for MiniMax-M3 (#46117)
  • 作者: Ethan Yang | +226/-42 | 3 个文件

  Numerically-equivalent Triton-kernel optimizations for the native MXFP8 path of MiniMax-M3 on AMD MI355X (CDNA4 / gfx950). Math is unchanged (tl.dot_scaled, fp32 accumulate; outputs numerically identical to the current kernels, tol 6e-2). AMD/ROCm-only: both kernels are gated on current_platform.is_rocm() and supports_mx() (gfx95x); on other platforms the oracle selects the existing paths, so …

• c3284c3 #47546 — [Perf][3/N] Expand Triton kernel warmup coverage, Qwen (#47546)
  • 作者: Roberto L. Castro | +3/-6 | 1 个文件

  Depends on: https://github.com/vllm-project/vllm/pull/47539 Fix: (Worker_TP0 pid=2149009) WARNING 07-03 06:43:45 [jit_monitor.py:125] Triton kernel JIT compilation during inference: _zero_kv_blocks_kernel (constexprs={BLOCK_SIZE=1024, N_SEGS=10, PAGE_SIZE_EL=270336}; signature={BLOCK_SIZE=‘constexpr’, N_SEGS=‘constexpr’, PAGE_SIZE_EL=‘constexpr’, block_ids_ptr=’*i64’, n_blocks=‘i32’, seg_addrs_ptr…

• d3e69fd #47081 — [Perf] Use blocking CUDA events to avoid busy polling cuda driver lock (#47081)
  • 作者: Summer Yang | +13/-6 | 3 个文件

  When async scheduling is enabled, the per-step CUDA-event syncs default to spin-wait events that busy-poll the CUDA driver lock. Under per-step TP collective contention the main-thread spin can balloon on a rotating rank, making it arrive late at the eager cross_device_reduce all-reduce while the other ranks spin-wait on it — the occasional “long all-reduce” straggle. This makes those events **b…

📖 Documentation

• dd0d74c #47374 — [Doc] Surface the –kv-cache-memory suggestion at INFO and document fast-startup knobs (#47374)
  • 作者: Nils Matteson | +9/-1 | 2 个文件

  Two small discoverability fixes for startup time, motivated by measuring a full boot-phase ledger on H100 (vLLM 0.24.0, Qwen3-8B + Qwen3-30B-A3B-FP8): 1. Promote the –kv-cache-memory suggestion from DEBUG to INFO. The worker already computes the exact value that reproduces the current KV allocation on every boot (gpu_worker.py), but emits it at DEBUG where operators never see it — making the …

• 47c40bf #47913 — [Doc] Fix manylinux tag in installation guide (#47913)
  • 作者: Nick Cao | +9/-9 | 4 个文件

  The manylinux tag in the installation guide is currently wrong, leading to 404 errors when installing vllm following these guides. See https://github.com/vllm-project/vllm/releases/tag/v0.24.0, check the tags on the wheels. ## Test Result Matches the diff. —

• c74e751 #36715 — [Doc] Fix grammatically incorrect error message in gpu_worker and xpu_worker (#36715)
  • 作者: Robin | +2/-2 | 2 个文件

  …supported’ Fix grammatically incorrect error messages in RuntimeError raises. - vllm/v1/worker/gpu_worker.py line 289: “Not support device type” → “Unsupported device type” - vllm/v1/worker/xpu_worker.py line 70: “Not support device type” → “Unsupported device type” “Not support” is grammatically incorrect English. “Unsupported” is the correct adjective form. Error message string change only, no …

🖥️ Kernel

• 5d23ca4 #47408 — [Kernel] Applies routed_scaling_factor internally (#47408)
  • 作者: Jee Jee Li | +56/-44 | 8 个文件

  Move the routed_scaling_factor for both M3 and DeepSeek V3 into the routing kernel. This has two benefits: - When torch.compile is not enabled(new model path), it eliminates the extra mul kernel introduced by routed_scaling_factor - It makes it easier to support the fusion of finalize + add + ar + norm #### before #### after ## Test Result —