共 57 个 commit，涉及 221 个文件，+8763/-992 行变动。

概要

统计项	数值
Commit 数	57
变更文件	221
新增行数	+8763
删除行数	-992

Commit 列表

🧪 CI/Tests

• 0885b51 #48746 — [CI][ROCm] Stabilize ci_base hash calculation and image handoff (#48746)
  • 作者: Andreas Karatzas | +69/-6 | 2 个文件

  • Calculate the ROCm ci_base content hash three times with five-second intervals. - Require every calculation to succeed and produce the same hash. - Fail when the BASE_IMAGE digest cannot be resolved instead of hashing unknown. - Pass the exact ci_base image selected by the ensure step to the test-image build. Separate Buildkite jobs could calculate different ci_base tags for the same commit. The…

• 5811ed6 #48545 — [Test][kv_offload] Fix flaky drain() helper in test_fs_tier.py (#48545)
  • 作者: Chaojun Zhang | +4/-18 | 1 个文件

  The drain() test helper in test_fs_tier.py used a polling loop that exited early after 20 consecutive empty rounds (200 ms). Under a loaded system, store tasks could complete after the 200 ms window, causing test_load_job_emits_no_event and test_partially_failed_store_emits_no_event to fail with assert 0 == 1. ## Fix Replace the polling loop with tier.drain_jobs() (which calls wait_idle() and bloc…

• adce068 #48676 — [ROCm][CI] fix test_common.py (#48676)
  • 作者: Charlie Fu | +10/-4 | 2 个文件

  Cause Newer Transformers requires a tokenizer when generate() uses stop strings. That can come from: Explicit stop_strings= in the call, or The model’s generation_config.json naver-hyperclovax/HyperCLOVAX-SEED-Think-14B (used in test_common.py) has this in its generation config: “stop_strings”: ["<|endofturn|>", “<|stop|>”] HfRunner in tests/conftest.py was calling: self.model.generate(**input…

• 3ad85e0 #48387 — [CI][AMD] Configure MI300 tests for native execution without DinD (#48387)
  • 作者: Andreas Karatzas | +631/-75 | 23 个文件

  • Explicitly set native_ci: true on the current MI300 jobs. MI250, MI325, and MI355 omit the flag and continue through legacy DinD. - Keep runtime selection configuration-driven rather than inferred from an AMD device family. - Harden the producer/consumer artifact contract with an exact producer step, retries, checksum validation, commit/base/wheel metadata, strict wheel cardinality, and fatal up…

• 6e07344 #47330 — [ROCm][CI] Remove mxfp4 test skips after amd-quark 0.12 release (#47330)
  • 作者: Micah Williamson | +11/-59 | 5 个文件

  Bump Quark to unblock mxfp4 functionality that was broken with Torch 2.11, and clean up test skips that were put in place to unblock CI.

• 05d4f8b #48647 — [ROCm][CI] fix flashinfer import check (#48647)
  • 作者: Divakar Verma | +6/-4 | 1 个文件

  cuda platform check needs to be done before from flashinfer backend import Resolves (buildkite test fail link):

• 0b54201 #48289 — [CI] Build macOS arm64 CPU wheel natively on the macmini queue (#48289)
  • 作者: Michael Goin | +90/-6 | 4 个文件

  Builds the macOS Apple Silicon (arm64) CPU wheel natively on the new macmini Buildkite queue, then publishes it to wheels.vllm.ai via the normal release upload path. Supersedes #46660: now that a macOS Buildkite agent exists, we build directly on it instead of dispatching a GitHub-hosted runner. Drops the GHA workflow, the actions:read+write dispatch token, and the gh-based bridge; keeps the VLLM_…

• f04d3f6 #47754 — [Test] Enable KV cache events for HMA models in CPU offloading test (#47754)
  • 作者: Itay Etelis | +14/-19 | 1 个文件

  KV cache events no longer force the hybrid KV cache manager off (the restriction was removed in #37688, with connector-side group support added in #43468), so this drops the stale uses_hma guard from test_cpu_offloading and enables the KV-event assertions for the HMA models as well.

• 0a9396a #47231 — [XPU][CI] Add tests/v1/e2e/general/test_correctness_sliding_window.py in Intel GPU CI (#47231)
  • 作者: xiangdong | +2/-0 | 2 个文件

  Test Result —

🖥️ Kernel

• 6036bf1 #48512 — [Kernel][Helion] Add Helion kernel benchmark script (#48512)
  • 作者: Xiaohong (Sean) Chen | +574/-28 | 6 个文件

  This PR is to add an utility script to benchmark Helion kernels. Major changes are: - Updated all Helion kernels’ baseline function inside kernel impl file to use native torch implementation. This baseline is intended to be used as the baseline for Helion autotuner and correctness unit tests. It is more stable to use native torch impl for this purpose. - Added a script to run ops-level benchmark f…

• 2fa63e0 #48264 — [Kernel][Helion] Helion kernel lazy registration (#48264)
  • 作者: Xiaohong (Sean) Chen | +31/-6 | 3 个文件

  Current implementation will eagerly register all existing Helion kernels under “vllm/kernels/helion/ops” directory during vLLM server startup even if they are not used. The vLLM server will also log the following message during startup which often confuses user that Helion kernel are used. An ideal approach is to lazily register Helion kernel when it is really used. This PR disabled the eager regi…

📦 Other

• 61141ed #41934 — [Hardware][XPU] Register batch-invariant kernels for XPU (#41934)
  • 作者: Tomasz Zielinski | +130/-58 | 6 个文件

  This pull request adds partial support for batch invariance on Intel XPU devices. This is a requirement for reinforcement learning on XPU. Unit tests are modified to cover the registered kernels on XPU. ## Test Result All newly added unit tests pass. ### Note The code was co-developed with GitHub Copilot.

• 05eed72 #48526 — [ROCm] Re-enable cudagraph memory profiling, captured on the current stream (#48526)
  • 作者: peizhang56 | +37/-8 | 3 个文件

  Re-enables cudagraph memory profiling on ROCm (is_cuda() -> is_cuda_alike()) that #48440 disabled as a stopgap, and fixes the decode-throughput regression that motivated that revert. profile_cudagraph_memory() captures throwaway graphs via graph_capture(), which by default allocates a fresh side stream (torch.cuda.Stream(…)). torch’s caching allocator pools free blocks per stream, so the side-st…

• 5810e88 #47991 — [Model] Add RobertaForTokenClassification / XLMRobertaForTokenClassification (#47991)
  • 作者: Gopala-Krishna Char | +118/-1 | 5 个文件

  vLLM’s model registry supports BertForTokenClassification and ModernBertForTokenClassification for the /pooling token-classify runner, but RoBERTa-family checkpoints have no token-classification entry. RobertaForSequenceClassification/XLMRobertaForSequenceClassification already exist, but a RoBERTa or XLM-RoBERTa checkpoint fine-tuned for token classification (e.g. NER) currently has no matching a…

• 615834e #47636 — [KVOffload][P2P] Well-known default host/port env vars and per-DP-rank control port (#47636)
  • 作者: liranschour | +413/-61 | 13 个文件

  Give the P2P KV-offload secondary tier (type: “p2p” under TieringOffloadingSpec) the same well-known-port ergonomics the NIXL connector already has, plus per-DP-rank port binding so data-parallel replicas don’t collide. Two commits: 1. Well-known default host/port env vars. Add VLLM_P2P_SIDE_CHANNEL_HOST (default localhost) and VLLM_P2P_SIDE_CHANNEL_PORT (default 5710), mirroring VLLM_NIXL_SID…

• 1b30ae4 #47873 — [Rust Frontend] Fix flaky tls_handshake_timeout_drops_silent_client test (#47873)
  • 作者: Tahsin Tunan | +12/-6 | 2 个文件

  Summary - Problem: the test manually advanced the paused clock and raced the handshake close against the socket read, making it flaky. - Reproduce: for i in $(seq 20); do cargo nextest run -p vllm-server tls_handshake_timeout_drops_silent_client; done - Fix: drop the manual advance/yield_now, let tokio’s paused clock auto-advance a bare read().await, and assert the close lands at the handshake…

• 4e04bcb #48034 — [Rust Frontend] Tolerate whitespace before the outer brace in JSON tool-call parsers (#48034)
  • 作者: Tahsin Tunan | +105/-2 | 3 个文件

  Summary - A Rust JSON parser for tool calls was failing when pretty-printed (with newlines/spaces between braces). Python’s json.loads tolerates that whitespace, but the Rust parser didn’t skip whitespace before the closing }. - Fix: add explicit whitespace skipping before both close braces to match Python’s behavior.

• 9dd2e72 #48206 — fix flaky multi example connector consistency (#48206)
  • 作者: Aarushi Jain | +22/-4 | 1 个文件

  Fixes a flaky failure in tests/v1/kv_connector/unit/test_multi_connector.py::test_multi_example_connector_consistency. The test asserts the exact per-request scheduler event sequence for each connector, expecting on_new_request at index 0 of every window: on_new_request get_num_new_matched_tokens 0 update_state_after_alloc num_blocks=[…] … build_connector_meta However, build_connector_meta is …

• d119beb #48159 — [ROCm] Add tuned selective_state_update config for AMD MI350 (#48159)
  • 作者: Giuseppe Grossi | +102/-0 | 2 个文件

  Adds tuned selective_state_update (Mamba SSU decode kernel) launch configs for the AMD Instinct MI350X for both the cache_dtype=float16 and cache_dtype=float32 variants, mirroring how the MI355 (#47767), and MI300X (#47945, #47947) configurations were done. vLLM bundles per-device SSU configs for NVIDIA parts (B200, GB200, H100, H200, RTX PRO 6000) and, recently, MI355 and MI300X. On MI350X ge…

• 12a8057 #48600 — [CI/Build] Split release artifact annotations by type (#48600)
  • 作者: Kevin H. Luu | +25/-24 | 2 个文件

  Split the shared release-artifacts Buildkite annotation into dedicated release-wheels, release-images, and release-manifests annotations. This preserves inline reporting after each artifact is produced while making release outputs easier to scan. The annotation helper now requires an explicit context, preventing future callers from silently adding unrelated artifacts back to a shared section. Dupl…

• e281ac6 #48554 — [Rust Frontend] Integrate MM audio support (#48554)
  • 作者: Bugen Zhao | +1274/-257 | 18 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Audio support in llm-multimodal was added in https://github.com/lightseekorg/smg/pull/1905. Similar to #47959, this PR adopts that to further extend the Rust frontend to support audio input and preprocessing. Related code is generalized more a bit to support modalities not limited to vision (image / video). End-to-end test with Qwen3-ASR ## Test Result T…

• b6770d7 #48527 — [ROCm] Run init test engine in-process to avoid KV-cache OOM (#48527)
  • 作者: djramic | +7/-1 | 1 个文件

  test_initialization.py::can_initialize patches EngineCore._initialize_kv_caches to cap the KV cache. On spawn platforms the engine subprocess re-imports EngineCore, drops the patch, allocates the full KV cache, and OOMs (RuntimeError: Test subprocess ‘can_initialize’ failed). Fix: on spawn platforms, run the engine in-process (VLLM_ENABLE_V1_MULTIPROCESSING=0) so the cap applies. ## Test R…

• 37aa528 #48174 — Build with ABI stable FlashMLA (#48174)
  • 作者: Jane (Yuan) Xu | +18/-13 | 1 个文件

  Part of https://github.com/vllm-project/vllm/issues/26946. Test out stable FlashMLA extensions by moving the pin past https://github.com/vllm-project/FlashMLA/pull/15 and https://github.com/vllm-project/FlashMLA/pull/16 CI should still all be green. In the standalone fork: 4748 tests passed. In vLLM: 5 sparse tests passed, the dense need H100 but I’m on a aarch64 GB200 rn —

• 96d2ced #44549 — [Security] Replace diskcache to eliminate pickle deserialization (#44549)
  • 作者: Russell Bryant | +162/-19 | 7 个文件

  The outlines structured output backend used diskcache (which relies on pickle) to cache compiled outlines_core.Index FSM objects on disk. Pickle deserialization allows arbitrary code execution, posing a security risk. Replace diskcache with a lightweight SQLite-backed cache that uses outlines_core’s native binary serialization (via Rust serde) through Index.from_binary(). This eliminates the pickl…

• fdf2cf6 #48632 — [LoRA][1/N] Integrate flashinfer MoE LoRA for BF16 model (#48632)
  • 作者: Jee Jee Li | +425/-0 | 2 个文件

  Part of https://github.com/vllm-project/vllm/pull/47226 ## Test Result —

• 9b2be4e #46390 — [Quant] Enable humming w[2-7]a[4,8] inference with compressed-tensors (#46390)
  • 作者: HDCharles | +544/-3 | 8 个文件

  while there exists wNa8o8 support already, we separate the new vanilla support per the discussion in https://github.com/vllm-project/vllm/pull/45185 e2e tests were run, full repro: https://github.com/vllm-project/llm-compressor/pull/2821 also new CI test AI assistance was used (Claude). 🤖 Generated with Claude Code

• 4f7fffb #48525 — [Core][LoRA] Support fp32 lm_head (head_dtype) on the LoRA path (#48525)
  • 作者: Karthik Kothuri | +11/-29 | 3 个文件

  Follow-up to #48390 (fp32 lm_head via head_dtype), requested by @aoshen02. That PR rejected head_dtype != model dtype when –enable-lora was set, because LogitsProcessorWithLoRA._get_logits computes the base projection itself and bypassed the head_dtype-aware path in LogitsProcessor. This PR makes the two work together, which RL + LoRA workflows need (RFC #48297, RFC #48305 §3.6). ## Change - …

• f7aadae #48385 — add pad-aware reduce path (#48385)
  • 作者: gnovack | +239/-34 | 7 个文件

  Adds a padding-aware reduce function for marlin MoE. When serving with expert-parallelism enabled, the MoE inputs and intermediate states are padded along the token dimension to handle the worst case (i.e. all tokens routed to a single rank; see https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/fused_moe/prepare_finalize/deepep_v2.py#L32-L33). The moe_sum reduce kernel curr…

• 7e95052 #48428 — fix: size FlashInfer prefill workspace to batch head footprint (#48428)
  • 作者: Joe Rowell | +19/-0 | 1 个文件

  The fixed 394 MiB workspace default overflows for wide-head models at the 8192-token prefill chunk on sm_120, where FlashInfer hard-errors on batch_prefill_tmp_v instead of growing. Size it to max_num_batched_tokens * num_qo_heads * head_dim, floored at the configured default. ## Test Result —

• 9182e86 #48030 — Log fully resolved pooling config at startup (#48030)
  • 作者: Taneem Ibrahim | +78/-2 | 3 个文件

  Pooling runner startup already includes pooler_config inside the large vLLM config dump, but that only shows the final object. It does not explain which fields came from user input, Sentence Transformers metadata, model defaults, or pooler defaults, and it does not tie the resolved config to the model’s supported pooling tasks. This change records pooler config source per resolved field during Mod…

• 32e632d #46662 — [Reasoning] Optimize TPOT for thinking budget when used with speculative decoding (#46662)
  • 作者: rishitdholakia13 | +7/-20 | 3 个文件

  Optimized TPOT regression when thinking budget and speculative decoding are used together. The sampler was merging output_token_ids with spec draft tokens on every step whenever penalties/bad words/thinking budget was active causing expensive Python list copies at long contexts. Result TB behavior is unchanged; per-step sampling overhead drops from ~38 ms to ~0.2 ms at long context 63000 (bs=64), …

• 7ffb98e #48373 — [ROCm] Retune MI355 selective_state_update float32 config on the unified effective_batch grid (#48373)
  • 作者: vanshbhatia-amd | +35/-19 | 1 个文件

  Re-tunes the AMD Instinct MI355X selective_state_update (Mamba SSU decode kernel) cache_dtype=float32 launch config onto the same effective_batch grid already used by the MI300X (#47947) and MI350X (#48159) configs, so all AMD Instinct SSU configs are sampled on a uniform set of points. The original MI355 config (#47943) was keyed on a narrower, deployment-specific grid (10 … 5120). This P…

• cdaa40d #47666 — [KV Offload] Split cpu_cache_usage_perc into write/read usage gauges (#47666)
  • 作者: Srinivas Krovvidi | +80/-0 | 4 个文件

  vllm:kv_offload_cpu_cache_usage_perc currently reports one combined gauge for CPU KV-cache space pinned by any in-flight transfer. This PR splits it into two additional gauges so write pressure and read pressure can be told apart: - vllm:kv_offload_cpu_cache_write_usage_perc : fraction of CPU cache space pinned by in-flight stores (GPU-to-CPU copies) not yet completed. - vllm:kv_offload_cpu_cache_…

• 1ff9429 #48036 — [CI Bug] Fully solve accuracy issue for DSv3.2 + MTP + Sequence Parallel (#48036)
  • 作者: Wentao Ye | +34/-3 | 3 个文件

  A through fix for https://github.com/vllm-project/vllm/pull/47902 vllm serve deepseek-ai/DeepSeek-V3.2 –trust-remote-code –kernel-config.enable_flashinfer_autotune=False –enable-expert-parallel –tensor-parallel-size 8 –tokenizer-mode deepseek_v32 –tool-call-parser deepseek_v32 –enable-auto-tool-choice –reasoning-parser deepseek_v3 –port 9256 –speculative-config ‘{“method”:“mtp”,“num_spec…

• 7a74a96 #47021 — [NIXL] Avoid reading expired blocks in bidirectional turn-2 read (#47021)
  • 作者: tomerg-nvidia | +264/-22 | 10 个文件

  Currently with the multi-turn bidirectional mode, the nixl connector can result with garbage reads on turn 2 (P reading from D). This is because D sets a constant TTL for the blocks which is not extended by the heartbeat like the P TTL. When the blocks are expired they can be reused for other requests. The problem is that subsequent turns don’t have a way to tell that the blocks on D are no longer…

• b6754f5 #48594 — [Model] Enable LoRA support for tower and connector in LlavaNextVideo (#48594)
  • 作者: karthik | +149/-3 | 3 个文件

  Part of #31479 (enable LoRA for the tower and connector in more multimodal models). Adds SupportsLoRA, packed_modules_mapping, get_mm_mapping(), and get_num_mm_encoder_tokens() / get_num_mm_connector_tokens() to LlavaNextVideoForConditionalGeneration, following the pattern used for LlavaForConditionalGeneration in #31513. LlavaNextVideoForConditionalGeneration currently has no LoRA support at all …

• 894ebb2 #48291 — Add Cosmos3 Edge Reasoner model (#48291)
  • 作者: adsridhar | +1474/-27 | 13 个文件

  This PR brings in a new Cosmos3 Edge Reasoner model (not yet released) to vllm. Cosmos3 is a mixture of transformers model consisting of a Reasoner and a Generator tower. This PR introduces custom model layers, processors, and performs checkpoint mapping. Checkpoint loading and input processing tests have been added. ## Test Result The new tests pass.

• c9a788e #47595 — fix(security): guard lm-format-enforcer regex compile with timeout (#47595)
  • 作者: Juan Pérez de Algaba | +14/-1 | 1 个文件

  The lm-format-enforcer backend was left unguarded after the fix for GHSA-rwxx-mrjm-wc2m. Route the RegexParser call through compile_regex_with_timeout and validate regex patterns early in validate_structured_output_request_lm_format_enforcer, matching the xgrammar and outlines backends.

🐛 Bug Fix

• 66b6c68 #48125 — [PD][Bugfix] Fix validation of cache shape for attn backends enforcing different kernel_block_size (#48125)
  • 作者: Nicolò Lucchesi | +6/-1 | 1 个文件

  I have started looking into heterogeneous attn backend selection here https://github.com/vllm-project/vllm/pull/48012 (an extension of hybrid ssm setup if you want). This exposed a latent bug in the validation of cache shapes that we have in the registration phase. Say we have 2 attn backends enforcing different block sizes: - user block_size=128 - FA kernel_block_size=128 - SWA kernel_block_s…

• c0302d9 #48098 — [Bugfix] Fix parallel_tool_calls=null crash in Responses API from_request() (#48098)
  • 作者: Mahad Rehman | +46/-2 | 2 个文件

  Fixes #48097. Sending “parallel_tool_calls”: null to the Responses API (/v1/responses) caused from_request() to pass None to ResponsesResponse.parallel_tool_calls (typed as bool, non-optional), raising a Pydantic ValidationError during response construction. vLLM’s error handling catches this and returns a 400 with a raw Pydantic error message, instead of resolving to the documented default li…

• 313fae3 #48711 — [Bugfix] Fix GLM5 config (#48711)
  • 作者: Jee Jee Li | +29/-0 | 1 个文件

  Fix the following error when running nvidia/GLM-5.2-NVFP4on the latest main branch. ## Test Result —

• 7aab6e2 #48688 — [ROCm][Bugfix] Enable the fp32 head_dtype torch.mm fast path on ROCm (#48688)
  • 作者: Turner Jabbour | +36/-4 | 2 个文件

  #48390 added a torch.mm(…, out_dtype=torch.float32) fast path so the fp32 head_dtype lm_head projection avoids materializing a full fp32 copy of the lm_head weight on every step. That fast path was gated on current_platform.is_cuda(), so ROCm always fell back to the cast path (F.linear(hidden_states.to(fp32), lm_head.weight.to(fp32), …)), which allocates and writes a full fp32 copy of the (lar…

• 3b39fd2 #48671 — [Bugfix][Spec Decode] Support heterogeneous QK fusion geometry (#48671)
  • 作者: aoshen02 | +98/-19 | 2 个文件

  Fix QK Norm+RoPE fusion when speculative decoding uses target and draft models with different attention geometries. QKNormRoPEFusionPass discovers attention metadata from CompilationConfig.static_forward_context, which contains modules from both the target and draft models. It previously selected only the first attention layer and registered patterns for that geometry. Compiling a draft graph with…

• 6472131 #48379 — [Bugfix] Set kv_quant_mode on the generic MLA KV-cache spec (#48379)
  • 作者: drakosha | +2/-0 | 1 个文件

  FIX #48378 MLAAttention.get_kv_cache_spec left kv_quant_mode at its KVQuantMode.NONE default, so the KV-cache reshape treated every MLA layer as skipped from quantization and sized the view by head_size (576 elements/token) while the raw tensor is allocated by the fp8_ds_mla byte layout (656 B/token): Any DeepSeek-V3.2 / GLM DSA model with –kv-cache-dtype fp8_ds_mla crashes at engine init on curr…

• 0bd6b85 #44371 — [Bugfix] Preserve unloaded non-persistent buffers during layerwise reload (#44371)
  • 作者: Joan | +143/-5 | 4 个文件

  Fix layerwise reload corrupting unloaded non-persistent direct buffers. I hit this as a silent Gemma4 divergence in a warm weight-sync loop: google/gemma-4-26B-A4B-it matched trainer-forward on cold load, but after warm /update_weights calls the trainer-vs-vLLM KL jumped from about 0.01 to >1.5. The corrupted tensor was Gemma4Router.root_size, a persistent=False runtime buffer that changed from bf…

• 3ca242d #48622 — [Bugfix][R3] Exclude draft routers from expert capture (#48622)
  • 作者: aoshen02 | +35/-7 | 2 个文件

  Fix routed-experts (R3) capture corruption when speculative decoding uses a MoE draft model such as MTP. GPUModelRunner._bind_routed_experts_capturer currently walks compilation_config.static_forward_context, which contains modules from both the target model and the draft model. The target and draft can both have an MoE layer with layer_id=0, so the later draft forward overwrites the target layer-…

• 0f0f28b #48654 — [Bugfix][CI] Fix test_head_dtype quant_method test on ROCm (#48654)
  • 作者: Micah Williamson | +13/-1 | 1 个文件

  This ROCm-specific test failure was introduced by #48390, which added both this test and the _apply_head path it exercises. Failure log: https://buildkite.com/vllm/ci/builds/77772/canvas?jid=019f5ac7-4a84-4f1a-9b09-3a047ad8cb91&tab=output test_head_dtype_equal_to_model_dtype_uses_quant_method fails on AMD CI: NotImplementedError: Could not run ‘vllm::rocm_unquantized_gemm’ with arguments from the …

• 520a20b #45222 — [Bugfix] MoRIIO toy P/D proxy: add /health (#45222)
  • 作者: chaeminlim-mb | +55/-1 | 2 个文件

  Add /health to the MoRIIO toy P/D proxy. Bench harnesses and load balancers probe the proxy URL, not the backend URLs. The backends already expose /health, but the proxy returned 404, so the harness could mark a live proxy as dead. This PR only touches examples/disaggregated/disaggregated_serving/moriio_toy_proxy_server.py and its route-level test. The decode-budget behavior is left unchanged here…

• 313d01f #48631 — [CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631)
  • 作者: Matthew Bonanni | +213/-36 | 5 个文件

  Alternative to https://github.com/vllm-project/vllm/pull/48609 FA2 and FA3 support GLM-5 dimensions. FA4 does not. Report this correctly. Fixes the following CI failures (build #77969): - Basic Models Tests (Extra Initialization) 1 — test_can_initialize_large_subset[Glm4MoeLiteForCausalLM] - Basic Models Tests (Extra Initialization) 2 — test_can_initialize_large_subset[Glm4MoeLiteMTPModel]…

• b2f7d25 #48520 — [Bugfix] Make MLA+SWA check the layer’s backend, not the model config (#48520)
  • 作者: Michael Goin | +2/-2 | 1 个文件

  Attention.get_kv_cache_spec asserted on the model-level model_config.use_mla, rejecting a regular sliding-window layer in a drafter merely because the target model is MLA. Replaced with a per-layer check as self.attn_backend.is_mla(), preserving the real invariant (an MLA layer must not use a sliding window) without the false positive. Fixes https://huggingface.co/shanjiaz/dspark-mistral-small-119…

• af453e5 #48262 — [Bugfix] Gemma4 parser: classify channel-less output consistently in streaming and non-streaming (#48262)
  • 作者: adhi29 | +124/-25 | 2 个文件

  Fixes #48217 With thinking enabled, the Gemma4 streaming parser pre initialized its engine to REASONING for any prompt ending in a new model turn. If the model then produced a direct answer with no channel markers, every streamed chunk was classified as reasoning and no content was ever emitted, while the non streaming path classified the exact same text as content. This PR makes the pre initializ…

• 32aef44 #48411 — [Bugfix] Include inline per-token-head scales in offloaded page transfer width (#48411)
  • 作者: Itay Etelis | +41/-6 | 3 个文件

  Per-token-head quantized KV pages (*_per_token_head kv-cache-dtypes) carry fp32 scales inline within each (head, token) cell, beyond real_page_size_bytes (budgeted in AttentionSpec.page_size_bytes). The offloading connector used real_page_size_bytes as the per-block transfer width, so offloaded pages were truncated mid-cell: tail heads/tokens (data and scales) were dropped and restores brought bac…

• 793cf79 #48583 — [Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation (#48583)
  • 作者: Juan Pérez de Algaba | +298/-13 | 5 个文件

  Serialize all uses of torch.sparse.check_sparse_tensor_invariants() behind a shared threading.Lock to prevent concurrent prompt-embedding parts from racing the process-global save/restore flag.

• 038ec29 #48473 — [Bugfix] Return 400 instead of 500 when multimodal data is sent to a text-only model (#48473)
  • 作者: Hoang Nguyen | +32/-2 | 2 个文件

  Sending multimodal data (e.g. an image_url) to a text-only model returns HTTP 500 InternalServerError. It is a client mistake and should be HTTP 400 BadRequest. BaseRenderer._process_multimodal reads self._mm_req_counter before resolving the processor: _mm_req_counter is only initialized inside the if mm_registry.supports_multimodal_inputs(…) block, so on a text-only model the attrib…

⚡ Performance

• 442c421 #48137 — [Perf] Remove redundant repeat and copy for dsv4, 1.8% E2E TPOT improvement. (#48137)
  • 作者: Wentao Ye | +316/-15 | 3 个文件

  Part of https://github.com/vllm-project/vllm/issues/45861 hidden_states = hidden_states.unsqueeze(-2).repeat(1, self.hc_mult, 1) we do a repeat and copy for each decode step, which is waste of resource. This PR optimize the behavior GPT5.5 used to generate mhc_pre_big_fuse_broadcast_with_norm_tilelang kernel vllm serve deepseek-ai/DeepSeek-V4-Flash -dp 4 -ep –kv-cache-dtype fp8 –tokenizer-mode d…

📖 Documentation

• ca3618b #45437 — [Doc] Sync four function docstrings with their signatures (#45437)
  • 作者: Daoyuan Li | +8/-4 | 3 个文件

  Audit of function docstrings against their actual signatures; four Args sections document parameters that don’t exist (or are misspelled), so the rendered API docs mislead callers: | Function | Problem | Fix | |—|—|—| | w8a8_triton_block_scaled_mm (fp8_utils.py) | entry spelled output_dytpe | → output_dtype (the actual parameter) | | scaled_fp4_quant (_custom_ops.py) | documents use_8x4_sf_l…

🔩 Misc

• 50ac1c7 #45781 — [Misc] Rename VLLM_TRITON_ATTN_USE_TD to VLLM_TRITON_USE_TD (#45781)
  • 作者: Artur Fierka | +26/-7 | 3 个文件

  [Misc] Rename VLLM_TRITON_ATTN_USE_TD to VLLM_TRITON_USE_TD Follow-up to the tensor-descriptor (TD) pilot #40327 and the TD-adoption RFC #42545. The pilot shipped the TD opt-in behind the attention-specific VLLM_TRITON_ATTN_USE_TD; the RFC extends TD across vLLM’s Triton kernels behind a single, general flag. This renames the variable to VLLM_TRITON_USE_TD. The old name stays registered (so it d…