共 59 个 commit，涉及 354 个文件，+27751/-2420 行变动。

概要

统计项 数值
Commit 数 59
变更文件 354
新增行数 +27751
删除行数 -2420

Commit 列表

🔧 Refactor

📦 Other

  • c4dd6d7 #48849 — Fix: Restore data_parallel_size > 1 for use_sequence_parallel_moe (#48849)
    • 作者: passtoor-agi | +1/-0 | 1 个文件

    Fix memory regression for MoE models on pure TP configurations (DP=1). Root cause: Commit 1ff942965 (PR #48036) removed and self.data_parallel_size > 1 from use_sequence_parallel_moe in vllm/config/parallel.py, causing Sequence Parallel MoE to incorrectly enable for TP-only (DP=1) configurations. SP MoE is designed to avoid token replication across DP ranks (see comment lines 633-639 in pa…

  • ce2aecc #48417 — [Performance] Use CuTe-DSL for FlashInfer MXFP4 quantization (#48417)
    • 作者: JooHo Lee | +6/-2 | 2 个文件

    Use the CuTe-DSL backend for activation quantization in FlashInferMxFp4LinearKernel. The kernel is already selected only on SM100+ when FlashInfer CuTe-DSL is available, and its following MXFP4 GEMM already uses the CuTe-DSL backend. This change makes the activation quantization use the same backend while keeping backend selection explicit at the kernel call site. Fixes #48205. The default CUDA ba…

  • d4b4562 #48942 — [XPU] Bump vllm_xpu_kernels to v0.1.11.1 (#48942)
    • 作者: Artur Fierka | +1/-1 | 1 个文件

    Bumps vllm_xpu_kernels pin from v0.1.11 to v0.1.11.1. v0.1.11 registers moe_sum with the old 2-arg XPU schema. Commit f7aadae5e5 added 2 extra args (topk_ids, expert_map) to the CUDA-side moe_sum op, but the XPU-side registration was never updated to match — every XPU MoE forward pass now crashes: v0.1.11.1 includes vllm-xpu-kernels#462, which fixes this. Verified on B70: reproduced the crash on t…

  • 4c6e2e4 #47516 — [XPU][UT]fix _POSSIBLE_KERNELS error on XPU (#47516)
    • 作者: Yejing Lai | +2/-2 | 1 个文件

    Root cause: choose_scaled_mm_linear_kernel did a direct dict index possible_kernels [current_platform._enum]. POSSIBLE INT8 KERNELS has no PlatformEnum.XPU entry, so this raised an unhandled KeyError instead of the intended “no kernel found” error. Fix: Changed both lookups to .get(current_platform._enum, []), so an unsupported/unregistered platform falls through to the existing ValueError (“Faile…

  • 8502958 #47975 — [XPU] support HND layout (#47975)
    • 作者: liuzhenwei | +0/-8 | 1 个文件

    with PR https://github.com/vllm-project/vllm/pull/44455 and https://github.com/vllm-project/vllm-xpu-kernels/pull/432 Now we can support HND layout in XPU, this unblocks PD disaggregation with heterogeneous TP. ## Test Result —

  • d5b1ec2 #48828 — [XPU] allow forcing flash attn for mm_prefix (#48828)
    • 作者: liuzhenwei | +12/-6 | 1 个文件

    On XPU, multimodal prefix-LM models (e.g. Gemma4-26B-A4B-it) were unconditionally forced to Triton Attention (https://github.com/vllm-project/vllm/pull/47688) because XPU Flash Attention lacks FA4 and cannot apply the bidirectional mask for vision tokens. This prevented users from using Flash Attention even for text-only workloads for better performance where the mask is irrelevant. This PR allows…

  • 109b736 #48839 — [docs] preserve page path in stable-docs announcement link (#48839)
    • 作者: Sage | +1/-1 | 1 个文件

    the banner always sends users to the stable docs homepage instead of the matching page. verified locally

  • 426d48b #48281 — [KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
    • 作者: Chang Guo | +326/-10 | 11 个文件

    Add optional locality metadata to KV events emitted by the built-in FS and OBJ secondary tiers. locality describes storage placement relative to the publishing vLLM instance. LOCAL means the storage is local to that instance; REMOTE means it is not. The field does not define an access path, routability, or latency. - Accept LOCAL or REMOTE in FS and OBJ tier configuration. - Treat an omitted value…

  • 26c909e #41599 — [Model] Support TranslateGemma-12b-it (#41599)
    • 作者: Zhang Jian | +174/-3 | 2 个文件

    Closes https://github.com/vllm-project/vllm/issues/41540. Related to https://github.com/vllm-project/vllm/issues/32446 This PR is copied over from https://github.com/vllm-project/vllm/pull/32819, with comments addressed. - TranslateGemma’s chat template requires extra metadata (source_lang_code, target_lang_code) that is not compatible with exsiting OpenAI chat schema. This PR allows extra fie…

  • fb1d8cc #48042 — [rl] Stateful Trainer Send: New Abstractions [1/N] (#48042)
    • 作者: Aaron Hao | +644/-24 | 7 个文件

    Context This is the first of a three-PR split of the trainer-side weight-transfer rework originally proposed as one large PR. The split is: - PR 1 (this one): introduce the new trainer-side abstractions (WeightSource / ModuleSource, VLLMWeightSyncClient, TrainerWeightTransferEngine, WeightTransferTrainerFactory). Purely additive — the existing worker-side WeightTransferEngine and every back…

  • 472d330 #48878 — Add blocks_per_chunk configuration for KV offloading to support heterogeneous KV cache groups (#48878)
    • 作者: Debasish Mohanty | +55/-1 | 2 个文件

    This PR introduces support for configuring KV offloading using blocks_per_chunk in kv_connector_extra_config as an alternative to the existing token-based block_size configuration. Previously, the offloading chunk size was specified in terms of tokens (block_size), which was internally translated into blocks_per_chunk based on the GPU KV cache block size. This translation assumes that all KV cache…

  • 4d4e04f #48617 — [Render] Add round trip parity test and docs for derender (#48617)
    • 作者: Martin Hickey | +397/-2 | 7 个文件

    Adds a new e2e test (ref: https://github.com/vllm-project/vllm/issues/42729#issuecomment-4691079526) that test the chat completions path /v1/chat/completions (parses tokens as they stream out on a normal server) against the disaggregated derender path /v1/chat/completions/derender (parses the same tokens all at once out of process). Its the same generation, same tokens and both parsers should ther…

  • ee8f36d #48881 — [Warmup] Show CuTeDSL compilation progress (#48881)
    • 作者: Woosuk Kwon | +4/-0 | 1 个文件
    • Show progress while registered CuTeDSL kernels compile during startup. - Display the progress bar only on global rank zero to avoid duplicate output from distributed workers. ## Duplicate-work check I searched open PRs for CuTeDSL warmup progress tqdm and cutedsl_warmup. PR #48807 changes exception handling around warmup calls and does not overlap with this rank-zero progress reporting. ## Valid…
  • f3e9497 #48884 — [Model] Add Inkling LoRA support [4/N] (#48884)
    • 作者: Woosuk Kwon | +338/-48 | 12 个文件
    • Mark Inkling as LoRA-capable and map adapter names for its packed QKVR projection, shared sink experts, and LM head. - Add a LoRA-capable linear implementation of the shared sink experts while retaining the fused implementation when LoRA is disabled. - Support Inkling’s shared-outer MoE adapter layout, including expert-parallel slicing and stride-zero expansion of shared factors. - Apply Inkling…
  • fe784ff #48582 — [M3] Improve indexer for long-context decode (sm100) (#48582)
    • 作者: Thien Tran | +756/-63 | 7 个文件

    This PR adds a CuteDSL indexer decode kernel for Minimax M3, optimized for long-context. - TMA + mma.sync design, instead of tcgen05 as I find saturating memory bandwidth with high occupancy (+ hardware scheduling) is easier than tcgen05 software pipelining, especially when there is context imbalance (different requests have different context lengths) - BF16 and FP8 Indexer cache support - Specula…

  • 2cab53d #42749 — [Model][Hardware][AMD]: Part 1/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_AITER_FA, and ROCM_AITER_UNIFIED_ATTN (#42749)
    • 作者: Jack Hu | +1361/-4 | 15 个文件

    This is Part 1 of a stacked PR series that splits the original jhu96/optimize-qwen30b branch (PR: https://github.com/vllm-project/vllm/pull/39527) into two reviewable PRs. - Here, we enabled runtime inductor fusion substitution for both the ROCM_AITER_FA and ROCM_AITER_UNIFIED_ATTN attention backends. As for ROCM_ATTN (coming in part 2), it requires kernel-level changes, thus deferring to anot…

  • ab0a20d #46396 — [Docs] Add Phi-3.5-mini-instruct to batch invariance tested models (#46396)
    • 作者: yuvalluria | +1/-0 | 1 个文件

    Adds microsoft/Phi-3.5-mini-instruct to the list of tested models for batch invariance. ## Test Results Model: microsoft/Phi-3.5-mini-instruct (3.8B) Results: Test Configuration: - Hardware: 4x NVIDIA A10G GPUs (23GB each) - Tensor Parallelism: 4 - Backend: FLASH_ATTN - vLLM Version: v0.21.0 - Environment: VLLM_BATCH_INVARIANT=1, VLLM_USE_FLASHINFER_SAMPLER=0 - Temperature: 0.0 (greedy…

  • 4a394bf #47216 — [Spec Decode][DSpark] Add Gemma4-12B DSpark draft model (#47216)
    • 作者: Hangrui Cao | +417/-4 | 6 个文件

    Adds DSpark (semi-autoregressive block speculative decoding, built on #46995) for Gemma4-12B, for the deepseek-ai/dspark_gemma4_12b_block7 checkpoint. Following the Laguna DFlash review, this is a thin subclass reuse of the existing stacks, not a standalone model. ## Design - Gemma4DSparkModel(DFlashQwen3Model) — inherits the fused context-KV precompute. Gemma4’s attention_k_eq_v i…

  • c95c663 #48538 — [Quant] Add nvfp4_per_token online MoE quantization (#48538)
    • 作者: Michael Goin | +285/-5 | 7 个文件

    Adds a new online-quantization shorthand, –quantization nvfp4_per_token, that quantizes MoE expert weights to NVFP4 at load time from a bf16 checkpoint and runs them through the FlashInfer TRTLLM fused-MoE kernel with dynamic per-token global activation scales. Similar in spirit to https://github.com/sgl-project/sglang/pull/26083, but fits right in vLLM’s existing online-quant framework, so i…

  • fb5ec0d #48869 — [Model] Add Inkling MTP=1 support [3/N] (#48869)
    • 作者: Woosuk Kwon | +710/-6 | 11 个文件

    Why This is the third Inkling enablement slice after #48799 and #48822. It adds only the first MTP draft layer so Inkling can use one speculative token with CUDA graphs. Duplicate-work searches for “Inkling MTP” and “Inkling speculative decoding” found only #48768. This PR intentionally supersedes only the MTP=1 portion of that draft: it excludes LoRA, MTP>1 scheduler/KV-cache work, and unrelat…

  • efa2e42 #48868 — [Helion] Fix degenerate scale_ub in kernel input generators (#48868)
    • 作者: Shangdi Yu | +35/-4 | 4 个文件

    The generate_inputs() for the per-block/per-token fp8 quant Helion kernels set scale_ub = torch.mean(input). The inputs are zero-mean (torch.randn), so mean(input) ≈ 0 (often slightly negative). Since scale_ub clamps the per-group/per-token amax used to derive the quant scale, a ~0 upper bound collapses every scale to min_scaling_factor, saturating essentially the entire fp8 output. The kernel and…

  • 02bf9c7 #46757 — Fix Quark mxfp4 quantized model loading issue under mtp (#46757)
    • 作者: xiao-llm | +26/-2 | 2 个文件
    • Fix Quark quantization exclusion handling for GLM-5.2 MTP layers. - Treat excluded aggregate MoE modules as unquantized when their child expert projections are listed in quantization_config.exclude. - This prevents the GLM-5.2 MTP layer from being allocated with MXFP4-packed expert tensors when the checkpoint stores that layer in bf16. GLM-5.2 MXFP4 checkpoints can keep the MTP layer unquantized…
  • f61163e #48858 — [Model] Add Hopper FA4 relative attention for Inkling (#48858)
    • 作者: Woosuk Kwon | +109/-5 | 2 个文件

    Inkling’s current relative-attention path uses the tml-fa4 sheared-bias interface, which is a Blackwell-specific path. On Hopper, that reaches incompatible SM90 call signatures and split-KV constraints. This change: - uses regular FA4 score_mod plus aux_tensors for Inkling relative bias on Hopper (SM90); - keeps the optimized tml-fa4 sheared-bias path on Blackwell (SM100/SM110); - keeps num_splits…

  • 251f7e4 #48822 — [Model] Add PW CUDA graph support for Inkling [2/N] (#48822)
    • 作者: Woosuk Kwon | +98/-32 | 10 个文件

    This is the second independently mergeable slice carved out of #48768. It adds breakable (piecewise) CUDA graph support for Inkling on Model Runner V2: - automatically enables breakable CUDA graphs for InklingForCausalLM and InklingForConditionalGeneration - marks the Inkling attention boundary as eager and makes short-convolution metadata safe for uniform batched execution - preserves padded requ…

  • ce65385 #47679 — [KV Offload] Split tiering_lookup_delay into sync/async histograms (#47679)
    • 作者: Srinivas Krovvidi | +208/-5 | 4 个文件

    Splits the planned vllm:kv_offload_tiering_lookup_delay_seconds metric into two histograms on TieringOffloadingManager: - vllm:kv_offload_tiering_lookup_sync_delay_seconds : cost of a single secondary-tier lookup call. - vllm:kv_offload_tiering_lookup_async_delay_seconds : total time a request’s lookup stays deferred across a tier promotion, from the first RETRY until it resolves (or the request f…

  • 75bdad4 #48507 — [Bug][Quantization] Fix humming is_layer_skipped for compressed-tensors “re:” ignore entries (#48507)
    • 作者: Zhongdongming Dai | +67/-2 | 2 个文件

    HummingConfig.is_layer_skipped decides which layers stay unquantized. It reads the checkpoint’s ignore / ignored_layers / modules_to_not_convert list and matches each entry against the layer prefix using a plain substring check: compressed-tensors checkpoints express ignored layers as regex patterns prefixed with re: (e.g. “re:vision_tower.*”, r"re:.mlp.(gate|up|gate_up|down)_proj.”). T…

  • 7cd1d57 #47442 — [CI/Build][Docker] Bump nvidia-cutlass-dsl to 4.6.0 and drop packaging workarounds (#47442)
    • 作者: Artem Perevedentsev | +8/-120 | 7 个文件

    In nvidia-cutlass-dsl wheel package there was a bug in using the same paths in filesystem by two different sub-wheels that resulted in race conditions during uv pip install process in vLLM. Original bug reports https://github.com/NVIDIA/cutlass/issues/3259, https://github.com/NVIDIA/cutlass/issues/3170 in cutlass repository. To overcome this issue vLLM used some temporary workarounds https://githu…

  • d803b44 #47559 — [NIXL] Bump nixl to 1.3.1 (#47559)
    • 作者: ovidiusm | +1/-1 | 1 个文件

    Bump NIXL version to 1.3.1. requirements/kv_connectors.txt is in run_all_patterns, so editing it triggers the full NixlConnector P/D sweep; the -nixlconnector-pd-accuracy- jobs exercise the NixlConnector against the new nixl wheel. ## Test Result TBD — awaiting the -nixlconnector-pd-accuracy- sweep on this branch. —

  • 530852f #48481 — [KV Connector] Fix PD async scheduling race condition for hybrid attn models (#48481)
    • 作者: Artem Perevedentsev | +158/-7 | 4 个文件

    IMPORTANT: PR was fully copied from https://github.com/vllm-project/vllm/pull/47373 This PR fixes an accuracy regression in PD disaggregation with async scheduling for hybrid attention + SSM/GDN models. When a decode-side request receives KV blocks from a remote prefill worker, the newly allocated attention blocks are overwritten by the incoming NIXL transfer. For hybrid models, vLLM also sc…

  • a9531ed #48150 — [KV Offload] Define clean backend configuration boundary (#48150)
    • 作者: Chang Guo | +1275/-843 | 27 个文件

    Part of #47929. This PR defines plain offloading configuration structs in vllm/v1/kv_offload and moves the existing KVCacheSpec and KVCacheConfig parsing for block geometry, CPU sizing inputs, and file identity into the offloading connector, which translates the raw configs into those structs. The struct shape (model-, cache-, and parallel-level sub-configs with tokens_per_block, blocks_per_chunk,…

  • 9f8cbfd #48209 — Vectorize prep xfer list creation (#48209)
    • 作者: Ilia Yastrebov | +85/-74 | 3 个文件

    In P/D disaggregation with the NIXL connector, the first KV transfer to each new peer is very slow (up to several seconds), spiking first-token latency. The decode-side handshake (add_remote_agent) is dominated by vLLM Python code that builds the per-block NIXL descriptor list — a Python loop creating ~6.5M (addr, len, dev) tuples — which NIXL then ingests element-by-element. This one-time ~1….

  • ea1d65f #47741 — [Rust Frontend] Add Seed-OSS tool parser (#47741)
    • 作者: Chao-Ju Chen | +332/-18 | 7 个文件

    This PR is the tool-parser half of Seed-OSS support in the Rust frontend. The reasoning side is already there, but the Seed-OSS tool-call wrapper is different from Qwen3-Coder, so a Seed-OSS model can’t return tool calls yet. This finishes it. Roadmap item is seed_oss under #44280. The tool-call format is just Qwen3-Coder with a different wrapper. vllm/parser/seed_oss.py says as much: <tool_call> …

  • f44f3d6 #47965 — [Rust Frontend] Wait for mock engine endpoints before ZMQ connect (#47965)
    • 作者: Reid | +34/-24 | 1 个文件

    The mock engine can race ahead of frontend endpoint binding during startup. For TCP endpoints, it previously attempted the ZMQ connect without any readiness wait. For IPC endpoints, it only waited for the socket path to exist, which does not always mean the listener is ready to accept connections. When the connect happens too early, zeromq treats the refused connection as retryable and backs off b…

  • 85e2969 #47973 — BF16x3 router GEMM (#47973)
    • 作者: Thien Tran | +555/-10 | 5 个文件

    This PR is inspired by CuBLAS BF16x9: https://developer.nvidia.com/blog/unlocking-tensor-core-performance-with-floating-point-emulation-in-cublas/ ## Background All FP32 numbers, including subnormals, can be represented exactly by a (weighted) sum of 3 BF16 numbers. This is because FP32 has 24 mantissa bits (including the implicit 1.xxx for normal numbers) and BF16 has 8 mantissa bits. A simple pr…

  • dc9f845 #48738 — [Rust Frontend] Fix mock engine test shutdown race (#48738)
    • 作者: Reid | +1/-1 | 1 个文件

    The mock-engine abort test can complete its functional assertions and then fail during cleanup with errors Connection reset by peer. The cleanup helper previously closed the client transport before cancelling the mock engine. If the mock engine processed the transport closure first, normal test shutdown was reported as an unexpected I/O failure. Cancel the mock engine before shutting down the clie…

🐛 Bug Fix

  • f38f3d1 #48596 — [Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
    • 作者: AlexHuang | +116/-31 | 3 个文件

    Fix: last KV block that fills at request finish time is never offloaded, causing prefix cache misses for that block. Also fixes a data corruption race when the block is reused. Bug principle: _build_store_jobs runs during schedule() — at that point the finishing token (EOS) hasn’t been processed yet, so the last block is still partial and skipped. request_finished is called after EOS is append…

  • 7b31925 #48699 — [Bugfix]Fix transformer backend failed: AttributeError: ‘Parameter’ object has no attribute ‘weight_loader’ (#48699)
    • 作者: Yejing Lai | +13/-12 | 1 个文件

    Reproduce: python3 -m vllm.entrypoints.openai.api_server –model deepseek-ai/DeepSeek-R1-Distill-Llama-8B –enforce-eager –port 8000 –host 0.0.0.0 –model_impl transformers –trust-remote-code –gpu-memory-util=0.9 –no-enable-prefix-caching –max-num-batched-tokens=8192 –max-model-len=8192 –quantization fp8 will meet transformer backend failed: AttributeError: ‘Parameter’ object has no attrib…

  • ce4bdcb #48855 — [Bugfix] Enable FlashAttention MLA prefill for Mistral Small 4 head dims (#48855)
    • 作者: Julien Denize | +10/-5 | 3 个文件

    Mistral Small 4 uses MLA head dimensions (qk_nope_head_dim=64, qk_rope_head_dim=64, v_head_dim=128). FlashAttnPrefillBackend.supports_mla_dimensions did not list these dims, so the model was excluded from the FlashAttention MLA prefill backend. This adds the config to the supported set (FA2/FA3/FA4), extends the selector unit test, and updates the attention backends doc. - A/B probe on H200 (FA3) …

  • 69d4f5e #46213 — [Bugfix][Multimodal] Fix Qwen3-Omni use_audio_in_video with mixed image/video inputs (#46213)
    • 作者: wendadawen | +266/-83 | 8 个文件

    Fix three bugs in Qwen3-Omni multimodal input validation when use_audio_in_video=True: 1. Single video + image inputs (V+I, I+V) crashed EngineCore because the deepstack interleaved branch only treated video positions as vision positions and missed image positions. 2. Interleaved embedding merge inferred modality from token counts and could misclassify image embeddings as video in mixed image/vide…

  • 17fdd42 #48251 — [Bugfix][Attention] Preserve post-load tensors across weight reloads (#48251)
    • 作者: aoshen02 | +80/-4 | 4 个文件

    Preserve attention runtime tensors derived by process_weights_after_loading() across layerwise weight reloads. This fixes stale FlashInfer attention sinks and prevents standard MLA CUDA graphs from retaining obsolete W_UV/W_UK_T addresses. ## Root cause Two post-load paths replaced tensors instead of refreshing their existing storage: - Standard MLAAttention reassigned derived W_UV and W_UK_T view…

  • 3b6c96a #48901 — [Bugfix][Pooling] Fix wrong scores for chunked prefill under torch.compile (#48901)
    • 作者: Sebastian Woo | +52/-0 | 2 个文件

    Fixes #48831. LAST-pooling models (e.g. Qwen/Qwen3-Reranker-0.6B) return wrong relevance scores when a query+document pair is long enough that its prefill is split across chunks — but only under torch.compile (the default). –enforce-eager was always correct, and the offline batch path (which happened not to chunk) was correct, which is why this hid so easily. ## Root cause I traced this end-t…

  • 67f9046 #48642 — [Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642)
    • 作者: Matthew Bonanni | +516/-85 | 13 个文件

    PR #47327 added dense-MHA prefill routing for sparse MLA, exposing two fp8_ds_mla mixed-batch bugs: - req_id_per_token could exceed the top-k tensor length and cause out-of-bounds writes. - Dense MHA could not gather the packed 656-byte FP8 cache, and FlashMLA assumed its metadata covered the full batch. This PR bounds the converter input, adds arbitrary sequence starts to the packed FP8 gather/up…

  • ab3c1ae #48785 — [Bugfix] Fix activation quantization dispatch for WNA4Int/WNA8Int (#48785)
    • 作者: HDCharles | +13/-6 | 3 个文件

    CT humming path was pretending to do wNaM but was falling back to wNa16 ### Problem 1: HummingInputSchema -> BaseInputSchema if you call HummingInputSchema.from_config from a subclass it won’t enter into the branch. so it won’t get to CompressedTensorsInputSchema it will instead try to create keys based on humming names. Those names aren’t set up so instead they default to none. When that would ge…

  • 971dac2 #47495 — [Bugfix][KV-transfer] MoRIIO: retry RDMA send-queue-full backpressure instead of failing the read (#47495)
    • 作者: limeward | +91/-14 | 4 个文件

    At high concurrency the per-QP RDMA send queue fills up. read_remote_data posts the READ synchronously, but the mori executor joins its worker before returning and marks the status on the calling thread, so a send-queue-full rejection comes back as a Failed() status immediately. That Failed() status was then treated as a failed transfer. get_finished notifies the prefill to release its blocks …

  • 8c3393f #48134 — [Bugfix][Rust Frontend] Limit chat top_logprobs in responses (#48134)
    • 作者: Reid | +86/-5 | 4 个文件

    Fixes a Rust frontend chat response parity issue where top_logprobs entries were not truncated according to the requested count. For chat completions with logprobs=true, the Rust frontend previously emitted every decoded candidate returned by the engine for each output token. Python serving limits top_logprobs as follows: - omitted or 0: return an empty top_logprobs list - positive k: return the f…

  • cc706b0 #47707 — [Bugfix][Rust Frontend] Detokenizer: avoid leaking prompt on zero-generated-token completions (#47707)
    • 作者: JinYan Su | +34/-5 | 1 个文件

    DecodeStream::flush() called before any push_token() (i.e. a request that finishes with zero generated tokens — the engine suppressed the only emitted EOS) leaks the entire prompt text into cumulative_output, because seed_prefix never ran and prefix stays empty. This mirrors the Python V1 path, where BaseIncrementalDetokenizer.update() early-returns on empty new_token_ids, leaving output_text unto…

  • 12f2c51 #48530 — [Bugfix] Fix offloading set_ overflow for packed non-uniform KV caches (#48530)
    • 作者: Elvir Crnčević | +8/-7 | 1 个文件
    • When attention layers are packed into a shared KV cache tensor (e.g. MLA models with non-uniform page sizes), each layer’s page_size can be smaller than the block_stride. torch.Tensor.set_(storage, offset, size, stride) validates that offset + size_bytes <= storage_size, but for packed layouts the last block’s offset + page extends past the per-layer boundary into the next layer’s region — valid…

🧪 CI/Tests

  • 867ff69 #48772 — [CI] Gate non-default release wheel builds (#48772)
    • 作者: Kevin H. Luu | +69/-30 | 4 个文件

    Regular release-v2 builds currently build every wheel variant after each merge to main. This change limits the automatic wheel group to: - Linux x86_64, CUDA 13.0 - Linux aarch64, CUDA 13.0 CUDA 12.9, CPU, macOS, and ROCm wheel work remains available behind unblock steps for on-demand builds. Builds with NIGHTLY=1 bypass those blocks and run the full wheel set. The index scripts also avoid letting…

  • 67fe73b #48873 — [CI] Extend max-model-len for test_parsable_context to allow reasoning to finish (#48873)
    • 作者: Micah Williamson | +1/-1 | 1 个文件

    For this test test_parsable_context.py::test_basic, the max-model-len was arbitrarily limited to 5000. This is just barely too short to fit the reasoning needed to pass test_parsable_context.py::test_basic[Qwen/Qwen3-8B] sometimes. Here we extend the max model len to fit the context. This issue manifests on both ROCm and CUDA (see the BK links aboce) in a flaky manner. Sometimes it caps out at the…

  • 7d56fe2 #48015 — [ROCm][CI] Avoid HIP init at config time via lazy aiter import in Quark OCP-MX (#48015)
    • 作者: music-dino | +29/-21 | 1 个文件

    Fixes ROCm failures in the model-initialization tests, for Basic Models Tests(Extra Initialization) 3, 4, 5 and 6 .e.g.: These tests stub out V1EngineCore._initialize_kv_caches via monkeypatch to skip the dummy forward pass that profiles memory. The stub only takes effect if the engine core is forked (so it inherits the patched parent process). On ROCm the parent’s HIP context gets initialized dur…

🦀 Rust Frontend

  • 9354f22 #48107 — [Rust][Benchmark] Port in vllm-bench (#48107)
    • 作者: Roy Wang | +16926/-2 | 45 个文件

    Test Result —

  • 3e90d01 #47699 — [Frontend] Overlap preprocessing and computation for pooling models offline inference (#47699)
    • 作者: wang.yuqi | +673/-150 | 10 个文件

    TL;DR - Use multithreading to accelerate preprocessing. You can specify the number of threads using renderer_num_workers. - Tiling to overlap preprocessing and computation for pooling models offline inference. - Sadly, because of the GIL, multithreading can only speed up the tokenizer and can hardly accelerate multimodal preprocessing at all. We are looking for ways to use multiprocessing to sp…

🔩 Misc

  • b88abb5 #44749 — [Misc] Remove orphaned env vars and stale env-var references (#44749)
    • 作者: Daoyuan Li | +1/-33 | 3 个文件

    Five VLLM_* entries in vllm/envs.py no longer have any consumers anywhere in the tree, and one EP doc still tells users to tune an env var that was removed. Drop the dead definitions and stale references so the env-var surface and the docs match the code. Same shape as #44128. ## Findings (verified by grep -rl across the tree, excluding vllm/envs.py) | Env var | Why it’s dead | | — | — | | VLL…

  • a317bc5 #48717 — [Misc][Nixl] Unify _logical_to_remote_kernel_block_ids (#48717)
    • 作者: Nicolò Lucchesi | +41/-64 | 5 个文件

    Fold _logical_to_remote_kernel_block_ids into _logical_to_kernel_block_ids function + ratio as input, so we can use the same function for both local/remote expansion. cc @ZhanqiuHu

⚡ Performance

  • f17be06 #48143 — [Perf] Optimize clamp to clamp_ (#48143)
    • 作者: Wentao Ye | +48/-32 | 7 个文件

    Optimize clamp to clamp_ to reduce additional memory allocation.

  • d08eeba #47156 — [Perf][MoE] Write FlashInfer combine into final output (#47156)
    • 作者: Samuel Nordmann | +61/-2 | 3 个文件

    Pass vLLM’s preallocated MoE output tensor to FlashInfer one-sided combine, removing the temporary combine output and subsequent output.copy_(). Requires FlashInfer PR: https://github.com/flashinfer-ai/flashinfer/pull/3776 ## Performance Nemotron Ultra 550B A55B NVFP4, one GB200 node (4 gpus), TP1/DP4/EP4, FlashInfer one-sided, ISL/OSL 50000/2048, concurrency 16: | | Baseline | Direct output | Del…

  • b8168e3 #46275 — [ROCm][Perf][DSV4] Enable split sparse decode on gfx942 (#46275)
    • 作者: Tuukka Sarvi | +45/-42 | 2 个文件

    Enable the existing split partial/reduce Triton sparse decode path for DeepSeek-V4 sparse MLA decode on AMD gfx942. gfx950 already uses this path. gfx942 currently falls back to the monolithic _sparse_attn_decode_ragged_kernel, even though the split path runs correctly on gfx942. This PR extends the tuned-architecture guard to include gfx942 and updates the ROCm DeepSeek-V4 sparse attention tests …