共 63 个 commit，涉及 279 个文件，+7534/-1917 行变动。

概要

统计项 数值
Commit 数 63
变更文件 279
新增行数 +7534
删除行数 -1917

Commit 列表

📦 Other

  • 4f819f8 #38390 — [Model Runner v2] E/P/D disaggregation support (#38390)
    • 作者: Wentao Ye | +145/-8 | 7 个文件

    E/P/D disaggregation support for MRv2 ### Single image

  • f9c74b4 #50991 — [Mamba] enable prefix cache by default (#50991)
    • 作者: Jiangyun Zhu | +30/-36 | 4 个文件

    enable prefix cache for hybrid model by default and change the default mode to align ## Test Result —

  • 5789897 #49969 — [Spec Decode] Add top-k DSpark Markov projection (#49969)
    • 作者: Andrii Skliar | +235/-26 | 4 个文件

    DSpark computes the base logits for all draft positions in parallel, then applies the Markov bias sequentially over the full draft vocabulary. The repeated full-vocabulary Markov projection is on the serial drafting path. This PR adds dspark_draft_topk for Qwen3 DSpark. It selects the top-k candidates from the base logits once, gathers the corresponding markov_w2 rows, and evaluates the sequential…

  • 1eb3694 #51014 — [Docs] Fix two docs build warnings (#51014)
    • 作者: Harry Mellor | +2/-2 | 2 个文件

    Fixes two warnings emitted by mkdocs build: 1. Broken cross-reference in docs/design/moe_kernel_features.md. The naive row of the all2all backend table pointed at vllm.model_executor.layers.fused_moe.runner.MoERunner, but runner/init.py re-exports nothing — the class is defined in runner/moe_runner.py. The link rendered as plain text for readers. The link text (layer.py) was also stale fro…

  • 59b2fdf #48250 — Support MLA properly in the Transformers modeling backend (#48250)
    • 作者: Harry Mellor | +699/-52 | 9 个文件

    Requires the following Transformers side PRs: - https://github.com/huggingface/transformers/pull/47435 - https://github.com/huggingface/transformers/pull/47451 - https://github.com/huggingface/transformers/pull/47460 Portions of this PR that were split into their own PRs: - https://github.com/vllm-project/vllm/pull/49957 - https://github.com/vllm-project/vllm/pull/49982 — vLLM side changes: - Ad…

  • 24c939c #47104 — [XPU] fix collecting oneccl version info (#47104)
    • 作者: Yan Ma | +2/-3 | 1 个文件

    oneccl version info on XPU platform should not be collected from apt lib. Here we refer the mpirun –version as it reflects the correct oneCCL system really uses. python vllm/collect_env.py ## Test Result —

  • e98a877 #50368 — [Rust Frontend][gRPC] Add multimodal image inference (#50368)
    • 作者: Connor Carpenter | +553/-69 | 12 个文件
    • Add image inputs to the Rust frontend gRPC Inference service. - Extend GenerateRequest with media items supporting HTTP(S) URLs, data URIs, and raw bytes. - Reuse the existing multimodal preprocessing pipeline to fetch and process images, expand placeholder tokens, and attach engine-facing multimodal features. - Preserve media UUID and MIME type metadata. - Require token-ID prompts when media is…
  • 0b1c151 #50540 — [Rust Frontend] Align tool rendering for Kimi K3 (#50540)
    • 作者: Bugen Zhao | +85/-9 | 4 个文件

    Align the Rust Kimi K3 renderer’s tool declarations with the Python and checkpoint encoding behavior. Related Python fix: #50228. That PR fixes the corresponding K3 renderer path in the Python frontend; this PR covers the Rust renderer behavior only. The renderer previously serialized a missing function description as an empty string and omitted strict for every tool. This change: - omits descript…

  • 385d4c0 #50859 — [ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859)
    • 作者: fxmarty-amd | +269/-1 | 2 个文件

    Companion AITER fix: https://github.com/ROCm/aiter/pull/4530 (making this PR fix redundant once the AITER fix is merged, released, and AITER pin in vLLM is updated). This PR hot-fixes that started failing in ROCm vLLM following AITER version bump from 0.1.16.post5 to 0.1.19 (https://github.com/vllm-project/vllm/pull/49361) as seen in mi355_1: Quantized Models Test, with: Previously passing, e.g. s…

  • 413e70d #48825 — Perf/h20 moe config e256 n512 (#48825)
    • 作者: zzt | +147/-0 | 1 个文件

    Add a tuned Triton MoE kernel configuration for E=256, N=512 on NVIDIA H20 GPUs. This shape is used by Qwen3.5-35B-A3B with TP=1. Without this file, vLLM reports that the device-specific configuration is missing and falls back to the default MoE configuration. The configuration was generated with Triton 3.6.0 using vLLM’s official benchmark_moe.py tuner. It covers batch sizes 1, 2, 4, 8, 16, 24, 3…

  • 11f8826 #50818 — [Kimi-K3] Migrate FlashKDA to PyTorch stable ABI (#50818)
    • 作者: Thien Tran | +19/-7 | 3 个文件

    TP8 - GSM8K: 0.9674 - OCRBench: 89.2 ## Test Result —

  • f427612 #45043 — llmd+vllm+mori-ep(inter node wide-ep)+mori-io(write) for 2p2d with dp=ep=16 tp=1 (#45043)
    • 作者: shikamd123 | +711/-121 | 8 个文件

    This PR extends #44355 from single-pod intra-node Wide-EP (1P1D, DP=EP=8, TP=1) to multi-pod cross-pod Wide-EP (2P2D, DP=EP=16, TP=1), where one DP world is split across two 8-rank pods per role (master ranks 0..7 + headless child ranks 8..15). Same real-world deployment: disaggregated prefill/decode serving with MoRI-IO as the KV connector, fronted by the llm-d routing sidecar on AMD …

  • 0b37d83 #48861 — fix: NVFP4 quantization out_dtype should match model dtype, not torch default (#48861)
    • 作者: Chris Fontes | +3/-3 | 1 个文件

    Problem torch.get_default_dtype() returns float32 by default. However, vLLM models typically run in bfloat16 (or float16). The NVFP4 (ModelOpt FP4) quantization linear methods were setting self.out_dtype = torch.get_default_dtype(), causing the NVFP4 kernel to output float32 tensors. This creates a dtype mismatch with downstream consumers: - LoRA adapters expect inputs matching the model dt…

  • f57123a #48048 — feat(frontend): session id plumbing into requests (#48048)
    • 作者: Karen Chung | +438/-4 | 39 个文件

    PR #1 of #48049 Add a first-class request-level session_id to vLLM so related requests in the same conversation, agent run, or session can carry stable identity without overloading request_id or storing session identity in SamplingParams.extra_args. This PR plumbs session_id through OpenAI-compatible request bodies, HTTP X-Session-ID, a temporary compatibility fallback from vllm_xargs[“session_id”…

  • e578de3 #50327 — [ModelRunnerV2] Fix scalar Mamba state update with int32 mappings (#50327)
    • 作者: Varun Vinayak Shenoy | +58/-16 | 2 个文件

    Fix a Model Runner V2 failure in pipeline-parallel hybrid Mamba models when a non-last pipeline rank processes a request: InputBatch.idx_mapping is intentionally an int32 CUDA tensor. The scalar branch of MambaHybridModelState.postprocess_state passed that mapping to Tensor.index_fill_, which only accepts an int64 index. This change adds a dedicated Triton scalar-fill kernel. It consumes the exist…

  • f43e1d2 #43615 — [ROCm] Enable AITER and FP8 inference on GFX120x (#43615)
    • 作者: skysnow2001 | +172/-44 | 12 个文件

    Enable AITER-backed FP8 and attention fast paths on AMD RDNA4 / gfx12x (R9700, RX 9070 XT), while preserving existing MI3xx behavior. Previously several AITER gates were effectively MI3xx-only, so gfx12 fell back to the generic ROCm/Triton paths even when AITER Triton support was available. This PR separates AITER Triton support from MI3xx-only CK support so gfx12 can use the optimized Triton path…

  • 42ab184 #50721 — [MRV2] Enable routed-experts capture (#50721)
    • 作者: aoshen02 | +335/-227 | 10 个文件

    Enable the existing routed-experts (R3) return path with Model Runner V2 (MRV2), while preserving the MRV1 protocol and behavior. ## Validation ### Focused tests - ruff check and format check: passed - git diff –check: passed - focused routed-experts tests: 13 passed, 2 skipped without CUDA - Ray zero-copy detachment tests: 2 passed - pre-commit, including mypy: passed - H200 CUDA capture tests: …

  • 755513d #48120 — [Hybrid] Stage the postprocess inputs with a single loop over the request list (#48120)
    • 作者: Francesco Fusco | +147/-76 | 2 个文件

    #40172 introduced stage_postprocess_inputs_to_gpu to feed the Triton kernel that runs the mamba state copies (prefix caching + MTP path). The current implementation walks req_ids[:num_reqs] twice and factors the work into two helpers (stage_mamba_state_idx_to_gpu, stage_postprocess_metadata_to_gpu) that are only ever called from stage_postprocess_inputs_to_gpu itself. This PR inlines both helpers …

  • 8f50685 #50582 — [ROCm][Kimi-K3] aiter moe environment variable cleanup (#50582)
    • 作者: Hongxia Yang | +71/-56 | 5 个文件

    Rebase on main for the closed PR: https://github.com/vllm-project/vllm/pull/50162 which was approved. cc @tjtanaa Changes: Env rename + hot-path caching — AITER_SITUV2_A8W4 → VLLM_ROCM_USE_AITER_MOE_SITUV2_A8W4, following vLLM naming. Cached in rocm_aiter_ops as _MOE_SITUV2_A8W4 with is_fused_moe_situv2_a8w4_enabled() (requires is_fused_moe_enabled()) and wired into refresh_env_variables(). Remove…

  • 5df9999 #50656 — [Kimi-K3] Add option to shard the shared expert instead of replicating (#50656)
    • 作者: Tyler Michael Smith | +154/-3 | 3 个文件

    Shard the shared-expert weights to avoid redundant work. In order to do so we need to re-distribute the activations, which adds an extra AllGather and ReduceScatter. This has been tested e2e to work and speed up decodes. So far, preliminary perf measurements show this is a decode win, and for prefill it’s a throughput vs kv cache size tradeoff. FYI the performance and memory footprint improvements…

  • 005fa01 #50624 — docs: document reasoning_content output removal as a breaking client change (#50624)
    • 作者: Fede Kamelhar | +2/-1 | 1 个文件

    reasoning_content was renamed to reasoning (RFC #27755, #27752) and then removed from the output entirely in #33402. The input side kept the old name alive — #42664 accepts reasoning_content and normalizes it to reasoning. This PR documents the resulting asymmetry as a breaking change for output consumers, since the current one-line warning (“To migrate, directly replace reasoning_cont…

  • 9ae11a6 #50524 — [Model] Add K-EXAONE-2.0-750B-A37B (#50524)
    • 作者: Kyungmin Lee | +161/-11 | 6 个文件

    This PR adds support for the K-EXAONE-2.0-750B-A37B, developed by LG AI Research. Hugging Face: https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B Technical Report: https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B/blob/main/assets/K-EXAONE-2.0-Technical-Report.pdf Transformers: https://github.com/nuxlear/transformers/tree/add-k-exaone2 SGLang: https://github.com/lkm2835/sglang/tree…

  • b977407 #50424 — Support quantized DSpark Markov heads (#50424)
    • 作者: Andrii Skliar | +4/-0 | 1 个文件

    Add quantization configuration support to the DSpark Markov head. DSparkMarkovHead now accepts and forwards quant_config to its ParallelLMHead-based markov_w2 projection. Qwen3DSparkModel passes its model quantization configuration when constructing the Markov head. This allows quantized markov_w2 weights, including W4A16 configurations with weight_scale_2, to load through the normal quantization …

  • ec40f6a #49960 — [CPU] Fix torch.compile crash from torch.accelerator.synchronize on CPU-only hosts (#49960)
    • 作者: Ganesh R | +13/-2 | 2 个文件

    On CPU-only hosts, torch.compile (e.g. gpt-oss) crashes during aot_compile with AssertionError: No accelerator available for torch.accelerator.synchronize. torch.accelerator.synchronize() has no accelerator on CPU, so it breaks both eager (raises) and torch.compile (Dynamo’s handle_synchronize asserts, even though a CPU sync is a no-op). This no-ops it for CPU at platform import.

  • 32c42c4 #50750 — [UX] remove torch compile warning when using breakable cudagraph (#50750)
    • 作者: Jiangyun Zhu | +12/-8 | 1 个文件

    Test Result —

  • c8602c7 #50801 — [CPU] Refine CPU kernel dispatch (#50801)
    • 作者: Li, Jiang | +47/-30 | 6 个文件
    • Make the existing SGLang-derived AMX kernel for INT8 W8A8 linear (symmetric weight quant + shape-aligned) unconditional instead of gated behind the experimental VLLM_CPU_SGL_KERNEL opt-in flag. Benchmarking shows no downside to always enabling it within its existing eligibility window. oneDNN remains the fallback for every ineligible case (asymmetric quant, misaligned shapes, non-AMX x86) and is…
  • b9d1e24 #50678 — K3: Move LatentMoERunner (#50678)
    • 作者: Jee Jee Li | +4/-5 | 2 个文件

    Test Result —

  • d9dac2b #46870 — fix: remove stray duplicate from serving benchmark config (#46870)
    • 作者: Christopher Miyai | +5/-0 | 1 个文件

    Fixes #43537 Removed a copy-paste artifact from #43262 that left a bare duplicate of client_parameters in the tests array, breaking JSON parsing for the default serving benchmark config. Also added a check-json pre-commit hook scoped to .buildkite/**/*.json to catch invalid JSON in benchmark configs before they reach main. AI assistance was used (Claude). No duplicate PRs exist for this issue. Tes…

  • d83eb0b #50755 — fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755)
    • 作者: Juan Pérez de Algaba | +35/-0 | 3 个文件

    DeepStream was not registered as a GPU-requiring codec, allowing request-level activation of NVDEC decoding without startup configuration or VRAM reservation. The decode path also skipped VLLM_MAX_IMAGE_PIXELS enforcement and accepted request-controlled pool_size for the process-wide DecodePool singleton. - Register “deepstream” as a GPU codec in VIDEO_LOADER_REGISTRY so merge_kwargs strips it fro…

  • 4635cc3 #50383 — Shard the K3 Latent-MoE up-projection on large batches (#50383)
    • 作者: Jee Jee Li | +176/-72 | 2 个文件

    This PR depends on https://github.com/vllm-project/vllm/pull/50000 and should be merged after it. ### GSM8K - This PR ### PERF(input: 8K,output:1K) - TTFT | Concurrency | main median | branch median | Delta | main p99 | branch p99 | Delta | | —: | —: | —: | —: | —: | —: | —: | | 1 | 515.27 ms | 492.59 ms | −4.40% | 518.59 ms | 496.04 ms | −4.35% | | 4 | 553.51 ms | 531.34 ms | **…

🦀 Rust Frontend

  • 7743486 #50580 — [Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580)
    • 作者: Bugen Zhao | +205/-51 | 5 个文件

    Align DeepSeek V4 Flash prompt rendering with DeepSeek-V4-Flash-0731 and the current hosted API’s model-specific effort mapping. The renderer accepts the canonical 0731 levels: low emits no prefix, high emits the “Absolute maximum” prefix, and max emits the “Beyond maximum” prefix. DSML parsing and output syntax retain their existing behavior. The request normalization follows the public deepseek-…

  • 72cd542 #50868 — [Rust][Benchmark] Preserve UTF-8 across benchmark stream chunks (#50868)
    • 作者: Reid | +30/-10 | 1 个文件

    The Rust benchmark SSE handler decoded every incoming network chunk independently with String::from_utf8_lossy. When TCP or HTTP chunking split a multibyte UTF-8 character, each incomplete byte sequence was replaced with U+FFFD before the following chunk arrived. For example, splitting 中 (E4 B8 AD) after E4 corrupted the streamed text: Buffer incoming data as raw bytes and decode it only after a c…

🔩 Misc

  • 7b50d2c #50879 — [Misc] Avoid importing nixl_ep on every vllm serve config (#50879)
    • 作者: Nicolò Lucchesi | +24/-23 | 1 个文件

    Avoid importing nixl_ep all the time we boot up vllm. Instead, only do it lazily when needed (dpep configuration). Right now if you do vllm serve … import_utils will try to resolve nixl_ep, and in this case even log “unrelated” import failures as this configuration can then go on and run just fine, as no DPEP is needed (I have no libcudart 12 on this machine, but it only matters if it’s actual…

  • dd11df0 #49389 — [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389)
    • 作者: wangxiyuan | +16/-263 | 19 个文件

    Remove the deprecated calculate_kv_scales option (runtime fp8 k/v scale estimation) across config, attention, quantization, runner, and related tests/docs. fp8 KV cache scales now resolve via a single path: loaded from the checkpoint if present, otherwise defaulting to 1.0. related change: #37201 ## Test Result —

🐛 Bug Fix

  • 122b3d4 #50915 — [Bugfix][CPU] Fix macOS build: std::sqrt is not constexpr under libc++ (#50915)
    • 作者: Harjoth Khara | +2/-2 | 1 个文件

    The nightly macOS Apple Silicon Smoke Test has failed every night since 2026-07-31. Both runners — macos-15 (required) and macos-26 — fail while building: Latest failing run: https://github.com/vllm-project/vllm/actions/runs/30782456002 ### Why it happens The code asks for a compile-time constant built from std::sqrt: std::sqrt is not constexpr until C++26. GCC allows it anyway as an extension…

  • 199644d #50906 — [Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906)
    • 作者: yimdev | +115/-13 | 3 个文件

    Sparse MLA masked-MHA builds a bit-packed top-k mask whose size is approximately: The existing workspace was fixed at 64 MiB, while the routing matrix allows a single 32K prefill to enter masked-MHA. Such a request requires exactly 128 MiB. Heterogeneous batches can require even more despite staying within max_num_batched_tokens. Attempting to reshape the fixed workspace for an oversized mask can …

  • 7c40d61 #50250 — [Bugfix] Flatten >2D multimodal embeddings, not just 3D (#50250)
    • 作者: Michał Ganczarenko | +2/-2 | 1 个文件

    MultiModalMixin._split_embeddings (Transformers modeling backend, vllm/model_executor/models/transformers/multimodal.py) only flattened vision/audio encoder output to 2D when embeddings.ndim == 3. CohereLabs/aya-vision-8b’s get_image_features returns embeddings shaped [num_items, H, W, hidden] (4D — the spatial grid is never flattened by the model), so the tensor skipped the flatten step and reach…

  • 5f2ee2f #50462 — [Bugfix][Core] Log KV cache capacity after block-size resolution (#50462)
    • 作者: Tanmay Dixit | +19/-23 | 2 个文件

    Fixes #50456. KV-cache capacity was logged during worker KV-cache configuration, while the authoritative values exposed through cache_config.kv_cache_size_tokens and kv_cache_max_concurrency are published later from the resolved scheduler configuration. This PR centralizes those observability updates after scheduler KV-cache configuration generation and effective block-size resolution: - removes t…

  • 6a9fdf0 #50950 — [Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints (#50950)
    • 作者: Lazurite | +71/-2 | 2 个文件

    Multimodal sequence-classification checkpoints cannot be served: the score head is built with the wrong number of labels, so weight loading fails with AssertionError: Tried to load weights of size torch.Size([20, 2560]) to a parameter of size torch.Size([2, 2560]) Reported against vLLM in #47956 (Qwen3-VL, 20 labels) and downstream as modelscope/ms-swift#9704. ## Root cause as_seq_cls_model.init…

  • a1657a0 #51003 — [Bugfix][Build] Fix DeepGEMM CUDA 12.9 FP8 header visibility (#51003)
    • 作者: Kevin H. Luu | +2/-2 | 2 个文件

    Fix the CUDA 12.9 release build failure in DeepGEMM’s host extension while preserving the exact DeepGEMM f5a76426 code baseline. The pinned mqa_logits.cuh directly names __nv_fp8_e4m3, but it does not include the CUDA header that declares the type. CUDA 12.9 host compilation therefore fails with: This updates both synchronized vLLM DeepGEMM pins from f5a76426 to e21c821f. The new commit has f5a764…

  • adbf08d #50886 — [Bugfix][Reasoning] kimi_k3: O(delta) reasoning-end check on the decode path (#50886)
    • 作者: Bowen Wang | +157/-10 | 2 个文件

    KimiK3ReasoningParser never overrode is_reasoning_end_streaming, so it inherits the base implementation — return self.is_reasoning_end(input_ids) — which throws delta_ids away and re-derives the answer from the entire sequence. is_reasoning_end then runs two backward subsequence scans over request.all_token_ids, allocating two lists per probed position (list(haystack[i:i+n]) == list(needle)) a…

  • 74295e3 #50926 — [CI][Bugfix] Fix flaky test_store_orders_after_compute_write (#50926)
    • 作者: Nick Hill | +7/-0 | 1 个文件

    The no-barrier control phase leaves a ~800ms backlog of sleep+fill kernels in flight on its compute stream (the host loop only waits on the store-copy events). Those leftover fills race the barrier phase’s fill->copy window on the shared gpu tensor and flakily corrupt one iteration, tripping the ‘store raced compute even with the barrier’ assertion. Drain the compute stream before returning from _…

  • 41ba11b #50567 — [Bugfix][Kimi-K3] Enforce packed rows and op availability in AttnRes dispatch (#50567)
    • 作者: namgyu-youn | +11/-1 | 2 个文件

    Follow-up to #50090 and #50000 1. The kernel hardcodes row stride H, but the dispatch only checked stride(-1) == 1, not stride(0) == hidden_size. A row-padded input silently corrupts output instead of failing (max|out - ref| = 6.66 vs. the suite’ 8e-2 tolerance on a repro case). Packed rows are an intended precondition (per @gau-nernst), so this adds a STD_TORCH_CHECK to fail loudly. 2. torch.ops….

  • 6a9109d #48420 — [Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True (#48420)
    • 作者: Ryan Hamby | +57/-1 | 3 个文件

    Fixes #48383 use_audio_in_video=True can be requested by a caller that doesn’t know in advance whether a given video has an audio track. When the video has no audio, no audio items are extracted from it, but Qwen3OmniMoeThinkerProcessingInfo._call_hf_processor forwarded use_audio_in_video=True to the underlying HF Qwen3OmniMoeProcessor regardless. The upstream HF processor assumes use_audio_in_vid…

  • c2881ce #50432 — [Bugfix][Hybrid] Fix cross-block race on num_accepted in MRv2 align prefix cache (#50432)
    • 作者: Francesco Fusco | +15/-10 | 1 个文件

    PR https://github.com/vllm-project/vllm/pull/42406 introduced the support of aligned prefix cache in MRv2 by reusing the postprocess_mamba_fused_align kernels developed for MRv1. However there is a race condition that needs to be fixed. run_fused_postprocess_align (the MRv2 align-mode postprocess path) passes num_accepted_tokens_gpu as both the postprocess_mamba_fused_align kernel’s *read source…

  • 4a3447d #50417 — [Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417)
    • 作者: Raphaël Rialland | +317/-58 | 12 个文件

    Separate the target model ability to produce multimodal embeddings from the drafter ability to consume them. - Add a SupportsMultiModalEmbeddings protocol and typed capability helper. - Make SupportsMultiModal inherit the protocol. - Explicitly implement it on supported Eagle and Inkling drafters. - Centralize capability detection and fallback warnings in DraftModelSpeculator.load_model(). - Alloc…

  • c4e9f09 #50746 — [Bugfix][Frontend] Reject empty gRPC stop strings (#50746)
    • 作者: zcxGGmu | +80/-8 | 4 个文件
    • Reject empty stopping.stop_strings during gRPC request conversion. - Return InvalidArgument before invalid stop strings reach the text decoder. - Add conversion and unary gRPC regression coverage. Fixes #50725 - cargo test -p vllm-server empty_stop_string_is_rejected – –nocapture - cargo test -p vllm-server unary_generate_empty_stop_string_returns_invalid_argument – –nocapture - cargo test -…
  • 1c0d207 #50777 — [Bugfix] Default Gemma3 Model intermediate_tensors to None (#50777)
    • 作者: Taneem Ibrahim | +1/-1 | 1 个文件

    Currently Gemma3Model.forward declares intermediate_tensors as a required positional parameter: Every other pooling-capable model defaults it, e.g. BertModel: Callers that never produce intermediate tensors, for instance the pooling path, omit the argument. Binding the compiled forward then fails at engine start: This makes the model unusable for pooling. ## How it was found Surfaced by CI on #482…

  • 7f2e78b #49056 — [Bugfix] Emit a valid media type from encode_{audio,image,video}_url (#49056)
    • 作者: Vineeth Sai Varikuntla | +21/-4 | 2 个文件

    encode_audio_url, encode_image_url and encode_video_url in vllm/multimodal/utils.py build a data URL and resolve its media type with: When the format’s extension is not registered in mimetypes.types_map, the fallback is a bare top-level type (“image”, “audio”, “video”). A media type with no subtype is not well-formed, so formats missing from the table produce an invalid data URL such as data:image…

  • 952694e #49230 — [Bugfix] Validate NIXL speculative config compatibility (#49230)
    • 作者: Tzu-Ling Kan | +165/-2 | 3 个文件

    Overview: Fail the NIXL prefill/decode handshake early when speculative decoding is set up incompatibly on the two workers, instead of crashing later during inference. #### Details: ##### Problem In disaggregated serving, prefill builds the KV cache and NIXL transfers it to decode. If decode enables a model-based speculative method such as EAGLE3, but prefill does not (or uses a different dra…

  • f0de1a6 #50823 — [Bugfix] Shard UniformTypeKVCacheSpecs block table width under DCP (#50823)
    • 作者: drakosha | +42/-0 | 2 个文件

    Fixes #50825. UniformTypeKVCacheSpecs never overrode max_num_blocks_per_req, so it used the base cdiv(max_len, block_size) while AttentionSpec divides by decode_context_parallel_size. The two sides that must agree read different specs: - the runner sizes the block table from the group spec (gpu_model_runner.py, may_reinitialize_input_batch), which for DSA models is UniformTypeKVCacheSpecs — no…

  • 68ca6fd #50869 — [Bugfix] Remove bad startup assertion (#50869)
    • 作者: Benjamin Chislett | +0/-25 | 1 个文件

    Removes an invalid assertion added erroneously in https://github.com/vllm-project/vllm/pull/47419

🧪 CI/Tests

  • cb81048 #50199 — [XPU][CI]Adjust Samplers test ENV for Intel GPU (#50199)
    • 作者: xiangdong | +3/-3 | 1 个文件

    Adjust Samplers test ENV for Intel GPU ## Test Result —

  • 8adc840 #50917 — [ROCm][Test] Use BF16 for Jina v5 nano MTEB test (#50917)
    • 作者: Andreas Karatzas | +2/-0 | 1 个文件
    • Run jinaai/jina-embeddings-v5-text-nano in BF16 only on ROCm while retaining the existing automatic FP16 selection on NVIDIA. - Keep the Hugging Face FP32 baseline and long-context embedding comparison active instead of bypassing them with a cached MTEB score. The failure in AMD Buildkite build 11591 motivated this change after the newly added Jina nano case exposed long-context FP16 numerical d…
  • e279f71 #50728 — [ROCm][Test] Fix AITER MXFP4 oracle contract (#50728)
    • 作者: Andreas Karatzas | +11/-4 | 1 个文件

    PR #46298 switched this test to the GPT-OSS MXFP4 converter but left it configured for ordinary SILU, and the reference calculation reused tensors after the converter had modified their layout in place. The newer AITER version from PR #49361 exposed that inconsistent setup as an unsupported small-M kernel configuration. - Use SWIGLUOAI for the AITER_MXFP4_BF16 test case, matching the activation an…

  • 76d995d #50726 — [CI][ROCm] Export Helion benchmark script in test artifacts (#50726)
    • 作者: Andreas Karatzas | +3/-0 | 2 个文件

    PR #48968 added Helion numerical tests that import scripts/benchmark_helion_kernels.py. CUDA runs these tests from a full checkout, but ROCm CI reconstructs its workspace from a smaller exported artifact where that script was missing, so the AMD job failed during test collection. The ROCm test workspace is assembled from the export_vllm or export_vllm_wheel_release stage rather than from a full ch…

  • 8ba87e0 #46844 — [CI] Mooncake PD integration tests (#46844)
    • 作者: Nicolò Lucchesi | +413/-4 | 9 个文件

    Let’s start tracking MC functionalities e2e with proper integration tests, starting with basic PD functionality. I separated the buildkite jobs so that we can turn them on/off more easily, given the feature matrix compatibility isn’t yet at NixlConnector level, and that we can develop independently and save some CI runs.

  • ad0bf39 #50266 — [CI] KimiLinear PD in nightlies (#50266)
    • 作者: Nicolò Lucchesi | +24/-2 | 2 个文件

    Follow up to https://github.com/vllm-project/vllm/pull/49762 to cover Kimi KDA disagg on CI. moonshotai/Kimi-Linear-48B-A3B-Instruct is still a bit too big to run on each PR, so I am adding it to nightlies as a fast proxy to track K3 P/D.

  • 9acb7b3 #50839 — [CI] And PPL test for multimodal generation models (#50839)
    • 作者: wang.yuqi | +198/-0 | 4 个文件

    Following #24485 And a fast but sensitive test to test the entire pipeline of a multimodal model, including the processor, vision encoder, and language model. The dataset lmms-lab-encoder/llava-bench-in-the-wild has a total file size of 9.78 MB and contains only 60 rows. I believe it already includes enough tokens to test multimodal models. Help confirm the accuracy of #50411. Add the following co…

🖥️ Kernel

  • c810937 #49791 — [Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 (#49791)
    • 作者: xiaozhoupy | +619/-51 | 7 个文件

    CuTe DSL skinny GEMM - Add an optional compile-time K specialization to the existing CuTe DSL skinny GEMM. - On the static-K path, use a compile-time K-tile loop and prefetch the first two B tiles before the PDL wait. - Convert each loaded A/B register tile to FP32 once before FP32 accumulation. - Keep the existing dynamic-K path used by Kimi-K3 unchanged. ### GLM-5.2 mapping - Extend the exis…

  • 65cf127 #50776 — [Kernel] Skip fully masked key blocks in windowed Triton prefill (#50776)
    • 作者: Almog Tavor | +10/-1 | 1 个文件

    _fwd_kernel in triton_prefill_attention.py walks every key block from 0 to end_n and masks out-of-window keys after loading them. The masking is correct, but the work is not skipped, so a sliding-window pass still costs O(seq_len^2). For a query block covering [start_m * BLOCK_M, (start_m + 1) * BLOCK_M), only keys in [start - SLIDING_WINDOW_Q, end + SLIDING_WINDOW_K] can survive the mask, so the …

🔧 Refactor

  • 0cf49a5 #50285 — [Refactor] Remove multiple dead codes (#50285)
    • 作者: Wentao Ye | +1/-405 | 22 个文件

    Remove multiple dead codes

⚡ Performance

  • 89ac407 #50716 — [Perf] Speed up multimodal placeholder and token-match scanning (#50716)
    • 作者: Hareesh Gali | +157/-44 | 2 个文件

    Placeholder discovery and prompt-update matching scan token prompts position-by-position, allocating an O(len(match)) list slice at every position. _iter_placeholders also re-resolves content token ids at every position, and _find_matches re-scans the prompt per unmatched item per round even when items share a target. This runs per MM request even on processor-cache hits, serialized on the MM …