共 32 个 commit，涉及 152 个文件，+4165/-1983 行变动。

概要

统计项 数值
Commit 数 32
变更文件 152
新增行数 +4165
删除行数 -1983

Commit 列表

📦 Other

  • 5df9999 #50656 — [Kimi-K3] Add option to shard the shared expert instead of replicating (#50656)
    • 作者: Tyler Michael Smith | +154/-3 | 3 个文件

    Shard the shared-expert weights to avoid redundant work. In order to do so we need to re-distribute the activations, which adds an extra AllGather and ReduceScatter. This has been tested e2e to work and speed up decodes. So far, preliminary perf measurements show this is a decode win, and for prefill it’s a throughput vs kv cache size tradeoff. FYI the performance and memory footprint improvements…

  • 005fa01 #50624 — docs: document reasoning_content output removal as a breaking client change (#50624)
    • 作者: Fede Kamelhar | +2/-1 | 1 个文件

    reasoning_content was renamed to reasoning (RFC #27755, #27752) and then removed from the output entirely in #33402. The input side kept the old name alive — #42664 accepts reasoning_content and normalizes it to reasoning. This PR documents the resulting asymmetry as a breaking change for output consumers, since the current one-line warning (“To migrate, directly replace reasoning_cont…

  • 9ae11a6 #50524 — [Model] Add K-EXAONE-2.0-750B-A37B (#50524)
    • 作者: Kyungmin Lee | +161/-11 | 6 个文件

    This PR adds support for the K-EXAONE-2.0-750B-A37B, developed by LG AI Research. Hugging Face: https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B Technical Report: https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B/blob/main/assets/K-EXAONE-2.0-Technical-Report.pdf Transformers: https://github.com/nuxlear/transformers/tree/add-k-exaone2 SGLang: https://github.com/lkm2835/sglang/tree…

  • b977407 #50424 — Support quantized DSpark Markov heads (#50424)
    • 作者: Andrii Skliar | +4/-0 | 1 个文件

    Add quantization configuration support to the DSpark Markov head. DSparkMarkovHead now accepts and forwards quant_config to its ParallelLMHead-based markov_w2 projection. Qwen3DSparkModel passes its model quantization configuration when constructing the Markov head. This allows quantized markov_w2 weights, including W4A16 configurations with weight_scale_2, to load through the normal quantization …

  • ec40f6a #49960 — [CPU] Fix torch.compile crash from torch.accelerator.synchronize on CPU-only hosts (#49960)
    • 作者: Ganesh R | +13/-2 | 2 个文件

    On CPU-only hosts, torch.compile (e.g. gpt-oss) crashes during aot_compile with AssertionError: No accelerator available for torch.accelerator.synchronize. torch.accelerator.synchronize() has no accelerator on CPU, so it breaks both eager (raises) and torch.compile (Dynamo’s handle_synchronize asserts, even though a CPU sync is a no-op). This no-ops it for CPU at platform import.

  • 32c42c4 #50750 — [UX] remove torch compile warning when using breakable cudagraph (#50750)
    • 作者: Jiangyun Zhu | +12/-8 | 1 个文件

    Test Result —

  • c8602c7 #50801 — [CPU] Refine CPU kernel dispatch (#50801)
    • 作者: Li, Jiang | +47/-30 | 6 个文件
    • Make the existing SGLang-derived AMX kernel for INT8 W8A8 linear (symmetric weight quant + shape-aligned) unconditional instead of gated behind the experimental VLLM_CPU_SGL_KERNEL opt-in flag. Benchmarking shows no downside to always enabling it within its existing eligibility window. oneDNN remains the fallback for every ineligible case (asymmetric quant, misaligned shapes, non-AMX x86) and is…
  • b9d1e24 #50678 — K3: Move LatentMoERunner (#50678)
    • 作者: Jee Jee Li | +4/-5 | 2 个文件

    Test Result —

  • d9dac2b #46870 — fix: remove stray duplicate from serving benchmark config (#46870)
    • 作者: Christopher Miyai | +5/-0 | 1 个文件

    Fixes #43537 Removed a copy-paste artifact from #43262 that left a bare duplicate of client_parameters in the tests array, breaking JSON parsing for the default serving benchmark config. Also added a check-json pre-commit hook scoped to .buildkite/**/*.json to catch invalid JSON in benchmark configs before they reach main. AI assistance was used (Claude). No duplicate PRs exist for this issue. Tes…

  • d83eb0b #50755 — fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755)
    • 作者: Juan Pérez de Algaba | +35/-0 | 3 个文件

    DeepStream was not registered as a GPU-requiring codec, allowing request-level activation of NVDEC decoding without startup configuration or VRAM reservation. The decode path also skipped VLLM_MAX_IMAGE_PIXELS enforcement and accepted request-controlled pool_size for the process-wide DecodePool singleton. - Register “deepstream” as a GPU codec in VIDEO_LOADER_REGISTRY so merge_kwargs strips it fro…

  • 4635cc3 #50383 — Shard the K3 Latent-MoE up-projection on large batches (#50383)
    • 作者: Jee Jee Li | +176/-72 | 2 个文件

    This PR depends on https://github.com/vllm-project/vllm/pull/50000 and should be merged after it. ### GSM8K - This PR ### PERF(input: 8K,output:1K) - TTFT | Concurrency | main median | branch median | Delta | main p99 | branch p99 | Delta | | —: | —: | —: | —: | —: | —: | —: | | 1 | 515.27 ms | 492.59 ms | −4.40% | 518.59 ms | 496.04 ms | −4.35% | | 4 | 553.51 ms | 531.34 ms | **…

  • 0a64460 #50133 — [CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133)
    • 作者: Li, Jiang | +972/-839 | 20 个文件

    The unquantized (BF16/FP16/FP32) CPU MoE was the last unquantized backend still on the legacy monolithic path. oracle/unquantized.py short-circuited before backend selection with # TODO: migrate to MK structure, and UnquantizedFusedMoEMethod carried three CPU escape hatches (is_monolithic, a bespoke process_weights_after_loading branch, and a bespoke apply_monolithic branch) to keep it alive. Ever…

  • e481da9 #49664 — [XPU] [Linear] add torch as xpu linear backend (#49664)
    • 作者: zofia | +15/-8 | 5 个文件

    Usage: vllm serve –model deepseek-ai/DeepSeek-R1-Distill-Llama-8B –enforce-eager –port 8000 –trust-remote-code –no-enable-prefix-caching –quantization fp8 –linear-backend torch

  • 2755489 #50807 — [INC] fix w4a4 model (#50807)
    • 作者: Qiming Zhang | +4/-3 | 2 个文件

    INCMxfp4LinearMethod called init_mxfp4_linear_kernel() without an activation_quant_key, leaving it as None. This caused platform-specific failures: XPU (hard crash): XPUMxFp4LinearKernel.can_implement strictly requires activation_quant_key == kMxfp4Dynamic and rejects None, causing model loading to fail with: CUDA (silent regression): MarlinMxFp4LinearKernel.can_implement accepts None as a…

  • 5e35a6f #50547 — cpu_model_runner.py: skip the warm up if CompilationMode.NONE (#50547)
    • 作者: YAMAMOTO Takashi | +6/-1 | 1 个文件

    for cpu backend, there seems to be little reason to perform warm up unless compiling. speed up tests. ## Test Result this reduced the wall time of cached runs of examples/disaggregated/example_connector/run.sh on my cpu-only environment from 85 seconds to 68 seconds. —

  • 9a4fd57 #50688 — [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) (#50688)
    • 作者: Omkar Shewale | +155/-91 | 6 个文件

    Adds support for jinaai/jina-embeddings-v5-text-nano. The jina-embeddings-v5 family uses two different backbones under the same JinaEmbeddingsV5Model architecture: - jina-embeddings-v5-text-small uses a Qwen3 decoder backbone. - jina-embeddings-v5-text-nano uses a bidirectional EuroBERT encoder backbone with is_decoder=false. This PR dispatches based on is_decoder: - -small retains the existing Qw…

  • 0055b8b #50032 — [Attention][MiniMax-M3] Add MSA speculative decode verification (#50032)
    • 作者: Jason Li | +1275/-59 | 15 个文件

    Add an opt-in MSA/CUTLASS sparse decode path for MiniMax M3 speculative verification on SM100 and SM103 GPUs: MSA planning and metadata remain NVIDIA-backend-specific. Unsupported platforms, KV formats, and graph shapes continue through the existing Triton decode path without constructing an MSA plan. The CUTLASS path is limited to FP8 E4M3 KV cache, page size 128, top-k 16, query lengths 2-32, at…

  • 55c98e3 #50661 — [Model Runner v2] Enable BGE M3 pooling embed token_classify (#50661)
    • 作者: Taneem Ibrahim | +29/-9 | 2 个文件

    Enable BGE-M3’s combined embed&token_classify pooling task on Model Runner V2 for encoder-only models. This is the fourth step in the MRV2 pooling series. Once this PR lands, encoder-only pooling on MRV2 is complete for every in-tree pooling task: | Task | Status | | — | — | | embed, classify | #48791 (merged) | | token_classify | #50293 (merged) | | token_embed | #50574 (merged) | | embed…

  • 96add73 #50641 — [Elastic EP] Fix non-contiguous weight transfers (#50641)
    • 作者: Itay Alroy | +11/-7 | 1 个文件

    Elastic EP weight transfer passed each weight tensor directly to PyNccl send API, but PyNccl transfers each tensor as a flat contiguous memory and does not account for tensor strides. For non-contiguous tensor views, such as some DeepSeek-V3 weights, this caused new workers to receive incorrect weights during scale-up. Instead, pack non-contiguous weights into a temporary contiguous buffer before …

🔩 Misc

  • dd11df0 #49389 — [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389)
    • 作者: wangxiyuan | +16/-263 | 19 个文件

    Remove the deprecated calculate_kv_scales option (runtime fp8 k/v scale estimation) across config, attention, quantization, runner, and related tests/docs. fp8 KV cache scales now resolve via a single path: loaded from the checkpoint if present, otherwise defaulting to 1.0. related change: #37201 ## Test Result —

🔧 Refactor

  • 0cf49a5 #50285 — [Refactor] Remove multiple dead codes (#50285)
    • 作者: Wentao Ye | +1/-405 | 22 个文件

    Remove multiple dead codes

🐛 Bug Fix

  • 68ca6fd #50869 — [Bugfix] Remove bad startup assertion (#50869)
    • 作者: Benjamin Chislett | +0/-25 | 1 个文件

    Removes an invalid assertion added erroneously in https://github.com/vllm-project/vllm/pull/47419

  • f5bb701 #50764 — [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764)
    • 作者: Omkar Shewale | +49/-0 | 2 个文件

    test_openapi_stateless[POST /v1/messages] fails intermittently with a 500. Seen on an unrelated PR in build 81852: AnthropicMessagesRequest.cache_salt (added in #49498) declares no length constraint, so its generated OpenAPI schema advertises "” as a valid value. Schemathesis runs in positive-data mode and generates requests from that schema, so it eventually emits cache_salt: “”. The handler pass…

  • 5c4fe4b #49069 — [Bugfix][KV Connector] Propagate EAGLE state across merged Mooncake store groups (#49069)
    • 作者: Yifan Qiao | +118/-39 | 2 个文件
    • propagate EAGLE/MTP handling to every Mooncake store cache group merged under the same KV cache spec - reuse the core coordinator’s SpecGroup representation so external-store lookup and save masks follow the same grouping semantics - add a DeepSeek-V4-shaped SWA regression test covering the store-to-lookup round trip ## Root cause DeepSeek V4 annotates only the cache group containing the MTP lay…
  • e42c230 #50766 — [Bugfix] serving_llama70B_tp4 benchmark was silently running at tensor_parallel_size=1 (#50766)
    • 作者: Turner Jabbour | +1/-0 | 1 个文件

    While building an AMD-specific variant of the performance-benchmark test configs (serving-tests-rocm.json, mirroring serving-tests.json), verifying the copied serving_llama70B_tp4_random_128_128 entry through the actual merge_serving_tests_stream merge logic in run-performance-benchmarks.sh surfaced that its server_parameters never sets tensor_parallel_size. It silently inherits tensor_parallel_si…

  • c666810 #50761 — [ROCm][Bugfix][Kimi-K3] Preserve MoE correction bias in FP32 (#50761)
    • 作者: Fangzhou Ai | +4/-1 | 1 个文件

    Kimi-K3 checkpoints store e_score_correction_bias in FP32 and the AMD router already emits FP32 logits. The AMD model currently constructs the correction-bias parameter under the ambient BF16 model dtype, so the standard weight loader copies the FP32 checkpoint into BF16 storage. A later AITER runtime upcast cannot restore the discarded low bits. Construct the small correction-bias parameter in FP…

🧪 CI/Tests

  • 8ba87e0 #46844 — [CI] Mooncake PD integration tests (#46844)
    • 作者: Nicolò Lucchesi | +413/-4 | 9 个文件

    Let’s start tracking MC functionalities e2e with proper integration tests, starting with basic PD functionality. I separated the buildkite jobs so that we can turn them on/off more easily, given the feature matrix compatibility isn’t yet at NixlConnector level, and that we can develop independently and save some CI runs.

  • ad0bf39 #50266 — [CI] KimiLinear PD in nightlies (#50266)
    • 作者: Nicolò Lucchesi | +24/-2 | 2 个文件

    Follow up to https://github.com/vllm-project/vllm/pull/49762 to cover Kimi KDA disagg on CI. moonshotai/Kimi-Linear-48B-A3B-Instruct is still a bit too big to run on each PR, so I am adding it to nightlies as a fast proxy to track K3 P/D.

  • 9acb7b3 #50839 — [CI] And PPL test for multimodal generation models (#50839)
    • 作者: wang.yuqi | +198/-0 | 4 个文件

    Following #24485 And a fast but sensitive test to test the entire pipeline of a multimodal model, including the processor, vision encoder, and language model. The dataset lmms-lab-encoder/llava-bench-in-the-wild has a total file size of 9.78 MB and contains only 60 rows. I believe it already includes enough tokens to test multimodal models. Help confirm the accuracy of #50411. Add the following co…

  • 0033211 #44972 — [Test][V1] Add sleep/wake correctness regression test for hybrid GDN/… (#44972)
    • 作者: chun-wan | +100/-0 | 2 个文件

    Hybrid Mamba/gated-delta-net (GDN) models (e.g. Qwen3-Next) keep a persisted conv + recurrent state cache. With sleep mode (RLHF reuse: sleep -> weight update -> wake_up), the state-cache tag is discarded on sleep and its device memory is re-created on wake. If a new sequence consumes that slot before it is reset, the gated-delta-rule kernel propagates the (non-zeroed) memory; when it holds NaN/in…

⚡ Performance

  • 89ac407 #50716 — [Perf] Speed up multimodal placeholder and token-match scanning (#50716)
    • 作者: Hareesh Gali | +157/-44 | 2 个文件

    Placeholder discovery and prompt-update matching scan token prompts position-by-position, allocating an O(len(match)) list slice at every position. _iter_placeholders also re-resolves content token ids at every position, and _find_matches re-scans the prompt per unmatched item per round even when items share a target. This runs per MM request even on processor-cache hits, serialized on the MM …

🦀 Rust Frontend