共 57 个 commit，涉及 208 个文件，+13754/-3940 行变动。

概要

统计项 数值
Commit 数 57
变更文件 208
新增行数 +13754
删除行数 -3940

Commit 列表

📦 Other

  • bf2a393 #46053 — Temporarily remove @markmc from CODEOWNERS (#46053)
    • 作者: Mark McLoughlin | +0/-10 | 1 个文件

    Removing myself temporarily while I’m on leave. For reference, much of the current observability review backlog is tracked by: - https://github.com/orgs/vllm-project/projects/44 - Prometheus metrics - https://github.com/orgs/vllm-project/projects/55 - OpenTelemetry tracing

  • d682968 #45990 — [Model] Remove BambaForCausalLM (#45990)
    • 作者: Tiezhen WANG | +6/-529 | 5 个文件
    • Remove BambaForCausalLM (IBM Bamba-9B, hybrid Mamba-2 + attention model) - The Bamba architecture served as the basis for nemotron_h.py (NemotronHForCausalLM) which remains fully supported and is the actively maintained successor - Add BambaForCausalLM to _PREVIOUSLY_SUPPORTED_MODELS pointing to vLLM v0.23.0 ## Files changed - vllm/model_executor/models/bamba.py — deleted - vllm/model_executor/m…
  • 021cdf7 #43179 — Fix _riscv_supports_rvv_vlen128() to detect RVV on hardware without zvl flags (#43179)
    • 作者: lyd1992 | +30/-3 | 3 个文件

    Fix _riscv_supports_rvv_vlen128() so that RISC-V hardware without zvlb flags in /proc/cpuinfo can still correctly detect RVV support. On some RISC-V hardware (e.g. SG2044), /proc/cpuinfo advertises the V extension (rv64imafdcv) and zve* / zvfh flags but does not report any zvlb VLEN hint. The existing Python-only check required zvl128b to be present: This caused _get_attn_isa() to fall b…

  • 4cb5e74 #44801 — [Rust Frontend]: Add /get_world_size route with static parallel size (#44801)
    • 作者: Ashar | +258/-2 | 11 个文件

    Add parallel config for world size in EngineCoreReadyResponse that is captured and persisted during startup and served directly. This makes the implementation fast but there are cases such as Elastic DP where ranks can change after startup. For those cases. We have two options: - Use a utility function that provides current rank each time a request arrives. - Add a new Output Stream message like c…

  • afdcbd5 #45681 — [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
    • 作者: Tuukka Sarvi | +545/-52 | 8 个文件

    This PR fixes DeepSeek V4 / DeepSeek V4 Flash functional issues on ROCm gfx942 (MI300X), while preserving the gfx950 path. The main fixes are: - Avoid unsupported FP8 arithmetic when converting UE8M0 block scales. - Make the fused DeepSeek V4 qnorm/RoPE/KV-insert path emit the correct FP8 bytes on ROCm: - gfx942 uses FNUZ encoding. - gfx950 uses OCP encoding. - FNUZ uses max value 224.0. - Propaga…

  • 8d4f549 #42727 — fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config (#42727)
    • 作者: AlexHuang | +579/-428 | 20 个文件

    This PR fixes the AWQ dequantize issue on Intel XPU (#41469) and refactors the AutoAWQ config for better XPU performance support. The refactoring approach is inspired by #38288 (consolidate GPTQ). Key changes: 1. Fix AWQ dequantize on XPU: Added AutoAWQXPULinearMethod using Intel’s oneDNN int4 GEMM kernel for proper XPU support 2. Unified AutoAWQConfig: Merged awq_marlin.py and awq.py into…

  • 351c72d #44991 — [CPU] Skip Triton kernel monkey-patches when Triton-CPU is available (#44991)
    • 作者: Jonathan Mamou | +26/-6 | 3 个文件

    When Triton-CPU is installed and active (the default in x86 Docker images since 0.22.0), the @triton.jit kernels in rejection_sampler.py, block_table.py, and llm_base_proposer.py can run natively on CPU — but they fail to compile due to int32/int64 type mismatches that Triton-CPU’s stricter compiler catches. This PR: 1. Fixes the type mismatches in 5 Triton kernels by replacing 0 (int32) with …

  • 7299e65 #45950 — [Rust Frontend] Return model metadata fields in /v1/models (#45950)
    • 作者: Tahsin Tunan | +153/-26 | 8 个文件

    The Rust frontend’s GET /v1/models returned only id, object, created, and owned_by, dropping the vLLM extension fields the Python frontend includes on each model card. Most visibly, it omitted max_model_len. This brings the Rust /v1/models response into parity with the Python frontend so it stays a drop-in replacement. This was an unlisted gap (not on the #44280 roadmap); /v1/models itself was alr…

  • 0898535 #45093 — Fix Stale Encoder Cache After Weight Update (#45093)
    • 作者: littlecircle0730 | +12/-0 | 2 个文件

    Fixes #44910. After a native weight update (update_weights() / finish_weight_update()), the V1 multimodal encoder cache could continue serving stale encoder outputs. Encoder cache entries are keyed only by mm_hash. As a result, repeated multimodal inputs could still hit existing cache entries and reuse encoder outputs generated before the weight update. The cache remained stale until reset_encoder…

  • 5fd3b27 #45444 — [Mooncake] Skip KV lookup for non-reachable SWA blocks (#45444)
    • 作者: Wei Zhao | +120/-18 | 4 个文件

    This PR adds some optimizations for reducing overhead in Mooncake KV offloading. - In lookup, skip SWA blocks that are not eligible to be considered cache hit, using kv cache group’s reachable_block_mask. - Use None as all-True mask for store_mask, saving list construction overhead for full cache: masks.append([True] * num_chunks if mask is None else mask). ## Performance benchmark: DeepSeek v4 TP…

  • 1e9f04d #44602 — fix(anthropic): preserve inline system message position for prefix caching (#44602)
    • 作者: MrFan | +103/-28 | 2 个文件

    Problem PR #44283 merged all inline role: system messages from the messages array into a single leading system message. This changes the conversation prefix, breaking KV-cache hits in multi-turn dialogues. #44048 (currently open) moves the same merge logic to the protocol layer but retains the same prefix-breaking behavior. ### Example of the problem ## Fix - Remove inline system message extrac…

  • a331589 #40287 — [XPU] Update nixl to v0.10.1 in Dockerfile (#40287)
    • 作者: liuzhenwei | +6/-5 | 2 个文件
    • Update nixl to v0.10.1 in Dockerfile - Add validation script of PD disaggregation into ci ## Test Result —
  • 421c1ec #45905 — [KV Offloading] Remove dummy worker-side stats from OffloadingConnector (#45905)
    • 作者: AlexHuang | +0/-21 | 2 个文件

    Remove the dummy OffloadingConnectorStats() return in the worker branch of get_kv_connector_stats(). This was a temporary workaround (marked TODO(orozery): Remove once PR #43877 lands). PR #43877 decoupled scheduler-side stats collection from worker-side stats existence, making this workaround unnecessary. - vllm/distributed/kv_transfer/kv_connector/v1/offloading_connector.py: Remove 5 lines (TODO…

  • d57888e #39726 — [SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)
    • 作者: Jonathan Chen | +251/-9 | 3 个文件

    This pull request adds reset_cache() functionality by clearing scheduler side state + resetting all prefix cache During Scheduler.reset_prefix_cache() it runs kv_cache_manager.reset_prefix_cache() which will fail for in-flight DMA transfers as there are still ref_cnt > 0 blocks in the pool. This will need reset_prefix_cache to run twice in that case. These changes purposely are done this way to al…

  • ed938ad #45757 — [CPUOffloading] Guard CPU eviction check (#45757)
    • 作者: Varun Sundar Rabindranath | +114/-0 | 2 个文件

    Eviction can be expensive when the CPU cache is big and full. The eviction logic walks all the blocks to check if there is a candidate. The PR adds logic to track the idle cache blocks based on the load and store requests and escapes the check if the eviction is guaranteed to fail. ## Comparison with main Legend: “deferred” - requests waiting on lookup “capacity” - requests waiting due to max_num_…

  • 731fb33 #45876 — [Rust Frontend] Validate tokenized bad_words vocabulary range (#45876)
    • 作者: Reid | +61/-0 | 2 个文件

    Rust sampling parameter lowering. This matches Python behavior and rejects out-of-vocab bad word token IDs before they are forwarded to engine-core. Fix a Rust frontend validation gap for bad_words. The Rust lowering path tokenizes bad_words into _bad_words_token_ids, but previously did not validate those token IDs before forwarding sampling params to engine-core. Python validates the tokenized ba…

  • 8dd8b6e #43958 — [XPU] Fix FP8 block-scaled scheme selection on non-CUDA platforms (#43958)
    • 作者: Yejing Lai | +2/-1 | 2 个文件

    Fix test_compressed_tensors.py::test_compressed_tensors_fp8_block_enabled to meet ValueError: XPUFP8ScaledMM only support per-channel and per-tensor quantization. Root cause: get_device_capability() will return None on xpu. The _check_scheme_supported helper previously returned False in this case, which caused CompressedTensorsW8A16Fp8 to be selected instead of CompressedTensorsW8A8Fp8. Fixed: ret…

  • 1797576 #45309 — Revert “[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture” (#45309) (#45972)
    • 作者: Woosuk Kwon | +30/-37 | 1 个文件

    The main point that should break is indexer We can expand the cuda graph range by having wq_b_kv_insert and compressor_aux_stream, so that we can get perf improvement export MODEL=“deepseek-ai/DeepSeek-V4-Pro” vllm serve deepseek-ai/DeepSeek-V4-Pro –trust-remote-code –kv-cache-dtype fp8 –block-size 256 –enable-expert-parallel -dp 8 –compilation-config ‘{“cudagraph_mode”:“FULL_AND_PIECEWISE”,"…

  • 5fd21eb #45849 — [BUG] fix hidden states nan for hybrid attention models (#45849)
    • 作者: shanjiaz | +9/-0 | 2 个文件

    Fix ExampleHiddenStatesConnector returning NaN hidden states on hybrid attention models (e.g., Qwen3.5 mixing linear_attention and full_attention layers). Resolves #45734 Fixed three main issues: 1. _hs_group_idx was hardcoded to 0. For hybrid models with multiple KV cache groups, the hidden-states group is not always group 0. This caused request_finished_all_groups to pass the wrong group’s block…

  • 2a47a9f #45309 — [DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309)
    • 作者: Wentao Ye | +37/-30 | 1 个文件

    The main point that should break is indexer We can expand the cuda graph range by having wq_b_kv_insert and compressor_aux_stream, so that we can get perf improvement export MODEL=“deepseek-ai/DeepSeek-V4-Pro” vllm serve deepseek-ai/DeepSeek-V4-Pro –trust-remote-code –kv-cache-dtype fp8 –block-size 256 –enable-expert-parallel -dp 8 –compilation-config ‘{“cudagraph_mode”:“FULL_AND_PIECEWISE”,"…

  • 9c7c74b #45857 — [Log] Update deepgemm log (#45857)
    • 作者: Wentao Ye | +11/-6 | 3 个文件

    After https://github.com/vllm-project/vllm/pull/37980 landed, we already has deepgemm installed automatically, some logs are out-of-date and should be updated.

  • 0a7bacd #45863 — [DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863)
    • 作者: Wentao Ye | +184/-18 | 3 个文件

    Part of https://github.com/vllm-project/vllm/issues/45861 We rebuild the same metadata for every layer, this can be easily optimized using a small cache vllm serve deepseek-ai/DeepSeek-V4-Pro –trust-remote-code –kv-cache-dtype fp8 –block-size 256 –enable-expert-parallel -dp 8 –compilation-config ‘{“cudagraph_mode”:“FULL_AND_PIECEWISE”, “custom_ops”:[“all”]}’ –attention_config.use_fp4_indexer…

  • 8b2b566 #43853 — Feature: Enable Flashinfer non-gated MoE bf16 (#43853)
    • 作者: amirkl94 | +30/-8 | 3 个文件

    Add support for flashinfer non-gated MoE bf16. Note that this PR makes flashinfer-trtllm backend the new default for non-gated MoE bf16 models for sm100. Shows perf gain of ~15% e2e for below benchmark. ## Test Result MODEL=nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 on 1 gb200 node. flashinfer: serve cmd: result: gsm8k: Verified for tp4 as well. triton: serve cmd: result: gsm8k:

  • 0b131b1 #44626 — [ROCm][AITER][Quark] Tag per-channel FP8 weights as PER_CHANNEL so AITER pre-shuffled GEMM is selected (#44626)
    • 作者: xaguilar-amd | +7/-3 | 1 个文件

    On ROCm with AITER enabled, FP8-quantized attention projections produced by Quark silently fall back to the generic torch.scaled_mm (hipBLASLt/Tensile CijkF8BS) path instead of AITER’s faster pre-shuffled GEMM (gemm_a8w8_bpreshuffle). The root cause is a bug in vLLM’s Quark integration: a per-output-channel FP8 weight scale (one fp32 value per weight row, length N) was tagged as GroupSha…

  • bcb518a #40601 — [quant][autoround]Refactor INC quantization into package with INCScheme orchestrator (#40601)
    • 作者: Yi Liu | +1940/-803 | 13 个文件

    Follow the compressed-tensor style by replacing the monolithic inc.py with an inc/ package that uses scheme-based dispatch, paving the way for adding more schemes in the near future. Part of https://github.com/vllm-project/vllm/issues/37979 ### Local test models - Qwen3-30B-A3B-Instruct-2507-W4A16 - Qwen3-32B-W4A16 - Qwen3-Next-80B-A3B-Instruct-W4A16 cc @hshen14 @thuang6

  • 06e1e08 #44469 — [XPU] Fix test_logprobs_e2e import error: pin lm-eval[api]>=0.4.12 (#44469)
    • 作者: Chaojun Zhang | +6/-9 | 3 个文件

    Fix ImportError in test_logprobs_e2e caused by an incompatible lm-eval version on XPU CI. - requirements/test/xpu.in: pin lm_eval[api]>=0.4.12 - .buildkite/intel_jobs/misc_intel.yaml: install lm_eval[api]>=0.4.12 before running test_logprobs_e2e.py on XPU CI

  • 1a59078 #45654 — [CI/Build] Avoid duplicate ViT CG test introduced by accident (#45654)
    • 作者: Isotr0py | +8/-2 | 1 个文件
    • Currently, ViT CG tests run on both Multi-Modal Models (Standard) 2: qwen3 + gemma and Multi-Modal Models (Standard) 4: other + whisper, which waste CI resource. - This PR remove the ones in qwen+gemma suite. ## Test Result —
  • e28e8c8 #45854 — [ROCm][Quant] Minimax-M3: Enable fp8_per_channel for bf16 weights on mi300x (#45854)
    • 作者: Hongxia Yang | +11/-0 | 5 个文件

    Improve the perf of Minimax-M3 bf16 model on MI300x (gfx942)/ The fp8 w8a8 MoE quant config dropped the SwiGLU-OAI alpha/beta that models such as MiniMax-M3 pass to FusedMoE (swiglu_alpha=1.702, swiglu_beta=1.0). Only swiglu_limit was forwarded, so the silu_and_mul_with_clamp kernel ran with its default alpha=1.0/beta=0.0 and produced garbage (gsm8k 0.00) on both the serialized (Fp8MoEMethod) and …

  • ee0fd69 #45279 — docs, kv_offloading: add docs for selective offload (#45279)
    • 作者: Angelo Ruocco | +23/-0 | 1 个文件

    This commit adds explaination and guidance on how to use the selective_offload knob in kv_transfer_params introduced with commit 864990e8d Add documentation None ## Test Result None cc @kfirtoledo @orozery — - [] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”.

  • 3d20275 #45908 — fix(security): enforce audio decode duration limit in chat completions path (#45908)
    • 作者: Juan Pérez de Algaba | +11/-2 | 1 个文件

    AudioMediaIO.load_bytes() and load_file() called load_audio() without max_duration_s, bypassing the decompression-bomb guard for audio loaded via /v1/chat/completions (input_audio content parts). Wire VLLM_MAX_AUDIO_DECODE_DURATION_S into both methods to enforce the same limit that already protects the /v1/audio/transcriptions path.

  • 6d8fff5 #45595 — [KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
    • 作者: Itay Etelis | +49/-81 | 6 个文件

    When all tracked requests finish, OffloadingConnectorScheduler.build_connector_meta flushed every in-flight job through a blocking worker.wait() (added in #42611). That wait runs on the engine-core thread, so a request arriving while it runs cannot be scheduled until the offload transfers complete. This replaces the blocking idle-flush with a non-blocking drain: the engine keeps stepping while the…

  • e2c5857 #45805 — [Rust Frontend] Support hybrid/external DP LB in Python supervised bootstrap (#45805)
    • 作者: Bugen Zhao | +159/-6 | 8 个文件

    Signed-off-by: Bugen Zhao i@bugenzhao.com Add Python-supervised Rust frontend support for DP LB modes where each Rust frontend owns an explicit engine rank range. This covers external DP LB with one frontend per DP rank and hybrid DP LB with one frontend per node managing multiple local DP ranks. Rust managed-engine mode continues to require a remote-capable DP coordinator service before it can …

🖥️ Kernel

  • 22cc891 #46006 — [Kernel] Add PDL support for DeepGEMM kernel (#46006)
    • 作者: Jee Jee Li | +87/-23 | 3 个文件

    Test Result —

  • e945169 #45999 — Revert “[Kernel] Add PDL support for DeepGEMM kernel” (#45999)
    • 作者: Micah Williamson | +20/-62 | 3 个文件

    Reverts vllm-project/vllm#42996 This broke AMD build

  • 4403af8 #42996 — [Kernel] Add PDL support for DeepGEMM kernel (#42996)
    • 作者: Jee Jee Li | +62/-20 | 3 个文件

    Test Result —

  • 46f74e1 #34432 — [Kernel][Helion][1/N] Add Helion kernel for rms_norm_dynamic_per_token_quant (#34432)
    • 作者: Xiaohong (Sean) Chen | +6750/-0 | 4 个文件

    This PR is to add Helion kernel for operation. It follows the implementation from the vllm c version. This is a subtask for https://github.com/vllm-project/vllm/issues/32962. Autotuning and benchmarking in H200 and B200 were contributed by @gmagogsfm. ### Kernel level benchmark Environment Python: 3.12.12 Pytorch: 2.11.0 Cuda: 13.0 Helion: 1.0.0 Benchmark Setup Latency measure: Baseline: a…

🐛 Bug Fix

  • 7022141 #44725 — [Bugfix][Frontend] Fix Anthropic count_tokens decorator order driving server load negative (#44725)
    • 作者: Ting SUN | +1/-1 | 1 个文件

    The Anthropic /v1/messages/count_tokens route nests its decorators in the reverse order of every other load-aware route: On a client disconnect this double-counts the load decrement: - load_aware_call increments server_load_metrics on entry. - listen_for_disconnect sees http.disconnect and decrements it once. - with_cancellation cancels the handler and returns None (it does not raise). - The outer…

  • f428718 #45823 — [Fix][KV offload] Defer on_request_finished until in-flight transfers drain (#45823)
    • 作者: Ronen Schaffer | +174/-7 | 4 个文件

    Currently submit_store() can be called on a SecondaryTierManager after its on_request_finished() was triggered for the same request: the connector calls on_request_finished() eagerly while GPU→primary stores are still in flight, and their later completion drives complete_store() → submit_store() cascade to the secondary tiers. This PR fixes that. The connector now defers on_request_finished() …

  • b409217 #45448 — [Bugfix] Complete one-shot fused all-reduce PDL at end to avoid NaN (#45448)
    • 作者: Ace Eldeib | +11/-1 | 1 个文件

    The FlashInfer one-shot Lamport all-reduce signals programmatic-dependent- launch (PDL) completion before committing its output buffer when trigger_completion_at_end=False, so the next PDL-launched kernel reads the uninitialized buffer and produces NaN. The fused AR+RMSNorm pass set the flag to num_tokens <= PDL_ADVANCE_LAUNCH_TOKENS (#43103) – exactly the batch=1 / spec-decode shapes, where the …

  • ebbb2d5 #45941 — [CI/Build][Bugfix] Fix SD LoRA (#45941)
    • 作者: Jee Jee Li | +8/-0 | 1 个文件

    FIX https://buildkite.com/vllm/ci/builds/72627/list?jid=019ed455-89ce-479e-9715-9e3db365ce64&tab=output ## Test Result —

  • 0d339cf #45879 — [Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879)
    • 作者: Bryan Shan | +141/-15 | 2 个文件

    Fixes #45330. Supersedes #45337 (same root cause, stale with merge conflicts). _validate_remote_agent_handshake asserts that SPLIT-region block_len scales linearly with tp_ratio between prefill and decode workers. But block_len is proportional to per-rank KV heads (max(1, total_kv_heads // tp_size)), not TP size. When tp_size > total_kv_heads, GQA replication caps per-rank heads at 1 on both s…

  • 9d4b87f #45196 — [Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors (#45196)
    • 作者: Ting SUN | +108/-7 | 5 个文件

    DefaultModelLoader / LoadConfig accept several invalid configs silently or fail late with confusing internal errors. This validates them at construction. Three small fixes in the same loader (bundled to avoid near-duplicate PRs; happy to split): 1. Config validation — type safetensors_load_strategy as Literal[“lazy”, “eager”, “prefetch”, “torchao”] | None (a typo used to silently fall back to …

  • 58b2e89 #45867 — [Bugfix][Gemma4] Render reasoning on assistant turns without tool_calls (#45867)
    • 作者: Luciano Martins | +2/-2 | 1 个文件

    Fixes a template regression introduced by #45553: the thinking channel guard and message.get(’tool_calls’) silently dropped reasoning content on assistant messages without tool calls — e.g. the final answer after a tool chain where the model reasons about the tool result before responding. Fix: remove the tool_calls guard. Reasoning is now rendered whenever thinking_text and thinking_gate are both…

  • 091386a #45794 — [Bugfix] MiniMax-M3 (AMD): add packed_modules_mapping and pass swiglu… (#45794)
    • 作者: wangjiaxin99 | +0/-0 | 0 个文件

    Running MiniMax-M3 on AMD with Quark MXFP4 quantization currently fails at weight loading with: AssertionError … param_data.shape != loaded_weight.shape The root cause is that the fused qkv_proj / gate_up_proj modules are not declared as packed modules, so the loader tries to copy a full fused-weight tensor into a single shard’s parameter, producing a shape mismatch. This PR fixes the loading pa…

  • 5e27b2b #45917 — [Bugfix] Pass TP group to FlashInfer all-reduce fusion (#45917)
    • 作者: danisereb | +2/-8 | 2 个文件

    Symptom Serving Nemotron-3-Ultra-NVFP4 with –tensor-parallel-size > 1 and –data-parallel-size > 1 hangs at startup during CUDA-graph capture / warmup: workers freeze and EngineCore repeatedly logs No available shared memory broadcast block found in 60 seconds. The server never becomes ready. Bug reproduced on both H100 and GB200. Using TP>1/DP=1, TP=1/DP>1, and TP=1/DP=1 works (with expert p…

  • eb0fdeb #45831 — [Bugfix][PD] Fix DSV4 disaggregated serving (#45831)
    • 作者: zhanqiuhu | +4/-1 | 1 个文件

    Bug Recent changes broke DSV4 PD disagg. The is_mla_region check in base_worker.py only recognizes MLAAttentionSpec, but DSV4’s compressor layers are SlidingWindowMLASpec. ### Fix Include SlidingWindowMLASpec in the isinstance check so these regions are treated as MLA region. ### Test ### - DSV4 Flash 4P4D on 8xH100 starts and serves requests after this fix. - PR #42310 adds test cases for DSV…

  • d537122 #45782 — [ROCm][Bugfix]: Fallback GFX942 sparse MLA ops to Triton (#45782)
    • 作者: vllmellm | +15/-34 | 1 个文件

    This PR resolves severe generation accuracy degradation, numerical instability occurring in sparse MLA-based models on ROCm platforms. During our internal evaluations on the main branch, we identified a critical regression: - GLM-5.1-FP8 suffered from a massive accuracy collapse (dropping to ~55% on GSM8K). - DeepSeek-V3.2 experienced a slight but noticeable accuracy degradation (~3% drop on GSM8K…

  • f694d43 #45913 — [Bugfix][test] Use Salesforce/wikitext for ppl tests (#45913)
    • 作者: wentian-byte | +1/-1 | 1 个文件

    datasets >= 5.0 + huggingface_hub >= 1.19 no longer accept the legacy single-segment id ‘wikitext’ (HfUriError: ‘Repository id must be namespace/name’). The dataset moved to ‘Salesforce/wikitext’. Without this fix, all PPL tests in tests/models/language/generation_ppl_test/ fail at load_dataset() before any PPL is computed: huggingface_hub.errors.HfUriError: Invalid HF URI ‘hf://datasets/wikitext@…

  • 3c6084b #45852 — [Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> (fixes #45834) (#45852)
    • 作者: Nikhilesh Chhetri | +270/-0 | 6 个文件

    Fixes #45834. After #45553 the Gemma4 tool chat template inserts <|channel>thought\n into the generation prompt when continuing a turn after a tool response with enable_thinking=True. The prompt token IDs therefore end inside an open reasoning channel (a <|channel> start with no matching <channel|> close). Gemma4Parser’s engine, however, always starts in ParserState.CONTENT. As a result the fi…

  • 68ff30d #42332 — [Bugfix] Fixes MiniCPM-O resampler device placement to avoid tensor device mismatch (#42332)
    • 作者: Joel Smith | +3/-1 | 1 个文件

    Fixes #42322. MiniCPM-O 2.6 crashes on startup with Expected all tensors to be on the same device. This is caused by Resampler2_5._set_2d_pos_cache creating the pos_embed buffer on CPU during init (found in minicpmv.py). Typically, it would be moved to GPU via MiniCPMVBaseModel._ensure_resampler_device which is generally called during the load_weights of MiniCPMVBaseModel. MiniCPMOBaseModel overri…

🧪 CI/Tests

  • 554352a #45679 — [Test][KV Connector] Add request_finished fence population tests for offloading scheduler (#45679)
    • 作者: AlexHuang | +302/-9 | 2 个文件

    Add tests covering request_finished() fence index (_block_id_to_pending_jobs) population and cleanup in OffloadingConnectorScheduler. Adapted for non-blocking drain (#45595) — tests now use block-reuse fence flush instead of all-finished flush. Related to #42761 (KeyError in _block_id_to_pending_jobs). While that bug was fixed by #42959, this PR adds regression tests for the fence mechanism. 3 new…

  • 2959a92 #44650 — [XPU][CI] add model runner v2 into CI (#44650)
    • 作者: liuzhenwei | +62/-2 | 2 个文件

    add model runner v2 into intel CI ## Test Result —

🔧 Refactor

  • b4c80ec #44681 — [Refactor] Remove dead cutlass mxfp8 code (#44681)
    • 作者: Wentao Ye | +0/-1369 | 10 个文件

    This is not used at all in production since #34448 CC @mgoin

  • 2659f60 #45454 — [Refactor] Remove dead quantization code and tests (#45454)
    • 作者: Wentao Ye | +0/-120 | 5 个文件

    Remove dead quantization code and tests

⚡ Performance

  • e1a5fc4 #45826 — [Rust Frontend][Perf] O(n) argument scan in tool parser (#45826)
    • 作者: Bugen Zhao | +408/-72 | 8 个文件

    Previously in many tool parsers, when we scan the argument body, we used a trivial take_until. This handles partial input correctly, but not that efficiently: if it returns Incomplete, we do not persist any scan state, so the next scan still starts from the beginning of the buffered input. Even though each scan is lightweight, i.e. simply finding the close marker, it is still O(n) per chunk and th…

  • fa85ead #41992 — [MM][Perf][CG] Support ViT full CUDA graph for Kimi-VL (#41992)
    • 作者: Oğuzhan KIR | +498/-39 | 5 个文件

    Metric | No CG | With CG | Δ – | – | – | – Mean TTFT | 132.91 ms | 111.13 ms | ↓ 16.4% Median TTFT | 130.88 ms | 102.64 ms | ↓ 21.6% P99 TTFT | 290.47 ms | 193.25 ms | ↓ 33.5%

✨ New Feature

  • d112eb1 #45896 — [feature] MiniMax-M3-MXFP4 support added (#45896)
    • 作者: qli88 | +44/-3 | 4 个文件

    Co-authored-by: wangjiaxin99 This PR is based on https://github.com/vllm-project/vllm/pull/45794. It updates the unfused_triton MoE backend to handle MiniMax-M3-MXFP4. Test Plan command line: lm_eval –model local-completions \ –tasks gsm8k \ –output_path ./results \ –log_samples \ –num_fewshot 20 \ –model_args model=/data/amd-MiniMax-M3-MXFP4/,base_url=http://localhost:8000/v1/completion…