共 64 个 commit，涉及 407 个文件，+37006/-3660 行变动。

概要

统计项	数值
Commit 数	64
变更文件	407
新增行数	+37006
删除行数	-3660

Commit 列表

🧪 CI/Tests

• 6f612fb #45722 — [ROCm][CI] Patch conftest to resolve occasional OOMs (#45722)
  • 作者: Micah Williamson | +15/-0 | 1 个文件

  Sometimes we see OOMs in CI when a test starts while a previous test is lazily releasing memory. This PR should work around that case by waiting for memory to settle before starting a test. Here is an example of the failing case: https://buildkite.com/vllm/ci/builds/70756/canvas?jid=019eabf3-e26f-4150-af1e-d7ea834f1278&tab=output#L660

• 040df8f #45728 — [CI] Fix attention benchmark smoke test (#45728)
  • 作者: Matthew Bonanni | +2/-2 | 2 个文件

  Forward fix for the CI failure that prompted the revert in #45486. — AI assistance (Claude) was used to prepare this change.

• 7e179e4 #41532 — [ROCm][CI] Gate incompatible HF references on Transformers v5 (#41532)
  • 作者: Andreas Karatzas | +217/-145 | 10 个文件

  This updates generation-test metadata for models whose HF reference path is incompatible with the installed Transformers v5 runtime. Test groups fixed: - mi355_1: Language Models Tests (Standard) for MiniCPM4. - mi355_1: Language Models Test (Extended Generation) for HyperCLOVAX. MiniCPM4: - Fails in HF remote code because it imports is_torch_fx_available, removed from the installed Transformers v…

• c69c734 #44372 — [XPU][CI] add intel xpu cases for nightly CI (#44372)
  • 作者: wenjun liu | +119/-0 | 2 个文件

  Add xpu cases for nightly in CI pipeline. PR ## Test Result —

📦 Other

• 506ec6d #45793 — Upgrade tpu-inference to v0.22.1 (#45793)
  • 作者: Sting Lin | +1/-1 | 1 个文件

  Upgrade tpu-inference to latest stable release v0.22.1 Verified on tpu-inference CI. ## Test Result Success. —

• a52205b #43098 — [Model] Add HrmTextForCausalLM (Hierarchical Reasoning Model — Text) (#43098)
  • 作者: yifei wu | +658/-0 | 8 个文件

  HRM-Text shipped in transformers 5.9.0 (released 2026-05-20, merged to main via huggingface/transformers#46025). The model performs a hierarchical recurrent forward over two transformer stacks (H slow, L fast) inside nested H/L cycle loops; each recurrence step consumes a distinct KV cache slot, matching transformers’ cycle_offset formula. Attention uses a sigmoid gate (Qwen3Next-style) and a fuse…

• 3d34f8c #44178 — [ROCm][Cleanup] Remove stale AITER FA hybrid KV-cache TODO (#44178)
  • 作者: Tuukka Sarvi | +0/-2 | 1 个文件

  Remove a stale TODO from the ROCm AITER FA shuffled KV-cache update path. The TODO said correct hybrid-model KV-cache handling still needed to be added because Mamba state could make the KV cache non-contiguous. Since I think #43660, the ROCm AITER FA backend uses block-first KV-cache layout and the shuffled writer uses the actual key/value cache block strides, so the TODO no longer describes the …

• eb04c76 #43050 — feat: MLA prefill enable FA4 fp8 output (#43050)
  • 作者: Carl Y | +489/-20 | 13 个文件

  Completes FlashAttn x Static FP8 in https://github.com/vllm-project/vllm/issues/35792 - Enables fused static FP8 output in FA4 backend - currently points at https://github.com/vllm-project/flash-attention/pull/135 ## Test Result Eval —

• cca3365 #45753 — [Rust Frontend] Add CORS support (#45753)
  • 作者: Tahsin Tunan | +732/-42 | 12 个文件

  Summary - Adds CORS to the Rust frontend (tracking: #44280), matching the Python frontend’s Starlette CORSMiddleware. - A tower_http::cors::CorsLayer is applied to every response with the same permissive defaults Python uses (allow_origins=[""], methods/headers [""], credentials off), plus a small middleware that keeps non-CORS responses clean. ### Notes - CorsLayer reproduces Starlette’s br…

• a8c86ee #45306 — [Quant] Support modelopt_mixed on Ampere (SM80/SM86) (#45306)
  • 作者: Mike G | +7/-1 | 1 个文件

  Support modelopt_mixed on Ampere (SM80/SM86) Enables modelopt_mixed checkpoints (NVFP4 routed experts + FP8 weight-only dense layers) on Ampere — A100/SM80 and RTX 30-series/SM86. ModelOptMixedPrecisionConfig rejected these GPUs at engine-config validation (“Minimum capability: 89”), even though modelopt_mixed does not require native FP8 tensor cores. Its layers run on Marlin, which needs only c…

• 405c7cf #42726 — [ZenCPU] Add zencpu Platform Runtime Logging and Docs (#42726)
  • 作者: Lalithnarayan C | +108/-3 | 6 个文件
• d53f459 #44528 — [KV Connector][Mooncake] Pipeline-parallel support for PD-disaggregated serving with Mooncake connector (#44528)
  • 作者: Hank Han | +680/-34 | 4 个文件

  This PR adds Mooncake-specific support for pipeline-parallel prefill in PD-disaggregated serving. It is intended for long-context workloads where the prefill side needs PP to fit or run efficiently on H20-class devices. Decode-side PP is intentionally not part of this PR. ### Unit tests Run the Mooncake unit suites from a Kubernetes validation pod using the PR source checkout: ### GLM-5.1-FP8 E2E …

• b2cfae7 #45631 — Add Triton recompile detection (#45631)
  • 作者: Thien Tran | +81/-15 | 6 个文件

  Many Triton kernels in vLLM recompile during runtime for various reasons: - tl.constexpr args - Pointer alignment: divisibility by 16 or not - Stride alignment: divisibility by 16 or not (or more precisely, making certain 2D tl.load() pattern have 16-byte alignment) The last 2 are not obvious when looking at a kernel. Usually in vLLM we can lose alignment because of tensor slicing (i.e. mixed pref…

• a7fdfee #45690 — [CPU] Support Gemma Diffusion (#45690)
  • 作者: Li, Jiang | +213/-100 | 7 个文件

  • Add dynamic causal support in CPU attention - Optimize CPU fused moe CI tests ## Test Result —

• 8bf3749 #41496 — [Bug Fix] Allow pinned memory for WSL2 (#41496)
  • 作者: Jimmy Lee | +431/-3 | 4 个文件

  Cuda graph restricts CPU and GPU tensor copy during graph capture to only pinned tensors. Therefore WSL’s current restriction for pinned memory is preventing the use of –cpu-offload-gb in conjunction with cuda graph. This change enables the use of pinned memory for WSL2, and warnings have been added to inform of the restrictions with pinned memory in WSL2. This PR improves WSL2 support and potent…

• 9096659 #45777 — [Cleanup] Remove dead env (#45777)
  • 作者: Cyrus Leung | +0/-3 | 1 个文件

  Fix a bad merge from https://github.com/vllm-project/vllm/pull/40183 ## Test Result —

• a9a8a32 #40299 — Register parsed config classes before tokenizer init (#40299)
  • 作者: Andrew Barnes | +85/-5 | 3 个文件

  Register the already-parsed HF config class with AutoConfig before cached_tokenizer_from_config() reads the tokenizer config again. This fixes #40249, where worker-side tokenizer init could still fail on qwen3_5_moe even though vLLM had already parsed the model config successfully. ## Why this is not a duplicate Checked before posting: - gh issue view 40249 –repo vllm-project/vllm –comments - gh…

• 9d808e2 #40183 — [Core] Use fastsafetensors ParallelLoader for weight loading (#40183)
  • 作者: gitbisector | +68/-56 | 3 个文件

  Replace the manual per-shard SafeTensorsFileLoader loop with ParallelLoader (added in fastsafetensors 0.2.0), routing through fastsafetensors’ copier registry so future copier improvements are picked up automatically. Adds VLLM_FASTSAFETENSORS_QUEUE_SIZE as an opt-in pipelining knob (default 0 preserves existing peak VRAM; queue_size=k costs k extra shard-sized buffers at peak during loading). Swi…

• 259ff89 #45696 — [Rust Frontend] Require ModelConfig.vocab_size to be present (#45696)
  • 作者: Bugen Zhao | +69/-126 | 8 个文件

  Signed-off-by: Bugen Zhao i@bugenzhao.com Tighten Rust frontend vocab-size handling so sampling validation treats model vocab as a resolved model-config invariant, matching Python frontend behavior. - Require ModelConfig::vocab_size() to return a resolved value and fail HF backend setup when config.json lacks vocab_size. - Store resolved model vocab in HfTextBackend / SamplingLimits and use it d…

• b8bd773 #45758 — [XPU] Fix Triton attn fp8/bf16 check failing (#45758)
  • 作者: liuzhenwei | +25/-22 | 2 个文件

  The compute capability checks added in https://github.com/vllm-project/vllm/pull/43914 are CUDA-specific and incorrectly reject XPU devices. This PR skip these checks on XPU. python3 examples/basic/offline_inference/generate.py –model Qwen/Qwen3-0.6B –enforce-eager -tp 1 –attention-backend TRITON_ATTN –kv-cache-dtype fp8 ## Test Result —

• e3cfea2 #44412 — [Multimodal] Add Qwen3-VL video loader (#44412)
  • 作者: Isotr0py | +100/-3 | 3 个文件

  • This PR adds Qwen3-VL video loader to align with Qwen3VLVideoProcessor’s sample_frames behavior by default. ## Test Result All tests pass —

• f99260d #45685 — [Rust Frontend] Lower out-of-vocab validation to text layer (#45685)
  • 作者: Bugen Zhao | +278/-212 | 14 个文件

  Background: https://github.com/vllm-project/vllm/pull/45569#issuecomment-4706255909 > Also, I think we may need some refactoring of the validation first: it is currently done within the handlers for different endpoints, so it can be very easy to miss some of them (just like that we missed it in gRPC, which this PR is going to address). Ideally we should push the validation down to a common/shared …

• 3f65e21 #45674 — [Rust Frontend] Support max_logprobs validation (#45674)
  • 作者: Bugen Zhao | +484/-98 | 15 个文件

  Add Rust frontend support for max_logprobs validation and configuration, aligned with the Python frontend semantics. - Validate logprobs, prompt_logprobs, and logprob_token_ids in the text layer, including -1 expansion to vocab size, per-request caps, token-id range checks, and the Python logprobs == len(logprob_token_ids) rule. - Introduce SamplingLimits for runtime validation bounds (max_model_l…

• 3afe659 #45275 — [EP] Enable DBO with NIXL EP (#45275)
  • 作者: Itay Alroy | +8/-6 | 2 个文件

  DBO works with NIXL EP because it supports async prepare/finalize, but the backend validation only allowed DeepEP backends. Also add nixl_ep to the CUDA graph guidance for decode optimized all2all backends. cc @tlrmchlsmth

• 16e9117 #45298 — [EP] Query NIXL EP top-k index dtype (#45298)
  • 作者: Itay Alroy | +3/-1 | 1 个文件

  NIXL EP can be built with different top-k index types. Query the dtype reported by the installed nixl_ep instead of hardcoding int64. ai-dynamo/nixl#1751 changes the default NIXL EP top-k index type from 64-bit to 32-bit. Following nixl_ep.topk_idx_t keeps vLLM compatible with both the upstream default and custom NIXL EP builds. cc @tlrmchlsmth

• ab8b0fe #45606 — nixl_ep: Skip post-receive quantization for NVFP4 (#45606)
  • 作者: Itay Alroy | +8/-10 | 2 个文件

  After #44992, NIXL EP checks the current vLLM config to decide whether NVFP4 receive quantization should be skipped. That breaks when the receive path runs without a current config. Instead of relying on –moe-backend, make the decision from the MoE quant dtype. This matches non-hybrid DeepEP low-latency behavior: NVFP4 receives a regular activation tensor, and FlashInfer CUTEDSL quantizes the inp…

• 25ee659 #44955 — Fix parallel_tool_calls: null treated as false instead of default true (#44955)
  • 作者: Zang Peiyu | +2/-2 | 1 个文件

  Fixes #44948: parallel_tool_calls: null was treated as false, trimming to a single tool call. The documented default is true, so explicit null should behave like the default. ## Fix Change truthiness check to explicit is not False: - None → keep all tool calls (default) - True → keep all tool calls - False → trim to first - vllm/entrypoints/serve/utils/tool_calls_utils.py: 2 lines Logic verificati…

• 7e612a0 #44541 — [KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541)
  • 作者: Ronen Schaffer | +255/-22 | 10 个文件

  TieringOffloadingManager didn’t override reset_cache(), so engine resets left state stale. The new override: 1. Drains in-flight primary↔secondary transfers before resetting primary state. Without it, an in-flight secondary→primary load could write into a reallocated primary slot, or a primary→secondary store could read junk from a replaced slot — silent corruption either way. 2. Discards defe…

• 0a1c503 #45381 — [Model] Add MiniMax M3 support (#45381)
  • 作者: youkaichao | +14746/-323 | 108 个文件

  • Add MiniMax M3 model support across config, processors, model registry, AMD/NVIDIA model implementations, MTP, sparse attention, and warmup paths. - Add MiniMax M3 reasoning and tool parsers, including Rust frontend registrations and Python-facing parser wrappers. - Add supporting kernels, quantization paths, router GEMM shape support, and targeted tests. ## Duplicate-work check - Open PR search…

• 0d80979 #45548 — [Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
  • 作者: Flora Feng | +47/-45 | 4 个文件

  Remove separate self.reasoning_parser_cls and self.tool_parser attributes from OpenAIServingChat, consolidating all parser access through the existing self.parser_cls

• fa63bb9 #43914 — Remove redundant Triton KV cache dtype asserts and enforce architectural support (fp8 >= sm89) (#43914)
  • 作者: Mike G | +33/-39 | 2 个文件

  On the Triton attention backend (TRITON_ATTN), an fp8 KV cache is stored via current_platform.fp8_dtype() == float8_e4m3fn (Triton fp8e4nv), which has no lowering before SM89; a bfloat16 KV cache needs native bf16 (SM80+). Neither is caught early today: on older GPUs the engine loads and then dies deep in inductor autotuning with type fp8e4nv not supported in this architecture, surfacing o…

• 5ed15f4 #43557 — Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel (#43557)
  • 作者: Xin He | +253/-17 | 3 个文件

  Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel, which currently produces severely incorrect quantization results. Related PR: #37463 ## Root Cause The original code computes the E8M0 scale factor via: This is mathematically wrong because: floor(log2(a / b)) != floor(log2(a)) - floor(log2(b)) Dividing by 6.0 before extracting the exponent shifts the mantissa boundary, c…

• b997071 #45510 — (security) Enforce audio upload size limit before full file materialization (#45510)
  • 作者: Juan Pérez de Algaba | +210/-2 | 4 个文件

  The speech-to-text routes (/v1/audio/transcriptions, /v1/audio/translations) called UploadFile.read() without bounds before checking VLLM_MAX_AUDIO_CLIP_FILESIZE_MB, allowing oversized uploads to fully materialize in memory before rejection. This adds a bounded chunked read that rejects over-limit files early, preventing memory exhaustion from malicious uploads.

• 1d88c4d #45676 — [Docs] Update the online serving docs. (#45676)
  • 作者: wang.yuqi | +83/-37 | 4 个文件

  Update the online serving docs. ## Test Result —

• 9872921 #44423 — [XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423)
  • 作者: Yejing Lai | +4/-0 | 1 个文件

  XPU matmul/attention kernels are not batch-invariant across chunked-vs-non-chunked prefill and sync-vs-async scheduling shapes, so token outputs can diverge slightly between configs.

• 40eac9a #44760 — [Rust Frontend] Support parallel_tool_calls = false (#44760)
  • 作者: FAUST | +135/-23 | 8 个文件

  part of https://github.com/vllm-project/vllm/issues/44280 Test | Purpose – | – structured_stream_suppresses_later_tool_calls_when_parallel_disabled | Verifies that when parallel_tool_calls is false, structured_chat_event_stream emits only the first tool call’s events and the final AssistantMessage contains a single tool call, suppressing all subsequent tool calls. prepare_chat_request_maps_paral…

• b5adb02 #45200 — [Models] Fix MiMo v2.x QKV TP sharding + FP4 support (#45200)
  • 作者: Giancarlo Delfin | +170/-5 | 2 个文件

  Context MiMo v2.5 Pro is currently not supported for TP != 8 because of an issue with the fused QKV projection weights sharding. Additionally, MiMo v2.5 Pro FP4 (used as the base model for the recently released DFlash repo), is not supported because the incorrect quantization config is currently used for MoE (MXFP4 is required for the expert weights). Attempting to serve either model does not re…

🖥️ Kernel

• ce3ef17 #36895 — [Kernel][Helion][1/N] Add Helion kernel for rms_norm_per_block_quant (#36895)
  • 作者: Xiaohong (Sean) Chen | +8573/-0 | 5 个文件

  This PR is to add Helion kernel for operation. It follows the implementation from the vllm c version. This is a subtask for https://github.com/vllm-project/vllm/issues/32962. ### Kernel level benchmark Environment Python: 3.12.12 Pytorch: 2.11.0 Cuda: 13.0 Helion: 1.0.0 Benchmark Setup Latency measure: Baseline: and Autotuning Setup Default “full” autotuning effort. i.e. LFBOTreeSearch…

🐛 Bug Fix

• bf5149b #36616 — [Bugfix] Fix FlashMLA sparse accuracy with topk_length and zero-init padding (#36616)
  • 作者: Ajay Anubolu | +13/-3 | 1 个文件

  Pass topk_length to flash_mla_sparse_fwd for precise attention masking and use new_zeros instead of new_empty for BF16 head padding. Closes #36524

• c5e5c33 #45707 — [Bugfix][MoE] Restore routed output unpadding before shared expert add (#45707)
  • 作者: Netanel Haber | +13/-18 | 2 个文件

  Fixes regression for https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 and https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4 running with FLASHINFER_TRTLLM MoE. Regression introduced in https://github.com/vllm-project/vllm/pull/41184. The test that would have caught this was skipped and wasn’t reenabled when FLASHINFER_TRTLLM was enabled for nano nvfp4: ht…

• ebf3a6d #45307 — [Bugfix] Fix trtllm fused allreduce+rms_norm for transformers backend (#45307)
  • 作者: Thomas Parnell | +7/-0 | 1 个文件

  The following currently crashes on main: gives: See reproducer above. ## Test Result Works fine after fix. —

• 7ad894c #44784 — [Bugfix] Prevent cuMemcpyBatchAsync segfault with MTP and KV offloading (#44784)
  • 作者: joshua abraham | +744/-5 | 2 个文件

  Bug: Currently, vLLM crashes with a cuMemcpyBatchAsync segfault when using native KV offloading (–kv-offloading-backend native) alongside MTP speculative decoding on long contexts. Root cause: I found that the OffloadingConnectorScheduler was accidentally including EAGLE/MTP draft-attention groups in both the load and store scheduling paths. Since these draft groups don’t have stable bloc…

• 6607a80 #45553 — [Bugfix][Gemma4] Fix offline parser truncation, adjust_request token leak, and chat template sync (#45553)
  • 作者: Luciano Martins | +94/-74 | 5 个文件

  Fixes remaining Gemma4 tool calling bugs not addressed by #45588 (ParserEngine rewrite): offline parser truncation, reasoning token leak when thinking is disabled, and chat template sync with upstream HuggingFace model PRs. Fixes #39069 Fixes #39130 ### Bugs fixed - #39069 — Offline _parse_tool_arguments in gemma4_utils.py truncates string values containing internal double quotes. The function…

• 2addbb9 #45673 — [BugFix] Support async scheduling with prompt embeds for multimodal models (#45673)
  • 作者: Ruinan Ma | +5/-20 | 2 个文件

  [BugFix] Support async scheduling with prompt embeds for multimodal models Follow-up to #45383. #45383 fixed prompt_embeds for multimodal models (e.g. Gemma3ForConditionalGeneration), but worked around a second issue (“Root Cause 2”) by disabling async scheduling whenever enable_prompt_embeds is set on a multimodal model. This PR fixes that underlying issue so async scheduling produces corre…

• d467a2a #45357 — [Bugfix] Defer block freeing until in-flight steps finish under async scheduling + PD KV consumer (#45357)
  • 作者: llx | +541/-18 | 9 个文件

  This PR fixes a race in async scheduling with PD KV consumers by deferring the actual return of freed KV blocks to the block pool until the newest in-flight scheduled step has finished. With async scheduling, the scheduler may dispatch step N+1 before step N’s output is processed. If a request finishes or is preempted based on step N’s output, step N+1 may already be in flight and may still write …

• eacff17 #35536 — [Model Runner V2][Bugfix] Fix MRV2 LoRA warmup (#35536)
  • 作者: Jee Jee Li | +227/-63 | 5 个文件

  WIP export VLLM_USE_V2_MODEL_RUNNER=1 pytest vllm/tests/lora/test_llm_with_multi_loras.py -v -s ## Test Result —

• 51ec5cf #45464 — [Bugfix] Chat Completions Harmony Refactor Clean up (#45464)
  • 作者: yzong-rh | +46/-38 | 3 个文件

  Clean up three issues from https://github.com/vllm-project/vllm/pull/45171 1. Align the streaming path parse_delta() with non-stremaing path parse() to not emit reasoning or tool calls if the reasoning or tool call parser isn’t set. 2. Add ParserManager.get_parser is_harmony flag to to OpenAIServingRender (newly added in #45003). Doesn’t change behavior since harmony still uses a special path that…

• a3195fa #43981 — [AMD][Bugfix][Quantization] Honor fused-name match in is_layer_skipped (#43981)
  • 作者: RoyWang | +97/-1 | 2 个文件

  PR #41892 (https://github.com/vllm-project/vllm/pull/41892) introduced a bug that breaks Step-3.5-Flash-FP8 on vLLM v0.22.0+: every prompt degenerates to a stream of <｜begin▁of▁sentence｜> tokens regardless of input. Reproduced on ROCm vLLM 0.22.0+rocm722, TP=4, MI308X with the official stepfun-ai/Step-3.5-Flash-FP8 checkpoint. Root cause: #41892 registered a packed_modules_mapping = {qkv_proj: [q_…

• 588db18 #44409 — [Bugfix] Two-phase KV allocation for cross-group prefix cache hits (supersedes #33775) (#44409)
  • 作者: Saddss | +255/-34 | 4 个文件
• 6c5872e #45417 — [Bugfix] Unset HF’s default max_new_tokens for DiffusionGemma (#45417)
  • 作者: Martin Kukla | +14/-0 | 1 个文件

  DiffusionGemma model’s generation_config.json sets default max_new_tokens to 256. The rationale being that the HuggingFace users should never hit OOMs when running the model. For vLLM implementation, we don’t need to limit the max length by default in order to limit OOMs. Run the model without specifying max len, and generate more than one canvas. ## Test Result Generates more than one canvas — …

• c17e2f7 #45465 — [Bugfix][Rust Frontend] Make metrics respect –served-model-name (#45465)
  • 作者: Reid | +118/-9 | 2 个文件

  The Rust frontend currently accepts –served-model-name, but frontend Prometheus metrics can still be labeled with the backend model path instead of the public served model name. For example, serving Qwen/Qwen3-0.6B with: This change resolves served model names once in server state setup and uses the primary served name for frontend-side metrics labels. If no served model name is configured, it pr…

⚡ Performance

• ced32bb #42425 — [Perf] Add VLLM_TRITON_FORCE_FIRST_CONFIG to skip Triton autotuning (#42425)
  • 作者: Francesco Fusco | +209/-0 | 4 个文件

  Triton’s @triton.autotune benchmarks every candidate config on the first launch per key, then caches the winner. Three things make this painful when debugging or measuring: 1. Autotune is timing-driven, so its winner is non-deterministic. Run-to-run jitter can promote a different (BLOCK_M, num_warps, …) tuple each time, and a different tuple reduces partial sums in a different order — so ide…

• ad32608 #43586 — [MM][Perf][CG] Support dual-path ViT full CUDA graph for DeepSeek-OCR (#43586)
  • 作者: Shanshan Shen | +809/-69 | 16 个文件

  This PR implements the SupportsEncoderCudaGraph protocol for DeepseekOCRForCausalLM, enabling full CUDA graph capture of the vision encoder with a dual-path graph architecture. DeepSeek-OCR uses a two-tower ViT (SAM + CLIP) with a dynamic tiling mechanism — global images at 1024×1024 and optional local patches at 640×640. Rather than capturing a single monolithic graph for both paths, this PR …

• e18fe93 #45061 — [Perf] Optimize DSv4 prefill chunk planning, 4.0% E2E Throughput Improvement (#45061)
  • 作者: Wentao Ye | +133/-23 | 3 个文件

  Optimize DSv4 prefill chunk planning, instead of fixed 4, we can pour in as much requests as possible for one chunk For example We will get chunks This saves memory and reduce kernel launch directly! ### Acc Covered in current unit tests + lm_eval ### Perf

🔧 Refactor

• 3f53e21 #45463 — [Refactor] Remove Fp8OnlineLinearMethod as scheduled (#45463)
  • 作者: Wentao Ye | +25/-82 | 5 个文件

  A follow up for https://github.com/vllm-project/vllm/pull/32189 Remove the Fp8OnlineLinearMethod as scheduled Covered in unit tests

🔩 Misc

• 3f1ff1f #45792 — [Misc]Clean up useless test (#45792)
  • 作者: wangxiyuan | +0/-25 | 1 个文件

  seed_everything platform api has been removed already. Related test code can be cleaned up as well. ## Test Result —

• 81d8f4e #45640 — [Misc] Added validation for Cohere /v2/embed input field exclusivity (#45640)
  • 作者: Taneem Ibrahim | +65/-1 | 2 个文件

  This PR adds request validation for the Cohere-compatible /v2/embed API so that requests must provide exactly one non-empty input field among texts, images, or inputs. Previously, invalid requests could provide multiple input fields, and the server would silently choose one based on preprocessing order. Empty lists could also pass request parsing and fail later or produce unclear behavior. Since t…

• b00e76f #45210 — [Misc][Model] add io processor for query/document embeddings from ColBERT (jinaai/jina-colbert-v2) (#45210)
  • 作者: xx-thomas | +479/-0 | 7 个文件

  Address #42234 @noooop run vllm using existing code 1. git clone https://github.com/vllm-project/vllm.git 2. VLLM_PRECOMPILED_WHEEL_COMMIT=nightly VLLM_USE_PRECOMPILED=1 uv pip install –editable . –torch-backend=auto –verbose 3. call /pooling as specified in issue to understand current behavior run vllm with new plugin installed 1. uv pip install ./tests/plugins/colbert_query_plugin (after foll…

• f4359a7 #44892 — [DSV4][Minor] Fix supported KV cache dtypes (#44892)
  • 作者: Woosuk Kwon | +8/-8 | 3 个文件

  Fixes a bug introduced in #44699 FlashMLA doesn’t support bfloat16, and FalshInfer doesn’t support fp8_ds_mla

🦀 Rust Frontend

• c4fd979 #45759 — [Frontend] Remove AsyncMicrobatchTokenizer. (#45759)
  • 作者: wang.yuqi | +12/-219 | 2 个文件

  Remove AsyncMicrobatchTokenizer. 1. AsyncMicrobatchTokenizer uses a 2ms wait_timeout, which is too large for preprocessing, as the normal time for most tokenizer.encode calls is ~0.1ms. This has a significant impact on Frontend preprocessing, especially for pooling models and the render API. 2. The batch acceleration of tokenizer.encode comes entirely from multi-threading; without TOKENIZERS_PARAL…

• f3858d5 #45755 — [Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine (#45755)
  • 作者: Ben Browning | +768/-169 | 12 个文件

  Replace the hand-written NemotronV3ReasoningParser with a NemotronV3Parser built on the ParserEngine framework, reusing the Qwen3 config with Nemotron-specific adjustments. Wire up adapters via make_adapters() and update the reasoning parser registry. This exposed some onerous per-parser wiring in test_delegating_replay.py and test_replay.py, so I took this chance to also clean that up to minimize…

• 76a373e #45588 — [Frontend] Replace legacy Gemma4 parsers with engine-based implementation (#45588)
  • 作者: Ben Browning | +2808/-1332 | 20 个文件

  Migrate Gemma4 from separate hand-coded reasoning and tool parsers to the unified ParserEngine framework introduced for Qwen3. A single state machine in vllm/parser/gemma4.py now handles both channel-based reasoning extraction and custom tool call parsing with declarative configuration. The streaming engine parser adapters are registered under the existing gemma4 name so no user-facing configurati…

• cd9078f #45600 — [Frontend] Skip structural tags for auto tool_choice without strict mode (#45600)
  • 作者: Flora Feng | +111/-24 | 8 个文件

  • Add strict field to FunctionDefinition protocol model to match the OpenAI API spec - Gate structural tag generation on strict: true when tool_choice=“auto” — if no tool opts into strict schema enforcement, return None instead of constraining output - This prevents structural tags from over-constraining model generation when the caller hasn’t explicitly requested strict function calling

📖 Documentation

• 25c53d1 #45671 — [ROCm][Doc] Add installation notes about python version requirement (#45671)
  • 作者: vllmellm | +13/-0 | 1 个文件

  Added a note to the ROCm pre-built wheel installation section about python version requirement. Addressing #44660. ## Test Result —