共 57 个 commit，涉及 194 个文件，+4216/-2329 行变动。

概要

统计项 数值
Commit 数 57
变更文件 194
新增行数 +4216
删除行数 -2329

Commit 列表

🧪 CI/Tests

  • b0d7875 #49450 — [CI] Increase timeout of pytorch-compilation-unit-tests (#49450)
    • 作者: Nick Hill | +1/-1 | 1 个文件

    Similar to https://github.com/vllm-project/vllm/pull/49374. This one is also taking longer, e.g. https://buildkite.com/vllm/ci/builds/79339#019f88d2-9b09-4081-a473-5e9f6bd9416c

  • 53c2f20 #49350 — [ROCm][CI] skip moe weight padding for eplb (#49350)
    • 作者: Divakar Verma | +4/-1 | 1 个文件

    This PR skips moe padding for eplb as contiguous weights are required, following a refactoring. pytest -s -v tests/distributed/test_elastic_ep.py::test_elastic_ep_scaling[sync_eplb] This PR is the 3rd piece for solving this test case. 1. https://github.com/vllm-project/vllm/pull/47206 -> Fixes the weight transfer issue. 2. https://github.com/vllm-project/vllm/pull/49251 -> Resolves the final accur…

  • 2f75e7f #49374 — [CI] Increase timeouts for jobs exceeding current limits (#49374)
    • 作者: Kevin H. Luu | +14/-14 | 8 个文件

    Increase CI timeouts for jobs that exceeded their configured limits in Buildkite builds 79241 and 79299. For each job with a true Buildkite timed_out state, the new timeout is ceil(current_timeout * 1.5) because the pipeline schema requires integer minutes: - 45 -> 68: Basic Correctness, OpenAI Part 1, MTEB, Multimodal Standard 1 - 30 -> 45: Benchmarks CLI - 35 -> 53: e2e Scheduling - 50 -> 75: AP…

  • 05781e2 #49329 — [ROCm][CI] Fix order-dependent failure in test_flash_attn_accepts_handled_fp8_variants (MI355) (#49329)
    • 作者: stefankoncarevic | +5/-2 | 1 个文件

    tests/kernels/attention/test_attention_selector.py::test_flash_attn_accepts_handled_fp8_variants[fp8|fp8_e4m3] fails on the Kernels (B200-MI355) CI group when the file is run in full, while passing when the two cases are run in isolation: This is a test-ordering bug, not a production regression. ### Root cause The test verifies that FlashAttentionBackend accepts fp8/fp8_e4m3 on platforms that …

  • 08e5067 #49359 — [CI] Bump timeout of entrypoints-integration-api-server-openai-part-2 (#49359)
    • 作者: Nick Hill | +2/-2 | 2 个文件

    The timeout seems to now be too tight, it occasionally legitimately exceeds it. Bumping from 45 to 55 mins.

  • b8fb56d #49243 — [CI] Add gemma-4-E4B-it-assistant to CI gsm8k for GemmaMTP (#49243)
    • 作者: Michael Goin | +4/-1 | 1 个文件

    To better cover against regression like https://github.com/vllm-project/vllm/pull/47953 ## Test Result —

  • 1dca300 #49339 — [CI] Fix and wire encoder/manager cudagraph unit tests (#49339)
    • 作者: Nick Hill | +9/-0 | 3 个文件

    tests/v1/cudagraph/test_encoder_cudagraph.py and test_cudagraph_manager.py were not referenced by any Buildkite job, so they never ran. The encoder test was also broken: get_num_graphs_to_capture() reads self.config.enable_dual_path_graph (added with dual-path ViT graph support), but the _make_manager_with_budgets test helper builds the manager via object.new without setting self.config, raisi…

  • 61e10f0 #48845 — [ROCm][CI] Fix AITER MLA fp8 decode metadata regression test (#48845)
    • 作者: stefankoncarevic | +5/-4 | 1 个文件

    The gfx950 AITER MLA persistent-decode regression test (tests/kernels/attention/test_rocm_aiter_mla_decode_metadata.py) started failing on main due to a collision between two bugfixes: - #46997 added the test and the dtype_q/dtype_kv forwarding to aiter.get_mla_metadata_v1. At that point the decode query dtype was bf16, so the test pinned EXPECTED_Q_DTYPE = torch.bfloat16. - #47276 (…

  • 5aab491 #49325 — [CI] Wire tests/models/inkling into a B200 job (#49325)
    • 作者: Nick Hill | +13/-0 | 1 个文件

    The tests/models/inkling suite (8 files: FA4 kernels, sconv metadata, MoE weight layout, contract validation, MTP input fusion, QKV prep, MM towers) was not referenced by any CI job, so none of it ran in CI - including the contract-validation tests that #48822 itself modified. Add a B200 job (the FA4 tests require SM100; they skip elsewhere) to the Models - Basic group. Suite runs in ~6.5 min on 1…

  • 5812e1a #41653 — [Test] Add DeepSeek MTP parallel-load tests (#41653)
    • 作者: Stefano Castagnetta | +265/-0 | 2 个文件

    test_mtp_correctness[deepseek] covers TP=1 only. This adds parallelism coverage for DeepSeek MTP weight loading and basic correctness: TP=2, EP=2, EP=2 + EPLB via the inline LLM constructor, plus DP=2 via AsyncLLM (mirrors tests/v1/distributed/test_eagle_dp.py). Each parametrize entry builds spec and no-spec engines at the same parallelism shape, runs greedy decode under VLLM_BATCH_INVARIANT=1, an…

📦 Other

  • 37e370f #48957 — [DSv4 Perf] Skip empty c128 kernel launch, around 2x kernel performance improvement. (#48957)
    • 作者: Wentao Ye | +53/-2 | 2 个文件

    We do compress in SparseAttnCompressC128Block8Kernel and store in SparseAttnNormRopeStoreFullKernel We will skip it inside the kernel by taking a look at boundary, but this skip could be move ahead to bypass the full kernel. Acc covered in unit test Perf through this AI generated script And we get

  • c79ff5f #49326 — [Build] Bump vllm-flash-attn to C++20-compatible commit for torch-nightly (#49326)
    • 作者: Andrey Talman | +1/-1 | 1 个文件

    Bumps the vllm-flash-attn pin so vLLM builds against torch nightly, which now requires C++20. ed4b7342 = flash-attention#168 (“Require C++20 to match PyTorch ATen headers”), which sets CMAKE_CXX_STANDARD 20 in the vllm-flash-attn build. ## Why Recent PyTorch added a hard guard in ATen.h: vLLM’s main extensions already build at C++20, but the vendored vllm-flash-attn subproject pinned C++…

  • 1a659a0 #49431 — Upgrade tpu-inference to v0.25.0 (#49431)
    • 作者: Teresa Chen | +1/-1 | 1 个文件

    Upgrade tpu-inference to latest stable release v0.25.0 Verified on tpu-inference CI. ## Test Result Success. —

  • 0f6cf7f #49045 — [Rust Frontend] Extract request preparation from the inference path (#49045)
    • 作者: Sage | +190/-109 | 2 个文件

    Extracts the existing chat and text request preparation into engine-free processors for normal inference and a future renderer. This implements Stage 1 of #49047.

  • c79ad3a #49255 — [Rust Frontend][gRPC] Add abort control RPC (#49255)
    • 作者: Connor Carpenter | +177/-29 | 5 个文件

    Add an idempotent gRPC control RPC for aborting active Rust frontend requests by ID. - Add vllm.Control.Abort with repeated request IDs and an empty response. - Register vllm.Control on the existing gRPC listener. - Resolve caller-facing request IDs through the frontend’s external-to-internal request-ID mapping before aborting EngineCore work. - Treat empty, unknown, repeated, and completed reques…

  • d6dbdb9 #49408 — [XPU] WA of topk_softplus_sqrt arg mismatch on XPU (#49408)
    • 作者: Liangqiusong | +15/-0 | 1 个文件

    This PR adds a workaround to fall back to old interface for XPU, This will be removed once xpu-kernels is updated.

  • 06da482 #49395 — [XPU] WA of topk_softmax arg mismatch on XPU (#49395)
    • 作者: liuzhenwei | +11/-0 | 1 个文件

    PR #48979 added the is_padding argument to _moe_C.topk_softmax, but the current XPU kernel does not yet support it. This PR adds a workaround to fall back to old interface for XPU, This will be removed once xpu-kernels is updated. ## Test Result —

  • 9df2f91 #49396 — [Renderer] Offload derender CPU work to renderer thread pool (#49396)
    • 作者: Guan-Ming Chiu | +24/-0 | 1 个文件

    The /derender endpoints detokenize and resolve logprobs synchronously inside async handlers, blocking the event loop for all concurrent requests. Offload that work to the renderer’s shared thread pool in one hop; public async API unchanged. Not a duplicate: only open PR touching this area is #47931, which addresses a different problem. - pytest tests/entrypoints/scale_out/derender/test_derender.py…

  • 387189c #47992 — [ROCm] Remove redundant AITER fused_qk_rmsnorm probe (avoids config-time HIP init) (#47992)
    • 作者: stefankoncarevic | +16/-77 | 3 个文件

    enable_mla_dual_rms_norm_fusion probed AITER (check_aiter_fused_qk_rmsnorm) while building the VllmConfig in the parent process. That probe imports aiter, which initializes the HIP context and forces the multiprocessing start method to spawn. When VLLM_ROCM_USE_AITER=1, this breaks any flow that relies on fork — e.g. in-process registration of an out-of-tree model. It reproduces as a ROCm-only…

  • 75576c6 #49398 — Add auto label for xpu relate issue (#49398)
    • 作者: Kunshang Ji | +29/-0 | 1 个文件

    Test Result —

  • 16aca63 #49251 — [ROCm] Upgrade NIXL and UCX (#49251)
    • 作者: Andreas Karatzas | +127/-102 | 10 个文件
    • Upgrade the ROCm image to pinned upstream NIXL and UCX revisions. - Build the ROCm NIXL wheel and set UCX_RMA_PPLN_ENABLE=y. - Prefer nixl_rocm while retaining legacy RIXL compatibility.
  • 6049424 #49364 — [MRV2] Always build attn metadata at capture time (#49364)
    • 作者: Woosuk Kwon | +35/-21 | 2 个文件
  • ec59c15 #44120 — [MoE Refactor] Migrate MoeWNA16Method quantization method over to using the new MK oracle scheme. (#44120)
    • 作者: bnellnm | +805/-195 | 11 个文件

    This PR is an fixed + updated version of #42647 Migrate MoeWNA16Method quantization method over to using the new MK oracle scheme. Additional fixes - Fix Triton backend weight conversion when original weights are from auto_gptq. - Fix marlin zero point conversion code when original ZPs come from auto_gptq. Also fixes broken quantization import when humming module is not installed. This was broken …

  • 85f638a #48979 — skip cudagraph/DP padding in topk (#48979)
    • 作者: gnovack | +302/-99 | 10 个文件

    This PR modifies the VLLM_MOE_SKIP_PADDING logic introduced by https://github.com/vllm-project/vllm/pull/46428/, removing the separate call to torch.where and instead passing the is_padding mask into the topk kernel. The topk kernel then reads from is_padding to determine whether to set any given topk_ids entry to -1. ### Unit tests Ran Top-K kernel unit tests after adding new test cases to valida…

  • 33178f9 #49292 — Fix Qwen3-VL M-RoPE on the Transformers modeling backend (grids + compile) (#49292)
    • 作者: Aritra Roy Gosthipaty | +3/-7 | 1 个文件

    I wanted to serve the Qwen/Qwen3-VL-4B-Instruct-FP8 using the transformers modeling backed. I came across two distinct issues, one with compilation and other with eager. In this PR I will talk about them seperately. ## Enforcing Eager This command was able to get the engine running but failed when prompted. The issue stemmed from Video inputs are not implemented yet, so this line gets triggere…

  • 6e96891 #48683 — [ROCm] Bump AITER to v0.1.16.post5 (#48683)
    • 作者: Fangzhou Ai | +1/-1 | 1 个文件

    Advance the ROCm image build from AITER v0.1.16.post3 to v0.1.16.post4. This brings the gfx950 MXFP4 MoE path, top-k and MLA metadata/reduction optimizations, and AITER’s FlyDSL 0.2.2 dependency. FlyDSL does not need a separate vLLM pin because the AITER build stage installs AITER’s requirements.txt before building its wheel. The tested AITER source commit was 72eedddd921d0fb8f274e7b9291dd04306d08…

  • 47f1b47 #49241 — Ci/add laguna xs gsm8k (#49241)
    • 作者: Michael Goin | +10/-0 | 2 个文件

    Just porting over https://github.com/vllm-project/vllm/pull/42071 to use the 1xB200 instead of 8xH200 runner ## Test Result —

  • c67650f #45991 — [XPU][DeepSeekV4]Add DeepSeek-V4 fuse_index_q SYCL kernel path (#45991)
    • 作者: Avery Miao | +150/-0 | 2 个文件

    This PR wires DeepSeek-V4 fused indexer Q (RoPE + quantization) to the SYCL kernel path on XPU, replacing the Triton fallback. ## Code change - vllm/models/deepseek_v4/common/ops/fused_indexer_q.py: - Add current_platform.is_xpu() dispatch before the existing has_cutedsl() / Triton branches. - XPU calls torch.ops._xpu_C.deepseek_fused_indexer_q_rope_fp8 (FP8 path) or torch.ops._xpu_C.deepseek_fuse…

  • 6700813 #44456 — [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions (#44456)
    • 作者: Lucas Wilkinson | +115/-225 | 10 个文件

    PR #42374 (first part of RFC #42082) has been split into 4 PRs: #44454 [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config #44455 [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends -> #44456 [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions #44458 [4/N][KV-Cache Layout Refactor] Standardize KV cache layout Move M…

  • 1134545 #48641 — Revert “[Sampler] Stop upcasting logits to fp32 in apply_sampling_params” (#48641) (#49033)
    • 作者: vllm-agent | +25/-85 | 5 个文件

    apply_sampling_params copies the logits into a new fp32 tensor before applying sampling params. Generation models emit bf16 logits by default (except in RL), so this copy is a pure upcast that costs num_logits * vocab * 4 bytes. Under spec decode there are num_reqs * (1 + num_spec_tokens) logits instead of num_reqs, so the copy reaches multiple GiB. On Qwen3-8B + a dflash speculator it tries to al…

  • adfbbc1 #49177 — Propagate Flash Attention cache configuration to Ray workers (#49177)
    • 作者: Stefan Kaestle | +1/-0 | 1 个文件

    This change adds FLASH_ATTENTION_ to the environment-variable prefixes propagated from the vLLM driver to Ray workers. Flash Attention’s compilation cache is configured through environment variables. Without propagating them, Ray workers do not use the configured cache and may repeat expensive compilation during worker startup or scaling operations. This keeps the cache configuration consistent be…

🐛 Bug Fix

  • 2dc5a72 #49400 — [Bugfix][Renderer] Rebuild vision chunk UUIDs in async render path (#49400)
    • 作者: Guan-Ming Chiu | +2/-0 | 1 个文件
    • HfRenderer.render_messages_async skips the rebuild_mm_uuids_from_mm_data call that the sync path performs for unified-vision-chunk models (Kimi-K2.5) - Online serving therefore attaches stale pre-chunking UUIDs to multi_modal_uuids, corrupting mm-cache identity for chunked videos - Add the missing call, mirroring the sync path - Not a duplicate: no open PR touches this path - Verified with a tem…
  • 7c21548 #49297 — [PD][Bugfix] Fix NIXL hybrid MLA+mamba heterogeneous TP (#49297)
    • 作者: Ziming Huang | +10/-8 | 2 个文件

    Fixes KV transfer for hybrid MLA + SSM (Mamba) models under heterogeneous TP (prefill TP > decode TP) with the NIXL connector. ## Test Result —

  • 060b5f6 #49294 — [Bugfix][Attention] Ignore empty MLA context chunks during merge (#49294)
    • 作者: Lucas Wilkinson | +196/-0 | 5 个文件

    FIX for https://github.com/vllm-project/vllm/issues/49334 Alternative to https://github.com/vllm-project/vllm/pull/49196 - mark zero-context MLA attention states with -inf LSE before merging - use a single in-place Triton kernel driven by the existing ragged query and context offsets - resolve query-block ownership 32 request boundaries at a time with a warp-local reduction, amortized across all a…

  • ba18929 #49178 — [Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178)
    • 作者: aoshen02 | +131/-37 | 14 个文件
    • Skip only MTP completeness assertions while checkpoint-format IPC/NCCL weights arrive in buckets. - Keep normal startup loading and immediate parameter validation strict. - Use a scoped context that restores automatically on exceptions. Checkpoint-format IPC/NCCL transfer sends a complete update in multiple buckets. Each model.load_weights() call therefore sees only part of the checkpoint, but M…
  • 0500ca6 #49380 — [CI][Bugfix] Fix ROCm FP8 KV cache dtype in attention backend test (#49380)
    • 作者: peizhang56 | +5/-2 | 1 个文件

    tests/v1/attention/test_attention_backends.py::test_causal_backend_correctness[fp8*] fails on ROCm (gfx94x / MI300) with [AttentionBackendEnum.TRITON_ATTN] produced non-finite values. Root cause is in the test, not the backend. The test stores the FP8 KV cache using a hardcoded torch.float8_e4m3fn, but at runtime the attention backends reinterpret the raw cache bytes with current_platform.fp8_dtyp…

  • a1c15bc #49356 — [CI][Bugfix] Fix and wire streaming-input tests (#49356)
    • 作者: Nick Hill | +13/-5 | 4 个文件

    The tests/v1/streaming_input/ suite ran in no CI job and had drifted out of sync with the code: - test_scheduler_streaming.py: the create_scheduler() mock’s model_config lacked is_encoder_decoder/is_diffusion, so the truthy MagicMock tripped the scheduler’s encoder-decoder assertion; and one assertion checked scheduler.waiting, but paused streaming sessions now route to the skipped_waiting (blocke…

  • a7d00ec #48524 — [Bugfix] DFlash fc sized wrong when num_target_layers != num_hidden_layers (#48524)
    • 作者: Michael Goin | +61/-11 | 3 个文件

    Uses get_eagle3_aux_layers_from_config as a shared utility to more robustly get the aux-layer list. Fixes https://huggingface.co/shanjiaz/dspark-mistral-small-119b where there are 3 aux layer ids and 5 drafter layers. ## Test Result —

  • 96a7392 #49016 — [Bugfix] fix cutalss version upgrade bug, need update MSG new commit (#49016)
    • 作者: rongfu.leng | +2/-2 | 2 个文件

    Fixs: https://github.com/vllm-project/vllm/issues/49005 Wait https://github.com/vllm-project/MSA/pull/8 this pr merge after update new commit. ## Test Result —

  • 60d443f #48655 — [CI/Build][The Rock][BugFix] Use fork method in test_multiproc_executor_multi_node for py 3.14 compat and fix test_multiproc_executor_shutdown_cleanup (#48655)
    • 作者: rasmith | +19/-5 | 2 个文件

    The distributed/test_multiproc_executor.py has multiple problems. I was fixing this for The Rock on MI325 but encountered an additional issue. 1. test_multiproc_executor_multi_node uses multiprocessing and tries to pickle a nested function which is not compatible with Python 3.14 forkserver start method * set start method to fork for Python 3.14 compatibility 2. test_multiproc_executor_shutdown_cl…

  • fca252d #49351 — [CI][Bugfix] Reduce max_model_len in OOT embedding test to fix KV-cache OOM on small GPUs (#49351)
    • 作者: Flora Feng | +1/-1 | 1 个文件

    Fixes https://github.com/vllm-project/vllm/issues/49333 test_oot_registration_embedding fails at engine startup on the L4 CI GPU - ValueError: To serve at least one request with the model’s max seq len (2048), 0.66 GiB KV cache is needed, which is larger than the available KV cache memory (0.64 GiB). After the 9B Gemma2 embedding model’s weights and the CUDA-graph memory profiler’s ~0.5 GiB reserv…

  • b2b8f67 #47953 — [Bugfix][Spec Decode] Restrict embedding-width share guard to EAGLE drafts (#47953)
    • 作者: Evan Takahashi | +6/-2 | 1 个文件

    Fixes #47794 — Gemma4 MTP fails engine initialization on nightly with a linear shape mismatch ([s47, 3840] x [5632, 1024]). Root cause: #43957 added an embedding-width guard to _maybe_share_embeddings that disables target↔draft embedding sharing when the widths differ. That is correct for EAGLE drafts, which consume input_embeds in their decoder layers at their own hidden size (the Eagle3MiniMaxM2…

  • de6ec29 #49302 — [Bugfix] Fix DSA crash under breakable piecewise cudagraphs (#49302)
    • 作者: Nick Hill | +30/-12 | 3 个文件

    #48822 made breakable piecewise graph replay pad batch metadata to the captured shape (num_reqs padded to min(num_tokens, max_num_reqs), with zero-length trailing requests). That breaks backends whose attention runs eagerly as graph break points and reads the real batch from the forward context: the DSA indexer decode path expands the padded request count by next_n (deepgemm asserts batch_size * n…

  • 8950394 #48860 — [Bugfix] Prefix-cache metrics double-counted when a KV connector defers requests (#48860)
    • 作者: Seiji Eicher | +204/-41 | 5 个文件

    Fixes #43736. Whenever the scheduler re-evaluates a waiting request that still has num_computed_tokens == 0, it re-runs the lookup and calls record again, adding the request’s num_tokens to queries and its num_hits to hits. The metrics skew toward the multiply-counted request’s statistics. Sequence: 1. The scheduler runs the prefix-cache lookup for a waiting request with num_computed_tokens == 0 a…

  • 7bb49be #49306 — [Bugfix] Handle MLA fallback during FA4 JIT warmup (#49306)
    • 作者: Roberto L. Castro | +5/-1 | 1 个文件

    Follow-up to #47451. As @mgoin correctly noted in https://github.com/vllm-project/vllm/pull/47451#discussion_r3616102551, JIT warmup must handle models for which no generic MLA prefill backend is available. Related auto-revert PR: #49268.

  • f890e1d #48843 — [BugFix] Set graph_pool_id before FULL CUDA graph capture in ModelRunner V2 (#48843)
    • 作者: Ilya Markov | +116/-0 | 2 个文件

    Fixes #48840: AssertionError: graph_pool_id is not set under graph capture when VLLM_USE_NCCL_SYMM_MEM=1 is enabled with ModelRunner V2 and FULL_DECODE_ONLY CUDA graphs. Add set_graph_pool_id() before torch.cuda.graph() in CudaGraphManager.capture() for the FULL capture path. Adds a CPU unit test that asserts graph_pool_id is set before entering torch.cuda.graph(). pytest tests/v1/cudagraph/test…

  • 5b3762a #49021 — [Bugfix][CPU] Fix Clang OpenMP build on macOS (#49021)
    • 作者: Sya | +3/-1 | 1 个文件

    Clang does not support capturing structured bindings in OpenMP regions, causing the CPU attention extension to fail to build on macOS. Replace the structured binding with regular local variables to fix. Tested on Apple M2,

  • 4d30c51 #49190 — [bugfix] Fix Cosmos3 Edge checkpoint weights filtering, video loading, prompt expansion (#49190)
    • 作者: bastefaniak | +30/-3 | 5 个文件
    1. Skip k_norm_und_for_gen weights in generator tower. 2. Route Cosmos3EdgeVideoProcessor through the Qwen3-VL video backend 3. Correct video placeholder expansion to produce timestamped, per-frame tokens Verify the new checkpoint weight is skipped in tests/models/multimodal/test_mapping.py Verify the processor produces the expected number of video tokens and frame wrappers in tests/models/multimo…
  • 3e0c887 #47298 — [Bugfix] Fix Ovis2_5 special tokens for transformers v5 (#47298)
    • 作者: Miłosz Grunwald | +12/-19 | 2 个文件

    Fix Ovis2_5 processor vllm serve AIDC-AI/Ovis2.5-2B –dtype bfloat16 –tensor-parallel-size 1 –max-model-len 8192 –trust-remote-code 2>&1 | tee “serve.log” ## Test Result Without the fix: (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] █ █ █▄ ▄█ (APIServer pid=12810) INFO 07-01 12:59:40 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀…

🦀 Rust Frontend

  • 61c9ef9 #49153 — [Frontend] Parallelize preprocessing within the same request for pooling models online serving. (#49153)
    • 作者: wang.yuqi | +530/-812 | 17 个文件

    TL;DR - Use multithreading to accelerate preprocessing. You can specify the number of threads using renderer_num_workers. note: - We have not implemented tiling to overlap preprocessing and computation for online serving as we did for offline inference (#47699), in order to avoid unnecessary complexity. Therefore, it is not recommended to submit an extremely large number of requests at once. Yo…

  • eb44b3a #49295 — [Rust][Benchmark] Use async HTTP clients (#49295)
    • 作者: Bugen Zhao | +200/-122 | 15 个文件

    Uncached downloads created reqwest::blocking::Client while the benchmark was running inside a Tokio runtime. Dropping the blocking client could shut down its internal runtime from async context and panic. This PR changes to use async reqwest for dataset downloads and hf-hub access, so that we can remove its blocking feature from vllm-bench. The tokenizer interface remains synchronous. As a result,…

🔩 Misc

  • 1750e44 #49322 — [Misc] Move PyNvVideoCodec stuff out of gpu worker (#49322)
    • 作者: Isotr0py | +200/-190 | 6 个文件
  • 4809de7 #49344 — [Misc] Fix terminal output logo coloring (#49344)
    • 作者: Nick Hill | +5/-4 | 1 个文件

    This changes it to use the default color bolded which will end up as white on black and black on white, which is the proper vLLM colors!

  • 040cbf9 #49214 — [Misc] Use VLLMValidationError in chat completion tool and batch validators (#49214)
    • 作者: Umut Polat | +16/-11 | 1 个文件

    Follow-up to #36254, continuing the migration of user-facing request validation in the OpenAI-compatible protocol layer to VLLMValidationError. A few checks in ChatCompletionRequest.check_tool_usage and BatchChatCompletionRequest.check_batch_mode still raised plain ValueError, while sibling checks in the same validators already use VLLMValidationError. The custom exception carries a parameter name…

  • 7a98c7a #49244 — [Misc] Remove old now unsupported max_num_partial_prefills and max_long_partial_prefills (#49244)
    • 作者: Nicolò Lucchesi | +4/-60 | 3 个文件

    Removes the max_num_partial_prefills and max_long_partial_prefills scheduler config fields. I believe we have been on v1 for 1+ year now, but we still have these args which are leftovers from v0/v1 hybrid period where an oracle was needed. These fields were introduced for the V0 scheduler and explicitly rejected by the V1 enablement oracle in #13726. With V0 removed, the fields are now dead config…

  • 0d9e606 #49299 — [Misc][Docs] Fix XPU compute-runtime driver link version mismatch (#49299)
    • 作者: Lena Onyshchenko | +1/-1 | 1 个文件

    The XPU installation guide (docs/getting_started/installation/gpu.xpu.inc.md) links the text “compute runtime 26.18” to release tag 26.14.37833.4, which matches neither the link text nor the driver version actually installed in docker/Dockerfile.xpu (26.18.38308.1). This updates the link to point at the matching 26.18.38308.1 release so users install the recommended driver version. Docs-only chang…

  • adc98f0 #49298 — [Misc] Add @esmeetu to codeowners for rust/src/bench (#49298)
    • 作者: Roy Wang | +1/-0 | 1 个文件

    #48107 ## Test Result —