共 57 个 commit，涉及 222 个文件，+19881/-1395 行变动。

概要

统计项 数值
Commit 数 57
变更文件 222
新增行数 +19881
删除行数 -1395

Commit 列表

⚡ Performance

  • 04502de #49607 — [Perf] Hash videos by source bytes (#49607)
    • 作者: Guan-Ming Chiu | +94/-21 | 8 个文件
    • Video requests hash the full decoded frame buffer; images already hash their compressed source - Hash the smaller of source bytes and decoded frames, plus the sampling metadata - Byte-count comparison, so it is hardware-independent and always picks the cheaper buffer - Follow-up: decode options absent from the metadata still stay out of the key, like images - pytest tests/multimodal/test_parse.p…
  • 56c96b0 #48774 — [Perf] Tune LL BF16 Router GEMM (#48774)
    • 作者: Roberto L. Castro | +93/-20 | 2 个文件

    This PR tunes the ll_bf16_gemm router GEMM dispatch heuristic for critical models: GLM5.2, DSv3.2, DSv4-Flash, and DSv4-Pro on Hopper and Blackwell GPUs. Router GEMM has shown up as an expensive kernel if it is not treated carefully. For the most critical models, we have historically used ad-hoc, PTX-based kernels focused on LL scenarios. However, those kernels are usually limited to a single shap…

  • a89015c #48739 — [Perf] Make merge attention context count a runtime argument (#48739)
    • 作者: liminfei-amd | +2/-2 | 1 个文件

    Makes merge_attn_states_kernel’s batch-varying prefill_tokens_with_context a runtime argument instead of a tl.constexpr, preventing a new Triton specialization for each distinct value. Partial fix for #48650. ## Why this is not a duplicate Rechecked on 2026-07-15 immediately before preparing this draft: - #48650 is open, unassigned, with no comments. - PR #48734 now implements the issue’s count_ex…

  • 96fa3f4 #49659 — [Perf] Skip ll_bf16 router GEMM warmup for non-MoE models (#49659)
    • 作者: neweyes | +1/-1 | 1 个文件

    Skip unnecessary ll_bf16 router GEMM warmup for non-MoE models. _warmup_ll_bf16_router_gemm() initializes router GEMM kernels that are only used by MoE models. Previously, the warmup was executed on all Hopper/Blackwell GPUs regardless of model type, introducing unnecessary startup overhead for dense models. This change adds an is_moe check so that router GEMM warmup only runs when the loaded mode…

  • c314af1 #48637 — [CPU][Perf] INT8 Fused MoE Kernel for Arm CPUs (#48637)
    • 作者: Fadi Arafeh | +1742/-216 | 12 个文件

    Enable W8A8 (per token input, channelweise weights) MoE checkpoints like RedHatAI/Qwen3-30B-A3B-Thinking-2507-quantized.w8a8 on Arm CPUs To achieve this, we: - reuse the structure of the existing bf16 grouped gemm kernel in the int8 kernel (I tried to extend the existing bf16 grouped gemm kernel to support int8 quantization, but the code looked horrible) - add AdvSIMD smmla gemm micro kernels # Pe…

  • 8d28b48 #49524 — [Perf] Isolate MM preprocessing on its own executor (#49524)
    • 作者: Guan-Ming Chiu | +31/-20 | 3 个文件
    • One in-flight image/video preprocessing currently blocks every concurrent request’s tokenization for its full duration (1s+ for large images), because both share a single-worker executor - Give MM preprocessing its own executor so the two never contend; composes with #44786 - Narrow the #38418 guard to pooling models, whose preprocessing still runs on the renderer workers - pytest tests/renderer…

📦 Other

  • d2ca300 #47711 — [MRV2][Performance] Skip no-op FP32 logits materialization (#47711)
    • 作者: Song Zhixin | +21/-0 | 1 个文件

    MRV2 currently materializes the logits tensor as FP32 before applying sampling parameters, even when every active request is a no-op: temperature is 0 or 1, top-k equals the vocabulary size, top-p is 1, min-p is 0, and there are no penalties, logit biases, or bad words. The Gumbel sampling kernel already converts loaded logits to FP32 in registers. This PR adds a no-op fast path that returns the o…

  • 59a6b04 #49204 — [Core] Fix internal LB load-balancing (#49204)
    • 作者: Nick Hill | +188/-59 | 7 个文件

    Previous PR https://github.com/vllm-project/vllm/pull/30739 added support for efficient DP without EP. The coordinator was kept to propagate engine queue stats for loadbalancing, but there was an omission whereby these stats were not actually getting published, which could result in significant imbalance between the ranks. As well as fixing this (by also calling _maybe_publish_request_counts in th…

  • dbccc5a #48912 — [Model] Enable EVS for Qwen3.5 (#48912)
    • 作者: Rui “Garry” Gao | +32/-12 | 1 个文件

    Add Qwen3.5 support for Efficient Video Sampling (EVS), which was previously disabled in a hard-coded fashion. Per the test results below, on Qwen3.5 EVS works, with characteristics (accuracy decay, TTFT improvement) comparable to Qwen3-VL. Test accuracy decay with –video-pruning-rate 0.75 on Qwen3.5-9B model on Video Bench (subsets MSRVTT+ActivityNet+Driving-decision-making). Test TTFT on random…

  • 92e8518 #49957 — Improve Transformers modelling backend fx tracer (#49957)
    • 作者: Harry Mellor | +307/-128 | 9 个文件
    • Trace with meta tensors: every traced op also runs on meta tensors, so len, iteration and .shape unpacks come from PyTorch’s meta kernels instead of the hand-written rules in _infer_len/_rank. - Record untraceable attention interfaces as opaque leaf calls (flagged in the node’s meta, with the declared return length). vllm_attention_forward used to end the trace early; the graph now reaches the m…
  • 77cba02 #48123 — [KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
    • 作者: Ronen Schaffer | +301/-52 | 16 个文件

    Adds per-request tier filtering to KV cache offloading. Requests can specify which secondary tiers to load from via kv_transfer_params[“kv_load_tiers”], filtering by medium and locality. See detailed design discussion: https://github.com/vllm-project/vllm/pull/48123#issuecomment-4979058299 Key changes: - Medium enum: CPU/STORAGE (coarse granularity, both FS and OBJ → STORAGE) - TierMatcher(NamedTu…

  • 30fbd05 #49909 — [ROCm] Use backend-default dot precision for ReplaySSM (#49909)
    • 作者: Andreas Karatzas | +33/-21 | 2 个文件
    • Let ROCm Triton select its native dot precision instead of passing the unsupported CUDA-only tf32x3 value. - Keep tf32x3 on CUDA and move the ROCm Mamba kernel group from MI250 to MI300 so the default ROCm path is exercised.
  • 0906123 #48841 — [ROCm] [Model] Enable TML inkling (#48841)
    • 作者: TJian | +9213/-9 | 33 个文件

    This is the first enablement PR for inkling model on ROCm. This PR also targets the more efficient variant of the model which is mxfp4. There are possible path for loading and use nvfp4 weight, however, we could only support it through inefficient emulation code path. The weight is generated and distributed by https://huggingface.co/lightseekorg/Inkling-MXFP4 instead. There is a continuous effort …

  • cbc3a87 #49907 — [Tokenizer] Use HF config for HF tokenizers (#49907)
    • 作者: Andreas Karatzas | +10/-4 | 1 个文件
    • Resolve the tokenizer backend before loading config and request HF config for HF-backed tokenizers. - Keep native Mistral auto-detection while preventing dual-format repositories from selecting the wrong tokenizer class.
  • fd9d2ed #49944 — [Rust Frontend] Keep --max-model-len engine-owned (#49944)
    • 作者: Bugen Zhao | +63/-17 | 3 个文件

    max_model_len is an engine launch input, not frontend runtime state shared by the Python-bootstrapped and Rust-managed paths. This change: - removes max_model_len from SharedRuntimeArgs, so vllm-rs frontend –args-json no longer deserializes Python’s engine-owned value - moves it to ManagedEngineArgs, where only vllm-rs serve needs it to launch the Python engine - forwards the original string to P…

  • 5d07e26 #47514 — [Quantization][INC]Add MXFP8 Linear Support (#47514)
    • 作者: Zhenzhong Xu | +322/-9 | 7 个文件

    python3 examples/basic/offline_inference/generate.py –model ~/Qwen3-8B-MXFP8-AR/ –block-size 64 –max-model-len 4096 –gpu-memory-utilization 0.6

  • d742856 #49502 — [3/N][Core][KV Connector] Support reliable partial-tail KV offload for sub-block prompts (#49502)
    • 作者: Dao007forever | +1448/-49 | 13 个文件

    see https://github.com/vllm-project/vllm/issues/45702 Enable fine-grained prefix lookup and MooncakeStore offload when a prompt ends before the shared block boundary. Handle lazy hash sequences and local-only partial hits correctly so sub-block requests can reuse cached state without triggering an invalid remote load. ### max_tokens=1 failure mode A producer’s partial-tail marker previously became…

  • 544cb72 #48577 — [CPU][Spec Decode] Optimize GDN conv path for speculative decoding (#48577)
    • 作者: Tianmu Li | +767/-122 | 5 个文件

    Improve CPU serving performance for Qwen3.5 GDN speculative decoding with MTP by replacing fallback torch conv1d path with cpp implementation when AMX is available. Perf: Server: Client: Accuracy: ## Test Result Performance | config | before output tok/s | after output tok/s | delta | | — | —: | —: | —: | | no spec decode | 70.29 | 70.16 | -0.18% | | MTP k=1 | 63.81 | 74.82 | +17.26% | | M…

  • f19ee27 #49571 — [Hardware][Power] Add FAST_EXP for Power (#49571)
    • 作者: Akash kaothalkar | +24/-4 | 3 个文件

    Optimize expf computation on PowerPC CPUs via VSX vectorization (DEFINE_FAST_EXP). 1. CPU Attention Microbenchmark Run benchmark_cpu_attn.py. 2. End-to-End Throughput Benchmark Run benchmark_throughput.py to measure the overall token generation speedup. ## Test Results ### 1. CPU Attention Microbenchmark | Metric | main | feat/fast_exp | Improvement | |——–|——–|—————–|-…

  • 49f31d7 #49913 — [ROCm] Make vllm_c RMSNorm output contiguous (#49913)
    • 作者: Andreas Karatzas | +26/-1 | 2 个文件
    • Allocate contiguous ROCm vllm_c RMSNorm output when the input has transposed strides. - Cover the singleton-batch VibeVoice layout with an IR regression case.
  • 8de50e4 #49376 — [Docs] Document NVFP4 GEMM kernel selection and Marlin weight-only fallback (#49376)
    • 作者: Harjoth Khara | +14/-0 | 1 个文件

    Users hitting slow NVFP4 performance have no docs explaining which GEMM kernel vLLM picked or why. This adds a short note to the ModelOpt guide covering: - Kernel selection happens automatically at load time, based on what the GPU supports. - GPUs without a native FP4 GEMM kernel fall back to weight-only (W4A16) Marlin, which logs a warning and can cost throughput. - –linear-backend overrides the…

  • bf4f633 #49394 — [XPU] Enable QK Norm + RoPE fusion pass on XPU (#49394)
    • 作者: Chaojun Zhang | +24/-3 | 3 个文件

    Performance & Accuracy Test ## Commands Server (fusion_on adds –compilation-config or set , eager adds –enforce-eager instead): Bench client: ## Benchmark report (input=4096, output=1024, num-prompts=32, max-concurrency=8) Case | Duration (s) | Output tok/s | Total tok/s | Mean TTFT (ms) | Mean TPOT (ms) – | –: | –: | –: | –: | –: fusion_on | 321.33 | 101.98 | 509.89 | 17651.13 | 53.39 fus…

  • 439f336 #49736 — [Core] Fix gpu<->cpu syncs in MRV2 mamba_hybrid.py (#49736)
    • 作者: Nick Hill | +4/-4 | 1 个文件

    Found by @benchislett

  • ffc4f08 #46116 — [Core][KV-transfer] MoRIIO: heterogeneous TP<->DP prefill/decode read routing (#46116)
    • 作者: limeward | +467/-7 | 3 个文件

    Part of RFC #46107. MoRIIO today assumes the prefill and decode engines share the same parallelism layout. This PR enables heterogeneous disaggregated prefill/decode, where the two phases run different parallelism (TP vs DP+EP). The performance results in the RFC which encompasses this PR shows that heterogeneous parallel PD setups can achieve higher throughput and lower TTFT at high concurren…

  • fdaa0d9 #49331 — [ModelRunner V2] Support encoder-only attention (#49331)
    • 作者: Nick Hill | +276/-4 | 8 个文件

    This adds encoder-only attention support to MRV2 in preparation for BERT/RoBERTa model pooling support. Intentionally isolates the encoder-only related changes to minimize changes to non-pooling files / code paths.

  • 0934b26 #49901 — [CI/Build] Refresh tags before building macOS wheel (#49901)
    • 作者: Kevin H. Luu | +3/-0 | 1 个文件

    Refresh Git tags before building the native macOS arm64 CPU wheel. The macOS wheel job runs on a persistent Mac mini checkout. Buildkite fetched the requested commit for release-v2 build 4253, but did not refresh tag refs. As a result, setuptools-scm used the stale v0.23.1rc0 tag and produced 0.23.1rc1.dev1290+gf2654939e.cpu even though the commit was tagged v0.26.0. The Linux wheel jobs use fresh…

  • b5b61c6 #46877 — [Core][Distributed] Add process-checkpoint lifecycle hooks for communicators (starting with Flashinfer) (#46877)
    • 作者: Schwinn Saereesitthipitak | +185/-4 | 10 个文件

    Add a focused process-checkpoint lifecycle for FlashInfer-owned communication resources. - Forward V1 GPU worker prepare/restore calls through distributed state to registered device communicators, with accelerator synchronization around each transition. - Have the CUDA communicator transition FlashInfer all-reduce workspaces owned by its CPU process group independently of the optional standalone F…

  • 3f1d409 #49285 — [KV Offload] Fix num_tokens_after_batch for different termination types (#49285)
    • 作者: AlexHuang | +4/-2 | 1 个文件

    Fix incorrect num_tokens_after_batch computation in offloading scheduler by using num_computed_tokens for aborted requests (where partial KV data may exist) instead of always using num_tokens. ## Problem The original code used num_tokens for all finished requests: For aborted requests, num_computed_tokens < num_tokens (partial KV data). Using num_tokens would attempt to store chunks without actual…

  • da3a252 #48021 — [KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)
    • 作者: liranschour | +2673/-396 | 11 个文件

    Adds a generic peer-to-peer (P2P) secondary tier for KV-cache offloading, letting a vLLM engine both fetch KV blocks from and serve KV blocks to remote peers over a NIXL data transport with a ZMQ control channel. It generalizes the existing prefill/decode (PD) path into a symmetric P2P model: a consumer issues a LookupMsg for the blocks it needs; the producer answers against its local tier…

  • 21fd9e8 #45429 — [Model] Support top_k and top_p sampling for DiffusionGemma (#45429)
    • 作者: Guan-Ming Chiu | +18/-2 | 1 个文件

    Support per-request top_k/top_p for DiffusionGemma by filtering logits before the compiled denoise step, mirroring the AR sampler. Masked tokens become -inf; committed argmax (top-1) unaffected. Padding switched to masked_fill_ to avoid -inf * 0 = NaN on truncated canvases. Related: #45163. Validation relaxation lives in the companion PR. GPU behavioral check: per-request filtering, default fast p…

🐛 Bug Fix

  • 27d7061 #49963 — [Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online (#49963)
    • 作者: Umut Polat | +69/-1 | 2 个文件

    Fix a regression where truncate_prompt_tokens (and truncation_side) is silently dropped on the Jina rerank/score online path, so a request that asked to truncate is instead rejected with HTTP 400. JinaRankingIOProcessor.get_request_factory_online swaps ctx.request for a proxy PoolingCompletionRequest and delegates to the base factory, which reads truncation off ctx.request: The proxy was built…

  • ef9975d #45828 — [Bugfix] Reject pipeline parallelism for DiffusionGemma (#45828)
    • 作者: Guan-Ming Chiu | +0/-6 | 1 个文件

    Canvas state is advanced only by the sampler on the last PP rank, but every rank reads it for forward inputs, and the generic PP path broadcasts only token ids. Fail closed at startup instead of crashing in warmup. Thus, would like to remove SupportsPP interface to prevent error in advance - set PP=2 and run with DiffusionGemma ## Test Result - PP=2: fast ValueError, no warmup crash. PP=1: unchang…

  • 81962bb #49043 — [Bugfix]Reject invalid FlashInfer MNNVL workspaces (#49043)
    • 作者: rongfu.leng | +6/-0 | 1 个文件

    Fixs: https://github.com/vllm-project/vllm/issues/49041 On topologies without symmetric-memory multicast support, FlashInfer can construct an MNNVL allreduce workspace whose mc_ptr is null. vLLM treated the workspace as initialized and passed it into fused allreduce kernels, which dereferenced the null multicast mapping and failed with a CUDA illegal memory access. Validate mc_ptr immediately afte…

  • 394beb6 #49843 — [Bugfix][ROCm] Use batch DMA for CPU KV cache loads (#49843)
    • 作者: Andreas Karatzas | +16/-2 | 2 个文件
    • Add ROCm to the existing XPU batch-DMA fallback for CPU-to-GPU KV transfers. - Keep CUDA, XPU, and every other prior dispatch path unchanged. The ROCm Triton path directly loaded a raw shared-mmap host pointer. The reproduced fault address matched the mmap base plus the selected block offset in the first case above the Triton descriptor threshold. Buildkite: V1 Core + KV + Metrics
  • 7f599d7 #46913 — [communication] [bugfix] fix quickreduce acc error in cudagraph mode (#46913)
    • 作者: haoyangli0109 | +154/-8 | 2 个文件

    1. cause: Once flag_color is fixed by graph, it remains unchanged for each round → The written flag value repeats in each round and cannot be distinguished from the residual value of the previous round → The waiting party is prematurely satisfied by the old value and is immediately granted access → At this point, since the data for the current round has not yet been fully transmitted, the syst…

  • eb290ab #49591 — [Bugfix][CPU] Zero-pad MoE intermediate size for grouped-gemm TP alignment (#49591)
    • 作者: Li, Jiang | +211/-14 | 2 个文件

    The CPU AMX/vector grouped-gemm MoE kernels (csrc/cpu/cpu_fused_moe.cpp) tile the expert intermediate dimension in fixed 32-wide blocks with no tail/remainder handling. CPUFusedMOE.check_grouped_gemm only enables the fast kernel when the per-partition moe_intermediate_size (i.e. moe_intermediate_size // tp_size) is a multiple of 32; otherwise it silently falls back to cpu_fused_moe_torch, a per-ex…

  • 8061dc2 #49392 — [Bugfix] Normalize sparse MLA warmup compression ratios (#49392)
    • 作者: Xiaochang Wu | +24/-2 | 2 个文件

    Normalize DeepSeek V4 compression ratios before generating sparse-MLA Triton warmup keys, matching the runtime attention path. DeepSeek V4 configs use 0 to represent uncompressed/SWA-only layers. Runtime attention already converts those values with max(1, ratio), but BuildPrefillChunkMetadataKernel.get_warmup_keys() used the raw config values. This generated a COMPRESS_RATIO=0 specialization conta…

  • 53397fb #49823 — [Bugfix][KV Offload][P2P] Fix EngineCore crash reconnecting to a reaped peer (#49823)
    • 作者: Jason | +167/-13 | 3 个文件

    Fixes #49809. With the OffloadingConnector P2P secondary tier, EngineCore dies with AssertionError: ZmqConnection to :7777 already exists when a request reconnects to a peer whose session was just reaped. Relaxing that assertion is not sufficient. Three distinct defects sit on the same path, and fixing any one of them alone converts the crash into a different crash. ## Root cause ### Defect …

  • 74d3b79 #49429 — [Bugfix] Fix mHC block-M prenorm GEMM cross-row reduction carry-over (#49429)
    • 作者: Nick Hill | +8/-6 | 1 个文件

    In hc_prenorm_gemm_block_m_tilelang, T.alloc_var(init=0.0) declared inside the unrolled i_m loop is hoisted and initialized only once, so the cross-warp reduction accumulator carries over between the rows of a block: every odd output row (and its sqrsum) is the sum of both rows in its block. The block-M kernel is selected for num_tokens >= 1024, so DeepSeek-V4 mHC prefill produces corrupted mixes …

  • 50aa830 #49751 — [BugFix][MRV2] Don’t create dummy requests longer than max_model_len (#49751)
    • 作者: Nick Hill | +72/-7 | 4 个文件

    InputBatch.make_dummy and GPUModelRunner._dummy_run gave every dummy request num_tokens // num_reqs tokens and dumped the entire remainder on the last request, so a single dummy request could have seq_len = query_len up to num_tokens - num_reqs + 1 tokens, far exceeding max_model_len. Such a request cannot be backed by the block tables (width cdiv(max_model_len, block_size), alignment-padded): any…

  • f055388 #48366 — [Bugfix] Prevent NaN poisoning in xpu_mla_sparse for fully-masked index chunks (#48366)
    • 作者: Nick Iusiumbeli | +57/-2 | 2 个文件

    FIX #48364 _bf16_mla_sparse_kernel (the XPU sparse-MLA kernel behind XPU_MLA_SPARSE, DeepSeek-V4 XPU prefill, and the fp8 decode wrapper) NaN-poisons its output whenever the first BLOCK_N (=16) topk index entries of a row are all masked, even though valid keys follow later: - the running max starts at -inf, and a fully-masked chunk sets every logit to -inf; - re_scale = exp2(-inf - -inf) = NaN the…

  • 9e50e10 #49857 — [Bugfix][CuMem] Make KV-cache wake cleanup tag-safe (#49857)
    • 作者: aoshen02 | +3/-0 | 1 个文件

    Release transient Python and accelerator allocator memory before CuMem wake remaps sleeping allocations. ## Problem The weight-update lifecycle can create temporary CUDA tensors during tensor conversion, copying, and post-load processing. After those operations finish, PyTorch’s caching allocator may still retain a large amount of memory as reserved but not currently allocated. This creates a mism…

  • b68d7ef #49438 — [Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#49438)
    • 作者: Jonguk Cheong | +7/-1 | 1 个文件

    Persistent native KV-offload tiers currently fingerprint the unresolved cache_dtype argument. With the default cache_dtype=“auto”, FP16 and BF16 engines for the same model therefore share a filesystem or object-store namespace even though their KV bytes are incompatible. Because both formats have the same page size, a BF16 engine can accept FP16 pages and silently return incorrect output after res…

  • 7154856 #47791 — [Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)
    • 作者: Daniel Socek | +5/-1 | 1 个文件

    PR Purpose Fixes IndexError in kv_postprocess_layout_on_receive() for 5D blocks-first KV caches. kv_postprocess_layout_on_receive() (added in #30275) permutes a received KV cache from HND to NHD layout. It assumes the cache is 4D. But since #42095 is recently merged, non-MLA backends allocate a 5D cache (num_blocks, 2, block_size, H, D). This causes the following error: (This was also flagged a…

  • 0da6e7f #49134 — [Bugfix] Reject contradictory custom-op directives (#49134)
    • 作者: Taneem Ibrahim | +59/-11 | 3 个文件

    CompilationConfig.custom_ops accepted an operation that was both explicitly enabled and disabled, deferring the conflict to a runtime assert in CustomOp.enabled(). The existing all/none conflict check was also an assert; under python -O, both checks disappeared and the invalid configuration ran with the disable directive silently. Fusion settings could also append an enable directive after validat…

  • 5559679 #49052 — [Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)
    • 作者: coltonottley | +1/-0 | 1 个文件

    Fixes #48959. CPU KV offload can fatally abort EngineCore on a valid unaligned sliding-window external load because the scheduler sanity-checks pending physical GPU blocks using a chunk-aligned footprint. For the production geometry: - sliding window: 4,096 tokens; - GPU block: 32 tokens; - offload chunk: 256 tokens (blocks_per_chunk=8); - cached prefix: 98,012 tokens; - valid pending span: 129 GP…

🧪 CI/Tests

  • bc3629b #49732 — [ROCm][CI] Skip three torchao tests of gfx950 until torchao==0.18 is released (#49732)
    • 作者: fxmarty | +25/-0 | 1 个文件

    Submitting from this account as @fxmarty-amd is getting open pull request limit has been reached for this user —- On AMD MI350/gfx950, test_online_quant_config_dict_json, test_online_quant_config_file, and test_reload_weights in tests/quantization/test_torchao.py use Float8DynamicActivationFloat8WeightConfig, which fails on torchao releases up to v0.17.0 because that check only allows is_sm_at_l…

  • 312ea82 #49837 — [CI][ROCm] Make hf-xet reconstruction safe on shared NFS (#49837)
    • 作者: Andreas Karatzas | +13/-0 | 1 个文件
    • Keep Xet and the persistent Hub cache enabled while moving ephemeral Xet state to local storage. - Disable high-performance vectored reconstruction only when HF_HOME is NFS-backed.
  • afc9452 #49939 — [XPU][CI] Use platform device in InputBatch V2 test (#49939)
    • 作者: liuzhenwei | +2/-1 | 1 个文件

    Fix the InputBatch V2 test to use the current platform device instead of hard-coding CUDA. ## Test Result —

  • e099004 #49915 — [CI][ROCm] Reduce kernel test runtime (#49915)
    • 作者: Andreas Karatzas | +101/-37 | 3 个文件
    • Filter unsupported fused RMSNorm parameters during collection and skip the unsupported FP8 scaling module before GPU setup. - Shard MI300 kernel-core coverage three ways and increase Transformers Processing parallelism from four to eight.
  • 5f89a03 #49910 — [CI] Explicitly tear down speculative decode runners (#49910)
    • 作者: Andreas Karatzas | +43/-33 | 1 个文件
    • Run the speculative max-length cases through VllmRunner contexts so each parametrized engine shuts down before the next starts. - Preserve the direct LLM settings explicitly.
  • 29fdeab #49422 — [XPU][CI] Add more test cases in Intel GPU CI (#49422)
    • 作者: xiangdong | +219/-1 | 5 个文件

    Add more test cases in Intel GPU CI ## Test Result —

  • ff61739 #49895 — [CI] Add kimi and k3 auto-labeling rules (#49895)
    • 作者: jcotant-inferact | +44/-0 | 2 个文件

    Adds auto-labeling for Moonshot’s Kimi model family, in both places vLLM does labeling: - .github/mergify.yml — a label-kimi rule for PRs, plus a temporary label-k3 rule for K3 launch triage. - .github/workflows/issue_autolabel.yml — matching kimi and k3 entries in labelConfig for issues. Kimi is now a sizable surface in the tree (22 files across vllm/model_executor/models/, vllm/parser/, …

  • da99ffc #49516 — [ROCm][CI] Keep native datasets cache off shared NFS (#49516)
    • 作者: Andreas Karatzas | +6/-2 | 1 个文件
    • Keep persistent Hub blobs in HF_HOME while placing the processed Datasets cache in pod-local storage. - Avoid POSIX lock failures from the shared NFS-backed cache during native tests.
  • 854c33f #49911 — [CI][ROCm] Keep global GPU memory cleanup opt-in (#49911)
    • 作者: Andreas Karatzas | +3/-7 | 1 个文件
    • Require VLLM_TEST_CLEAN_GPU_MEMORY=1 before enabling the global cleanup fixture; function-scoped cleanup cannot release longer-lived fixtures. - Keep the targeted Qwen-VL and Nixl teardown from #49242.
  • ac87549 #49916 — [CI][ROCm] Reduce V1 attention test runtime (#49916)
    • 作者: Andreas Karatzas | +66/-51 | 2 个文件
    • Remove V1 attention from MI250 and split its MI300 and MI355 coverage into two pytest shards. - Classify unsupported MLA prefill combinations during collection and include the ROCm AITER FA prefill adapter.

🦀 Rust Frontend

  • 8040ef2 #49754 — [Frontend] expose stream_interval as req sampling param (#49754)
    • 作者: Walter Beller-Morales | +114/-0 | 5 个文件

    Expose stream_interval as a request time sampling parameter. This is a relatively small change since stream interval is already configured as a server startup command. The benefit of allowing it to be set at request time is that the workload can decide what to prioritize dynamically (shorter stream_interval for interactivity sensitive work, longer stream_interval for throughout/batch workloads) in…