chore(deps): update dependency flashinfer-python to v0.7.0

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
flashinfer-python	==0.6.16.post3 → ==0.7.0
flashinfer-python	==0.6.18 → ==0.7.0

---

Release Notes

flashinfer-ai/flashinfer (flashinfer-python)

v0.7.0

Compare Source

These highlights are also published at flashinfer.ai/releases.

v0.7.0 Highlights

FlashInfer 0.7.0 makes the unified mixture-of-experts (MoE) API official, brings TRT-LLM Gen MoE kernels into readable Python source with PrimTS, and introduces Autotuner v2 and a formal experimental-API policy. It also expands sparse and linear attention, expert-parallel serving, and diffusion workloads across Blackwell GPUs.

Read the v0.7 overview and the accompanying deep dives on MegaMoE, Autotuner v2, and the experimental path.

Unified MoE API is official

MoELayer is now an official FlashInfer API, with the lower-level kernel entry points supported alongside it. QuantConfig gives weights, activations, and output explicit QuantFormat fields: QuantConfig(weight=QuantFormat.MXFP4) selects MXFP4 weights with BF16 activations, while adding activation=QuantFormat.MXFP8 selects W4A8. Backend coverage expands with quantization-specific CUTLASS runners, cuTile BF16 and NVFP4 MoE, and CuTe-DSL BF16 MoE on Hopper.

• #​5200
• #​4952
• #​5061
• #​4610
• #​4646
• #​4878

TRT-LLM Gen MoE kernels as Python source with PrimTS

The experimental PrimTS MoE backend exposes kernels built with the CUTLASS DSL Primitives and Task Scheduling APIs, making the expert GEMMs available as readable Python source. It supports BF16, per-tensor and block-scaled FP8, and NVFP4/MXFP4 combinations, while reusing TRT-LLM Gen routing and finalization. Accuracy qualification targets B200; B300 qualification is still pending, and supported configurations have backend-specific restrictions. PrimTS attention also gains paged block-sparse attention, variable-window attention, and a unified plan()/run() contract for reusable wrappers.

• #​4361
• #​4474
• #​4599
• #​4829

Autotuner v2 tunes the way you serve

autotune_v2() lets applications measure candidates in eager or CUDA-graph execution and persist the results in an environment-specific cache managed by FlashInfer. Atomic cache entries support concurrent rank writes, and autotune_v2_reload() lets homogeneous ranks converge on shared results. In the reported vLLM validation, Qwen3-8B-FP8 at TP2 on B200 reduced its tuning window from 128 seconds on a cold start to 1 second on restart with the same cache; total startup was 410.7 seconds and 165.3 seconds, respectively. Applications opt in through the new API; autotune() remains available.

• #​3861

Experimental APIs and backends have an explicit opt-in path

Experimental APIs are marked with @flashinfer_experimental_api, and experimental backends have a dedicated flashinfer.experimental namespace. Calling an experimental API or explicitly selecting a marked backend emits a warning; automatic selection includes marked experimental backends only when FLASHINFER_ALLOW_EXPERIMENTAL_AUTO_BACKENDS=1 is set. The policy defines admission and graduation criteria and keeps experimental implementations JIT-only, outside prebuilt packages.

• #​4880

Expert-parallel MoE expands across Blackwell

moe_ep adds an unquantized BF16 MegaMoE backend and a W4A8 split backend on B200, with MXFP8-packed dispatch for the latter. RTX PRO 6000 and DGX Spark gain an MXFP8 MegaMoE backend with functional correctness validated and performance tuning ongoing. The NCCL-EP split path supports CUDA-graph capture through a reusable handle with per-step update(), validated through vLLM on four B200 GPUs.

• #​4386
• #​4529
• #​4387
• #​4795

DeepSeek-V4 and MiniMax-M3 sparse attention

Blackwell gains paged FP8/MXFP4 indexer logits and more top-K choices, including a CUB backend with variable-length support. DeepSeek-V4 Flash sparse MLA supports an NVFP4 KV cache on SM120/SM121 through kv_cache_format="nvfp4". MiniMax-M3 sparse attention gains source-distributed CAKE-generated kernels on SM100/SM103; the B200 kernel benchmark reports a 2.45× geometric-mean speedup over the MiniMax baseline across 11 comparable prefill, decode, speculative, and boundary cases.

• #​4365
• #​4811
• #​4621
• #​4442
• #​4955
• #​4355

Linear attention for Kimi K3, Qwen 3.6, and Nemotron-H

Kimi K3 gains a speculative-verification entry point compatible with vLLM's recurrent verifier, plus CuTe-DSL recurrent prefill on B200/B300 and SM120. The experimental RecurrentKDAPrefillWrapper supports planning and graph-safe prefix checkpoints. Qwen 3.6 gains experimental fused GDN decode steps on SM120 that combine projection, convolution, gating, and recurrent state updates. Nemotron-H gains source-built Mamba SSD-combined and selective-state-update backends on B200/GB300.

• #​4709
• #​4605
• #​4633
• #​5021
• #​5040
• #​4481
• #​4708
• #​4576
• #​4616

Diffusion and video attention on Blackwell

Video Sparse Attention gains generated SM100/SM103 kernels, and the block-64 path adds a native CuTe-DSL implementation with Sage FP8 support. SM120 gains a generated Sage block-sparse backend and an optimized NVFP4 attention path for Cosmos workloads. MiniMax-H3 gains a prepared MXFP8 pre-attention pipeline on B200/B300, while PrimTS block-sparse attention adds proxy compensation for Sol-Attn workloads.

• #​4593
• #​4612
• #​4951
• #​4502
• #​5060
• #​4872

Communication for PCIe and NVLink deployments

PcieIpcAllReduceWorkspace adds an intra-node CUDA-IPC all-reduce for two, four, or eight ranks on PCIe machines without NVLink. Blackwell fused all-gather matmul gains a backend="cake" option and a prepared callable for packed-QKV workloads, with tensor parallelism up to eight GPUs.

• #​4393
• #​4722
• #​4822

Notice: packaging and dependencies

flashinfer-jit-cache is now a small shim that depends on architecture-specific provider wheels. The usual installation command installs the provider set for the selected CUDA and CPU platform; to reduce image size, install only the provider needed for your GPU. If you mirror or vendor wheels, include the provider wheels as well as the shim, and keep their CUDA-specific versions aligned.

Dependency	0.6.18.post1	0.7.0
nvidia-cudnn-frontend	>=1.25.0	>=1.29.0
apache-tvm-ffi	>=0.1.6,!=0.1.8,!=0.1.8.post0,<0.2	>=0.1.11,<0.2
NCCL-EP Python packages	nccl4py>=0.3.1	nccl4py>=0.4.1 and nccl-extensions>=0.1.0
nvidia-cutlass-dsl through [cu12] / [cu13]	>=4.6.2a0	>=4.7.0a0

The base nvidia-cutlass-dsl requirement remains >=4.6.2a0; the CUDA extras have the higher floor. Environments pinned to CUTLASS DSL 4.6.2 need compatible dependency pins before installing those extras.

• #​4514
• #​5096
• #​5182
• #​5016

Notice: small-batch FP8 groupwise GEMM

⚠️ The CUTLASS gemm_fp8_nt_groupwise path on SM100/SM103 can intermittently produce incorrect output for M <= 32 with scale_granularity_mnk=(1, 128, 128). This pre-existing issue remains in 0.7.0; see #​4396 for the investigation. Validate affected workloads before deployment. A different backend requires its own supported shapes and scale layout; there is no documented switch to disable only this CUTLASS fast path.

Notice: API removals and behavior changes

Update callers of the removed APIs before upgrading:

Removed	Replacement
comm.trtllm_custom_all_reduce	comm.trtllm_allreduce_fusion
comm.trtllm_create_ipc_workspace_for_all_reduce	comm.trtllm_create_ipc_workspace_for_all_reduce_fusion
BatchDecodeMlaWithPagedKVCacheWrapper	mla.BatchMLAPagedAttentionWrapper
No-op end_forward() methods on decode, prefill, sparse, cascade, and POD wrappers	Remove the call
fused_moe.QuantVariant, QuantConfig(variant=...), QuantConfig.from_variant	QuantConfig(weight=..., activation=...) using QuantFormat

mamba.checkpointing_ssu replaces the per-step old_x, old_B, old_dt, old_cumAdt, and cache_buf_idx arguments with ring-buffer caches (x_cache, B_cache, dt_cache, ring_start). Update cache allocation and argument binding; the caller-provided out tensor remains required.

For the TRT-LLM backend of BatchPrefillWithRaggedKVCacheWrapper, skip_all_rows_active_check defaults to True. Calls on the default fast path must have positive query and KV lengths for every row; pass False to request device-derived checking when CPU length mirrors are absent.

gated_delta_rule_mtp still resolves an omitted disable_state_update to True in 0.7.0. Pass disable_state_update=True or False explicitly to select the intended behavior and suppress the stale warning announcing a change in this release.

• #​5218
• #​5061
• #​4545
• #​5039

What's Changed

• fix(sampling): reject unsafe multi-CTA top-k launches on low-SM GPUs by @​bkryu in #​4595
• feat(cake_kda): optimize small-BH recurrent-KDA prefill by @​yyihuang in #​4571
• test: enable the unified MoE fuzzer by default and prune legacy UTs by @​feih-nv in #​4475
• feat(gdn): add pooled state and state checkpointing and dtype support for feature parity by @​guangyunh-nv in #​4436
• feat(attention): add PrimTS Q64/KV256 and paged GQA block-sparse attention by @​heyuhhh in #​4474
• add backward-compatible aliases for bsa_attn_fwd and bsa_attn_blk64_fwd by @​hsr1234563 in #​4590
• fix(moe): restore SM12x MoE kernels broken by self-resolved helper in borrowed dense methods by @​lucifer1004 in #​4602
• Add @​Anerudhan to CODEOWNERS for core review by @​aleozlx in #​4622
• fix(moe_ep): fix in_kernel_fc2_reduce livelock on zero-token launches (MXFP8 + NVFP4) by @​mhoqueanik in #​4531
• feat(moe_ep): SM100 BF16 CuTeDSL MegaMoE kernel by @​mhoqueanik in #​4386
• feat: CuTe DSL kernels for Rubin (SM107) and batched FP8 GEMM for Blackwell by @​Vinnie6167 in #​4526
• Add back TRTLLM Gen MoE split-K by @​jiahanc in #​4617
• feat(kda): add CuTe DSL recurrent prefill backend by @​Observer007 in #​4605
• feat(cake_backend): accelerate DeepSeek fused routing by @​yyihuang in #​4587
• Stop legacy nightly release publishing by @​dierksen in #​4623
• feat(cake_backend): add Blackwell Router GEMM by @​yyihuang in #​4594
• [feat]custom all reduce kernel by @​qsang-nv in #​4393
• fix(attention): handle extreme negative logits in masked softmax by @​shoutoutuoadi325 in #​4401
• feat: trtllm-gen FMHA features for sm107 (spcompress, fp16softmax) by @​jimmyzho in #​4596
• feat(moe_ep): SM100 W4A8 (MXFP8xMXFP4) CuTeDSL split kernel backend with MXFP8 packed dispatch by @​mhoqueanik in #​4529
• Add Qwen fused GDN decode step for sm120 by @​nv-yunzheq in #​4481
• feat: collect a union of TEST_PATH targets in unit CI by @​kahyunnam in #​4641
• fix(moe): correct unified fuzzer references by @​feih-nv in #​4639
• [perf] split TRT-LLM Gen routing kernels to reduce compile time by @​jiahanc in #​4635
• feat(moe): add SiTU-GLU activation to the CUTLASS fused-MoE backend by @​xuanyu-mistral in #​4460
• feat(cake_mamba): add Blackwell selective state update backend by @​yyihuang in #​4616
• ci: consolidate CUDA coverage and validate candidate images by @​dierksen in #​4469
• feat(kda): add SM120a CuTe DSL prefill backend by @​JimpleMa in #​4633
• feat(moe): standalone trtllm-gen routing op + decomposed tests/moe routing matrix by @​aleozlx in #​4082
• feat(cake_vsa): add optimized SM100/SM103 block-sparse attention (VSA) by @​yyihuang in #​4593
• feat(cake_kda): add paired recurrent training for SM100a and SM103a by @​yyihuang in #​4636
• perf(gemm): optimize CuTe DSL W4A16 dense GEMM by @​zianglih in #​4686
• fix(sm120): align MXFP8 plain tactic and FP8 moe stage policy by @​CarstyYou in #​4660
• feat(moe): align unified MoE do_finalize behavior with flat API by @​feih-nv in #​4614
• perf(gdn): reduce non-CP CuTeDSL launch overhead by @​guangyunh-nv in #​4699
• feat(cake_mamba): add Blackwell Mamba SSDCombined by @​yyihuang in #​4576
• feat: SM120 NVFP4 SVDQuant Gemm in CuteDSL by @​rosenrodt in #​4420
• ci: disable sccache for cu134 nvcc by @​dierksen in #​4682
• Add paged MQA logits (attn_scores) kernels for Blackwell SM100 by @​dhiraj113 in #​4365
• perf(sm120): optimize NVFP4 attention with N64 score-slot reuse by @​tiffany940107 in #​4502
• feat(cake_msa): add Blackwell minimax sparse attention source kernels by @​yyihuang in #​4355
• perf(cake_kda): further optimize recurrent-KDA prefill on Blackwell by @​yyihuang in #​4675
• feat(moe): allow B12xMoEWrapper to share pre-allocated workspaces by @​lucifer1004 in #​4603
• (perf) add fused_GDN_step support for Qwen 3.6 35B A3B on sm120 by @​nv-yunzheq in #​4708
• fix: SageAttention support block size doesn't divide sequence; support K-smoothing by @​xrq-phys in #​4654
• ci: coordinate CUDA dependency policy by @​dierksen in #​4711
• perf(msa): chunked top-k and in-kernel causal offsets for the SM12x indexer by @​yichengj0 in #​4030
• perf(cake_kda): add recurrence-piece persistent M128 prefill by @​yyihuang in #​4728
• feat(moe): add remaining CUTLASS unified MoE runners by @​feih-nv in #​4610
• fix: Correctly wire scale_qkvo to cute-dsl fmha backends by @​xrq-phys in #​4665
• fix(kda): fall back to Cake when CuTe DSL predates cutlass.experimental by @​kahyunnam in #​4667
• fix(ci): skip source-only CUDA config test in nightlies by @​dierksen in #​4750
• perf(activation): cap act_and_mul_kernel block size for ~17-19% speedup at large hidden dims by @​yekerr in #​4733
• fix(fmha): select CGA reduction for MLA H512 decode by @​yihwang-nv in #​4702
• feat(norm): fused Add+RMSNorm+1x128 fp8 block-quant producer by @​NVShreyas in #​4480
• chore(aot): exclude single prefill/decode modules from jit-cache prebuilds by @​bkryu in #​4760
• build(jit): reduce JIT-cache wheel size by @​dierksen in #​4757
• feat(moe): add unified activation parity by @​feih-nv in #​4613
• Support per-token NVFP4 ReLU2 MoE by @​xuantengh in #​4618
• feat: CUB DeviceBatchedTopK top-k backend with variable-length support by @​NaderAlAwar in #​4442
• feat(cake_fmha): add native Blackwell DCP speculative decode by @​yyihuang in #​4518
• fix(cute_dsl): consult the arch gate in the GEMM and GDN dispatchers by @​Vinnie6167 in #​4649
• feat(cake_activation): add fused Blackwell SwiGLU MXFP8 quantization by @​yyihuang in #​4638
• Adding var-window FMHA context support for PrimsTS. by @​mingxu1067 in #​4599
• Add bias support to cublast gemm backend and as fallback backend for cutedsl gemm backend by @​jiahanc in #​4772
• misc: multi-arch cubins (sm100, 103, 107) in a single artifact by @​jimmyzho in #​4648
• refactor(mla): isolate planned FA2, FA3, and CUTLASS backends by @​saltyminty in #​4697
• Enable CuTe DSL MLA benchmarks for low head counts by @​lunarz-dev in #​4656
• fix(gemm): never move the shared cuDNN GEMM workspace by @​yanqinz2 in #​4666
• feat(cake_kda): add recurrent training template dispatch by @​yyihuang in #​4726
• fix(norm): limit add RMSNorm FP4 launch config heuristics to SM100 and SM103 by @​soodoshll in #​4494
• perf(cake_vsa): refresh Blackwell block-sparse WS kernel by @​yyihuang in #​4804
• feat(decode): add prims-ts backend and is_causal to paged decode by @​elwhyjay in #​4739
• [Bugfix] Skip .item() readback for trtllm_ragged_attention_deepseek during CUDA graph capture (#​4609) by @​zhang-keliang in #​4703
• feat(cake_comm): Add a Cake Blackwell all-gather matmul backend by @​yyihuang in #​4722
• feat(cake_bgmv): add deterministic prepared Blackwell MoE backend by @​yyihuang in #​4821
• feat(cake_backend): add grouped MXFP8 quantization by @​yyihuang in #​4820
• feat(attention): add variant_owns_mask for JIT variants that own the full mask by @​lesj0610 in #​4695
• feat(cake_kda): add native unbounded-softplus Kimi-Linear kernels by @​yyihuang in #​4535
• feat(cute_dsl): replace SM100 blk64 BSA C++/CUTLASS kernel with CuTe-DSL by @​hsr1234563 in #​4612
• feat(mla): support variable-Q decode with DCP in CuTeDSL by @​Observer007 in #​4719
• fix: support class method re-exports in API checker by @​saltyminty in #​4717
• perf(sm12x): optimize and unify static MoE by @​EricChen02 in #​4718
• fix(attention): make PrimTS paged block-sparse metadata live by @​heyuhhh in #​4664
• feat(MoE): FP8 MoE per-channel quant support by @​raayandhar in #​2809
• fix(ci): skip subprocess torch.compile tests when kernels are absent from jit-cache by @​bkryu in #​4783
• Unify MoE CuTe DSL dispatch to be dtype agnostic by @​PetersonGuo in #​4793
• fix(cute_dsl): make the optional-dependency guard independent of cutlass by @​Vinnie6167 in #​4753
• perf(attention): cut NVFP4 KV dequant instructions on targets without native FP4/FP8 conversion by @​lesj0610 in #​4746
• feat(moe): add cuTile fused MoE backend for BF16 and NVFP4 Unified MoE by @​bkryu in #​4646
• fix: skip SM107 low-latency GEMM cubins on Blackwell (#​4773) by @​kahyunnam in #​4848
• feat(prims_ts): support no-padding MLA query rows by @​PerkzZheng in #​4830
• Fix TRTLLM MOE per-token NVFP4 TileN 192 and update cubins by @​jiahanc in #​4741
• fix(benchmark): align MLA metadata with backend contracts by @​lunarz-dev in #​4835
• Add PR labeler rules for op: linear attention and op: misc by @​kahyunnam in #​4852
• [bug fix] Issue 3700 sparse mla sm121 hang by @​kahyunnam in #​4732
• feat(jit): log cold nvcc module builds by @​mgoin in #​4844
• docs: fix 0.6.19 document check failures by @​kangbintNV in #​4871
• Rubin open issue fixes by @​Vinnie6167 in #​4787
• fix(attention): count NVFP4 scale-factor staging in the batch smem budget by @​lesj0610 in #​4767
• fix(attention): scope the NVFP4 split-KV workaround to the arch it was found on by @​lesj0610 in #​4747
• fix: honor sparse MLA KV page strides by @​bvolpato in #​4362
• feat(gemm): enable tinygemm2_sm100 on SM107 (Rubin) by @​Vinnie6167 in #​4849
• test(moe): generate activation matrix and add backend contract fuzzers by @​feih-nv in #​4805
• refactor(moe): bind TRTLLM launch state to each call by @​feih-nv in #​4797
• fix(cudnn): return base-2 LSE from prefill (+ paged head_dim_qk != head_dim_vo O offset) by @​egilliam-nv in #​4663
• feat: Add cute-dsl-prims backend for SM120 FP8 GQA prefill by @​Tom-Zheng in #​4714
• Enable top_k_varlen on Rubin (SM107) and add the DKG filtered-radix backend (radix_filter) by @​dhiraj113 in #​4621
• chore: update TRTLLM-Gen GEMM cubins by @​bobboli in #​4840
• feat(moe): add CuTe DSL SM90 BF16 MoE backend by @​Aneureka in #​4878
• fix(gemm,moe): restore bmm_fp8 auto fallback and drop an over-strict SM107 tactic guard by @​Vinnie6167 in #​4853
• docs(moe): regenerate the Unified MoE activation matrix (unblocks CI on main) by @​Anerudhan in #​4903
• feat(moe): enable tile_size=256 for Rubin MoE autotuning by @​Vinnie6167 in #​4851
• perf(sparse mla): Update sparse MLA half-Q-tile heuristic to apply to short queries only by @​emascarenhas-nv in #​4752
• fix(gdn): stream-scope WY staging buffers and prefill launch workspace by @​elwhyjay in #​4476
• fix(moe): disable PDL in the trtllm-gen fused-MoE path on SM107 (Rubin) by @​Vinnie6167 in #​4806
• feat(moe_ep): SM120 MXFP8 swap-AB CuTeDSL MegaMoE kernel by @​mhoqueanik in #​4387
• feat(moe_ep): make the nccl_ep split path CUDA-graph capturable by @​Anerudhan in #​4795
• feat(gemm): add router GEMM ops for Kimi-K2 / Kimi-K3 and bf16 output by @​Vinnie6167 in #​4630
• feat(prims_ts): accept SM107 (Rubin) in the attention-ts device gate by @​Vinnie6167 in #​4755
• feat(moe): expand cuTile fused MoE activation support by @​bkryu in #​4888
• fix(gemm): remove SM120 MXFP8 tile configs that exceed the SM12x shared-memory limit by @​yichengj0 in #​4013
• Add CuteDSL Low Latency GEMM tactic to mm_fp4, mm_fp8, mm_mxfp8, tgv_gemm_sm100 backends (SM100/SM103) by @​PetersonGuo in #​4685
• feat(cake_warp_decode): add SM103 NVFP4 warp-decode backend by @​yyihuang in #​4855
• feat(cake_comm): extend Blackwell all-gather matmul to TP8 by @​yyihuang in #​4822
• Summary improvement by @​righthandabacus in #​4895
• fix(kda): accept SM107 (Rubin) in the FlashKDA family guards by @​Vinnie6167 in #​4710
• ci: enable patched sccache for CUDA 13.4 by @​dierksen in #​4808
• Modify CODEOWNERS - Anik by @​aleozlx in #​4937
• feat(kda): output-only KDA decode + drop-in for vLLM Kimi K3 recurren… by @​ameynaik-hub in #​4709
• Refactor Sparse MLA SM120 by @​lucifer1004 in #​4802
• Add ReplaySSM autotuning by @​askliar in #​4545
• feat(attention): cuTile paged/ragged prefill + paged/MLA decode by @​yifeis-nv in #​4018
• docs: teach agents the PR guidelines in CLAUDE.md by @​aleozlx in #​4948
• [feat] Add warp level split k bf16 gemm by @​jiahanc in #​4908
• feat: add experimental API/backend policy, @flashinfer_experimental_api, and flashinfer.experimental namespace by @​bkryu in #​4880
• fix(ci): publish the test scope with GITHUB_TOKEN, not the bot PAT by @​aleozlx in #​4962
• test: skip MonoMoE on GPUs with fewer than 128 SMs by @​cindyzxq in #​4954
• feat(topk): self-sampling GVR V2 backend (SM100/103/107), oracle-tracking auto, and the V1 threshold repair by @​dhiraj113 in #​4811
• ci: provision cuTile compiler in CUDA 13 images by @​saltyminty in #​4939
• Add option to skip checking active rows in TRTLLM ragged prefill by @​wzhao18 in #​4931
• fix(ci): anchor bot command matching to line start, outside code fences by @​aleozlx in #​4956
• feat(autotuner): Autotuner v2 (autotune_v2) — managed persistence, deployment-matched measurement, runner contract by @​YangXu1990uiuc in #​3861
• feat(cake_concat_mla): add SM100/SM103 concat MLA K backend by @​Kathryn-cat in #​4860
• feat(cake_kda): add fp32 state & long context kda prefill kernels by @​yyihuang in #​4845
• feat(cake_sage): add SM120 block-sparse attention backend by @​yyihuang in #​4951
• feat(prims-ts): support proxy-compensated block-sparse attention by @​heyuhhh in #​4872
• feat(prims-ts): support decode GQA ratios up to 128 by @​PerkzZheng in #​4915
• feat(mla): expose DSv4 TRTLLM-GEN RopeQuant by @​PerkzZheng in #​4918
• fix(moe): address review findings for PR #​4610 by @​cindyzxq in #​4914
• perf(attention): repack NVFP4 KV tiles to 16-bit on targets without a native E2M1 convert by @​lesj0610 in #​4769
• feat(sm120): add NVFP4 sparse MLA support for DeepSeek V4 Flash by @​tiffany940107 in #​4955
• Refactor SM120 grouped GEMM as SM12x GEMM by @​CarstyYou in #​4838
• feat: add NVFP4 quantize-append path for the MLA paged KV cache by @​elwhyjay in #​4676
• docs: resolve documentation check failures by @​kangbintNV in #​4927
• perf(moe): phase NVFP4 dispatch payload copies at EP4/EP8/EP16 by @​samnordmann in #​4067
• fix(docs): deploy stable documentation from release tags by @​cindyzxq in #​4945
• fix(kda): compile direct serving modules as C++20 by @​migarci2 in #​5017
• feat(cake_mega_moe): add optimized MXFP8 MegaMoE EP16 backend by @​hzfan in #​4970
• feat: Add valid_hidden_size and valid_intermediate_size params t… by @​aleozlx in #​2482
• Nccl extensions dependency update by @​Anerudhan in #​5016
• fix(trace): model the SM100/103 cute-dsl mm_bf16_fp4 prepared layout by @​kahyunnam in #​4620
• fix(mla): scope NoPE sparse_mla_top_k_lens requirement to non-sparse backends by @​lucifer1004 in #​4947
• fix(moe): unify the NVFP4 profiler workspace predicate and fix gated fc1 SF sizing by @​yichengj0 in #​4010
• feat(gdn): adopt the CuTe-DSL disk cache for TVM-FFI GDN kernels by @​elwhyjay in #​4912
• fix(gdn): support WY ucache kernels on SM121 by @​hebo1221 in #​4528
• feat(cake_gdn): add gdn prefill decode mtp Cake backend for SM100 and SM103 by @​yyihuang in #​4581
• perf(prims-ts): Optimize&refine PrimsTS block sparse attention by @​heyuhhh in #​5002
• feat(attention): opt-in LDTM.STAT for PrimTS context row_max by @​kris1025 in #​4839
• chore: remove unused benchmark test by @​yufeiwu-nv in #​5046
• feat(moe): wire unified UnpackedPrecomputed for TRTLLM BF16 and block-FP8 by @​feih-nv in #​4909
• feat(kda): support graph-safe CuTe DSL prefix checkpoints by @​Observer007 in #​5021
• feat(benchmarks): measure TRTLLM ragged row-check overhead by @​yufeiwu-nv in #​4998
• test(kda): restore the frozen module-ident consistency check by @​kahyunnam in #​5035
• feat(cake_gqa): Add an experimental SM110 GQA decode kernel by @​yyihuang in #​5052
• feat(moe): split QuantConfig into weight/activation/output format axes by @​feih-nv in #​4952
• fix(decode): reserve max(K+V, st.o) smem for FA2 FP8+GQA decode by @​ir1ka in #​5038
• fix: apply calibration scales to ragged FP8 KV prefill by @​yuchenwang3 in #​4984
• perf(attention): Refresh DSL_FMHA cubins by @​xrq-phys in #​4997
• fix(bench): quote benchmark CSV fields by @​lunarz-dev in #​5045
• fix(attention): enable SM121 FMHA v2 prefill by @​aeichler-ac in #​4661
• fix(mla): prevent intermittent hang in SM120 sparse-MLA swapAB prefill on DGX Spark by @​bkryu in #​5048
• refactor(attention): unify PrimTS plan/run and page-table contracts by @​yuxianq in #​4829
• Replayssm prefix materialize by @​hnover-nv in #​4815
• fix(bench): use float32 cos_sin_cache in apply_rope_with_cos_sin_cache by @​aeichler-ac in #​5066
• feat(cake_kernel): add MiniMax-H3 MXFP8 pre-attention for SM100a/SM103a by @​yyihuang in #​5060
• fix(jit): avoid invalidating bgmv_moe cache on source staging by @​200lz in #​4798
• ci: log installed Python packages before tests by @​dierksen in #​5062
• fix(jit): preserve source mtimes when staging rmsnorm_silu and monomoe sources by @​bkryu in #​5063
• fix(kda): make recurrent_kda backend="auto" decode fall back to CuTe DSL by @​kahyunnam in #​5037
• feat: Test runner summary prints cudnn backend version by @​righthandabacus in #​5068
• feat(jit-cache): split cache into architecture provider wheels by @​dierksen in #​4514
• bump version to 0.7.0 by @​aleozlx in #​4938
• fix(release): drop the stale diffusion_ops.minimax_h3 trace-registry entry by @​aleozlx in #​5216

New Contributors

• @​shoutoutuoadi325 made their first contribution in #​4401
• @​JimpleMa made their first contribution in #​4633
• @​NaderAlAwar made their first contribution in #​4442
• @​mingxu1067 made their first contribution in #​4599
• @​PetersonGuo made their first contribution in #​4793
• @​bvolpato made their first contribution in #​4362
• @​emascarenhas-nv made their first contribution in #​4752
• @​Kathryn-cat made their first contribution in #​4860
• @​migarci2 made their first contribution in #​5017
• @​kris1025 made their first contribution in #​4839
• @​yufeiwu-nv made their first contribution in [#​5046](https://redirect.github.com/flash

❗ Important

✂ PR body was truncated to here.

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[renovate]

Renovate Ignore Notification

Because you closed this PR without merging, Renovate will ignore this update (==0.7.0). You will get a PR once a newer version is released. To ignore this dependency forever, add it to the ignoreDeps array of your Renovate config.

If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR.