NVIDIA AI Updates: September 3, 2026

1. NVIDIA Publishes Co-Design Guidelines for Speculative Decoding

NVIDIA. The post gives five concrete rules for sizing speculative decoding rather than treating draft length as a tuning knob: push linear-layer work into compute-bound territory without straining KV cache capacity, set draft length to D = 128/G - 1 for attention-dominated workloads where G is query heads per KV head, align larger draft lengths to the attention kernel’s 128-element tile boundary, expand draft length at minimum latency only when acceptance gains outweigh overhead, and pick the draft mechanism by balancing acceptance length against draft overhead and implementation cost. It compares six mechanisms — external draft models, EAGLE-3, MTP, DFlash, DSpark, and suffix/n-gram — and points to ready-to-run EAGLE-3, DFlash, and DSpark implementations in the Model-Optimizer repository plus SPEED-Bench for measurement. Source

2. NVIDIA Walks Through a Modern CUDA Optimization Pass

NVIDIA. A companion developer post steps through optimizing a kernel with the current CUDA toolchain, showing which tool to reach for at each stage of the profiling and tuning loop. Source