Meta AI Updates: September 3, 2026

1. PyTorch 2.14 Lands NVGEMM, a nccl2 Backend, and First-Class Fault Tolerance

Meta. The release brings CuTeDSL-generated CUTLASS kernels into Inductor via NVGEMM, with epilogue fusion, scaled and NVFP4 GEMM, and grouped-reduction epilogues autotuned alongside Triton and ATen. PyTorch Distributed gains a nccl2 backend ported from torchcomms that implements the full collective contract with nonblocking communicators and eager communicator splitting, while fault tolerance becomes a c10d concept in its own right: in-place process-group reconfiguration, one-sided RMA windows, and a Flight Recorder that works across backends instead of only NCCL. Apple Silicon picks up native linear algebra including Jacobi-kernel SVD, eigh, QR and Cholesky; torch.switch generalizes torch.cond to multi-way branching; torch.while_loop can be captured in a CUDA graph; and @dynamic_spec gives declarative dynamic shapes shared across torch.compile, torch.export, and make_fx. Complex-valued tensor support in torch.compile ships as opt-in and experimental. Source