Meta AI Updates: July 10, 2026

1. PyTorch 2.13 Ships With Memory-Saving Loss, Native Safetensors, and Python 3.15 Support

PyTorch released version 2.13, a release with 3,328 commits from 526 contributors since 2.12. The release adds an nn.LinearCrossEntropyLoss that fuses the final linear projection with cross-entropy to cut peak GPU memory by up to roughly 4x for large-vocabulary language models, lets torch.load read .safetensors files directly without external libraries, and adds binary support for Python 3.15 including the experimental free-threaded 3.15t build. On the distributed side it introduces a new torchcomms communications backend for improved fault tolerance and partial-group recovery, plus a CuTeDSL code-generation backend for Inductor alongside Triton. Source

2. FlexAttention Lands on Apple Silicon and Gains Deterministic Backward on CUDA

PyTorch 2.13 brings FlexAttention to Apple Silicon (MPS) with hand-written Metal kernels for sparse prefill and decode paths, reporting up to about 12.3x speedup over scaled dot-product attention on sparse patterns such as a sliding-window shape that runs in roughly 35ms versus 431ms. The release also migrates many large MPS operations to native Metal to remove per-op compilation overhead. For CUDA, FlexAttention gains an opt-in deterministic backward pass that replaces atomic operations with precomputed write ordering to guarantee bit-for-bit reproducible gradients at less than 1 percent overhead. Source