AWS AI Updates: July 7, 2026

1. SageMaker HyperPod adds disaggregated prefill and decode for LLM inference

AWS. Amazon SageMaker HyperPod now supports Disaggregated Prefill and Decode (DPD), which splits the compute-bound prefill phase and the memory-bandwidth-bound decode phase of LLM inference onto separate GPU pools and transfers the key-value cache between them over Elastic Fabric Adapter using GPU-Direct RDMA. Intelligent routing sends long-context requests through the disaggregated path while shorter prompts go straight to decoders, which AWS says delivers more consistent per-token latency under sustained concurrency and higher throughput at strict latency targets. The feature is configured via pdSpec in the InferenceEndpointConfig resource and runs on EFA-capable instances across all regions where HyperPod uses the EKS orchestrator. Source

2. SageMaker Studio adds one-click Hugging Face model deployment and customization

AWS. Amazon SageMaker Studio now integrates directly with Hugging Face, letting practitioners select any supported model and choose “Customize on SageMaker AI” or “Deploy on SageMaker AI” to land in a pre-configured Studio environment with the model loaded. The workflow removes the prior steps of manually configuring environments, setting IAM permissions, and requesting GPU quota increases, and it provisions permissions for serverless customization jobs such as fine-tuning with custom reward functions, model evaluation, and deployment to SageMaker or Bedrock endpoints. Verified customers get default GPU access to G5, G6, and G4dn instances without quota requests, with quota and utilization visible inside Studio, and the feature is available in all AWS Commercial Regions where SageMaker Studio is supported. Source