AWS AI Updates: September 12, 2026

1. SageMaker HyperPod Caches Weights on Local NVMe to Cut Inference Cold Starts by 60%

AWS shipped model caching for SageMaker HyperPod inference, splitting the problem into two independent caches: a weights cache that stages model weights on node-local NVMe so pods read locally instead of pulling from S3 or FSx, and an image cache that pre-pulls the container image to skip the ECR download. AWS benchmarks across models from 57 GB to 145 GB report roughly 60% faster scale-out, with image-pull time down 97%, and pods that land on a cold node fall back to the original source rather than failing. It is configured through a modelCacheConfig block on the HyperPod Inference Operator’s InferenceEndpointConfig or JumpStartModel resource, and is GA in every region where HyperPod runs. Source

2. Bedrock Knowledge Bases Can Now Embed Video and Audio Directly Instead of Transcribing First

Bedrock Managed Knowledge Base added TwelveLabs Marengo 3.0 as an embedding model, which encodes visual scenes, speech, and video cues into 512-dimensional multimodal vectors rather than routing media through transcription and text embeddings. Results carry segment start and end timestamps, so an application can jump to the exact moment in a video that matched the query. The practical difference is retrieval on things transcription never captures, such as a play in game footage or an on-screen diagram in a lecture. Source

3. AWS DevOps Agent Runs Full Investigations Inside a Slack Thread

AWS DevOps Agent now supports bidirectional Slack communication, so on-call engineers can start and steer an investigation by @mentioning the agent in a connected private channel. Agent findings, recommended actions, and team-contributed context all accumulate in one thread, covering questions about AWS resources, system metrics, alarm status, deployment history, and incident patterns. It is available in all commercial regions where DevOps Agent already runs. Source