Daily News · 4 min read

AWS AI Updates: October 20, 2026

aws bedrock anthropic claude structured-outputs sagemaker trainium vllm inference pytorch open-source hugging-face

1. Amazon Bedrock Adds Structured Outputs and Strict Tool Use for All Claude Models

AWS. Amazon Bedrock now supports structured outputs for all Anthropic Claude models on both the bedrock-runtime endpoint (InvokeModel and Converse via Boto3) and the bedrock-mantle endpoint (Messages API). Bedrock uses constrained decoding to enforce a supplied JSON schema during token generation, so responses always parse and required fields are always present; on InvokeModel the schema goes in output_config.format. Setting strict: true on a tool definition constrains tool inputs to the declared schema, which AWS positions for agentic workflows on Bedrock Mantle that use server-side tools. Compiled grammars are cached per account for 24 hours, and only structural schema changes (not name or description edits) trigger recompilation. Source

2. New SageMaker AI Customers Can Sign Up With Google, GitHub, or Apple and Land in Studio Without a Credit Card

AWS. SageMaker Studio is now integrated with the updated AWS sign-up experience, so new customers can create an account with an existing Amazon, Apple, GitHub, Google, or email identity in two to three steps, without phone verification or (for most customers) a credit card. SageMaker AI then provisions a Studio environment in about 20 seconds and drops the user on the page they started from, such as a pre-configured deployment page when coming from a Hugging Face “Deploy on SageMaker AI” link. New customers get $100 in Free Tier credits usable for training, model customization, notebooks, and endpoints, with a chance to earn another $100; the Free Tier ends after six months or when credits run out, and usage stops automatically at that point. Source

3. AWS Contributes Push-Based and Bi-Directional KV Cache Transfer to vLLM for Disaggregated Inference

AWS. AWS detailed two features it built on Trainium clusters and contributed upstream to vLLM as accelerator-agnostic capabilities for prefill-decode disaggregation over EFA via the NIXL connector. Push mode has the decode instance register destination blocks ahead of time so the prefill instance writes KV blocks as soon as they are ready, which AWS measured at up to 3x better time-to-first-token and up to 30% better time-per-output-token for Qwen3-32B on a 1P1D EC2 P5en deployment. Bi-directional transfer lets decode nodes return previously computed KV to prefill nodes on later turns of a conversation, governed by a configurable recompute threshold, cutting TTFT in multi-turn benchmarks by up to 1.6x for Qwen3-32B and 3.3x for GLM-5-FP8. Both posts include steps to reproduce the benchmarks. Source (bi-directional)

4. Deepgram Ports Nova-3 Speech-to-Text to Trainium2 for 11 to 70 Percent More Audio per Dollar

AWS. Deepgram and the AWS Forward Deployed Engineering team ported Deepgram’s Nova-3 speech-to-text model from standard PyTorch to Trainium2 using TorchNeuron and torch.compile, then tuned it with NKI Library kernels, the Neuron Explorer profiler, and the Neuron Agentic Development toolkit. Hand-written kernels delivered a 2x lower-latency FFN and a 1.5x faster depthwise conv1d, and a from-scratch conv2d kernel unlocked batch sizes above 32. Running eight replicas on a single trn2.3xlarge with a Rust data-parallel runtime, the setup processed 11 to 70 percent more audio per dollar than Deepgram’s previous compute on offline batch workloads such as contact-center analytics and medical transcription. Source

5. TorchNeuron Brings a Native PyTorch Backend to Trainium, Entering Public Beta at PyTorch Conference

AWS. AWS introduced TorchNeuron, an open-source (Apache 2.0) native PyTorch backend for Trainium2 and Trainium3 that supports eager mode, an experimental adaptive eager mode, torch.compile, and custom NKI kernels. Models move to the accelerator with .to('neuron'), and FSDP2, tensor parallelism, and pipeline parallelism are supported. The public beta opens October 20 at PyTorch Conference North America via pip install torch-neuronx (source at github.com/aws-neuron/torch-neuronx), and Deepgram reports roughly 60% more audio-seconds per dollar than its existing GPU baselines. In a companion post, Hugging Face said Transformers, Diffusers, Accelerate, PEFT, and TRL now run on Trainium unchanged, with 89% of the top 100 text-generation architectures working out of the box via device_map="neuron" (versus 46% for the XLA-based Optimum Neuron v0.4.5, which it now recommends users move off). Source (Hugging Face)