AWS Nitro Sandboxes: How One Nitro Switch Isolates EC2, ECS and EKS Workloads
By Fabio Douek
Human-led, AI-assisted. Research and implementation done with AI tools, led, tested and edited by me. How posts are labeled
Jump to section
Explain (TLDR) like I am...
Imagine every toy in the playroom gets its own little house with a door that locks. Before, the toys all shared one big room, so a naughty toy could reach everyone else's things. Now each one plays in its own house, and the playroom is still the same size.
Some houses also come with a badge that proves exactly which toy lives inside. The grown-up with the cookie jar checks the badge before opening it. The houses do take up a bit of floor space, so fewer toys fit.
Nitro Sandboxes move the boundary between workloads on one instance from shared software to the hardware hypervisor, at no additional charge. For multi-tenant or untrusted code, that changes the residual risk from a shared kernel to a much narrower surface that AWS operates.
Confidential mode adds evidence: a signed measurement of what runs, which an AWS KMS key policy can require before releasing data. Diligence points remain: the pod spec or task definition is still trusted input, network egress is still the customer's to restrict, and the parent can still deny service.
The condition is untrusted code sharing a kernel with neighbours, where one exploit can spread across a host. The treatment is a separate hardware virtual machine per pod, task or workload on the same instance, started in seconds.
In my measurements the boundary held and the attestation worked on all three services. Side effects are capacity, not price: a few hundred MiB of memory per sandbox on ECS and EKS, about 18 MiB on EC2, at most five sandboxes per vCPU, and some host-level settings stop working. GPU workloads are not candidates yet.
For teams running other people's code, or code an AI agent wrote a minute ago, there is a low hum of worry about what one bad workload could reach. This launch quiets some of that, because the neighbour on the same machine is now behind a hardware wall.
It also asks for honesty about what remains. Someone still owns egress rules, image choices and the attestation pipeline, and confidential mode means accepting that operators lose, by default, the ability to exec into a workload. That trade deserves a conversation, not a default.
Picture one rehearsal studio split into soundproof booths. Before, every player shared the room, so one loud amp bled into every mic. Now each player gets a booth with its own walls, and the engineer can even get a signed copy of what was played.
The booths need floor space, so the room fits fewer players, and the sound check takes a few seconds longer per booth. On EC2 you build the booths yourself; on ECS and EKS you just ask for one on the setlist.
The headline is hardware isolation per container, without a separate instance per tenant and without an extra line item. On ECS and EKS it is a single field per task or pod (plus a Nitro-enabled capacity provider on ECS), which is the kind of time to value platform teams can sell internally in a week.
The differentiator for regulated buyers is attestation: a key that only opens for a measured workload. Be precise about the cost, though. It shows up as density and a few lost host features, not on the invoice.
Overview
On 2026-11-09 AWS launched Nitro Sandboxes: lightweight virtual machines that the Nitro hypervisor runs next to an EC2 instance, out of that instance’s own CPU and memory. You can put one around an EKS pod, an ECS task, or anything you boot yourself on EC2. AWS announced it on most Nitro instance types, in all commercial Regions, at no additional charge, though the EKS guide now limits isolated and confidential pods to the US commercial Regions.
The problem it targets is old, and AI agents made it urgent. Containers on the same host share one kernel, so a kernel exploit or a container escape in one workload can reach the others. AWS’s own ECS and EKS docs say so plainly. Until now the AWS-native ways around that were a separate instance per tenant, Fargate, or a VM-per-pod setup that needed bare metal or nested virtualization. Nitro Sandboxes give each pod or task its own kernel, behind the same boundary that separates EC2 instances, while it still shares the instance with its neighbours.
This post, part 1 of 4, explains how the pieces fit and summarizes what I measured. The hands-on parts go deep on each entry point:
- Part 2: EC2. The raw primitive, driven with Firecracker, including attestation and a memory-read test from the parent.
- Part 3: ECS. Isolated and confidential tasks on ECS Managed Instances, mostly from the console.
- Part 4: EKS. Isolated and confidential pods on EKS Auto Mode.
I ran all three on 2026-11-09 and 10 in us-east-1, on one m8g.xlarge (Graviton4, 4 vCPU, 16 GiB) per part (ECS briefly launched a second one when I overfilled the first), with AWS CLI 2.39.7. The short version: the boundary and the attestation work as advertised on all three; the opt-in really is one field per task or pod on ECS and EKS, once ECS has a capacity provider created with Nitro Enclaves on; the cost shows up as memory and density, not dollars; and in several places the docs on November 9 were wrong or silent, which the parts call out one by one. AWS has since revised the docs, the EC2 guide most of all, and they now close some of those gaps; I note below where.
Architecture
Under all three entry points is the same EC2 primitive, built on the Nitro Enclaves plumbing: the instance’s “Nitro Enclave” setting, the nitro_enclaves driver in a 6.18 kernel with Amazon’s sandbox extensions (AL2023 on EC2, where the guide now also points to an Ubuntu PPA; AWS’s Bottlerocket-based images on ECS and EKS), the Nitro Secure Module and its attestation documents, and the same KMS condition keys. What’s new is that the guest is a general-purpose VM. A Nitro Enclave talks to its parent over a single vsock channel and has no network or disk. A sandbox has virtio network and block devices and boots a normal kernel, so it can run an ordinary container image.
Firecracker 1.18, released the same day, can now drive the Nitro hypervisor through /dev/nitro_enclaves instead of KVM. On EC2 you run it yourself. On EKS, AWS runs it for you behind a containerd shim and an in-guest agent, both distributed as signed binaries, not open source. The ECS docs name an ECS sandbox agent inside each guest, but not the VMM that boots it.
Concepts
It’s not nested virtualization
A sandbox is a first-level guest of the Nitro hypervisor, a peer of its parent instance rather than a VM inside it. The EC2 launch wizard makes this concrete: on the m8g.xlarge I used, it showed “Nitro Enclave: Enable” a few fields above “Nested virtualization is not supported with the selected instance type”. Sandboxes worked anyway, because they don’t use it. You don’t need a .metal instance or KVM either. The revised EC2 guide adds that the two don’t mix: you can’t use Nitro Sandboxes on an instance that has nested virtualization enabled.
Modes: three naming schemes, two trust models
Each service names the modes differently, and no page maps them to each other. Here’s what I could establish, by trust model:
| No sandbox | Sandbox, parent trusted | Sandbox, parent untrusted | |
|---|---|---|---|
EC2 (Firecracker enclave key) | (a process or container on the instance) | {"direct_mode": true} | {} (the default) |
ECS (isolationMode) | SHARED (default) | ISOLATED_SANDBOX | CONFIDENTIAL_SANDBOX + attestationPolicy |
EKS (runtimeClassName) | (none) | isolated-sandbox | confidential-sandbox + sandbox.amazonaws.com/policy |
The columns are trust models, not memory models. Only EC2’s direct mode lets the parent read guest memory. The ECS and EKS isolated modes still run on confidential memory underneath (both had a Nitro Secure Module; details below), so “parent trusted” there means the parent keeps exec and a readable disk overlay and no policy of yours is enforced, not that it can read the sandbox’s RAM.
On EC2 the difference between the modes is memory. In part 2 I planted a secret in a guest and searched the parent’s view of it: 1,994 hits in direct mode, none in confidential mode. Direct mode has no Nitro Secure Module.
On ECS and EKS, even the “isolated” mode had a Nitro Secure Module device in every task and pod I checked, and both could produce attestation documents. That points to the confidential memory model underneath both of AWS’s managed modes; the ECS and EKS security pages also say the parent “cannot read or write guest memory” without qualifying the mode. What separates isolated from confidential there is a Cedar policy that you supply and that pins the workload (isolated mode runs a permissive built-in one), plus an encrypted disk overlay:
- Isolated: no policy of your own. The guest runs a permissive built-in one, whose PCR17 (
8d833c2f…) was the same on ECS and EKS. The parent is trusted: you cankubectl exec(I did on EKS) or use ECS Exec (per the docs), any container spec is accepted, and the disk overlay isn’t encrypted. - Confidential: an attestation policy, generated by AWS’s
eks-nitro-sandbox-genpolicyorecs-nitro-sandbox-genpolicy, pins the image digest, command, environment, user and capabilities. It denies exec and stdin, and the agent inside the guest enforces it. Its hash lands in PCR17, which a KMS key policy can require.
Generate that policy off the instance. In confidential mode the instance is the untrusted party, and a compromised one could hand back a permissive policy together with its matching PCR17, which you’d then paste into the key policy. AWS’s CLI tools page says to run the tools “on your workstation or in your build pipeline, not on the instance”. In parts 3 and 4 I ran the generators on the cluster itself as a demo shortcut; in production, run them in CI, in CloudShell, or on a Linux workstation or container.
Attestation in one paragraph
A confidential sandbox can ask its Nitro Secure Module for a document signed by AWS that lists measurements:
| PCR | What it measures |
|---|---|
| PCR0, PCR1, PCR2 | the boot image (kernel, initrd, command line); PCR0 also records launch flags such as CPU overcommit |
| PCR3, PCR4 | the parent’s IAM role and instance ID |
| PCR5, PCR6 | the PCI devices present at boot, and the whole image file (new for sandboxes; the EC2 guide says to pin PCR0, not PCR6, which can change between builds) |
| PCR8 | the certificate that signed the guest image; KMS maps the managed images’ PCR8 to ecs.amazonaws.com or eks.amazonaws.com through PCR8:ServiceName |
| PCR17 | the Cedar policy, on ECS and EKS |
AWS KMS accepts that document as the Recipient of a Decrypt call, checks the PCRs against conditions in the key policy, and returns the plaintext encrypted to a public key inside the document. The caller, which is often the parent, can’t read it; only the guest holding the private key can. I got this working on all three surfaces, including two things the November 9 docs didn’t cover on EC2 (a guest kernel with the NSM driver, and a PCR0 to pin: Firecracker doesn’t report it, so I read it once from a first boot). The EC2 guide’s new attestation page now covers both: it loads the nsm module of the AL2023 kernel, and it computes PCR0 in advance with eif_build. In each case, a changed workload was refused.
What changes compared with a plain container
| Plain container | In a Nitro Sandbox | |
|---|---|---|
| Kernel | the host’s, shared with every neighbour | its own (e.g. 6.18.54 in an EKS sandbox on a 6.18.56 node) |
What nproc and MemTotal show | the whole host (4 vCPU, 16 GiB) | the sandbox (1 vCPU, ~650 MiB for a 512 Mi request) |
hostNetwork / host network mode | allowed | EKS: pod stuck with hostNetwork is not supported for nitro-sandbox pods; ECS: rejected at registration |
hostPID / pidMode: host + privileged | the host’s processes | the sandbox VM’s own systemd and ~170 processes |
| Instance metadata | depends on your blocking | EKS: a surrogate with placement and service-endpoint paths only (no instance ID or credentials); ECS: no answer in my test; EC2: the parent’s IMDS, unless you block it |
| Who can read its memory | root on the host | nobody on the host (confidential memory) |
What doesn’t change
The pod spec or task definition is still trusted input: whoever writes it chooses the image and the mounts. In isolated mode the parent still controls the sandbox, can exec into it and can read its disk overlay. On EC2, the revised guide adds that the parent can read every block device it presents to a sandbox, encrypted EBS volumes included, so confidential data has to be encrypted inside the guest. Network egress is still yours to restrict with security groups or network policy, now enforced on the parent where the guest can’t touch it. And in every mode the parent can deny service or tamper with traffic that isn’t encrypted. Credentials are exposed too, in different ways. On EKS the parent can mint Pod Identity credentials for any pod on the node, so treat a pod’s role as available to the node. On ECS it can inject arbitrary task-role credentials and redirect the task’s AWS calls to an attacker-controlled account.
Density is the real price
There’s no charge for the feature, but each sandbox carries memory that a plain container doesn’t, and an instance holds a fixed number of them:
| Per-sandbox memory overhead | Max sandboxes on m8g.xlarge | |
|---|---|---|
| EC2 (self-managed) | 2 MiB, plus 16 MiB per virtio device in confidential mode | 20 (with cpu_overcommit and the slot budget raised from 4; 3 with dedicated cores) |
| ECS | 262 MiB for a 512 MiB or 1 GiB task, up to 509 MiB for 8 GiB | not measured; CPU ran out first in my test (a fifth 1-vCPU task launched a second instance), and memory binds only for larger tasks |
| EKS | 256 Mi (isolated) or 512 Mi (confidential) RuntimeClass overhead | 20 advertised; 18 to 20 ran in my tests |
Memory is donated whole and isn’t overcommitted (Firecracker’s docs say memory overcommit “is not yet supported”), and the cap is five sandboxes per vCPU, with at most 896 on one instance. On Intel and AMD instances with two threads per core, AWS also documents that usable sandbox compute is about half the advertised vCPUs; Graviton doesn’t have that penalty, which is one reason I used it.
How it compares with AWS Lambda MicroVMs
Earlier this month I covered AWS Lambda MicroVMs, the other new AWS way to run untrusted code in a VM. The EC2 guide itself points people there if they want to run none of the machinery, “with no instances or networking to manage”. They solve different halves of the problem:
| Lambda MicroVMs | Nitro Sandboxes | |
|---|---|---|
| Where it runs | AWS-managed, serverless | on your EC2 instances, ECS Managed Instances or EKS nodes |
| Start | ~1.8 s from a snapshot (my test) | ~2 s on EC2, ~5 s as an EKS pod on a warm node; cold boot every time |
| Suspend and resume with memory | yes | no (the EC2 guide rules out snapshot, pause, resume and live migration) |
| Attestation and KMS gating | not documented | yes, all three entry points |
| Hidden from your own operators | n/a (no parent you run) | yes, in confidential memory mode |
| Pricing | per second of vCPU and memory (pricing post) | no additional charge; you pay for the instances, and density is the cost |
| GPU | no | no |
If your sandboxes sit idle a lot and must come back with their memory, or you don’t want instances at all, Lambda MicroVMs fit better. If you already run ECS or EKS, want to bin-pack untrusted workloads on your own capacity, or need to prove what’s running, Nitro Sandboxes fit better.
Setup
There’s nothing to install for the overview. These are the checks I’d run before picking a path.
Does my instance type support it? Nitro Sandboxes need an instance type that runs Nitro Enclaves, so NitroEnclavesSupport is the first field to check:
aws ec2 describe-instance-types --region us-east-1 \
--instance-types m8g.large m8g.xlarge m7i.large m7i.xlarge \
--query 'InstanceTypes[].[InstanceType,VCpuInfo.DefaultVCpus,NitroEnclavesSupport]' --output table
| m7i.xlarge | 4 | supported |
| m7i.large | 2 | unsupported |
| m8g.xlarge | 4 | supported |
| m8g.large | 2 | supported |
The pattern from the EC2 guide’s table: most current families are supported except their smallest one or two sizes and the .metal sizes. In practice that means .xlarge and up on Intel and on 5th- to 7th-generation AMD (m5a, m6a, m7a and their c and r siblings), and .large and up on Graviton and on 8th-generation AMD (m8a, c8a, r8a). GPU instances appear in the list, but GPU passthrough into a sandbox isn’t available. The two lists aren’t identical, though, and the EKS guide calls sandbox types “a subset” of the Enclaves ones. For CPU families the EC2 guide’s sandbox table matches the Enclaves table today; for accelerated instances it’s shorter (no P5 or Trn2, for example), so check the sandbox table there.
Is my tooling new enough?
aws --version # 2.39.7 or later for the ECS fields (Homebrew had 2.39.6 on November 9)
kubectl version # EKS needs Kubernetes 1.36 or later on the cluster
Which path?
- EKS on Auto Mode: nothing to enable on a cluster in a US commercial Region; set
runtimeClassNameon the pod. On Graviton you need a NodePool of your own, because the built-ingeneral-purposepool is amd64 only. - ECS: only on Managed Instances; create a capacity provider with Nitro Enclaves turned on, then set
isolationModein the task definition. - EC2: enable Nitro Enclave on the instance, reserve a memory pool, and run Firecracker yourself.
Testing
The three hands-on parts each have their own tests. These are the headline numbers side by side, all on m8g.xlarge in us-east-1:
| EC2 | ECS | EKS | |
|---|---|---|---|
| Turn it on | Nitro Enclave: Enable + Firecracker | capacity provider checkbox + isolationMode | runtimeClassName |
| First sandbox, including the instance launch | n/a (manual) | 34 s, task created to RUNNING | 50 s, pod created to Ready (85 s in my first run) |
| Sandbox on a warm instance | to SSH: 1.7 s direct, 1.9 s confidential, ~3.1 to 3.2 s with an AL2023 guest kernel | first task: ~6 s from image pulled to container started (no warm isolated run timed) | 4.4 to 5.4 s to Ready (plain pod: 1.7 to 2.1 s) |
| Memory overhead per sandbox | 2 MiB + 16 MiB per device | 262 to 509 MiB | 256 Mi / 512 Mi |
| Max sandboxes on m8g.xlarge | 20 (with cpu_overcommit) | not measured; CPU ran out first at 1 vCPU per task | 20 advertised; 18 to 20 ran in my tests |
| Instance metadata inside | parent’s IMDS, unless blocked | no answer | surrogate: placement and service-endpoint paths only |
| Attestation + KMS demo | worked (PCR0 pinned) | worked (PCR17 + PCR8:ServiceName pinned) | worked (PCR17 + PCR8:ServiceName pinned) |
| Changed workload | refused by KMS | policy denied Agent::Action::"CreateContainer" | policy denied Agent::Action::"CreateContainer" |
| Where AWS shows the sandbox | nowhere: only the parent’s “Enclaves Support: Enabled” | the task’s “Trusted execution” field and trustedExecutionSummary | the pod’s runtimeClassName, in the console’s Raw view |
EC2 CPUUtilization counts sandbox work | yes: 51% with 2 of 4 vCPUs busy | yes | yes: 26% with 1 of 4 busy |
| Monitoring inside the host sees it | no: the parent’s top stayed 95% idle | Container Insights reports the task, but not its memory overhead | no: node CPU and Container Insights stayed flat, with no records for sandboxed pods; kubectl top pod did see it |
Some of these contradict, or go beyond, the November 9 documentation:
- EC2: a 1 GiB sandbox fails unless you reserve a memory pool at boot; the revised guide now makes the pool mandatory. I ran 80 sandbox vCPUs on a 4-vCPU instance despite the documented five-times cap, which the revised guide still states; the limit that actually applied was 20 sandboxes. A reboot inside the guest left the sandbox running and holding its slot, though the guide says it stops.
- ECS: the console has the feature controls (a checkbox on the capacity provider, an “Isolation mode” control on the task definition) that the docs never mention. The documented metadata surrogate didn’t answer. Rolling updates of sandboxed services created from the console failed with an image pre-ingest error and rolled back, five times out of five, until I turned off the container’s version consistency; services created with the CLI didn’t fail.
- EKS: the RuntimeClass has no toleration, contrary to the launch blog. The service account token is mounted in isolated mode, contrary to the docs. The node advertises 20 sandbox slots, but only 18 to 20 sandbox pods ran in my tests, and I didn’t pin down why. Container Insights had no records for sandboxed pods.
On all three, CloudTrail’s KMS Decrypt record carries the attestation fields, with the PCRs in base64. On EC2, a denied call carried none, so CloudTrail doesn’t show which image KMS refused.
Verdict
Nitro Sandboxes are the most practical answer AWS has shipped for running untrusted code next to trusted code on shared capacity. The boundary is real (I could verify it from the parent’s side on EC2), the attestation works with plain AWS KMS on all three services, and on ECS and EKS the opt-in is genuinely one field per task or pod (once ECS has a capacity provider with Nitro Enclaves on), with no extra charge.
It isn’t free in capacity. Expect a few hundred MiB per sandbox on ECS and EKS, five sandboxes per vCPU at most, no GPU, no hostNetwork, and pods that start in about five seconds instead of two. On ECS and EKS, confidential mode also asks you to give something up: no exec, and a policy to regenerate in CI whenever the spec changes. Check your monitoring too: the instance’s CPU metric counts sandbox work on all three, but tools that read the host, like the parent’s top on EC2 or Container Insights on EKS, don’t see inside.
My recommendation for each path:
- EKS Auto Mode is the smoothest: no setup on the cluster, the best-behaved metadata surrogate, and Kubernetes-native policy generation.
- ECS has the best console experience and the clearest per-task accounting, but it only works on Managed Instances, and rolling updates of console-created services failed until I turned off version consistency, so test your deployment path first.
- EC2 is for teams that want the raw primitive and are ready to run Firecracker, a memory pool and their own attestation pipeline. That’s a platform team’s job, not a checkbox.
If you don’t want instances at all, Lambda MicroVMs remain the managed alternative, with snapshots instead of attestation.