Monday, November 16, 2026 Edition No. 240
Search
Review · Infrastructure · 42 min read

OpenSandbox Sandbox Lifecycle: Hands-On on Amazon EKS

By Fabio Douek

Jump to section
Explain (TLDR) like I am...
What is this?

Imagine a robot helper that works in a little room full of toys. At nap time, someone takes a photo of the room and puts it away, and when the robot wakes up, the room is built again from the photo. The toys come back, but whatever the robot was in the middle of doing is gone, and the guard at the door has forgotten every key.

If someone knocks the room down by accident, a new empty room appears right away, and nobody tells the robot its toys are missing. A room with thicker walls of its own works too. It just takes a little longer to build.

This post tests what happens to an AI agent's sandbox over its life in OpenSandbox 1.2.0, an Apache 2.0 project of the Agentic AI Foundation, on Amazon EKS: expiry, pause and resume, warm pools, a virtual machine runtime and failures. A pause saves the sandbox's files to the organization's own registry. Memory, running processes and stored credentials are not kept.

The material findings concern loss that nobody is told about. A deleted sandbox pod is replaced at once with a clean copy while the platform keeps reporting it as running. And nothing pauses an idle container sandbox, so it holds its capacity until its timeout.

The presenting question was what a sandbox keeps and what it loses over its life on Amazon EKS. I took OpenSandbox 1.2.0 through creation, expiry, pause and resume through a registry, warm pools, Kata virtual machines and deliberate failures, and recorded every run.

Findings: files survive a pause; memory, processes and the Credential Vault do not. A deleted pod was replaced within seconds by one with a fresh filesystem while the sandbox still reported healthy, a silent loss. Under Kata, the sandbox I inspected booted its own kernel with two CPUs, and a create took 4.2 seconds against 2.2 under the default runtime.

Pause and resume sounds like a promise that nothing is lost, and part of this work is adjusting that expectation gently. The files come back. What the sandbox was doing, what it held in memory, and the credentials it was given do not, so whoever resumes it has to hand the credentials over again.

The harder moment was a deleted pod. The platform replaced it within seconds and went on saying all was well, while the files written before were gone. It helps to know that in advance, and to keep what matters in storage outside the sandbox, rather than find out halfway through a session.

Think of a sandbox's life as a set with a hard stop: it starts on the count, plays until its time runs out, and can be extended mid-song. Pause is a recording of the stage, not of the performance. The gear comes back exactly where it stood, but the notes in the air, the players' place in the tune and the passes at the stage door are gone.

A warm pool is a band already tuned in the wings. Kata gives each act its own soundproof room with its own kernel, at about twice the setup time. And when a pod drops out, a stand-in walks on with blank sheet music while the board still says everything is fine.

The story is a sandbox lifecycle you run in your own AWS account: time limits, pause and resume through Amazon ECR, warm pools of pre-started pods, and a switch to Kata virtual machines, on OpenSandbox 1.2.0, an Agentic AI Foundation project, on Amazon EKS.

The honest positioning comes with caveats buyers will ask about. A pause keeps files, not memory, processes or stored credentials, and nothing pauses an idle container sandbox on its own. A deleted pod comes back empty while the sandbox still looks healthy. Kata gives each sandbox its own kernel at about twice the create time.

GitHub See the source code here: opensandbox-lifecycle

Overview

OpenSandbox has been a project of the Agentic AI Foundation, a directed fund of the Linux Foundation, since the announcement on October 22, 2026; part 1 covers that move.

This post, part 3 of 6, takes sandboxes on the EKS cluster from part 2 through their whole life: create, expiry and renewal, pause and resume through Amazon ECR, warm pools, a switch from runc to Kata virtual machines, and what happens when a pod is deleted or a node is cordoned. One of the six tests is the project’s own resume benchmark, run unmodified.

Three findings shape how I’d build on it:

  • A pause keeps the root filesystem and nothing else. The root filesystem came back from ECR. Memory, processes and the Credential Vault didn’t: the vault is wiped on every pause, so whatever resumes a sandbox has to write it again. That’s the default mode; an experimental QEMU VM-state mode for a QEMU guest inside a runc sandbox also restores the guest’s memory. I didn’t test it.
  • Nothing pauses an idle container sandbox for you. 1.2.0’s auto-pause and wake-on-access are for the Fast Sandbox profile only. A sandbox on the standard Kubernetes path keeps its pod, and the CPU and memory it reserves, until its timeout, a kill or your own pause().
  • A deleted pod comes back silently. The controller re-created it within seconds from the original image, while the server kept reporting Running. The files written before were gone, and nothing told the client its root filesystem was new.

Kata worked on the nested-virtualization nodes: the sandbox I looked inside had booted its own kernel, with 2 vCPUs and 3.9 GiB, and a create took about twice as long as under runc. gVisor is off the table once egress is on.

Part 4 covers the egress sidecar and the Credential Vault, part 5 signs AWS requests at the egress, and part 6 stress-tests the same two nodes. Everything here runs under runc except Test 5, which switches the server to Kata and back. Every number comes from my runs on November 1, 2026 (UTC); I say when something comes from the code or the docs instead.

The lifecycle at a glance

Part 1 drew the eight states from the lifecycle spec. This is the same diagram, with the transitions my runs went through highlighted, and one event that isn’t in the spec at all:

Pending Running Pausing Paused Stopping Resuming Terminated Failed created 1 pause 2 done 3 kill or TTL 4 resume resumed the TTL expires (or a kill) 5 cleaned up my run: then gone 6 pod deleted new pod, files lost on error seen in my runs in the spec, not watched here pod deleted: new pod, files lost on error
The lifecycle spec's eight states, as in part 1. Amber transitions are the ones my runs went through and watched; grey ones are in the spec but weren't watched here (02b deletes a paused sandbox, but nothing followed its states), and red dotted lines are the transitions to Failed. The dashed red loop on Running isn't a state change at all: a deleted pod is replaced, and the sandbox reports Running throughout.
  1. Create. The server creates a BatchSandbox, the controller a pod; create() returns once the pod is Running (Test 1: 2.3 seconds).
  2. Pause. A commit Job on the sandbox’s node commits every container’s root filesystem to an image (Test 2).
  3. Paused. The images are in ECR and the pod is gone: a paused sandbox holds no CPU or memory on a node.
  4. Resume. A new pod starts from the snapshot images under the same sandbox ID (Tests 2 and 3).
  5. TTL (or a kill). At the expiry the controller deletes the BatchSandbox (on a kill, the server deletes it); the sandbox reports Stopping until its pod is gone, then it’s gone from the API (Test 1 watched the TTL path).
  6. Pod deleted. The controller creates a replacement from the original image; the files are gone and the state never leaves Running (Test 6).

Two missing transitions matter as much. Nothing moves a container sandbox from Running to Pausing except your own pause(): auto-pause and wake-on-access cover “Fast Sandbox backend only”, and Docker and Kubernetes pod sandboxes “are not covered”. And a timeout moves only when you renew it, or through the opt-in renew-on-access, which extends it for a sandbox that receives traffic (part 1’s expiry table has the details).

Setup

Step 1: The companion code

The scripts run on the cluster from part 2 and read its env.sh, its API key and its source checkout from ../opensandbox-eks. Run part 2’s 01_cluster.sh to 05_gateway.sh first (06_console.sh is optional), and bring the nodes back with ../opensandbox-eks/08_scale.sh up if you scaled them down. The Python scripts run under uv; server_config.py runs on your own python3, which must be 3.11 or later (it uses only the standard library).

git clone https://github.com/fabiodouek/my2centsai-blog-samples.git
cd my2centsai-blog-samples/opensandbox-lifecycle
uv sync
FileWhat it does
00_snapshot_config.sh, values/controller-snapshot.yamlThe snapshot registry and its token (Step 2)
server_config.py, switch_runtime.sh, values/runtime-*.tomlEvery change to the server’s config (Step 3)
manifests/pool.yamlThe warm pool (Step 4)
01_states.py to 06_failure.py, plus 02b_token_fallback.pyOne script per test
03_resume_benchmark.shThe project’s resume benchmark, run unmodified
99_cleanup.pyRemoves what these scripts leave, and nothing else

Each Python script tags its sandboxes example=opensandbox-lifecycle, exits 1 when a check fails, and keeps a kubectl port-forward to the server on 127.0.0.1:18080 alive, starting a new one whenever /health stops answering. That matters here: switching the runtime replaces the server pod, which ends every port-forward to it.

Step 2: Snapshots in one ECR repository

Pause and resume are configured on the controller, not the server (pause and resume guide). The controller needs a registry to push snapshots to, a kubernetes.io/dockerconfigjson Secret to push with, and, optionally, a naming template. values/controller-snapshot.yaml adds them to part 2’s controller values:

controller:
  snapshot:
    registry: ${ECR_REGISTRY}/${ECR_PREFIX}/snapshots
    imageURITemplate: "{{.Registry}}:{{.SandboxName}}-{{.ContainerName}}-{{.SnapshotTag}}"
    snapshotPushSecret: opensandbox-snapshot-ecr
    resumePullSecret: opensandbox-snapshot-ecr

The template is the part worth copying. The default, {{.Registry}}/{{.SandboxName}}-{{.ContainerName}}:{{.SnapshotTag}}, puts the sandbox’s name in the repository path, so ECR ends up with one repository per sandbox. Mine puts it in the tag, and every snapshot lands in opensandbox-eks/snapshots as <sandbox id>-<container>-snap-gen2. The first pause is gen2, not the guide’s gen1, because the tag carries the BatchSandbox’s metadata.generation (snapshotImageTag), which the create and the pause patch bring to 2.

00_snapshot_config.sh does three things. It creates the repository with the settings of part 2’s ECR repository creation template, because an explicit aws ecr create-repository doesn’t apply the template; only a push to a missing repository does. It writes the Secret with a fresh ECR token. And it upgrades the controller, which replaces its pod on the first run.

The first run of 00_snapshot_config.sh, trimmed
$ ./00_snapshot_config.sh

== ECR repository opensandbox-eks/snapshots
  created, with the creation template's settings: MUTABLE, AES256, project=opensandbox-eks
  lifecycle rule: Expire untagged images after 7 days

== Secret opensandbox/opensandbox-snapshot-ecr
  ok    type kubernetes.io/dockerconfigjson, token issued 2026-11-01T06:10:23Z, valid until 2026-11-01T18:10:23Z
  ok    the registry accepts the token (GET /v2/ answered 200)

== controller release opensandbox-controller
  revision 10 -> 11 (deployed), manifests changed
  ok    --snapshot-registry=<registry>/opensandbox-eks/snapshots
  ok    --snapshot-image-uri-template={{.Registry}}:{{.SandboxName}}-{{.ContainerName}}-{{.SnapshotTag}}
  ok    --snapshot-push-secret=opensandbox-snapshot-ecr
  ok    --resume-pull-secret=opensandbox-snapshot-ecr
...
  total                               34s

An ECR token lasts 12 hours, so run the script at the start of every session that pauses sandboxes. A re-run took 19 seconds: a new token, identical manifests, the controller pod left alone. 02_pause_resume.py refuses a token that expires within 10 minutes, the benchmark one that expires within an hour. The template’s 7-day rule expires only untagged images, so snapshots stay until you delete them; deleting a SandboxSnapshot doesn’t, since the controller’s deletion handler removes only the commit and unpause Jobs. If you keep using snapshots, add a lifecycle rule that selects tagged images, by tag pattern or by age, or delete them yourself as 99_cleanup.py does. Pick an age longer than any sandbox stays paused: a paused sandbox’s snapshot image is its only copy. And re-running part 2’s 04_install.sh turns snapshots off; run this script again after it.

Step 3: One way to change the server

From here on, every change to the lifecycle server goes through one script, server_config.py: the runtime in this post, the egress image in part 5 and the egress mode in part 6. It exists because the server’s whole config.toml is one Helm value. Three scripts with three ideas of that file would undo each other’s changes, and an upgrade without part 2’s gateway values would quietly remove the gateway. So it asks first:

./server_config.py --show
runtime=runc egress_image=base egress_mode=dns+nft ingress=gateway

That’s part 2’s state (runc, the 1.2.0 egress image, dns+nft, the gateway), and the one this post leaves behind. A change rebuilds the opensandbox-server release from part 2’s values/server.yaml and values/server-gateway.yaml, edits only the lines it’s asked to in config.toml, and keeps every other setting at the value of the last deployed Helm revision, so switching the runtime keeps part 5’s egress image and the other way round:

./server_config.py --runtime runc|kata-qemu|kata-clh    [secure_runtime] from values/runtime-<name>.toml
./server_config.py --egress-image base|<image>          [egress] image (an ECR image is checked first)
./server_config.py --egress-mode dns|dns+nft            [egress] mode
./server_config.py --baseline                           runc, the base egress image, dns+nft
add --dry-run to see the config.toml change, checked with helm template, without upgrading

Guards keep one change from losing another: a lock shared by parts 3 to 6, a re-read of the revision just before the upgrade, a refusal to drop a live gateway, and checks of the rendered and the mounted config.toml, plus /health and /version. A change replaces the server pod (the Deployment uses Recreate), so the API is down while the new pod starts. switch_runtime.sh runc|kata is a thin wrapper around it.

Step 4: A warm pool

A Pool keeps Ready pods waiting, and a create with extensions={"poolRef": "osb-pool"} takes one instead of starting a pod. manifests/pool.yaml is the release’s pool example adapted to this cluster (trimmed):

apiVersion: sandbox.opensandbox.io/v1alpha1
kind: Pool
metadata:
  name: osb-pool
  namespace: opensandbox
spec:
  template:
    spec:
      automountServiceAccountToken: false
      initContainers:
        - name: task-executor-installer    # task-executor runs the request's entrypoint after allocation
          image: ${ECR_REGISTRY}/${ECR_PREFIX}/task-executor:release-1.2.0
        - name: execd-installer            # execd and bootstrap.sh
          image: ${ECR_REGISTRY}/${ECR_PREFIX}/execd:release-1.2.0
      containers:
        - name: sandbox
          image: ${ECR_REGISTRY}/${ECR_PREFIX}/python:3.12-slim
          command: ["/bin/sh", "-c", "/opt/opensandbox/task-executor -listen-addr=0.0.0.0:5758 -log-dir=/tmp"]
          resources:
            requests: { cpu: "1", memory: 2Gi }
            limits: { cpu: "1", memory: 2Gi }
  capacitySpec:
    bufferMin: 2    # Ready, unallocated pods kept waiting
    bufferMax: 3
    poolMin: 0      # pods in the pool in all, allocated or not
    poolMax: 5
  recycleStrategy:
    type: Delete    # a released pod is deleted and replaced, never reused

What changed from the example: every image comes from my ECR copy (the example points at a registry in China); the workload is the series’ python:3.12-slim with the 1 CPU and 2 GiB a regular create gets, so warm and cold creates run the same pod shape; and the tolerations: [{operator: Exists}] is gone, as in part 2’s BatchSandbox template, because it tolerates every taint, a cordoned node’s included (Test 6). The pod exists before any request, so it runs task-executor, and after allocation the controller starts bootstrap.sh, and with it execd, as a task.

Two limits come with pools. A pooled create can’t carry a networkPolicy or the credential proxy (Test 4 gets the 400). And the server’s [secure_runtime] doesn’t reach pool pods: the server’s provider sets runtimeClassName only on pods it builds itself (batchsandbox_provider.py, against L264-L265 for the template path), and the secure runtime guide says to set it in each Pool’s template. I read that in the code and didn’t run a Kata pool.

Testing

All output below is from the recorded runs, trimmed. Node names and sandbox IDs are real; the registry host is printed as <registry>.

Test 1: States, TTL, renew and expiry

01_states.py creates a sandbox with the shortest timeout the server accepts, 60 seconds, runs a command, renews it, and then leaves it alone until the controller removes it. The server answers the create call only once the pod is Running and has an IP (_wait_for_sandbox_ready, bounded by the 60-second sandbox_create_timeout_seconds), so a client never sees Pending from create() itself. To catch it, a second thread lists the run’s sandboxes four times a second:

$ uv run python 01_states.py
image opensandbox-eks/python:3.12-slim, timeout (TTL) 60 s, sandboxes tagged run=6a4c0ba14604

== create
  [06:46:35] +  0.9s  Sandbox.create(timeout=60 s)
  [06:46:35] +  1.5s  state: Pending (CREATING)
  [06:46:37] +  2.6s  state: Running (RUNNING)
  [06:46:37] +  3.2s  create() returned: 2bb29460-6ded-4870-8b6c-be52d09c17ec
  [06:46:38] +  4.1s  created_at 06:46:35, expires_at 06:47:35 (61 s), BatchSandbox expireTime 2026-11-01T06:47:35.550178+00:00

== renew(90 s)
  [06:46:40] +  5.6s  renew() -> expires_at 06:48:08; get_info() -> 06:48:08 (+33 s); BatchSandbox expireTime 2026-11-01T06:48:08.837187+00:00

== wait for the expiry
  [06:46:40] +  5.6s  nothing touches the sandbox now; it expires in 89 s
  [06:48:09] + 94.7s  state: Stopping (DELETING)
  [06:48:10] + 95.8s  gone from list_sandbox_infos 1 s after expires_at
  [06:48:10] + 96.1s  get_info() now: SandboxApiException: Get sandbox 2bb29460-6ded-4870-8b6c-be52d09c17ec failed: Sandbox '2bb29460-6ded-4870-8b6c-
  [06:48:11] + 97.3s  BatchSandbox and pods left in the cluster: 0

== states seen
  Pending -> Running -> Stopping -> gone

all steps passed

The create took 2.3 seconds from the call. The timeout is an absolute expiry, created_at plus 60 seconds, and the BatchSandbox carries the same instant as spec.expireTime.

renew() doesn’t add time; it sets a new expiry. The SDK computes now + timeout on the client’s clock (renew) and sends that instant, so renew(90 s) moved the expiry to 90 seconds after the call, 33 seconds later than before. A shorter duration than what’s left would shorten the sandbox’s life, and the server’s max_sandbox_timeout_seconds cap doesn’t apply to it.

At the expiry, the controller acted fast: Stopping (DELETING) within half a second of expires_at, gone from the list about a second after it, no BatchSandbox or pod left, and get_info() raised a SandboxApiException (part 1 has the release note: 404 once the pods are gone). I never saw Terminated. The 1.2.0 release notes say an expired BatchSandbox reports Terminated with the reason SANDBOX_EXPIRED, but a poll four times a second didn’t catch one on the way to deletion.

Test 2: Pause and resume, and what comes back

02_pause_resume.py creates a sandbox with a deny-by-default network policy and the credential proxy on, because the Credential Vault lives in the egress sidecar that the policy adds. Before the pause, it writes a random marker to /root/marker.txt (the root filesystem) and to /dev/shm/marker.txt (memory), starts a sleep 1000, and creates a vault with a placeholder apiKey for api.example.com. Then:

sb.pause()                       # 202 Accepted; the script polls until Paused
# ...
sb = SandboxSync.resume(sb.id, connection_config=connection("proxy"),
                        resume_timeout=timedelta(seconds=600))

Both calls are asynchronous: the server answers 202 and the state moves on its own, so the 60-second create timeout doesn’t apply. The SDK’s resume() then waits for execd to be healthy, with a default budget of 30 seconds (the script raises it to 600). The run:

$ uv run python 02_pause_resume.py
...
== what came back
  ok    /root/marker.txt (root filesystem): 2ff91ec1251542f5  (same as before)
  ok    /dev/shm/marker.txt (memory) did not come back: gone
  ok    the sleep process did not come back: sleep pid 1110 not running (its pid file is in the root filesystem, so the file came back)
  the egress sidecar answers: policy defaultAction deny, 1 rule(s)
  ok    Credential Vault did not come back: 404 from the egress sidecar: no vault
  ok    re-created from the trusted side: revision 1, credentials ['demo-api-key'], bindings ['demo-api']

== times
  pause() to Paused (commit and push included)      6.7 s
  resume() to Running                               5.0 s
  resume() to a healthy sandbox                     5.3 s
  resume() to the first command                     5.6 s
  snapshot images, total compressed size          179.7 MB

all checks passed: the pod was replaced by one running the snapshot images; the root filesystem came back; /dev/shm, the process and the vault did not, and the vault was created again
The rest of the run: create, pause, the ECR images and resume
== create, with a network policy (it adds the egress sidecar that holds the vault)
  [09:24:09] + 10.8s  created e8b91f94... in 6.6 s
  [09:24:11] + 12.4s  pod containers: sandbox python:3.12-slim, egress egress:release-1.2.0

== state before the pause
  rootfs: 2ff91ec1251542f5  tmpfs: 2ff91ec1251542f5  sleep pid 1110 running
  ok    vault before the pause: revision 1, credentials ['demo-api-key'], bindings ['demo-api']

== pause
  [09:24:12] + 13.3s  pause() accepted in 0.1 s
  [09:24:13] + 14.7s  sandbox Pausing | snapshot Committing
  [09:24:18] + 19.9s  sandbox Paused | snapshot Succeed
  [09:24:20] + 22.0s  commit Job: 5 s on ip-10-60-15-81.ec2.internal
  ok    pods of the sandbox while Paused: 0
  ok    the server still lists it while Paused: Paused

== ECR opensandbox-eks/snapshots
  :<id>-egress-snap-gen2    136.3 MB   pushed 09:24:14
  :<id>-sandbox-snap-gen2     43.4 MB   pushed 09:24:14
  ok    2 image(s) for this sandbox, one per container (sandbox, egress); 2 in the repository, every snapshot in this one repository
  snapshot status: sandbox -> opensandbox-eks/snapshots:<id>-sandbox-snap-gen2
  snapshot status: egress -> opensandbox-eks/snapshots:<id>-egress-snap-gen2

== resume
  [09:24:22] + 23.9s  sandbox Resuming
  [09:24:27] + 28.5s  sandbox Running
  [09:24:27] + 28.8s  SandboxSync.resume() returned (sandbox healthy) 5.3 s after the call
  [09:24:28] + 29.1s  first command answered 5.6 s after the call
  ok    a new pod (its uid differs from the paused one's)
  ok    container sandbox runs opensandbox-eks/snapshots@sha256:ec8d84d55bb7...
  ok    container egress runs opensandbox-eks/snapshots@sha256:b1133be70ce7...
  imagePullSecrets on the resumed pod: ['opensandbox-snapshot-ecr']
After the resumeWhy
/root/marker.txtBack, same valueThe root filesystem is the snapshot
/dev/shm/marker.txtGone/dev/shm is a tmpfs, memory
The sleep processGone (its pid file, on the root filesystem, came back)A new pod starts its processes from scratch
The Credential VaultGone: the sidecar answered 404It lived in the old sidecar’s memory

The pause took 6.7 seconds from the call to Paused, commit and push included; the commit Job itself ran 5 seconds on the sandbox’s node, and while the sandbox was Paused it had no pod at all. The resume took 5.0 seconds to Running, 5.3 to a healthy sandbox and 5.6 to the first command. The resumed pod’s containers run the snapshot images by digest, and the pod gets opensandbox-snapshot-ecr as its imagePullSecrets.

A pause commits every container in the pod, not just yours. With a network policy, that’s two images per pause: the sandbox at 43.4 MB and the egress sidecar at 136.3 MB, 179.7 MB in all. Each is about its base image plus a small diff: python:3.12-slim is 43.3 MB in ECR and the egress image 136.2 MB. A sandbox without a policy, like the benchmark’s in Test 3, pushes one.

⚠️ The vault is wiped on every pause. On Kubernetes the Credential Vault exists only in the egress sidecar's memory, a pause deletes the pod, and the resumed sidecar starts empty (#1366, open as of November 7, 2026; 1.2.0 carries the vault across a pause for the Fast Sandbox profile only). The same create request from the trusted side works again after the resume, and the script does that. Whatever resumes your sandboxes has to hold, or be able to fetch, the real credentials at that moment. Part 4, Test 8, shows that a resumed sandbox with no vault still sends its requests, without credentials.

Only a 404, or an empty vault, from a sidecar that answers counts as wiped: the script reads the sidecar’s egress policy back first, so an unreachable sidecar would fail the check instead.

Two more points for a control plane. Per the guide, the root filesystem is durable only once the snapshot reaches Succeed; anything written after the last snapshot is lost on an eviction, an OOM kill or a lost node. And #2526 reported that commands.run() on a paused sandbox hangs forever in the Python SDK. Its fix, PR #2527, was merged to main on November 1, after 1.2.0. The 1.2.1-rc.1 pre-release of November 4 lists it, but release candidates don’t publish SDK packages, and as of November 7, 2026 the latest Python SDK on PyPI is still 1.2.0, which sets no read timeout on that stream. I didn’t test it, but on 1.2.0 I’d put a timeout around such calls.

A gotcha that cost me a run: kubectl get sandboxsnapshots returns the Fast Sandbox CRD, sandboxsnapshots.sandbox.fast.io, which the base chart installs too. OpenSandbox’s own is sandboxsnapshots.sandbox.opensandbox.io, short name sbxsnap.

The token, and what it’s for. I expected an expired ECR token to break resume with ImagePullBackOff. On EKS, a pull with a broken pull secret still works. The controller adds the one shared Secret to every resumed pod, so 02b_token_fallback.py tests the pull in plain pods instead: it pauses a sandbox, copies the pull Secret with a token ECR answers 400 to (standing in for an expired one), and pulls the snapshot image three ways with imagePullPolicy: Always. That asks ECR with credentials every time. A resumed pod is gentler on the registry: it references the snapshot by digest, with the server’s default IfNotPresent, so on the node that committed the snapshot it may not contact ECR at all. The pull policy is what makes the test stricter than what the script’s heading calls “the way a resumed pod does”. The shared Secret isn’t touched:

$ uv run python 02b_token_fallback.py
...
== 1. a snapshot image to pull: create, write a marker, pause
  sandbox eb3b001f... paused; marker ab67071eba334de6; image opensandbox-eks/snapshots:<id>-sandbox-snap-gen2, committed on ip-10-60-15-81.ec2.internal

== 2. a broken copy of the pull Secret
  Secret opensandbox-lifecycle-bad-token-042162c255a3: a copy of opensandbox-snapshot-ecr's shape, with an invalid token; ECR answers 400 to it (opensandbox-snapshot-ecr's own token: 200)

== 3. pull the snapshot image the way a resumed pod does (imagePullPolicy: Always)
  ok    a. pull secret opensandbox-lifecycle-bad-token-04..., the committing node: Succeeded; marker: ab67071eba334de6
          Successfully pulled image "opensandbox-eks/snapshots:<id>-sandbox-snap-gen2" in 90ms (90ms including waiting).
  ok    b. no pull secret, the committing node: Succeeded; marker: ab67071eba334de6
          Successfully pulled image "opensandbox-eks/snapshots:<id>-sandbox-snap-gen2" in 68ms (68ms including waiting).
  ok    c. pull secret opensandbox-lifecycle-bad-token-04..., the other node: Succeeded; marker: ab67071eba334de6
          Successfully pulled image "opensandbox-eks/snapshots:<id>-sandbox-snap-gen2" in 415ms (415ms including waiting).

== 4. where the kubelet's ECR credentials come from
  node group role osb-nodes-...: AmazonEC2ContainerRegistryReadOnly, AmazonEKSWorkerNodePolicy, AmazonEKS_CNI_Policy

With the broken copy on the node that committed the snapshot, with no pull secret at all, and with the broken copy on the other node, the pull worked in 90, 68 and 415 ms, and each pod read the marker back. The kubelet falls back to the node role’s ECR credentials, and the node group role has AmazonEC2ContainerRegistryReadOnly. So a resume’s pull should get through on the node role too, which I infer and didn’t run, and on EKS the resumePullSecret is redundant for a registry in the same account; one in another account would need it, unless that repository’s policy lets this node role pull. The token matters for the push at pause time, the case I didn’t test, because it means breaking the Secret every other part pauses through. The guide’s troubleshooting table lists unauthorized in the commit Job’s log for wrong credentials, and a failed commit Job is followed by an <snapshot>-unpause Job that unpauses the containers. I’d expect an expired token to land there, but that’s the docs, not my run.

Test 3: The project’s own resume benchmark

The project ships a runner for exactly this measurement, tests/python/benchmarks/kubernetes_resume.py, planned in #1457, an issue about resume latency on Kubernetes, open as of November 7, 2026. Per its guide, each trial creates a sandbox, writes a marker outside any volume, pauses it, waits for its pods to disappear, resumes it, and reads the marker back with a foreground command. It reports four cumulative times, all from the resume call:

FieldEnds when
api_ack_msThe resume API returns (accepted, not ready)
running_msThe first lifecycle GET reports Running
ready_msA fresh connect passes execd’s health checks
command_msA foreground command prints the marker: the time a user waits

03_resume_benchmark.sh runs it as published, from part 2’s checkout of the tag, with 3 warmups and 30 samples, through the server proxy, and with no cache cleared or warmed (the guide’s “uncontrolled” scenario). The runner turns the SDK’s retries off, so one dropped connection fails a trial, and a failure during clean-up stops the run: my first full run, on the shared 18080 forward, stopped at trial 4 when another client’s port-forward ended. The script now keeps a private forward on 18081.

$ ./03_resume_benchmark.sh
image       opensandbox-eks/python:3.12-slim@sha256:aa943ce41142...
runtime     runc; nodes: 2 x m7i.2xlarge Amazon Linux 2023.12.20261102, kernel 6.18.56-125.164.amzn2023.x86_64, containerd://2.3.7+unknown, 8 vCPU, 30.8 GiB
trials      3 warmups + 30 samples, server proxy through 127.0.0.1:18081, output results/resume-uncontrolled-20261101T092534Z.jsonl
...
== Summary (run took 8m 03s)
  completed_requested_trials: true   measured 30, successful 30, failed 0, warmup failures 0, cleanup failures 0
  api_ack_ms    p50 125 ms   p95 357 ms
  running_ms    p50 1534 ms   p95 1967 ms
  ready_ms      p50 2067 ms   p95 2502 ms
  command_ms    p50 2396 ms   p95 2823 ms
  command_ms over 30 trials: min 1982 ms, max 3235 ms; restored on 30 x ip-10-60-15-81

All 30 measured trials succeeded, with completed_requested_trials: true, in 8 minutes 3 seconds. The time a user waits for, command_ms, had a median of 2,396 ms and a p95 of 2,823 ms (range 1,982 to 3,235 ms). Of that, the API acknowledged in 125 ms at the median, the sandbox reported Running at 1,534 ms and execd was healthy at 2,067 ms. All 30 snapshots were committed and restored on the same node, so this run says nothing about a resume on a node that never had the image.

Two cautions. The guide calls 30 samples “a starting point, not a stable tail-latency claim”, and it publishes no numbers of its own, so there’s nothing upstream to compare with. And these sandboxes have no network policy, so no sidecar and one image per pause. Test 2’s sandbox, with a sidecar and a second, larger image, took 5.6 seconds to its first command, measured with that script’s own polling; I wouldn’t compare the two sample for sample, and this run doesn’t isolate what the sidecar costs.

Test 4: Pools, warm against cold

04_pools.py applies the Pool, waits for its buffer, then alternates a cold create and a pooled one, three rounds, with the same image and resources:

$ uv run python 04_pools.py
...
== 1. apply the Pool osb-pool
  pool.sandbox.opensandbox.io/osb-pool created
  capacitySpec {"bufferMax": 3, "bufferMin": 2, "poolMax": 5, "poolMin": 0}, recycleStrategy Delete
  [09:34:19] +  0.9s  pool: total 1, allocated 0, available 0
  [09:34:21] +  2.7s  pool: total 2, allocated 0, available 1
  [09:34:23] +  4.6s  pool: total 2, allocated 0, available 2
  buffer filled (2 Ready pods) 5 s after the apply

== 2. cold and warm creates, 3 rounds
  round 1  cold   2.7 s   (586262b6-4b49-48c4-95aa-a211acb5fe37-0)
  round 1  warm   1.3 s   (osb-pool-lg82t, pool pod osb-pool-lg82t)
  round 2  cold   2.1 s   (b6ab63f6-f1f7-4f07-8aca-bf7ca4dbfff9-0)
  round 2  warm   1.3 s   (osb-pool-9g6zz, pool pod osb-pool-9g6zz)
  round 3  cold   2.8 s   (af75616d-e73b-434e-add8-c4f63e07d136-0)
  round 3  warm   1.3 s   (osb-pool-msk8x, pool pod osb-pool-msk8x)
  median  cold 2.7 s, warm 1.3 s: warm is 1.5 s faster

The buffer of two Ready pods filled 5 seconds after kubectl apply. Cold creates took 2.7, 2.1 and 2.8 seconds and pooled ones 1.3 each, medians of 2.7 and 1.3. A pooled sandbox’s hostname is the pool pod’s name, not <sandbox id>-0, which is how the script checks that it really got a pool pod. Part 6 timed the same creates from inside the cluster: most of the warm 1.3 seconds was the port-forward from my Mac, not the Pool.

recycleStrategy: Delete means a used pod never serves another sandbox. The script writes a file into a pooled sandbox, deletes the sandbox and watches the pool:

The recycle run
== 3. recycleStrategy Delete: a used pod is replaced, not reused
  [09:35:36] pool pods: osb-pool-fpmgt, osb-pool-lsjvl, osb-pool-msk8x
  [09:35:38] sandbox b3b71b9b... got pool pod osb-pool-lsjvl; pool: total 2, allocated 2, available 1
  [09:35:39] sandbox deleted
  [09:35:40] +   1s  pool pods: osb-pool-fpmgt, osb-pool-lsjvl, osb-pool-msk8x Terminating
  [09:36:05] +  25s  pool pods: osb-pool-8dj9n Pending, osb-pool-fpmgt, osb-pool-lsjvl Terminating
  [09:36:06] +  27s  pool pods: osb-pool-8dj9n, osb-pool-fpmgt, osb-pool-lsjvl Terminating
  [09:36:36] +  57s  pool pods: osb-pool-8dj9n, osb-pool-fpmgt
  osb-pool-lsjvl was deleted with its sandbox, /tmp/previous-user.txt and all; it never returns to the pool. The buffer holds 2 pods that never ran a sandbox (new: osb-pool-8dj9n)

The controller started deleting the used pod 25 seconds after the sandbox, file and all; a replacement that had never run a sandbox was Running at 27 seconds, and the used pod was gone at 57. Restart and Noop reuse the pod instead, and the controller docs warn that Noop leaks files and processes into the next allocation. For agent sandboxes, I’d keep Delete.

The last step asks for a pooled sandbox with a network policy:

== 4. a pooled create with a network policy
  ok    refused with HTTP 400: Create sandbox failed: networkPolicy cannot be used together with extensions.poolRef because pooled pods are pre-created. Remove 'networkPolicy' from the request or use template mode.
  (the script goes on: put an egress policy for pooled sandboxes in the Pool template)

The pod exists before the request does, so its egress sidecar can’t be added afterwards (_ensure_pool_mode_compatible), and the same function refuses credentialProxy.enabled. Per-request egress rules, and with them part 4’s Credential Vault, don’t mix with pools; an egress policy for pooled sandboxes lives in the Pool’s template, the same for every sandbox.

Test 5: runc against Kata, and why not gVisor

The runtime belongs to the server, not to the request: one server process has one [secure_runtime], and a create can’t choose. So the test is a run under runc, a switch, the same run under Kata, and a switch back. The two 05_runtimes.py runs below are from an earlier session that day, on the same server configuration; the switch shown is a later recording. values/runtime-kata-qemu.toml is the whole change:

[secure_runtime]
type = "kata"
k8s_runtime_class = "kata-qemu"
uv run python 05_runtimes.py      # runc
./switch_runtime.sh kata          # kata-qemu
uv run python 05_runtimes.py      # a VM per sandbox
./switch_runtime.sh runc          # parts 4 and 5 expect runc
The switch to kata-qemu
$ ./switch_runtime.sh kata
server now:    runtime=runc egress_image=base egress_mode=dns+nft ingress=gateway   (revision 15)
server wanted: runtime=kata-qemu egress_image=base egress_mode=dns+nft ingress=gateway
  config.toml: changes:
    --- running config.toml
    +++ new config.toml
    @@ -26 +26,6 @@
     path = "/data/opensandbox.db"
    +
    +# from opensandbox-lifecycle/values/runtime-kata-qemu.toml
    +[secure_runtime]
    +type = "kata"
    +k8s_runtime_class = "kata-qemu"
  helm upgrade: revision 15 -> 16 (deployed) in 17 s, manifests changed
  server pod opensandbox-server-65597b69c6-j569l replaced opensandbox-server-775f8ddf57-8k74w, ready=true
  ok    server /health: {"status":"healthy"}
  ok    server /version: {"version":"1.2.0"}
  ok    release revision 16 is deployed
  ok    the server pod runs with: runtime=kata-qemu egress_image=base egress_mode=dns+nft ingress=gateway
  ok    gateway kept: server.gateway and [ingress] mode = gateway, host from values/server-gateway.yaml
  The server pod was replaced: a kubectl port-forward started before this has ended. The Python
  scripts in this directory start their own; restart yours if you use one.
switched to kata-qemu in 29 s

The switch is a Helm upgrade with a pod replacement: 17 seconds for Helm and 29 in all, with the API down for part of it. The switch back took 28 seconds. A pod’s runtimeClassName can’t change, so sandboxes already running keep the runtime they were created with; only new ones change.

05_runtimes.py times three creates and then looks at the system from inside one sandbox, with the default limits of 1 CPU and 2 GiB:

What the sandbox saw under kata-qemu
$ uv run python 05_runtimes.py
server runtime: kata-qemu   (sandboxes tagged run=9c4db5614eb3)

== 3 creates under kata-qemu
  create 1:   4.2 s   on ip-10-60-15-81.ec2.internal
  create 2:   6.1 s   on ip-10-60-36-30.ec2.internal
  create 3:   4.1 s   on ip-10-60-15-81.ec2.internal

== what the sandbox sees under kata-qemu
  pod runtimeClassName     kata-qemu
  pod overhead             cpu 250m, memory 160Mi
  pod limits               cpu 1, memory 2Gi
  kernel (uname -r)        6.18.35   (node: 6.18.56-125.164.amzn2023.x86_64)
  CPUs it can use          2   cgroup cpu.max: 100000 100000
  CPU model                Intel(R) Xeon(R) Platinum 8488C
  memory (MemTotal)        3.9 GiB
  machine (DMI)            n/a / n/a
  its kernel has been up   4 s
  entries in /dev          15
  -> its own kernel, not the node's: the sandbox runs in a virtual machine
From inside the sandboxrunckata-qemu
Kernel6.18.56-125.164.amzn2023.x86_64, the node’s6.18.35, its own
Kernel up for22 minutes (the node’s)4 seconds (the VM had just booted)
CPUs it can use8, all of the node’s; the 1-CPU limit is a cgroup quota2
Memory it sees30.8 GiB, the node’s3.9 GiB
Machine (DMI)Amazon EC2 / m7i.2xlargeNo DMI data in the guest (/sys/class/dmi/id unreadable)
Entries in /dev1615
Pod overhead reservedNone250m CPU, 160Mi
Create, 3 runs2.7 / 2.0 / 2.2 s, median 2.24.2 / 6.1 / 4.1 s, median 4.2

The CPU model, a Xeon Platinum 8488C, shows through in both.

Part 2’s Kata gate pod, with no limits, had 1 vCPU. This one, with limits of 1 CPU and 2 GiB, had 2 vCPUs and 3.9 GiB. kata-deploy 3.32.0’s configuration on the nodes, /opt/kata/share/defaults/kata-containers/configuration-qemu.toml, sets default_vcpus = 1, default_memory = 2048 (MiB) and static_sandbox_resource_mgmt = false. In Kata’s documented sizing model, the VM boots with those defaults and the runtime then adds the vCPUs a container’s CPU limit asks for (vCPU sizing); with static sizing off, it hot-adds CPU and memory to the running VM rather than sizing it before boot (QEMU configuration). That’s 1 + 1 vCPUs and 2 + 2 GiB, which fits the 2 vCPUs and the 3.9 GiB of MemTotal the guest reported. I didn’t observe the hot-plug itself. The RuntimeClass reserves only 160Mi of overhead for all of that, while part 2 measured the QEMU process at about 272 MiB; part 6 looks at what that under-reservation costs on a full node.

A create under Kata took 4.2 seconds at the median against 2.2 under runc, and the one create that landed on the other node took 6.1. The difference includes a VM boot on nested virtualization; I didn’t break it down further.

Why not gVisor. I didn’t install it: it needs runsc and its containerd shim on every node through custom node bootstrap, which wasn’t worth it for one error message. The server rejects it the moment a request carries a network policy. ensure_egress_runtime_compatible answers HTTP 400 when [secure_runtime] type = "gvisor": “networkPolicy is not compatible with runtime ‘gvisor’: gVisor does not support the nftables rules required by the egress sidecar. Use a compatible runtime (e.g. kata) or remove networkPolicy.” The secure runtime guide gives the cause: gVisor’s network stack has no iptables nat table, and the sidecar’s DNS redirect needs one. The root-filesystem committer doesn’t support gVisor either, per the pause and resume guide, and the draft fix for the redirects, #1374, a draft as of November 7, 2026, waits on upstream gVisor changes. Once you want egress control or the vault, the docs leave runc or Kata: the secure runtime guide says the egress sidecar works unchanged under Kata, and the validator lets it through. I didn’t run Kata with a network policy, and pausing a Kata sandbox is in neither my runs nor the pause and resume guide.

After the switch back, ./server_config.py --show gave the same line as in Setup.

Test 6: A deleted pod and a cordoned node

06_failure.py starts with a sandbox, writes a marker file into it, and deletes its pod with kubectl, the way an eviction, a node failure or an operator would. Then it polls the server, the BatchSandbox and the pods, with a 2-second pause between polls, which came to about every 4 seconds here:

$ uv run python 06_failure.py
...
== 1. delete the pod of a running sandbox
  sandbox e57f1f60-f3a8-47bf-a687-ee486d9770a2: pod e57f1f60-f3a8-47bf-a687-ee486d9770a2-0 (uid 9eb9b5e8), marker file written
  [09:37:08] kubectl delete pod e57f1f60...-0   (polling every 2 s)
  [09:37:11] +   2s  server Running (RUNNING) | BatchSandbox phase Succeed, replicas 1, ready 1 | e57f1f60...-0 Pending uid 1a6e053a
  [09:37:15] +   6s  server Running (RUNNING) | BatchSandbox phase Succeed, replicas 1, ready 1 | e57f1f60...-0 Running uid 1a6e053a
  after 75 s: a new pod e57f1f60...-0 (uid 1a6e053a)
  a command through the SDK now: no /root/marker.txt
⚠️ A deleted sandbox pod comes back empty, and the sandbox's state doesn't change. At the first poll (+2 s), a new pod with the same name was already Pending; at the next one (+6 s) it was Running. The server reported Running on every line, the BatchSandbox stayed Succeed with one ready replica, and the SDK's next command reached the new pod. The marker file was gone: the new pod starts from the original image, not from the old pod's writable layer, and nothing was snapshotted. Data written to the root filesystem since the last pause is lost: the sandbox's state never left Running, and nothing told the client its root filesystem was new.

The traces are elsewhere, from the code and untested here: the controller logs the re-creation and records a succeed to create pod event on the BatchSandbox, and the diagnostics events API (GET /v1/sandboxes/{id}/diagnostics/events, get_diagnostic_events() in the SDK) lists the pod’s Kubernetes events, where the deletion would show.

I expected no replacement at all. 1.2.0 made restartPolicy: Never the default for sandbox pods because, as part 1 explained, an in-place restart wipes the writable layer while the sandbox keeps looking healthy. A deleted pod gets the same outcome by another route. The controller’s scaleBatchSandbox creates a pod for any replica index that has none, named <sandbox id>-<index>, from the BatchSandbox’s template. The pause and resume guide says as much about durability: a new container “does not inherit the old writable layer”. If it matters, keep it in a volume or outside the sandbox. Pooled sandboxes are a different case: #954, open as of November 7, 2026, reports that a pooled BatchSandbox whose pod is deleted never gets a replacement. I didn’t test that one.

The second half cordons the node that doesn’t run the server and creates four sandboxes:

== 2. cordon a node, then create sandboxes
  [09:38:31] kubectl cordon ip-10-60-36-30.ec2.internal
  ip-10-60-15-81.ec2.internal   schedulable
  ip-10-60-36-30.ec2.internal   unschedulable (cordoned)
  taints on it: node.kubernetes.io/unschedulable:NoSchedule
  sandbox 1:  3.4 s, on ip-10-60-15-81.ec2.internal
  sandbox 2:  3.0 s, on ip-10-60-15-81.ec2.internal
  sandbox 3:  3.0 s, on ip-10-60-15-81.ec2.internal
  sandbox 4:  3.1 s, on ip-10-60-15-81.ec2.internal
  ok    4 of 4 on ip-10-60-15-81.ec2.internal, 0 on the cordoned node

==    the release's default toleration, on a plain pod pinned to the cordoned node
  pod toleration-demo with tolerations [{operator: Exists}]: Running on ip-10-60-36-30.ec2.internal
  uncordoned ip-10-60-36-30.ec2.internal

All four landed on the other node, in 3.0 to 3.4 seconds each. Part 2’s BatchSandbox template has no tolerations, so the cordon’s node.kubernetes.io/unschedulable:NoSchedule taint kept them off. Then the script pinned a plain pod with the release’s default toleration, [{operator: Exists}], to the cordoned node, and it ran there. That’s what the default template allows: sandboxes could land on a node an operator is trying to empty.

I cordoned and didn’t drain: on two nodes, a drain would also move the server, the controller and the gateway. A drain evicts the sandbox pods on the node, and from the first half I’d expect each to come back on the other node with a fresh root filesystem and no word to the client. The script uncordons the node at the end, also on failure or interrupt.

Results summary

TestScriptRecorded runResult
1. States, TTL, renew01_states.py99.1 sPending, Running, Stopping, gone; deleted about 1 s after expires_at; renew() sets an absolute expiry
2. Pause and resume02_pause_resume.py, 02b_token_fallback.py35.8 s, 43.0 sRoot filesystem back; /dev/shm, processes and the vault gone; 2 images with the sidecar; pause 6.7 s, first command 5.6 s after resume; a broken pull secret doesn’t stop the snapshot image from pulling on EKS
3. Resume benchmark03_resume_benchmark.sh496.0 s (runner 8m 03s)30/30 trials; command_ms p50 2,396 ms, p95 2,823 ms
4. Pools04_pools.py148.8 sWarm 1.3 s against cold 2.7 s (medians of 3); used pods deleted; poolRef with networkPolicy refused with 400
5. runc and Kata05_runtimes.py, switch_runtime.sh20.0 s and 27.8 s; switch recordings 30.5 s and 29.5 sServer switch 29 s, back 28 s; Kata: own kernel, 2 vCPUs, 3.9 GiB; create median 4.2 s against 2.2 s; gVisor refused with egress (code)
6. Failures06_failure.py114.0 sDeleted pod replaced with a fresh root filesystem, sandbox Running throughout; cordon respected; the default toleration ignores it; drain not run (expected: the deleted-pod case for each sandbox on the node, not tested)

Every script exited 0. The vault wipe is a check in 02; the silent replacement is an observation, because 06 prints what happens to the deleted pod and asserts only the cordon.

Cleanup

Every script deletes its own sandboxes, and 04_pools.py deletes its Pool, because idle pool pods hold a full CPU each. After an interrupted run:

uv run python 99_cleanup.py

It deletes only what this directory made: sandboxes with example=opensandbox-lifecycle, the Pool, the pods and Secret of 02b and 06, a cordon 06 left behind, and the snapshot images of this directory’s sandboxes once those sandboxes no longer exist. It keeps the repository, the Secret and the controller’s snapshot settings, which parts 4 and 6 use.

Check ./server_config.py --show before you stop: it should say runtime=runc, which parts 4 and 5 expect. Between sessions, scale the nodes to zero from part 2’s directory (the cluster costs about $1.01 an hour idle, per part 2’s Pricing), and remove everything with its teardown at the end of the series:

../opensandbox-eks/08_scale.sh down
../opensandbox-eks/99_teardown.sh

Verdict

The lifecycle on 1.2.0’s Kubernetes path does what the spec says; plan around what it doesn’t spell out. Expiry was exact to about a second, renewal is a plain absolute timestamp, pause and resume through one ECR repository worked in every run, the project’s benchmark ran unmodified, and the switch to Kata on nested virtualization was one TOML table and a Helm upgrade. The weak spots are about state. In the default mode, a pause keeps the root filesystem and nothing else, the vault included. A deleted pod loses the root filesystem too, while the sandbox’s state never leaves Running. And no idle sandbox pauses itself on this path.

So the model that fits is a sandbox as disposable compute with its state elsewhere: files that matter in a volume or object storage, the vault re-created by your control plane after every resume, and your own idle policy calling pause() or kill(). On those terms the pieces are solid, and Kata gives you a VM per sandbox at about twice the create time (egress under Kata isn’t tested here; part 5’s Test 8 runs it).

What you sign up for

ConcernThe 1.2.0 lifecycle on EKS, as tested; what comes from code or docs is marked
ExpiryAn absolute timeout, 60 s minimum; renew() sets now plus a duration and can shorten it (code); deletion about 1 s after expiry
Idle sandboxesNo auto-pause outside Fast Sandbox (docs); a sandbox holds its pod until its timeout, a kill, or your pause()
PauseEvery container’s root filesystem to an OCI image (2 with the egress sidecar), then the pod is deleted; memory, processes and the vault are lost
ResumeA new pod from the snapshot images, same ID; 2.4 s median to a command without a sidecar; the vault must be written again
RegistryOne ECR repository with a custom imageURITemplate; a 12-hour token for the push; tagged snapshots are yours to delete
PoolsWarm creates in 1.3 s against 2.7 s cold; Delete recycling; no networkPolicy (tested) or credential proxy (code) per request; runtimeClassName in the Pool’s template (code and docs)
IsolationOne runtime per server; Kata VMs on nested virtualization at about twice the create time; gVisor refused once egress is on (code)
FailureA deleted pod is re-created from the image with a fresh root filesystem while the API keeps reporting Running

Who should use it

This fits a team running agent sandboxes as short-lived workers: create, run, maybe pause between turns, delete. Expect a resume to bring back the environment, not the session, and keep anything that has to survive in storage the sandbox mounts or calls. If you need VM isolation, Kata on m7i nested virtualization works without bare metal, and the egress sidecar works under it too: part 5’s Test 8 runs it there.

It doesn’t fit, yet, a workload that expects a resumed sandbox to carry on where it stopped, with its processes and memory, or one that relies on the platform to pause idle sandboxes and save the capacity. Nor one that trusts Running to mean the sandbox still has its files. Part 4 takes the egress sidecar and the Credential Vault through nine tests on the same cluster.

Comments