OpenShell Architecture: How the Pieces Fit Before You Deploy

See the source code here: openshell-architecture

Overview

On 2026-09-26 the CNCF Technical Oversight Committee voted to accept OpenShell as a Sandbox project (cncf/sandbox#522), with 72.73% of binding voters in favor and none against. The contribution agreement was recorded as signed on 2026-10-02. Onboarding (#535) was still open when I wrote this, and the repository, the image registry and the documentation still live under NVIDIA.

OpenShell is an Apache 2.0 runtime for running AI agents under rules they can’t change. You wrap an existing agent, or any process, in a sandbox. A trusted component outside that sandbox decides which files it can touch, which hosts it can reach, with which binary, and which credentials get attached on the way out. The agent itself gets placeholders, not real API keys; the supervisor swaps in the real value as the request leaves. The project calls itself “the safe, private runtime for autonomous AI agents”.

I’m using release v0.1.6 (2026-10-27). The repository was created in February 2026, went public in mid-March, and shipped 115 v0.0.x tags between March and August. Version 0.1.0 (2026-09-25) is the project’s exit from alpha, and it changed the architecture enough that 0.0.x installs can’t be upgraded in place. Anything written about OpenShell before September may describe a design that no longer exists.

This post, part 1 of 6, explains how the pieces fit and runs them locally on a Mac. The rest of the series runs OpenShell on Amazon EKS:

• Part 2: installing it on EKS 1.37
• Part 3: sandbox lifecycle and isolation
• Part 4: policy and egress
• Part 5: AWS credentials signed at the supervisor
• Part 6: stress testing two nodes

GPU sandboxes are out of scope for the whole series. OpenShell supports them, but nothing here uses a GPU.

Architecture

OpenShell has four parts you interact with, and one boundary that matters more than the rest.

• Clients: the openshell CLI, a terminal UI (openshell term), and SDKs for Python, Go and TypeScript. They all talk gRPC to the gateway.
• The gateway is the control plane. It stores sandboxes, policies, providers (credential sets) and their history, issues certificates and tokens, and asks a compute driver to create the actual sandboxes. Drivers exist for Docker, Podman, Kubernetes and microVMs.
• The supervisor is the trusted half of each sandbox. It holds the policy, terminates TLS for the sandbox’s outbound connections, applies HTTP-level rules, resolves credentials and opens every real upstream connection.
• The workload is the untrusted half: your agent and everything it starts, plus a small OpenShell runtime that runs as PID 1 inside it. That runtime is OpenShell code the docs call trusted, but it runs inside the workload boundary with the agent and makes no policy decisions.

Since 0.1.0, those two halves are separate compute units: two containers on Docker or Podman, two Pods on Kubernetes. On 0.0.x the supervisor ran inside the agent’s container. RFC 0012 explains the move in one sentence: “the privilege needed to build that boundary sits beside the code it confines.”

Three traffic paths run through this picture, and it helps to keep them apart.

1. Lifecycle. The CLI asks the gateway to create a sandbox, and the gateway asks its driver for the two compute units. On Kubernetes, the driver creates a Sandbox custom resource and a separate supervisor Pod, and the Agent Sandbox controller (a Kubernetes SIG project OpenShell depends on) creates the workload Pod from it.
2. Session. exec, SSH, file transfer and log streaming never open a port on the sandbox. The supervisor dials out to the gateway and keeps that session open; the gateway relays your command over it. That is why a sandbox behind NAT or a strict firewall still works, and why openshell sandbox exec goes through the gateway even on your own laptop.
3. Outbound. When the agent connects anywhere, the connection never leaves the workload directly. The workload has no network path except to its own supervisor. OpenShell’s runtime inside it traps the connect() call and the DNS lookup, identifies which binary made them, and hands the request to the supervisor over a single mutually authenticated HTTP/2 channel (the “OpenShell Sandbox Protocol”). The supervisor checks the policy, terminates TLS if the endpoint calls for it, applies HTTP rules and credentials, and makes the real connection itself.

Concepts

Two compute units, one protocol

The split is the core idea. The workload holds the agent and OpenShell’s small runtime, nothing more: no sandbox token, no TLS client key, no credentials, no CA private key. It has the gateway’s public key so it can verify what it’s told. The supervisor holds the privileges and the secrets.

Inside the workload, a seccomp user-notification broker intercepts TCP opens and DNS queries. For each one it reads the calling process from /proc and pins the binary by its SHA-256 on first use. The policy can then say “curl may reach this host” rather than “this container may reach this host”. Agent sockets are bound to loopback inside the workload, so there is nowhere else for them to go.

Two details from the architecture page are worth knowing before you debug anything:

• It fails closed: without its supervisor, the agent gets nothing rather than an open network. The workload won’t launch the agent until the supervisor confirms, and if the supervisor’s connection drops, the agent is frozen. Only the same supervisor process can reconnect and resume it, within 30 seconds by default; otherwise the workload is stopped (part 3 shows it on EKS). A new supervisor can’t take over a running sandbox.
• Tokens are per run. The gateway issues a gateway token and a sandbox token bound to one sandbox and one “generation”: one run of the sandbox’s compute units. Every restart, or a stop and start, begins a new generation with fresh tokens and certificates.

One documentation trap: OpenShell’s security best-practices page still describes a network namespace with a veth pair and an HTTP CONNECT proxy at 10.200.0.1. That is the 0.0.x design. The architecture page, the driver READMEs and the code at the v0.1.6 tag describe the broker model above, and that’s what my local run showed: the deny lines refer to “transparent” connections, and the workload’s environment has no proxy variables at all.

Phases and generations

A sandbox moves through a small set of phases, defined in the protocol:

A few things that surprised me:

• Stop and start, yes; suspend, no. openshell sandbox stop keeps the sandbox and its workspace, and start brings it back with a new generation. suspend and resume exist in the CLI (“retain its memory state”), but on my Docker gateway suspend returned “compute driver does not support sandbox suspend”. No built-in driver implements it in 0.1.6: the gateway checks the driver’s capabilities and refuses before the driver is asked, so Podman, Kubernetes and VM give the same answer (part 3 shows it on EKS).
• There is no idle timeout. I found no TTL or auto-expiry in the 0.1.6 docs or protocol. --no-keep deletes the sandbox when its first command exits; anything else runs until you delete it.
• Restarts belong to the gateway, not the container runtime. --restart-policy never|on-failure|always (default never) decides what happens when the main process exits.

Policy layers: what’s locked and what’s live

A sandbox policy is one YAML file with sections that behave differently (manage policies):

Live changes reach the supervisor in one of two ways, set by the gateway’s config_delivery_mode. The default is poll, and supervisors poll every 10 seconds. Version 0.1.5 added push, which sends configuration over the supervisor’s existing session. On my local gateway in the default mode, a policy update took about that long: in the recording, --wait reported the new version loaded 10.0 seconds after I submitted it.

Two more rules are worth knowing early:

• A new policy generation closes connections opened under the old one. That includes keep-alive connections, tunnels and long streams. It’s the safe choice, and it surprises long-running agents.
• HTTP rules default to audit, not enforce. An endpoint marked protocol: rest and access: read-only logs a POST and lets it through unless you also set enforcement: enforce. Part 4 tests this.

Egress and credentials: one request

Credentials are where the design pays off. You create a provider on the gateway (a named set of credentials, such as a GitHub token) and attach it to a sandbox. Inside the sandbox, the agent sees environment variables like GITHUB_TOKEN, but their values are opaque placeholders of the form openshell:resolve:env:.... The real value exists only on the gateway and in the supervisor.

Three properties of that flow, read in the 0.1.6 code and documented in providers:

• Binding is by endpoint, not by Host header. The supervisor picks the credential from the endpoint the connection was allowed to (host, port and path). A request whose Host header names a different host is refused with request_authority_mismatch.
• Binding is by endpoint, not by binary. Any binary the policy lets reach api.github.com can have the GitHub placeholder resolved. The policy decides who reaches the host; the provider decides what’s attached there.
• The supervisor resolves the destination itself. It dials addresses it resolved and pinned, never what the sandbox’s own resolver claims. Loopback and link-local addresses (169.254.0.0/16, which covers cloud metadata endpoints) are always blocked, whatever the policy says, and the Kubernetes control-plane ports (2379, 2380, 6443, 10250, 10255) are blocked for exact-host and IP endpoints.

For AWS, there’s a step beyond substitution: the supervisor can re-sign requests with SigV4, and the gateway can mint STS credentials itself. Part 5 is about that.

Providers, the prover and the advisor

• Providers come from profiles. Since 0.1.0 the gateway ships with none built in. You import a profile (GitHub, AWS, Anthropic and others live under providers/), then create a provider from it. Credentials can be static or refreshed by the provider: OAuth2 flows, Google service accounts, or aws_sts_assume_role.
• The prover encodes policies for the Z3 solver and answers one question: does a candidate policy allow anything a boundary policy doesn’t? It runs as a standalone CLI, openshell-prover (the Homebrew formula installs it), which fits CI. The gateway itself only runs a risk check on proposals from the advisor, the feature that lets an agent propose a policy change for a human to approve. Plain policy set and policy update run no prover. And the prover checks the policy, not the enforcement. The prover docs say so directly: a pass “does not mean that the policy is as narrow as it could be, that it is safe for a particular task, or that a running sandbox enforces it.”

Drivers and the isolation menu

With no explicit setting, the gateway picks Kubernetes, then Podman, then Docker. Every option needs a recent kernel: Landlock ABI 3 or newer (Linux 6.2+) and seccomp user notification. The sandbox runtime probes for these at startup and refuses to run if they’re missing. User namespaces are an opt-in extra on Kubernetes.

What runs where on Kubernetes

Parts 2 to 6 live here, so the shape is worth seeing once.

Every OpenShell Pod (gateway, supervisor, workload) runs as non-root with all capabilities dropped and the runtime-default seccomp profile: no privileged containers, no host paths. The cost is density. Each sandbox is two Pods and one EBS-backed volume, which matters on two nodes (part 6).

Setup

Part 1 runs OpenShell on my Mac (Apple Silicon, Docker Desktop with 10 CPUs) with the Homebrew-installed gateway and the Docker driver. The companion directory has five scripts and three config files.

Step 1: Remove 0.0.x first

My Mac already had OpenShell 0.0.63 from June, with two sandboxes. The 0.1.0 upgrade notes are blunt: “Local OpenShell 0.0.x installations cannot be upgraded in place.” I deleted the sandboxes, then uninstalled:

openshell sandbox delete <each sandbox>
brew services stop nvidia/openshell/openshell
brew uninstall nvidia/openshell/openshell
rm -rf "$(brew --prefix)/var/openshell"

The installer itself only guards upgrades from versions before 0.0.37, so it would have tried to upgrade 0.0.63 in place. 00_gateway.sh refuses instead and prints the steps.

Step 2: Pin the version and switch telemetry off

00_gateway.sh installs with the official script, pinned:

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/v0.1.6/install.sh \
  | OPENSHELL_VERSION=v0.1.6 sh

Two pinning traps, both printed by the script:

• Without OPENSHELL_VERSION, the installer resolves releases/latest. On the day I ran it that was v0.1.6, but a new release ships most Tuesdays.
• Release tags have a v; container image tags don’t. ghcr.io/nvidia/openshell/gateway:v0.1.6 returns HTTP 404, and :0.1.6 returns 200. Helm values and image mirrors need the second form.

Telemetry is on by default. The runtime switch is an environment variable on the gateway, OPENSHELL_TELEMETRY_ENABLED=false, which the gateway passes to every supervisor; the Helm chart’s server.telemetryEnabled sets the same variable, and source builds can compile telemetry out (telemetry). On macOS, the Homebrew service wrapper sources ~/.config/openshell/gateway.env before starting the gateway, so the script writes the opt-out there before the first start, then checks the running process:

ps eww -p "$(pgrep -f bin/openshell-gateway | head -1)" | tr ' ' '\n' | grep OPENSHELL_TELEMETRY
# OPENSHELL_TELEMETRY_ENABLED=false

Step 3: Fix the Docker Desktop network mode

My first sandbox failed with:

ControlSupervisorStartFailed: Docker supervisor exited before becoming ready
… Startup configuration fetch failed after 5 attempts: failed to connect to OpenShell server

The Docker driver starts each supervisor container in host network mode by default and points it at 127.0.0.1:17670. On Docker Desktop for Mac, “host” is the Docker VM unless Docker Desktop’s host networking setting is on, which it wasn’t. The other mode, bridge, reaches the gateway through host.docker.internal, which the gateway’s certificate already lists. One setting in ~/.config/openshell/gateway.toml fixed it:

[openshell]
version = 2

[openshell.drivers.docker]
supervisor_network_mode = "bridge"

Step 4: A workload image that isn’t root

OpenShell 0.1.x defaults to nvcr.io/nvidia/base/ubuntu:24.04, a minimal Ubuntu with no agent installed, so I built a small image with curl, python3, git and jq, pinned to the base’s digest (the 24.04 tag moves). My first build left USER root in place, and the sandbox was refused before it started:

IdentityResolutionFailed: descriptor error: workload identity must not contain UID or GID zero

The fix is one line at the end of the Dockerfile:

USER 1000:1000

Testing

Each test is a script in the companion directory, recorded as it ran for the video. The image was already cached, so the timings measure OpenShell, not a pull.

Test 1: A first sandbox from the CLI

01_first_sandbox.sh creates a sandbox, looks at it, runs two commands, moves a file in and out, reads the logs and deletes it.

openshell sandbox create --name first --from openshell-workload:0.1.6 --detach
openshell sandbox exec first -- uname -a
openshell sandbox exec first -- id
openshell sandbox upload first ./README.md /sandbox/
openshell sandbox download first /sandbox/README.md /tmp/out/
openshell logs first -n 15
openshell sandbox delete first

Create to Ready took 2.5 seconds. The kernel was Docker Desktop’s (6.12.76-linuxkit … aarch64), the identity uid=1000(ubuntu), and the file came back byte for byte. openshell sandbox get shows the parts of the model described above: the restart policy (never), a runtime generation ID, the isolation backend (openshell-sandbox), a Ready condition (reason DependenciesReady, message “Supervisor session connected”), and the effective policy.

$ openshell sandbox create --name first --from openshell-workload:0.1.6 --detach
✓ Sandbox allocated (0s)
✓ Image already present (0s)
create to Ready: 2.5 s

$ openshell sandbox list
NAME   CREATED              PHASE
first  2026-10-31 21:07:06  Ready

$ openshell sandbox exec first -- uname -a
Linux 353948387e06 6.12.76-linuxkit #1 SMP Fri Mar  6 10:10:19 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux

$ openshell sandbox exec first -- id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),20(dialout),…

$ openshell sandbox upload first ./README.md /sandbox/
✓ Upload complete: /sandbox/README.md
upload round trip: ok

$ openshell logs first -n 15
[sandbox] [OCSF ] [ocsf] CONFIG:LOADED [INFO] Acknowledged initial policy revision as loaded [version:1] …
[gateway] [INFO ] [openshell_server::supervisor_session] supervisor session: relay opened successfully
[gateway] [INFO ] [openshell_server::grpc::sandbox] ExecSandbox (relay): command started

The logs show the session path: every exec is a relay over the supervisor’s session, not a connection into the container.

Test 2: The same from the Python SDK

02_sdk.py does the lifecycle in code, using the CLI’s registration of the local gateway (openshell) and its mTLS bundle:

from openshell import SandboxClient

with SandboxClient.from_active_cluster(cluster="openshell") as c:
    ref = c.create(workspace="default", name="sdk-demo",
                   spec={"template": {"image": "openshell-workload:0.1.6"}})
    c.wait_ready("sdk-demo", workspace="default", timeout_seconds=300)
    result = c.exec("sdk-demo", ["python3", "--version"], workspace="default")
    deletion = c.delete("sdk-demo", workspace="default")
    c.wait_deleted("sdk-demo", workspace="default", expected_sandbox_id=deletion.sandbox_id)

It went from Provisioning to Ready in 1.8 seconds and printed Python 3.12.3 and Ubuntu 24.04.5 LTS. Two rough edges: the SDK returns the phase as a raw enum number and doesn’t re-export the enum (the script reads names from the generated protobuf module), and the package’s classifier still says “Alpha”. With the gateway stopped, the CLI and SDK test scripts print one line and exit:

OpenShell gateway not reachable: run ./00_gateway.sh first

Test 3: Deny by default, per binary, changed live

policies/curl-only.yaml has one network rule:

network_policies:
  pypi:
    name: pypi-curl-only
    endpoints:
      - host: pypi.org
        port: 443
    binaries:
      - { path: /usr/bin/curl }

03_network_policy.sh creates a sandbox with it, tries three requests, adds a host live, and reads the deny lines:

pypi.org (curl):    200
pypi.org (python3): denied
httpbin.org (curl): denied

$ openshell policy update netdemo --add-endpoint httpbin.org:443 --binary /usr/bin/curl --wait
✓ Policy version 2 submitted (hash: 0e14d8f0560b)
✓ Policy version 2 loaded (active version: 2)
httpbin.org (curl): 200 after the update (10.5 s)

The log lines say exactly why each request failed, and name the real binary behind the python3 symlink:

DENIED /usr/bin/python3.12(0) -> pypi.org:443 [reason:transparent_tcp_policy_denied]
DENIED httpbin.org [reason:policy_dns_ineligible]
DENIED /usr/bin/curl(0) -> httpbin.org:443 [reason:transparent_tcp_policy_denied]

Three observations:

• The binary matters, not just the host. python3 was refused a host that curl could reach. In OpenShell, an allowlist is a list of binary and destination pairs.
• DNS is mediated too. Names resolve to synthetic addresses in 198.18.0.0/15 that only the supervisor maps to real ones. A name outside the policy is flagged at the lookup (policy_dns_ineligible) but still gets a synthetic address, so the connection that follows can be denied with the binary’s name on it: httpbin.org became 198.18.0.3, and the deny line names /usr/bin/curl.
• Most of the 10.5 seconds is the poll. In the recording’s timestamps, --wait returned 10.0 seconds after the submit, and the first allowed request succeeded half a second later. That fits the supervisor’s default 10-second poll, though the delay isn’t fixed: part 4 measured 7.9 to 12.5 seconds on EKS. push delivery is the knob if an agent waits on policy changes.

My first attempt used example.com as the second host, and it never resolved, with the reason policy_dns_upstream_nxdomain. It turned out the network I was on doesn’t resolve example.com at all, inside or outside OpenShell. The reason code made that a two-minute diagnosis.

Verdict

OpenShell’s architecture is the most convincing part of it. Splitting every sandbox into a workload that holds nothing and a supervisor that holds everything removes a whole class of failure: there is no secret in the agent’s environment to leak (though a known 0.1.6 weakness can still put the real AWS session within the agent’s reach; see part 5), and no socket the agent can open that doesn’t pass a policy decision with the binary’s name on it. The local run backs up the network half of that; it didn’t attach any credentials. Sandboxes start in about two seconds, the deny lines are specific, and the policy model (locked filesystem rules, live network rules) is easy to reason about.

What you sign up for:

Who should look at it: teams that want agents to run unattended with real credentials and want the rules enforced outside the agent, in a form a reviewer can read. Who should wait: anyone who needs suspend and resume, idle expiry or signed images today, or who can’t accept a weekly release train whose stability policy is still in review and has already been waived once.

The open question for the rest of the series is whether that clean picture survives a real cluster: a CNI enforcing the outer fence, two Pods and a volume per sandbox on two nodes, and AWS credentials signed in the supervisor. Part 2 starts with the install.