OpenSandbox and AWS Credentials: SigV4 Signing at the Egress
By Fabio Douek
Jump to section
- Overview
- Architecture
- Setup
- Step 1: The companion code
- Step 2: Two IAM roles
- Step 3: The broker, on EKS Pod Identity
- Step 4: The derived egress image, with the SigV4 addon
- Step 5: A workload image with boto3
- Testing
- Test 1: Bedrock with a short-term key, and no signing
- Test 2: Who the sandbox is to AWS
- Test 3: S3, from 19 bytes to a 20 MiB multipart upload
- Test 4: DynamoDB, Bedrock Converse, and the 1 MiB limit
- Test 5: CloudTrail names the sandbox
- Test 6: Trying to get credentials out
- Test 7: Rotation, the one-hour cap and revocation
- Test 8: The same flow under Kata
- Results summary
- What this does not solve
- Cleanup
- Verdict
- What you sign up for
- Who should use it
Explain (TLDR) like I am...
Imagine a robot helper that needs a library card, but you never want it to hold the real one. So it carries a pretend card, and a guard at the door swaps in the real card on the way to the library desk, then takes it back.
The guard is strict: if the robot asks for a second card, or says it is going to the library while walking to the toy shop, the answer is no. The real card still sits with the guard right next to the robot, so the grown-ups give it a card that only works for one small job and swap it for a fresh one every half hour.
This post tests a way to let AI agent code call AWS without holding AWS keys: OpenSandbox 1.2.0, an Apache 2.0 project of the Agentic AI Foundation, on Amazon EKS. A broker assumes one IAM role session per sandbox, and a proxy beside the sandbox signs each request with those temporary credentials.
Attribution is clean: CloudTrail recorded the sandbox's STS and Bedrock calls under its ID. The residual exposure is material, though. Within the role's permissions the sandbox acts as the principal for as long as it runs, the credentials sit in the same pod as the untrusted code, and the signing proxy is custom code the operator owns.
The presenting problem is an agent that must call AWS but should never hold an AWS key. The intervention: a broker issues one-hour role credentials per sandbox, and a proxy in the sandbox's pod signs each request with them while the sandbox holds placeholders. I tested it on Amazon EKS with S3, DynamoDB, Bedrock and STS.
Every call within the limits worked, and every attempt to mint new credentials, spoof a host on the AWS binding or find a key failed. Contraindications: request bodies over about 1 MiB outside S3 are cut off, and the real keys share the pod (under Kata, the virtual machine) with the code.
There is relief in handing an agent AWS access without the quiet dread of a key sitting in its environment. In my runs the sandbox only ever held placeholders, and every AWS call still ran under a name you can trace: the sandbox's own ID, which CloudTrail showed for the calls it records.
The harder conversation is about what that access still means. For as long as it runs the sandbox can do whatever the role allows, the keys live one container away, and the signing proxy is code your team now owns and has to trust. The comfort comes from a narrow role, not from the proxy.
Think of the egress proxy as the band's tour manager. The player on stage works under a stand-in name, and the manager signs every booking on the way out with the band's real signature, so S3, DynamoDB and Bedrock all take the call.
The manager refuses three things: hiring a second manager, sending a contract to a venue other than the one on the envelope, and signing a contract too long to read first, past about a megabyte outside S3. The catch is that the real signature rides in the same van as the player.
The story is AWS access with no AWS key in the sandbox. A broker on Amazon EKS gives each OpenSandbox sandbox its own one-hour IAM role session, and the egress proxy signs requests with it, so the AWS CLI and boto3 work unchanged with placeholder keys. CloudTrail names the sandbox behind the calls it records.
The honest positioning has limits buyers will ask about. The signing proxy is companion code, not an OpenSandbox feature, non-S3 bodies over about 1 MiB are cut off, and the sandbox can do anything the role allows while it runs.
Overview
OpenSandbox has been a project of the Agentic AI Foundation, a directed fund of the Linux Foundation, since the announcement on October 22, 2026; part 1 covers that move.
This post, part 5 of 6, gives sandboxes on the EKS cluster from part 2 access to AWS without putting an AWS key in them. Part 4 showed the Credential Vault adding a credential as a header, but most AWS APIs need a SigV4 signature computed from the secret key, the time and the request itself, which no added header can supply. So the flow has three pieces: a broker that assumes one IAM role session per sandbox and writes the temporary credentials into the vault, which adds them to AWS requests as three headers, and a small addon in the egress sidecar that removes those headers and signs the request again. The sandbox runs the stock AWS CLI and boto3 with placeholder keys.
The broker and the addon are my companion code, not OpenSandbox features; 1.2.0 supplies the vault, the egress proxy and the extension point, an extra mitmproxy script baked into a derived egress image. What I found:
- The stock clients worked unchanged. STS, S3 (up to a 20 MiB multipart upload), DynamoDB and Bedrock answered the AWS CLI, and boto3 for STS, S3 and Bedrock, in a sandbox that held only placeholders. CloudTrail recorded the STS and Bedrock calls under the sandbox’s ID, and the same flow ran under Kata.
- Bedrock didn’t need the addon at all. A Bedrock short-term API key is a bearer token, and the vault replaced the placeholder
Authorizationheader with it. - The limits are specific. Request bodies over about 1 MiB to services other than S3 are cut off. And for as long as it runs, the sandbox can do anything the role allows, from a pod that holds the real credentials; a credential set taken from that pod stays valid for up to an hour.
Part 3 covers the sandbox lifecycle, pools and the switch to Kata, and part 6 stress-tests the same two nodes, the addon included. Everything here runs under runc except Test 8, with the server’s egress in dns+nft mode. Every number comes from my runs on November 1, 2026 (UTC); I say when something comes from the code or the AWS docs instead.
Architecture
A SigV4 signature is an HMAC over the method, path, query, a set of headers, the body’s hash and a timestamp, keyed from the secret access key (AWS docs). The session token can travel as a header; the signature has to be computed where the secret is. In a sandbox pod, the only place that sees every request and may hold a secret is the egress sidecar.
osb-broker. It assumes the role osb-sandbox once per sandbox and writes the result into the sandbox's vault. The sandbox signs with a placeholder key (grey); OpenSandbox's system.py adds the three temporary values as headers, and the addon removes them and signs the request again with the real values (green). A request the addon can't sign safely is refused with a 403 or 501, or its connection is closed (red). The green bracket stands for every AWS service the sandbox calls, STS included.- The broker creates the sandbox and assumes
osb-sandboxfor it: placeholder keys, a deny-by-default policy allowing eight AWS hostnames, the credential proxy, and an hour-long STS session named after the sandbox and narrowed by a session policy (Step 2). - It writes the vault through the lifecycle server’s proxy: the three temporary values, and one
customHeadersbinding,aws-sigv4, for those hosts. Every 30 minutes it replaces the values withpatch. - The AWS CLI or boto3 signs a request with the placeholder key, and the sidecar’s transparent proxy intercepts it.
- OpenSandbox’s
system.pymatches the binding by the request’sHostheader and adds the threex-osb-aws-*headers. Up to here this is part 4’s vault, unchanged. - mitmproxy then runs the addon, which removes the three headers, checks the request, and either signs it again or refuses it.
- The signed request leaves for the host named in the TLS server name, through nftables like any other packet.
- AWS sees the role session, and CloudTrail records the sandbox ID as its session name.
| Credential | Held by | Lifetime |
|---|---|---|
The broker’s own, role osb-broker | the broker pod, from the EKS Pod Identity agent | managed by EKS Pod Identity |
A sandbox’s temporary credentials, role osb-sandbox | the broker, only while it writes the vault; after that, only the sandbox’s vault, in the egress sidecar’s memory | one hour, replaced every 30 minutes |
| A Bedrock short-term key (Test 1 only) | the same two places | no longer than the session it was made from |
The placeholders (AKIA_OSB_PLACEHOLDER, osb-placeholder-secret; AWS_BEARER_TOKEN_BEDROCK in Test 1) | the sandbox’s environment | never valid |
sts:AssumeRole, ecr:GetAuthorizationToken, s3express:CreateSession and similar. That's why the role and every session policy carry an explicit Deny on those actions, the addon refuses them before signing, and the role allows little else (Steps 2 and 4, Test 6).
Setup
Step 1: The companion code
The scripts run on the cluster from part 2 and use two neighboring directories: ../opensandbox-eks for the server, env.sh and the ECR registry, and ../opensandbox-lifecycle from part 3 for the port-forward helper and server_config.py, the one script that changes the server’s configuration. Run part 2’s 01_cluster.sh to 04_install.sh and part 4’s 00_image.sh first. You also need Terraform 1.9 or later and uv, an existing S3 bucket in us-east-1 named by AWS_TEST_BUCKET in a secrets file outside the repository (the OSB_SECRETS_ENV variable points to it), and access in Bedrock to the us.anthropic.claude-haiku-4-5-20251001-v1:0 inference profile.
git clone https://github.com/fabiodouek/my2centsai-blog-samples.git
cd my2centsai-blog-samples/opensandbox-aws-credentials
uv sync
| File | What it does |
|---|---|
01_iam.sh, terraform/ | The two IAM roles, the Pod Identity association and a DynamoDB test table (Step 2) |
broker/, 02_broker.sh, manifests/broker.yaml | The broker (Step 3) |
egress/, build_egress.sh | The SigV4 addon, its tests and the derived egress image (Step 4) |
00_image.sh, image/Dockerfile | The workload image with the AWS CLI and boto3 (Step 5) |
01_bedrock_bearer.py to 08_kata.py | One script per test |
common.py | Shared helpers and the output scrubber |
The tests run as uv run python 02_identity.py and so on. Each asks the broker for its sandboxes, which carry the label example=opensandbox-aws-credentials, deletes them at the end (also on failure), and exits 1 if a check fails. The scripts read the secrets file with a parser, never source it, print the bucket as example-bucket and the account as 123456789012, and show an AWS error as its error code only.
Step 2: Two IAM roles
01_iam.sh runs Terraform with its own state, separate from part 2’s:
osb-brokeris trusted by EKS Pod Identity for one service account,osb-brokerin namespaceopensandbox-broker. It may assumeosb-sandbox, plus one read-only IAM action for Step 3’s check.osb-sandboxis trusted byosb-brokeronly; no pod has a Pod Identity association for it. It allows the test bucket (list, get, put, delete, abort a multipart upload), the test table’s item actions, query and scan, one Bedrock inference profile and its model, and Bedrock short-term keys. Then it explicitly denies every action on this list:
[
"sts:Assume*", "sts:GetDelegatedAccessToken", "sts:GetFederationToken",
"sts:GetServiceBearerToken", "sts:GetSessionToken", "sts:GetWebIdentityToken",
"sts:SetContext", "sts:SetSourceIdentity", "sts:TagSession",
"s3express:CreateSession",
"ecr:GetAuthorizationToken", "ecr-public:GetAuthorizationToken",
"codeartifact:GetAuthorizationToken",
"redshift:GetClusterCredentials", "redshift:GetClusterCredentialsWithIAM",
"redshift-serverless:GetCredentials",
"secretsmanager:GetSecretValue",
"iam:*"
]
That’s broker/vending_actions.json, reflowed. The Allow statements are in sandbox_permissions.json, and the role’s policy (Terraform) and every session policy (the broker) are built from both files, so they can’t drift apart. An explicit Deny wins over any Allow, including one added to the role later.
The trust policy has one non-obvious line:
data "aws_iam_policy_document" "sandbox_trust" {
statement {
actions = ["sts:AssumeRole", "sts:TagSession"]
principals {
type = "AWS"
identifiers = [aws_iam_role.broker.arn]
}
}
}
resource "aws_iam_role" "sandbox" {
name = "osb-sandbox"
assume_role_policy = data.aws_iam_policy_document.sandbox_trust.json
max_session_duration = 7200 # two hours on purpose: see Test 7
}
EKS Pod Identity tags the broker’s session with the cluster, namespace, service account and pod, and those tags are transitive: they travel with the broker’s AssumeRole, so the trust has to allow sts:TagSession. In a separate check, not in the recordings, removing it from the trust made the broker’s AssumeRole fail within 10 seconds with AccessDenied on sts:TagSession. The role allows two hours on purpose; Test 7 shows why.
01_iam.sh: Terraform and its checks
$ ./01_iam.sh
== terraform apply
cluster osb-eks, Region us-east-1 (full output: logs/terraform-apply.log)
Plan: 6 to add, 0 to change, 0 to destroy.
aws_iam_role.broker: Creation complete after 1s
aws_eks_pod_identity_association.broker: Creation complete after 0s
aws_iam_role.sandbox: Creation complete after 9s
aws_iam_role_policy.sandbox: Creation complete after 1s
aws_iam_role_policy.broker: Creation complete after 1s
aws_dynamodb_table.test: Creation complete after 17s
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
took 24s
== checks
ok IAM role osb-broker exists
ok IAM role osb-sandbox exists
ok Pod Identity association for opensandbox-broker/osb-broker
ok no Pod Identity association in namespace opensandbox (the sandboxes')
ok DynamoDB table osb-cred-test is ACTIVE
took 5s
Six resources in 24 seconds, 17 of them for the DynamoDB table, and no Pod Identity association in the sandboxes’ namespace. In the console, through a read-only sign-in with the account masked:

Step 3: The broker, on EKS Pod Identity
EKS Pod Identity works per service account, and so per pod: an association on the sandboxes’ service account would hand its credentials endpoint to the untrusted container (how Pod Identity works). So the sandboxes get none, and a separate broker does: a one-replica Deployment in its own namespace, with automountServiceAccountToken: false and a restricted Pod Security label. Its API listens on 127.0.0.1 inside its pod, so only kubectl port-forward reaches it; the test scripts open one themselves.
When a test asks for a sandbox, the broker does three things. Simplified from broker/broker.py:
AWS_HOSTS = [
"sts.amazonaws.com", "sts.us-east-1.amazonaws.com",
"s3.us-east-1.amazonaws.com", "*.s3.us-east-1.amazonaws.com",
"s3.amazonaws.com", "*.s3.amazonaws.com",
"dynamodb.us-east-1.amazonaws.com", "bedrock-runtime.us-east-1.amazonaws.com",
]
sb = SandboxSync.create(
image=image,
env={
"AWS_ACCESS_KEY_ID": "AKIA_OSB_PLACEHOLDER",
"AWS_SECRET_ACCESS_KEY": "osb-placeholder-secret",
"AWS_REGION": "us-east-1", "AWS_DEFAULT_REGION": "us-east-1",
"AWS_CA_BUNDLE": "/opt/opensandbox/merged-ca-certificates.pem",
"AWS_EC2_METADATA_DISABLED": "true",
},
network_policy=NetworkPolicy(
default_action="deny",
egress=[NetworkRule(action="allow", target=host) for host in AWS_HOSTS],
),
credential_proxy=CredentialProxyConfig(enabled=True),
connection_config=connection(), # the server's in-cluster Service, through its proxy
)
creds = sts.assume_role(
RoleArn=SANDBOX_ROLE_ARN,
RoleSessionName=sb.id, # what CloudTrail shows for every call
Policy=session_policy(), # the role's Allow statements and the Deny again
DurationSeconds=3600,
)["Credentials"]
sb.credential_vault.create(
credentials=[
{"name": "aws-akid", "source": {"type": "inline", "value": creds["AccessKeyId"]}},
{"name": "aws-secret", "source": {"type": "inline", "value": creds["SecretAccessKey"]}},
{"name": "aws-token", "source": {"type": "inline", "value": creds["SessionToken"]}},
],
bindings=[{
"name": "aws-sigv4",
"match": {"schemes": ["https"], "hosts": AWS_HOSTS,
"methods": ["GET", "HEAD", "POST", "PUT", "DELETE"], "paths": ["/*"]},
"auth": {"type": "customHeaders", "headers": [
{"name": "x-osb-aws-access-key-id", "credential": "aws-akid"},
{"name": "x-osb-aws-secret-access-key", "credential": "aws-secret"},
{"name": "x-osb-aws-session-token", "credential": "aws-token"},
]},
}],
)
The SDKs sign with whatever key they’re given, so the placeholder only has to exist; its underscores keep it from matching the AKIA[0-9A-Z]{16} pattern secret scanners look for. There’s no AWS_SESSION_TOKEN, and no AWS_ACCOUNT_ID, which would switch DynamoDB to an account-based endpoint outside the allowlist. AWS_CA_BUNDLE is part 4’s Test 3 lesson: the AWS CLI trusted the proxy’s CA only through it or the undocumented REQUESTS_CA_BUNDLE.
Every 1,800 seconds, half the session, the broker assumes the role again and replaces the three values in one patch carrying the vault’s revision as expectedRevision. If the vault answers 404, as after a sidecar restart or a pause, it writes the vault again.
02_broker.sh builds and pushes the broker image, deploys it (a 44-second rollout), and checks from inside the broker pod that Pod Identity gave it credentials and that the sandbox role’s Deny is real. I left out the recording’s whoami lines, which predate the role’s two-hour maximum; Test 7 shows the current result.
== checks
ok EKS Pod Identity injected its credentials endpoint into the broker pod
$ python broker.py deny-check
session osb-deny-check of osb-sandbox, with the session policy (key ASIA…, expires 2026-11-01T07:46:43Z)
sts:AssumeRole: AccessDenied on sts:TagSession with no reason; asking AWS why (authorization id <id>, about a minute)
sts:AssumeRole ok AccessDenied on sts:TagSession: explicit deny in the role policy and the session policy (AWS's authorization details)
ecr:GetAuthorizationToken ok AccessDenied: explicit deny in identity-based policy
secretsmanager:GetSecretValue ok AccessDenied: explicit deny in identity-based policy
iam:ListRoles ok AccessDenied: explicit deny in identity-based policy
all 4 refused, each shown to be by the explicit Deny
ok deny-check: AWS refused every credential-vending call (its last line says how many it showed the Deny refused)
deny-check assumes osb-sandbox with the session policy, as for a sandbox, and calls four actions the Deny covers. Three refusals named the explicit deny. AssumeRole was refused on sts:TagSession (Pod Identity’s tags ride along, and the Deny lists that action too) with only an authorization ID, so the broker asked IAM to explain it (GetRequestAuthorizationDetails, an API in preview): an explicit deny in both the role policy and the session policy. This is the only place the second layer shows, because inside a sandbox the addon refuses those calls before AWS sees them (Test 6).
Step 4: The derived egress image, with the SigV4 addon
OpenSandbox’s egress process loads its own system.py into mitmproxy first, then any scripts named in OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT (launch.go). A create request can’t set that variable: the server forwards only an allowlist of OPENSANDBOX_EGRESS_* variables and rejects the rest (constants.py, helpers.py; from the code, not tried). So the route is an image with the variable baked in, and the server’s [egress] image pointing at it:
ARG BASE
FROM ${BASE}
COPY sigv4_resign.py /var/egress/custom/sigv4_resign.py
# Fail the build, not the sidecar: load the addon with the image's own python3 and mitmproxy.
RUN PYTHONDONTWRITEBYTECODE=1 /usr/bin/python3 -c "import importlib.util as u; \
s = u.spec_from_file_location('sigv4_resign', '/var/egress/custom/sigv4_resign.py'); \
m = u.module_from_spec(s); s.loader.exec_module(m); \
assert all(callable(getattr(m, hook, None)) for hook in ('requestheaders', 'request', 'responseheaders', 'response'))"
ENV OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT=/var/egress/custom/sigv4_resign.py
BASE is part 2’s ECR mirror of the 1.2.0 egress image, pinned by digest. The image is server-wide, so every sidecar runs the addon, part 4’s GitHub and Anthropic traffic included; requests without the three headers pass through byte for byte, which its tests check.
The addon is about 800 lines of standard-library Python on the image’s mitmproxy 11.0.2. It removes the three headers from every request that carries them, then:
| A request that… | The addon | The sandbox sees |
|---|---|---|
carries a second Host, Authorization, x-amz-content-sha256 or X-Amz-Target header | refuses it | 403 |
has a Host that differs from the TLS server name (SNI), or an SNI outside .amazonaws.com | kills it (host-sni-mismatch) | a closed connection |
| is presigned (the signature in the query string) | forwards it as it came, never signed again | AWS’s answer |
has Authorization: Bearer | refuses it (bearer-on-sigv4-binding) | 403 |
| is a credential-vending call | refuses it (credential-vending) | 403 |
| uses SigV4a or a signed-chunk upload | refuses it | 501 |
| goes to a service other than S3 with a body that may stream (over 1 MiB, chunked, or HTTP/2 without a length) | kills it (body-may-stream-without-payload-hash) | a closed connection |
| anything else signed with SigV4 | signs it again with the real values | AWS’s answer |
A few rules need a sentence each:
- Host and SNI. Because the vault picks the binding from the
Hostheader (#1758), the addon requires theHostto equal the TLS server name and pins the upstream connection to that name before it signs. - Credential vending. STS is an allowlist: only
GetCallerIdentity,GetAccessKeyInfoandDecodeAuthorizationMessagepass. Beyond STS it’s a blocklist (ECR, CodeArtifact, S3 Express, IAM and Redshift token calls), checked again once the body has arrived. - What gets signed, and when. Region and service come from the client’s credential scope (
bedrock-runtimesigns asbedrock). S3 clients declare the body’s hash inx-amz-content-sha256, so an S3 request is signed when its headers arrive and its body streams. Other services are signed from the body’s SHA-256, so the addon must hold the whole body first, and this image’s mitmproxy streams any body over 1 MiB (stream_large_bodies: 1m), so a larger one is killed. - Failure. Any exception on a request that carried the headers kills it.
- Error bodies. On requests it signed, the addon holds AWS error bodies (4xx and 5xx) and replaces the real key ID and session token with
[REDACTED]if AWS echoes them; one it can’t scrub is killed. That’s covered by the tests, and no recorded run hit it.
Each decision is a log line that starts credential proxy: [sigv4], at warn level. Part 4 saw credential proxy: lines arrive in batches; with that prefix and a flush after every line, one request was enough in a check before these recordings. The tests are 197 unit tests and 17 end-to-end tests that run a real mitmdump behind OpenSandbox’s own system.py, on the laptop and inside the 1.2.0 egress image. build_egress.sh runs the unit tests, builds for linux/amd64, pushes, and switches the server through part 3’s server_config.py:
build_egress.sh: build, push, and the server switch
$ ./build_egress.sh
== preconditions
ok base image opensandbox-eks/egress:release-1.2.0 in ECR (sha256:a3d30246e557...)
ok addon unit tests: 197 passed in 0.94s
ok server now: runtime=runc egress_image=base egress_mode=dns+nft ingress=gateway
took 6s
== build and push opensandbox-eks/egress-sigv4:release-1.2.0-197d79bc310b
FROM opensandbox-eks/egress@sha256:a3d30246e557... (release-1.2.0), linux/amd64 (full output: logs/build-egress.log)
ok build-time check: the image's python3 loaded sigv4_resign.py with all four hooks
ok pushed opensandbox-eks/egress-sigv4:release-1.2.0 and :release-1.2.0-197d79bc310b
ok digest sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719
took 6s
== switch the server's [egress] image
server now: runtime=runc egress_image=base egress_mode=dns+nft ingress=gateway (revision 17)
server wanted: runtime=runc egress_image=opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719 egress_mode=dns+nft ingress=gateway
config.toml: changes:
--- running config.toml
+++ new config.toml
@@ -20,3 +20,3 @@
[egress]
-image = "opensandbox-eks/egress:release-1.2.0"
+image = "opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719"
mode = "dns+nft"
helm upgrade: revision 17 -> 18 (deployed) in 19 s, manifests changed
server pod opensandbox-server-7c8b74fc7c-fjxbl replaced opensandbox-server-775f8ddf57-z5ll2, ready=true
ok server /health: {"status":"healthy"}
ok server /version: {"version":"1.2.0"}
ok release revision 18 is deployed
ok the server pod runs with: runtime=runc egress_image=opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719 egress_mode=dns+nft ingress=gateway
ok gateway kept: server.gateway and [ingress] mode = gateway, host from values/server-gateway.yaml
The server pod was replaced: a kubectl port-forward started before this has ended. The Python
scripts in this directory start their own; restart yours if you use one.
took 42s
egress image: <registry>/opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719
new sandboxes with an egress sidecar now run the SigV4 addon; restart any port-forward to the server
The 197 unit tests passed, the build-time check loaded the addon with the image’s own Python, and the switch was one line of config.toml and a 19-second Helm upgrade (42 seconds with its checks). The server references the image by digest, so no node can run an older build under the same tag.
Step 5: A workload image with boto3
Part 4’s egress-tools:1 image has the AWS CLI v2 2.39.3 but not boto3, and a sandbox here can’t pip install anything: its policy allows AWS hosts only. 00_image.sh builds opensandbox-eks/aws-tools:1 from it with boto3==1.43.123, the broker’s pin, and pushes it to ECR.
Testing
The runs below are the recorded ones, minus port-forward and cleanup lines, with the broker’s key prefix shortened to ASIA…. A create through the broker (the sandbox, the AssumeRole and the vault write) took 6.0 to 6.7 seconds under runc, from the cast timestamps.
Test 1: Bedrock with a short-term key, and no signing
Bedrock accepts short-term API keys as bearer tokens, which boto3 and the AWS CLI send when AWS_BEARER_TOKEN_BEDROCK is set. A key is bedrock-api-key- plus the base64 of a SigV4 presigned URL for CallWithBearerToken, and it can’t outlive the session that made it. With bedrock_bearer on, the broker makes one from the sandbox’s session in the format of AWS’s token generator (locally; nothing goes to AWS), stores it in the vault, and binds the Bedrock host with a bearer binding instead of the SigV4 one. The sandbox gets a placeholder AWS_BEARER_TOKEN_BEDROCK.
Part 4’s Test 5 showed a bearer binding replacing an Authorization header the sandbox had set itself, against the echo server. The question here is whether that holds for the header the AWS SDKs build for Bedrock:
== 1. a sandbox whose Bedrock key is a bearer binding
sandbox d8ce9fc1-2a61-4c12-ae13-f4165408ea69 vault revision 1 credentials expire 2026-11-01T18:03:04Z bedrock_bearer=True
$ env | grep -E '^AWS_(BEARER|ACCESS|SECRET)' | sort
AWS_ACCESS_KEY_ID=AKIA_OSB_PLACEHOLDER
AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-osb-placeholder
AWS_SECRET_ACCESS_KEY=osb-placeholder-secret
exit code 0
PASS the sandbox holds a placeholder bearer token
PASS the bearer value is not a real key
== 2. Converse over the bearer path (boto3 reads AWS_BEARER_TOKEN_BEDROCK)
$ python3 (boto3 bedrock-runtime converse, pinned Haiku model)
HTTP 200 | reply: Ready
exit code 0
$ aws bedrock-runtime converse --model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 --messages ...
exit code 0
Then the vault’s side. The script makes one SigV4 call to STS as a control: the addon logs it, which flushes the sidecar’s log, so any line still waiting in it comes out:
== 3. what the sidecar did
$ aws sts get-caller-identity # a SigV4 call: the addon logs it, which flushes the sidecar's log
control: the addon's `re-signed` line for the STS call reached the log: True
PASS the sidecar log flushed (the STS call's re-signed line is in it)
log: credential proxy: [sigv4] re-signed POST sts.us-east-1.amazonaws.com/ service=sts region=us-east-1 key=ASIA... payload=sha256 hook=request
vault: `applied binding=bedrock-bearer` lines: 2
log: credential proxy: applied binding=bedrock-bearer revision=1 host=bedrock-runtime.us-east-1.amazonaws.com method=POST headers=Authorization substitutions=
log: ... 1 more line(s)
addon lines in this window (the STS control call included): 1; re-signed: 0; refused bearer-on-sigv4-binding: 0
== 4. the result
RESULT the vault replaced the placeholder Authorization header: the call worked over the bearer path,
and the addon did not re-sign it. A Bedrock short-term key needs no SigV4 re-signing.
PASS bearer path: the call succeeded with a placeholder token, the vault logged `applied binding=bedrock-bearer`, and the addon neither re-signed nor refused it
It does. The sandbox sent only the placeholder, and Bedrock answered: boto3 got HTTP 200 and a reply, and the AWS CLI exited 0. The vault logged two applied binding=bedrock-bearer lines for the two calls, the one shown with headers=Authorization: it replaced the placeholder with the real key. The addon’s only line in that window was the STS control call’s, so the script’s re-signed: 0 counts Bedrock requests only. In this mode the SigV4 binding leaves out the Bedrock host, so the Bedrock requests carried none of its three headers, and the addon neither signed nor refused them. An earlier take without the control call showed no applied binding line at all; with the addon’s line flushing the log, both appeared, which fits part 4’s finding that these lines arrive in batches.
So for Bedrock Runtime, the stock vault and a short-term key are enough. The role needs bedrock:CallWithBearerToken (mine allows only short-term keys), and per the docs, bearer keys cover Bedrock and Bedrock Runtime actions only, not Agents.
Test 2: Who the sandbox is to AWS
02_identity.py asks the same question two ways from inside the sandbox:
== 1. the broker makes a sandbox and gives it credentials
sandbox 94548e3a-ca99-447c-8b9b-1a24e92cea23 vault revision 1 credentials expire 2026-11-01T13:46:07Z bedrock_bearer=False
== 2. the sandbox's identity, from the AWS CLI and from boto3
$ aws sts get-caller-identity --query Arn --output text
arn:aws:sts::123456789012:assumed-role/osb-sandbox/94548e3a-ca99-447c-8b9b-1a24e92cea23
exit code 0
$ python3 -c 'print(boto3.client("sts").get_caller_identity()["Arn"])'
arn:aws:sts::123456789012:assumed-role/osb-sandbox/94548e3a-ca99-447c-8b9b-1a24e92cea23
exit code 0
PASS the AWS CLI returns the assumed-role ARN with the sandbox ID as the session name (session name 94548e3a-ca99-447c-8b9b-1a24e92cea23)
PASS boto3 returns the same ARN
session name = 94548e3a-ca99-447c-8b9b-1a24e92cea23
sandbox id = 94548e3a-ca99-447c-8b9b-1a24e92cea23
PASS session name equals the sandbox ID
Both clients got assumed-role/osb-sandbox/<sandbox ID>: their requests left signed with AKIA_OSB_PLACEHOLDER and reached STS signed by the addon. This is everything the sandbox holds:
== 3. what the sandbox holds: placeholders only
$ env | grep '^AWS_' | sort
AWS_ACCESS_KEY_ID=AKIA_OSB_PLACEHOLDER
AWS_CA_BUNDLE=/opt/opensandbox/merged-ca-certificates.pem
AWS_DEFAULT_REGION=us-east-1
AWS_EC2_METADATA_DISABLED=true
AWS_REGION=us-east-1
AWS_SECRET_ACCESS_KEY=osb-placeholder-secret
exit code 0
All 11 checks passed, including no AWS_SESSION_TOKEN or AWS_ACCOUNT_ID and no temporary key in any variable or in the files under /root, /home, /tmp and /etc.
Test 3: S3, from 19 bytes to a 20 MiB multipart upload
03_s3.py lists, uploads 19 bytes, 5 MiB in one PutObject and 20 MiB as a multipart upload, downloads the 20 MiB object, compares checksums, and does a put and a list with boto3:
The S3 operations
== 2. listing
$ aws s3 ls
aws: [ERROR]: An error occurred (AccessDenied) when calling the ListBuckets operation ...
exit code 254
PASS aws s3 ls without a bucket is denied (no s3:ListAllMyBuckets) (AccessDenied)
$ aws s3 ls s3://example-bucket/ | head -3
2026-10-30 02:38:37 32 from-sandbox.txt
2026-10-30 02:38:32 20 hello.txt
exit code 0
PASS aws s3 ls s3://example-bucket/ works
== 3. uploads: 19 bytes, 5 MiB (one PutObject), 20 MiB (multipart)
$ wc -c /tmp/small.txt /tmp/five.bin /tmp/twenty.bin
19 /tmp/small.txt
5242880 /tmp/five.bin
20971520 /tmp/twenty.bin
26214419 total
exit code 0
$ aws s3 cp /tmp/small.txt s3://example-bucket/osb-test/05db70cb-4284-4d96-9604-d5a8c7c64c16/small.txt --only-show-errors
(no output)
exit code 0
PASS 19-byte upload
$ aws s3 cp /tmp/five.bin s3://example-bucket/osb-test/05db70cb-4284-4d96-9604-d5a8c7c64c16/five.bin --only-show-errors
(no output)
exit code 0
PASS 5 MiB upload (single PutObject)
$ aws s3 cp /tmp/twenty.bin s3://example-bucket/osb-test/05db70cb-4284-4d96-9604-d5a8c7c64c16/twenty.bin --only-show-errors
(no output)
exit code 0
PASS 20 MiB upload (multipart)
$ aws s3 ls s3://example-bucket/osb-test/05db70cb-4284-4d96-9604-d5a8c7c64c16/
2026-11-01 12:46:47 5242880 five.bin
2026-11-01 12:46:46 19 small.txt
2026-11-01 12:46:48 20971520 twenty.bin
exit code 0
== 4. download the 20 MiB object and compare checksums
$ aws s3 cp s3://example-bucket/osb-test/05db70cb-4284-4d96-9604-d5a8c7c64c16/twenty.bin /tmp/twenty.back --only-show-errors
(no output)
exit code 0
sha256 before upload 8e7b25f19c282be8...
sha256 after download 8e7b25f19c282be8...
PASS 20 MiB round trip: the checksums match
== 5. boto3 from inside the sandbox
$ python3 (boto3 put_object + list_objects_v2)
put_object ok; list_objects_v2 sees: boto3.txt, five.bin, small.txt, twenty.bin
exit code 0
PASS boto3 put_object and list_objects_v2
Everything worked, and the 20 MiB object came back with the same SHA-256. aws s3 ls without a bucket was denied on purpose: listing every bucket needs s3:ListAllMyBuckets, which the role doesn’t have. Then the sidecar’s side:
== 6. what the sidecar did (its log, event and payload type only)
addon events: {'re-signed': 19}
payload types re-signed: {'sha256': 7, 'STREAMING-UNSIGNED-PAYLOAD-TRAILER': 6, 'UNSIGNED-PAYLOAD': 6}
PASS the addon re-signed the S3 requests (19 re-signed)
PASS the addon refused or killed none of them
Nineteen requests signed, none refused or killed. Since January 15, 2025 (botocore 1.36.0, AWS CLI 2.23.0), the AWS CLI and boto3 upload with aws-chunked encoding and a checksum trailer, declared as STREAMING-UNSIGNED-PAYLOAD-TRAILER (data integrity in the SDKs). The chunks aren’t signed, so signing again only changes headers, and the addon let the 20 MiB body stream. S3 clients declare one of these in x-amz-content-sha256, which is what lets S3 skip the 1 MiB limit.
Test 4: DynamoDB, Bedrock Converse, and the 1 MiB limit
Small bodies to other services work too, signed once the whole body has arrived. A DynamoDB PutItem and GetItem round trip and a Bedrock Converse call:
DynamoDB and Bedrock Converse over SigV4
== 2. DynamoDB PutItem and GetItem (SigV4, small body)
$ aws dynamodb put-item --table-name osb-cred-test --item '{"pk":{"S":"item-e7019e72"},"note":{"S":"written through the addon"}}'
(no output)
exit code 0
PASS DynamoDB PutItem
$ aws dynamodb get-item --table-name osb-cred-test --key ... --query Item.note.S --output text
written through the addon
PASS DynamoDB GetItem returns what PutItem wrote
== 3. Bedrock Converse (SigV4) on the pinned Haiku model
$ aws bedrock-runtime converse --model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 --messages ... --query output.message.content[0].text
ready
exit code 0
PASS Bedrock Converse returns a reply
Then the limit. DynamoDB caps one item at 400 KB, so the script uses BatchWriteItem with items of about 290 KB, three in one request and then four:
Building the two batches
== 4. the 1 MiB limit on a non-S3 body
$ python3 /tmp/build.py # 3 items of 290 KB
under-e7019e72: 3 items, request body 870,259 bytes (0.83 MiB)
exit code 0
$ python3 /tmp/build.py # 4 items of 290 KB
over-e7019e72: 4 items, request body 1,160,335 bytes (1.11 MiB)
exit code 0
$ aws dynamodb batch-write-item --request-items file:///tmp/under.json # 0.83 MiB
{
"UnprocessedItems": {}
}
exit code 0
$ aws dynamodb get-item ... under-0 --query Item.pk.S --output text
under-e7019e72-0
PASS a 0.83 MiB DynamoDB body is signed and forwarded
PASS DynamoDB processed every item (UnprocessedItems empty) and the first item can be read back (unprocessed key count 0)
$ aws dynamodb batch-write-item --request-items file:///tmp/over.json # 1.11 MiB
exit code 255; client saw: connection closed
log: credential proxy: [sigv4] killed POST dynamodb.us-east-1.amazonaws.com/ reason=body-may-stream-without-payload-hash service=dynamodb
log: credential proxy: [sigv4] killed POST dynamodb.us-east-1.amazonaws.com/ reason=body-may-stream-without-payload-hash service=dynamodb
log: ... 1 more line(s)
PASS the 1.11 MiB request fails on the client
PASS the client saw a closed connection, not an AWS answer or a 403 (connection closed)
PASS the sidecar log says killed ... reason=body-may-stream-without-payload-hash (3 line(s))
$ aws dynamodb get-item ... over-0 --query Item.pk.S --output text
None
PASS nothing reached DynamoDB: the item is not there
The 870,259-byte batch (0.83 MiB) was signed and written: UnprocessedItems came back empty, and its first item read back. The 1,160,335-byte batch (1.11 MiB) never reached DynamoDB: the client saw a closed connection, the sidecar logged three identical killed lines (the CLI’s retries, I think), and the batch’s first item wasn’t in the table afterwards. It’s not a 403 because mitmproxy can’t answer locally for a body it’s already streaming, so the reason is only in the sidecar’s log.
From the addon’s code rather than this run, the same limit applies to any service other than S3: any BatchWriteItem over about 1 MiB (DynamoDB itself allows 16 MB per request), a Bedrock Converse with large images or documents, and HTTP/2 requests with a body but no Content-Length. The AWS CLI and boto3 send HTTP/1.1 with a length.
Test 5: CloudTrail names the sandbox
05_cloudtrail.py makes an STS call and a Bedrock call from a fresh sandbox, then polls CloudTrail event history with my own credentials (the default profile) until the events show up:
== 2. CloudTrail event history (the default AWS profile)
waiting for CloudTrail ... 0 s (have: nothing)
waiting for CloudTrail ... 31 s (have: Converse)
waiting for CloudTrail ... 61 s (have: Converse)
== 3. the events
2026-11-01T13:46:29Z sts GetCallerIdentity role osb-sandbox session 853b2917-c187-4e4c-863b-6777a01acb59
2026-11-01T13:46:30Z bedrock Converse role osb-sandbox session 853b2917-c187-4e4c-863b-6777a01acb59
2026-11-01T13:46:27Z sts AssumeRole role osb-broker session eks-osb-eks-osb-broker-9548f802-b151-4252-90f8-750f9eb2cddf
AssumeRole: roleSessionName = 853b2917-c187-4e4c-863b-6777a01acb59, durationSeconds = 3600, caller = assumed-role/osb-broker/eks-osb-eks-osb-broker-9548f802-b151-4252-90f8-750f9eb2cddf
PASS STS GetCallerIdentity is in event history under the sandbox's session name
PASS the AssumeRole event names the sandbox ID as roleSessionName
PASS the AssumeRole was made by the broker's role session
PASS Bedrock Converse is in event history under the same session name
PASS the events are in this account
Converse was in event history by the second poll (31 seconds) and the other two by the fourth, about a minute and a half after the calls; CloudTrail’s docs say delivery averages about 5 minutes and isn’t guaranteed. The sandbox’s calls are under the user name <sandbox ID>, and the broker’s AssumeRole event links the two: roleSessionName the sandbox ID, durationSeconds 3600. The console shows the same filter:

Event history is free but holds management events only, and Bedrock’s Converse is one. S3 object and DynamoDB item calls are data events, recorded only by a trail you pay for ($0.10 per 100,000 data events delivered); CloudTrail Lake, the other route, stopped taking new customers on May 31, 2026, as the screenshot’s banner says. I didn’t set up a trail, so Test 3’s uploads aren’t here.
Test 6: Trying to get credentials out
06_attacks.py gives one sandbox the standard credentials and tries seven ways to get something real out of it:
| Attack | Result |
|---|---|
A. Credential vending from the sandbox: sts assume-role, get-session-token, get-federation-token, ecr get-login-password | The three STS calls refused by the addon with 403 before signing; ECR unreachable (not on the allowlist) |
| B. A presigned URL made in the sandbox with the placeholder key | Forwarded unchanged; S3 answered 403; no temporary key in the response |
C. The Host spoof from part 4, against the AWS binding, in five cases | A Host naming another bound host was killed; a connection dialed to an S3 address still reached STS (the upstream is pinned to the TLS name); nothing was added when no binding matched or the connection wasn’t decrypted |
| D. The instance metadata service and the Pod Identity agent | Both time out (curl exit 28); no AWS_CONTAINER_* variables |
| E. A search for a temporary key, a session token or the three header names | A planted canary was found, so the search works; then 0 files under /root, /home, /tmp, /etc, /var and /opt, and 0 in /proc/*/environ |
| F. The sidecar’s API without its token | 401, 403 and 401, as in part 4 |
G. A Bearer header to a SigV4-bound host | 403 with x-osb-egress-refused: bearer-on-sigv4-binding |
Credential vending first:
== A. credential-vending calls from inside the sandbox
$ aws sts assume-role --role-arn arn:aws:iam::123456789012:role/osb-sandbox --role-session-name escape
exit code 254; client saw: Unknown (the CLI cannot parse the addon's 403); the addon's reason: credential-vending
log: credential proxy: [sigv4] refused 403 POST sts.us-east-1.amazonaws.com/ reason=credential-vending service=sts
PASS sts assume-role: refused, no credential returned (1 log line(s))
$ aws sts get-session-token
exit code 254; client saw: Unknown (the CLI cannot parse the addon's 403); the addon's reason: credential-vending
log: credential proxy: [sigv4] refused 403 POST sts.us-east-1.amazonaws.com/ reason=credential-vending service=sts
PASS sts get-session-token: refused, no credential returned (1 log line(s))
$ aws sts get-federation-token --name escape
exit code 254; client saw: Unknown (the CLI cannot parse the addon's 403); the addon's reason: credential-vending
log: credential proxy: [sigv4] refused 403 POST sts.us-east-1.amazonaws.com/ reason=credential-vending service=sts
PASS sts get-federation-token: refused, no credential returned (1 log line(s))
$ aws ecr get-login-password --region us-east-1
exit code 255; client saw: could not connect; output 96 characters
PASS ecr get-login-password: no token (the ECR host is not on the allow list)
Each STS call got the addon’s 403 (credential-vending) and was never signed, so AWS never saw it; the CLI reports Unknown because the addon’s error body isn’t in STS’s format. ecr get-login-password couldn’t connect at all: no ECR host is on the allowlist. The role’s Deny stands behind this, but the addon answers first, which is why only Step 3’s deny-check shows that layer.
Then the Host spoof that leaked in part 4:
== C. Host / SNI spoofing (#1758)
PASS an allowed AWS IP resolved for the spoof tests
SNI sts, Host s3 (another bound host):
curl exit 92, no HTTP answer
log: credential proxy: [sigv4] killed GET s3.us-east-1.amazonaws.com/ reason=host-sni-mismatch sni=sts.us-east-1.amazonaws.com
PASS SNI sts, Host s3: connection killed, host-sni-mismatch logged (curl exit 92)
control, SNI sts, Host sts:
HTTP/2 302 ; binding applied: 1 log line(s)
PASS control: a request for the bound host shows 'applied binding' in the log
PASS an S3 address resolved for the pin test
3. SNI sts, Host other.example.com (no binding matches):
HTTP/2 302 ; answered by the sts host: True; binding applied: 0; request signed: 0
PASS 3. SNI sts, Host other.example.com (no binding matches): no credential applied; the answer is the connected host's own
4. SNI sts, dialed to an S3 address (the upstream is pinned to the SNI name):
HTTP/2 302 ; answered by the sts host: True; binding applied: 1; request signed: 0
PASS 4. SNI sts, dialed to an S3 address (the upstream is pinned to the SNI name): STS answers and the binding applied is sts's
5. allowed IP, no SNI, Host s3:
HTTP/1.1 302 Found; answered by the sts host: True; addon or vault log lines: 0
PASS 5. no SNI: passed through undecrypted, so neither the vault nor the addon saw it (no log line) and nothing was added
note what each case proves: 1 kills a decrypted request whose Host names a bound host other than the SNI; 3 and 5 show nothing is added when no binding matches or the connection is never decrypted; 4 shows the upstream follows the SNI name, not the dialed address
Each case shows something different, and only two of them are about the addon:
Hostnames another bound host: killed by the addon. Part 4’s leak, aimed at AWS: the connection went to STS, theHostsaid S3, andsystem.pyadded the three headers although theHostdidn’t match the TLS name. The addon acts only on requests that carry them, so its kill line (curl exit 92) proves they were added, over HTTPS.- The control: a plain STS request logged
applied binding, flushed out by the addon’s own line for the same request. Hostoutside AWS: nothing added. WithHost: other.example.com, no binding matched, although the TLS name, STS, is bound, and STS answered with its own redirect. Had the vault added the three headers, the addon would have killed the request. With case 1, this confirms over HTTPS what part 4 read in the code: the vault picks the binding from theHostand doesn’t compare it with the TLS server name.- Dialed to an S3 address: pinned back to STS. curl connected to an S3 IP address while sending STS’s name in TLS and in
Host. In transparent mode the upstream would be the dialed address, but the addon sets it to the TLS server name, so STS answered and the binding applied was STS’s. curl sent no signature, so nothing was signed; per the addon’s code, a SigV4 request is signed after the pin. - No server name: never decrypted.
system.pypasses a TLS connection without SNI through without intercepting it (system.py), so STS answered directly, over HTTP/1.1 where the decrypted cases got HTTP/2. That’s the egress, not the addon.
In no case did a request leave with the credentials for a host other than the one in its TLS server name.
This works around #1758 for the AWS binding only, because the addon checks only requests that carry its three headers. Another binding in the same vault, such as Test 1’s Bedrock bearer binding or a GitHub token, would still be exposed exactly as in part 4.
Attacks B, D, E, F and G, as recorded
== B. a presigned URL made with the placeholder key
$ python3 (boto3 generate_presigned_url, no network call)
https://example-bucket.s3.amazonaws.com/osb-test/never-written.txt?AWSAccessKeyId=AKIA_OSB_PLACEHOLDER&Signature=IEKFGdOzYhqh0ux26TKeJJPMpgw%3D&Expires=1793540969
exit code 0
$ curl <presigned URL>
HTTP 403
log: credential proxy: [sigv4] presigned GET example-bucket.s3.amazonaws.com/osb-test/never-written.txt reason=query-auth-forwarded-unchanged
PASS presigned URL with the placeholder key is rejected by S3 (403) (HTTP 403)
PASS the addon forwarded it unchanged and logged presigned ... query-auth-forwarded-unchanged
PASS the response holds no temporary key
== D. the metadata services
EC2 instance metadata (169.254.169.254): curl exit 28; HTTP -
PASS EC2 instance metadata is unreachable
Pod Identity agent (169.254.170.23): curl exit 28; HTTP -
PASS Pod Identity agent is unreachable
PASS no AWS_CONTAINER_* credentials variables in the sandbox
== E. search the sandbox for a temporary key or session token
PASS the scans can find a key (a made-up canary in a file and in a process environment) (1 file, 1 process)
$ grep -rIlE '<temporary key | session token | carrier header name>' /root /home /tmp /etc /var /opt | wc -l
0
$ cat /proc/*/environ | grep -cE '<same patterns>'
0
PASS no file holds a temporary key, a session token or a carrier header (0 file(s))
PASS no process environment holds one (0 match(es))
== F. the sidecar's API without its token
GET 127.0.0.1:18080/credential-vault -> HTTP 401
PASS sidecar API /credential-vault refuses a caller without its token (HTTP 401)
GET 127.0.0.1:18080/credential-vault/_active -> HTTP 403
PASS sidecar API /credential-vault/_active refuses a caller without its token (HTTP 403)
GET 127.0.0.1:18080/policy -> HTTP 401
PASS sidecar API /policy refuses a caller without its token (HTTP 401)
== G. a Bearer header on a SigV4-bound host
$ curl -X POST https://dynamodb.us-east-1.amazonaws.com/ -H 'Authorization: Bearer ...'
HTTP/2 403
x-osb-egress-refused: bearer-on-sigv4-binding
PASS 403 with x-osb-egress-refused: bearer-on-sigv4-binding
boto3 presigned the URL in B with SigV2, its default for S3; the addon forwarded it untouched and S3 rejected the placeholder key. E’s search first found a planted canary, so its zeros mean something. The Bearer refusal used a POST: in an earlier run, not recorded, a GET with the same header, which has no Content-Length over HTTP/2, was killed instead (curl exit 92). All 24 checks passed.
Test 7: Rotation, the one-hour cap and revocation
07_lifetime.py follows one sandbox’s credentials through a rotation, a conflict and a revocation:
== 1. a sandbox, and the clock on its credentials
sandbox 8683e68a-faf9-4321-bd05-72f9c2317b83 vault revision 1 credentials expire 2026-11-01T18:03:33Z bedrock_bearer=False
revision 1; expires in 3598 s (the role-chaining cap: one hour)
the broker's timer rotates in 1800 s (half the session: 1,800 s)
PASS session length is the one-hour cap (3598 s)
PASS rotation is due at half the session
$ aws sts get-caller-identity --query Arn --output text
arn:aws:sts::123456789012:assumed-role/osb-sandbox/8683e68a-faf9-4321-bd05-72f9c2317b83
exit code 0
PASS AWS call before the rotation
The session lasts an hour (3,598 seconds were left when the script read it), and the broker’s timer would replace it after 1,800. The script doesn’t wait half an hour; it forces rotations through the broker’s API:
Forced rotations and a stale revision
== 2. force a rotation: the same sandbox keeps working
refresh -> revision 1 -> 2; new expiry 2026-11-01 18:03:35 UTC
PASS the vault revision went up
PASS the credentials are new (a later expiry than before) (2026-11-01 18:03:33 UTC -> 2026-11-01 18:03:35 UTC)
PASS expiry moved out by a new full session (3598 s left)
$ aws sts get-caller-identity --query Arn --output text
arn:aws:sts::123456789012:assumed-role/osb-sandbox/8683e68a-faf9-4321-bd05-72f9c2317b83
exit code 0
PASS the sandbox's AWS call after the rotation
== 3. a patch with the right revision works, one with a stale revision conflicts
refresh with expected_revision=2 -> revision 3
PASS expected_revision = current revision: accepted
refresh with expected_revision=1 (stale) -> HTTP 409, vault_status 409, revision now 3
PASS a stale expected_revision is refused with 409 or 412 (HTTP 409)
PASS the body reports a revision conflict (not 'vault not ready')
PASS the vault's own revision (in the conflict answer) is the accepted one, and the broker kept it
$ aws sts get-caller-identity --query Arn --output text
arn:aws:sts::123456789012:assumed-role/osb-sandbox/8683e68a-faf9-4321-bd05-72f9c2317b83
exit code 0
PASS the sandbox's AWS call after the conflict
note 412 is also what the vault answers when it is not Ready (the sidecar is starting), so the broker's body tells the two apart
== 4. the broker's own timer
skipped: the broker patches the vault every 1,800 s from the last rotation, with the vault's expected revision.
(run with --wait-loop to watch one; the unit tests in broker/ cover the loop, including the retry after a failure)
A rotation moved the vault from revision 1 to 2 with a new full session (3,598 seconds left), and the sandbox’s next call worked with no restart. A rotation with the right expected_revision went through (revision 3); one with a stale revision got HTTP 409, as in part 4’s Test 7, and changed nothing. The vault uses 412 for “not ready yet”, so the broker passes its status through. Unit tests cover the broker’s own timer; --wait-loop watches one (about 30 minutes) and isn’t recorded.
The cap comes from the broker’s own identity, which whoami checks inside the broker pod:
== 5. why the cap matters: the broker's identity
$ python broker.py whoami (in the broker pod)
broker identity: arn:aws:sts::<account>:assumed-role/osb-broker/eks-osb-eks-osb-broker-722fe53a-7004-4ebf-85e7-5beefa9f2cbb
AWS_CONTAINER_CREDENTIALS_FULL_URI: set
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE: set
credentials from: container-role
a two-hour AssumeRole of osb-sandbox: The requested DurationSeconds exceeds the 1 hour session limit for roles assumed by role chaining.
PASS the broker's identity is a role session from EKS Pod Identity (container-role)
PASS a two-hour AssumeRole is refused for role chaining, not for the role's own maximum
note the role allows two hours (max_session_duration = 7200); the broker's session is a role session, so its AssumeRole is role chaining: at most 3,600 s
The broker’s credentials are a Pod Identity role session (container-role), so its AssumeRole calls are role chaining. With the role allowing two hours, whoami asked for two: AWS refused with “the 1 hour session limit for roles assumed by role chaining”, the cap the AssumeRole reference describes. That’s why the broker asks for 3,600 seconds and rotates at half that.
Then revocation, mid-session:
== 6. revoke mid-session: delete the vault
DELETE /sandboxes/{id}/vault -> vault: revoked; next rotation: None
PASS the broker reports the vault revoked and stops rotating
$ aws sts get-caller-identity --query Arn --output text
exit code 254; client saw: InvalidClientTokenId
PASS the sandbox's next AWS call fails after the vault is deleted: STS rejects the placeholder key (InvalidClientTokenId)
PASS the sidecar added no credential and signed nothing (0 line(s))
PASS the broker refuses to rotate a revoked sandbox (HTTP 409)
the sandbox itself is still running: still running 6
Deleting the vault cut the sandbox off at once: nothing was added or signed, so the next call reached STS with the placeholder signature and was rejected (InvalidClientTokenId). In part 4 a deleted vault stopped the credential, not the traffic; here AWS refuses the traffic. The sandbox kept running, and the broker refused to rotate it again (409). The credentials themselves stay valid at AWS for up to an hour; they’re just out of the sandbox’s reach.
Test 8: The same flow under Kata
08_kata.py switches the server to Kata (kata-qemu) with part 3’s switch_runtime.sh, runs the identity, S3 and Bedrock flow once, and switches back to runc:
== 2. a sandbox with AWS credentials, under Kata
sandbox 79edb7ec-37b9-492d-870d-6fe02b80f09c vault revision 1 credentials expire 2026-11-01T14:55:29Z bedrock_bearer=False
pod runtimeClassName: kata-qemu
node kernel 6.18.56-125.164.amzn2023.x86_64; the sandbox sees kernel 6.18.35
PASS the pod runs with runtimeClassName kata-qemu
PASS the sandbox's kernel is the VM's, not the node's
PASS placeholder keys only
== 4. the addon, inside the Kata pod
addon lines: 8; re-signed: 8; refused or killed: 0
PASS the addon re-signed the requests and refused none
The switch to Kata, and the flow inside the VM
== 1. switch the server to Kata (kata-qemu)
$ ../opensandbox-lifecycle/switch_runtime.sh kata
server now: runtime=runc egress_image=opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719 egress_mode=dns+nft ingress=gateway (revision 24)
server wanted: runtime=kata-qemu egress_image=opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719 egress_mode=dns+nft ingress=gateway
config.toml: changes:
--- running config.toml
+++ new config.toml
@@ -26 +26,6 @@
path = "/data/opensandbox.db"
+
+# from opensandbox-lifecycle/values/runtime-kata-qemu.toml
+[secure_runtime]
+type = "kata"
+k8s_runtime_class = "kata-qemu"
helm upgrade: revision 24 -> 25 (deployed) in 18 s, manifests changed
server pod opensandbox-server-66d4f8f6bb-5tdgk replaced opensandbox-server-7c8b74fc7c-wbqmg, ready=true
ok server /health: {"status":"healthy"}
ok server /version: {"version":"1.2.0"}
ok release revision 25 is deployed
ok the server pod runs with: runtime=kata-qemu egress_image=opensandbox-eks/egress-sigv4@sha256:658c0839bf3a66a7ffcc906d159b7fd37a5da542a822fda411896dcd31d2d719 egress_mode=dns+nft ingress=gateway
ok gateway kept: server.gateway and [ingress] mode = gateway, host from values/server-gateway.yaml
The server pod was replaced: a kubectl port-forward started before this has ended. The Python
scripts in this directory start their own; restart yours if you use one.
switched to kata-qemu in 30 s
== 3. identity, S3 and Bedrock from inside the VM
$ aws sts get-caller-identity --query Arn --output text
arn:aws:sts::123456789012:assumed-role/osb-sandbox/79edb7ec-37b9-492d-870d-6fe02b80f09c
exit code 0
PASS assumed-role ARN with the sandbox ID as the session name
$ aws s3 cp /tmp/five.bin s3://example-bucket/osb-test/79edb7ec-37b9-492d-870d-6fe02b80f09c/five.bin --only-show-errors
(no output)
exit code 0
$ aws s3 cp s3://example-bucket/osb-test/79edb7ec-37b9-492d-870d-6fe02b80f09c/five.bin /tmp/five.back --only-show-errors
(no output)
exit code 0
upload/download 5 MiB: identical
PASS S3 upload and download under Kata, bytes identical
$ aws bedrock-runtime converse --model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 --messages ...
Ready
exit code 0
PASS Bedrock Converse under Kata
The pod ran with runtimeClassName: kata-qemu, and the sandbox saw the guest’s kernel (6.18.35), not the node’s (6.18.56). The caller identity, a 5 MiB S3 round trip with identical bytes and a Bedrock Converse call all worked, and the addon signed 8 requests and refused none. The create through the broker took 15.2 seconds, about 2.4 times runc’s (part 3 measured 1.9 times, without the sidecar and the vault write). The switches took 30 seconds there and 29 back.
Kata changes where the sandbox’s processes run, not how its requests are signed, and it doesn’t separate the sandbox from its credentials: Kata represents a pod as one VM, sidecar and vault included.
Results summary
| Test | Script | Recorded run | Result |
|---|---|---|---|
| 1. Bedrock bearer | 01_bedrock_bearer.py | 26.6 s | HTTP 200 with a placeholder token; the vault logged its bearer binding for both calls; the addon neither signed nor refused them (4 checks) |
| 2. Identity | 02_identity.py | 18.8 s | assumed-role/osb-sandbox/<sandbox ID> from the CLI and boto3; placeholders only (11 checks) |
| 3. S3 | 03_s3.py | 35.4 s | 19 B to 20 MiB multipart, checksum match; 19 requests signed, none refused |
| 4. Other services | 04_nons3_body.py | 37.1 s | DynamoDB and Converse worked; 0.83 MiB batch written and read back, 1.11 MiB batch killed, nothing written (10 checks) |
| 5. CloudTrail | 05_cloudtrail.py | 110.0 s | GetCallerIdentity and Converse under the sandbox ID, and the broker’s AssumeRole naming it, in about 90 seconds |
| 6. Attacks | 06_attacks.py | 99.7 s | No credential reached the sandbox; a Host naming another bound host was killed, and the upstream stayed pinned to the TLS name (24 checks) |
| 7. Lifetime | 07_lifetime.py | 35.7 s | One-hour sessions, rotation at 1,800 s; a two-hour request refused for role chaining; stale revision 409; vault delete stopped the calls (18 checks) |
| 8. Kata | 08_kata.py | 102.5 s, both switches included | Same flow in a VM; 8 requests signed, none refused (8 checks) |
All eight scripts exited 0 with every check passing, in 7 minutes 46 seconds together (each run includes its port-forward start). Tests 1 and 4 to 8 are later takes from the same day, after I raised the role’s maximum to two hours and added checks.
What this does not solve
In no test did a real key show up in the sandbox: not in its environment, its process environments or the files I searched (Tests 2 and 6), and no credential-vending call got through (Test 6). What the design doesn’t do is just as specific.
- The real credentials live next to the code you don’t trust. The vault is in the egress sidecar’s memory, in the sandbox’s pod and network namespace, and under Kata in the same VM (Kata runs a pod as one VM, per its docs). In Test 6 the sidecar’s API refused the sandbox without its token, as in part 4. A breakout from the sandbox container lands next to credentials. Under Kata, that’s this sandbox’s credentials, inside its VM. Under runc it’s root on the node: every co-located sandbox’s vault, the node’s instance role, and, if the broker runs on that node, its Pod Identity token, with which an attacker can assume
osb-sandboxagain and again for as long as they hold the node. Nothing in this demo kept the broker off the sandbox nodes: its manifest has no affinity, and the cluster has two nodes. Run the broker on nodes that run no sandboxes, a separate node group or affinity plus taints. That’s from the manifest and how Pod Identity works; I didn’t test a breakout. - The sandbox can use the role’s permissions for as long as it runs. The addon signs what the sandbox sends, and the broker keeps rotating, so the access lasts the sandbox’s lifetime; any one credential set stays valid at AWS for up to an hour, revoked or not. My test role could read, write and delete objects in one bucket and items in one table, and call one model, so a misbehaving agent could have deleted that bucket’s objects or spent on that model. Scope the role to the task. And the allowlist is a credential boundary, not a data boundary:
*.s3.us-east-1.amazonaws.comreaches every bucket, and the addon forwards a request with noAuthorizationheader unsigned, so a sandbox could write to someone else’s bucket that accepts anonymous writes, or through a URL someone else presigned for their own bucket, which the addon forwards unchanged. That’s from the code and the allowlist; I didn’t try it. - Presigned URLs have to come from the trusted side. The sandbox can’t presign anything useful with its placeholder key (Test 6). A URL presigned on the trusted side (the broker would need a call for it) would pass through unchanged; the addon’s tests cover that, and I didn’t run it on the cluster. Such a URL is a credential for its one operation until it expires, and one presigned with temporary credentials carries the session token in its query string (S3 docs).
- OpenSandbox’s own design for this doesn’t sign AWS requests yet. OSEP-0025 adds an
exchangecredential: the egress asks a resolver service for headers on each request. The resolver gets the method, host, path and headers but not the body (the test resolver accepts exactly those four fields), and the OSEP lists body signing as follow-up work, with the first implementation aimed at the Fast Sandbox profile. Most AWS APIs sign a hash of the body, so a resolver that never sees it can’t sign them; S3, whose clients declare that hash in a header, is the exception by my reading. None of it is in a stable release as of November 7, 2026. The contracts merged intomainon October 30, 2026, asc2f55b2(#2495); the OSEP (now marked implementing), #2500 and #2501 followed on November 1, and #2534 and #2535 on November 2. The 1.2.1-rc.1 pre-release of November 4 is the first to carry it. - The addon has limits of its own. Bodies over about 1 MiB to services other than S3 (Test 4), S3 requests over 1 MiB that don’t declare their payload hash (the AWS CLI and boto3 always do), signed-chunk uploads and SigV4a are refused or killed. It also runs on the 1.2.0 egress image’s mitmproxy 11.0.2, with the open
h2alerts part 4 lists, and trusts mitmproxy’s parsing of theHost, the TLS name and the body. - The Bedrock bearer binding isn’t checked against the TLS server name. The addon only checks requests that carry its three headers, so Test 1’s
bearerbinding is outside it, like any binding in part 4’sHost-header case (#1758). A request whoseHostnamesbedrock-runtimecould draw the Bedrock key onto a connection to another allowed AWS host. If that host quotes theAuthorizationheader in a response body, the key reaches the sandbox (part 4’s reflecting-host case): OpenSandbox redacts response headers only, and the addon scrubs only responses to requests it signed. That’s from the code; I didn’t test it. - The vending list isn’t complete. The addon allows three STS actions and blocks a list of token calls, and the role denies more, but neither covers every way to turn a signed request into a credential;
cognito-identity:GetOpenIdTokenForDeveloperIdentityis one neither lists. What really holds the line is the role’s short Allow list and the eight-host allowlist, which reaches no Cognito host. - A pause still wipes the vault (#1366, parts 3 and 4). The broker writes it again when a rotation finds it missing. A rotation due during the pause fails and is retried every 60 seconds, so credentials come back within about a minute of the resume, or up to 30 minutes later if none came due. A sandbox paused longer than its credentials last gets none until the broker restarts, because the broker gives up once a sandbox’s credentials expire with rotation still failing. That’s from the broker’s code; I didn’t test it.
None of this undoes the main result. A sandbox that signs with a placeholder and gets a one-hour, single-sandbox, CloudTrail-named session beats one with a key in its environment, where env would print it.
Cleanup
Each test deletes its own sandboxes, S3 objects and table items. When you’re done with part 5:
kubectl --context osb-eks delete namespace opensandbox-broker # the broker
./01_iam.sh destroy # both roles, the association, the table
python3 ../opensandbox-lifecycle/server_config.py --egress-image base # the server back on the 1.2.0 egress image
../opensandbox-lifecycle/switch_runtime.sh runc # only if 08_kata.py was interrupted
The test bucket is yours and stays; delete the ECR repositories opensandbox-eks/broker, opensandbox-eks/aws-tools and opensandbox-eks/egress-sigv4 if you don’t need them (part 6’s SigV4 rung uses egress-sigv4, so keep it until then). Between sessions, scale the nodes to zero from part 2’s directory, and remove everything with its teardown:
../opensandbox-eks/08_scale.sh down
../opensandbox-eks/99_teardown.sh
Verdict
It works, with the stock clients. A broker, the vault and about 800 lines of addon let the AWS CLI and boto3 run unchanged in a sandbox that never held a key: STS, S3 up to a 20 MiB multipart upload, DynamoDB and Bedrock, CloudTrail naming the sandbox on the calls it records, and the identity, S3 and Bedrock calls under Kata. Bedrock needed only a short-term key in a bearer binding. Every attempt I made from inside to get a credential out failed, and part 4’s Host spoof is killed for the AWS binding.
The signing is mine, though, not OpenSandbox’s: 1.2.0 gives you the vault and an extension point, and the broker, the Host pinning, the vending checks and the signing are code you’d own and retest on every egress image. The project’s own design, OSEP-0025, reached the 1.2.1-rc.1 pre-release on November 4, and its resolver still never sees a request body. And the security model is the role: for as long as it runs, the sandbox can do what the role allows, from a pod that holds an hour’s worth of keys.
What you sign up for
| Concern | AWS credentials at the egress on 1.2.0, as tested |
|---|---|
| What runs it | OpenSandbox’s vault and egress proxy, plus a broker and a SigV4 addon you build and own |
| Clients | The AWS CLI v2 and boto3 unchanged, with placeholder keys and AWS_CA_BUNDLE |
| Services tested | STS, S3 (to a 20 MiB multipart upload), DynamoDB, Bedrock (SigV4 and bearer) |
| Where the secrets live | The sandbox’s vault in its egress sidecar, in the sandbox’s pod (and, by Kata’s design, its VM); the broker holds them only while it writes the vault; the broker’s own Pod Identity session, which can assume osb-sandbox, lives in the broker pod |
| Identity and audit | One role session per sandbox, named after it; event history for management events, a paid trail for S3 and DynamoDB data events |
| Lifetime | One hour (role chaining), replaced every 30 minutes with expectedRevision; deleting the vault cuts the sandbox off |
| Limits | Bodies over about 1 MiB to services other than S3 killed; no SigV4a or signed-chunk uploads; presigned URLs from the trusted side only |
| Blast radius | Everything the role allows, for the sandbox’s lifetime; a credential set taken from the pod works for up to an hour, even after revocation |
Who should use it
It fits a team that wants agents to call a few AWS services for a task without holding keys, say an agent that reads and writes one bucket and calls one Bedrock model. If Bedrock is all it needs, use Test 1’s bearer binding and skip the addon: you still need something like the broker to make and rotate the key, and the sandbox should be allowed only the Bedrock Runtime host (the companion broker allows all eight; narrow AWS_HOSTS for that mode), since that binding isn’t checked against the TLS name. Otherwise keep the role per task and the allowlist to the services it uses, keep the broker on nodes that run no sandboxes, set AWS_CA_BUNDLE, and treat the addon as code you own, tested on every new egress image.
It doesn’t fit, yet, workloads that send large bodies to services other than S3 (big Bedrock documents, big DynamoDB batches), anything that needs SigV4a, or a threat model in which the sandbox and its credentials can’t share a pod. Part 6 puts the addon on its egress overhead ladder, on the same two nodes.