For the three filesystem-mode comparison see Filesystem. This page focuses on sandbox mode usage.
What sandbox solves
Confines the agent’s file operations and command execution to an isolated environment; the host stays untouched. Plus three extra wins:- Execution boundary — untrusted input, suspicious scripts,
rm -rf-shaped commands all stay inside the sandbox. - Cross-call recovery — not just conversation state:
pip install,npm install, generated temp files (the executable environment itself) are snapshotted, so the nextcall()resumes in the same sandbox without reinstalling. - Multi-replica friendly — when multiple replicas serve the same logical user, sandbox state can share a single slot so any node can resume the same workspace.
A minimal example
Local Docker, isolated per user:userId across call() → automatically reuses the same sandbox (or restores from snapshot). Different userId → separate sandbox. When userId is absent, falls back to sessionId as the isolation key.
IsolationScope — who shares a sandbox
All sandbox configuration lives on theSandboxFilesystemSpec (e.g. DockerFilesystemSpec). The key parameter is isolationScope:
SESSION is naturally concurrency-safe (each session has its own slot). USER / AGENT / GLOBAL in multi-replica deployments should pair with a mutex (see “Concurrency control” below).
USER-scope fallback: when IsolationScope.USER is active (either explicitly or by default) but RuntimeContext.userId is absent, the framework automatically falls back to SESSION scope using sessionId. This means you don’t need to guard against missing userId — the sandbox degrades gracefully.
Cross-call recovery = snapshots
The sandbox snapshots its workspace at eachcall() end and restores at the next start:
- Container still alive + workspace still there → just continue (fastest)
- Container gone → reboot from snapshot, restore workspace
- No snapshot → full init from
WorkspaceSpec(cold start) - Snapshot read or restore failure → this
call()fails and can be retried; snapshot state is never silently discarded
snapshotSpec:
AGENTS.md / skills/ / subagents/ / knowledge/) are synced into the sandbox at each start, content-hash-gated. So if you edit a script under skills/, the next call() has the new version inside the sandbox.
Distributed deployment
When multiple replicas run the same agent and any replica must be able to pick up the same user’s conversation, you need:- A distributed
AgentStateStore(e.g. Redis-backed) — passed via.stateStore(...)on the builder - A non-
Noopsnapshot (OSS / Redis / remote store) — configured directly on the filesystem spec via.snapshotSpec(...) - An appropriate
IsolationScope(defaultUSERis usually correct)
AgentStateStore that holds agent runtime state. Providing a distributed store automatically enables cross-replica sandbox resume — no extra configuration needed.
If you’re using a local AgentStateStore (the default JsonFileAgentStateStore) with sandbox mode, the framework logs a warning at build time reminding you that sandbox state won’t survive JVM restarts and can’t be shared across instances.
Concurrency control (multi-replica)
InUSER / AGENT / GLOBAL modes across replicas, two replicas serving the same user concurrently both write to the same slot — last writer wins. If that’s not OK, you need a distributed lock.
Recommended: use distributedStore(...) — snapshot and execution guard are auto-injected:
SandboxFilesystemSpec:
RedisSandboxExecutionGuard (Redis SET NX PX), JdbcSandboxExecutionGuard (MySQL GET_LOCK()). You can also implement SandboxExecutionGuard to plug in Zookeeper, etcd, or other lock stores.
Self-managed sandbox instances (advanced)
By default the framework owns the whole sandbox lifecycle. Three “I’ll manage it myself” scenarios: 1. I already have a running container; I want the agent to use itexternalSandbox to each agent’s call(), then shutdown() it yourself when done.
Choosing a sandbox store
All stores implement the same interface; agent code, toolkit, and
AGENTS.md don’t change.
Workspace root
The sandbox paragraph in the system prompt and file-tool path normalization use the workspace root of the sandbox bound to the current call. A restored session keeps its saved directory rather than adopting a new agent configuration. Absolute paths under that root become workspace-relative; other paths remain subject to the existing filesystem policy. For E2B, the default root is/home/user. Set new E2bFilesystemSpec().workspaceRoot("/home/user/project") to change it. Alternatively, supply a WorkspaceSpec with that root. If both are explicitly supplied, their roots must match (trailing slashes are ignored); conflicting values fail when the sandbox context is built. The effective workspace specification is copied, so caller-owned entries and environment are preserved without modifying the supplied object. When older E2B state has inconsistent root fields, its saved execution directory remains authoritative and the manifest is aligned with it; no files are moved. Docker’s default remains /workspace.
Supplying a WorkspaceSpec only for entries or environment does not explicitly configure its root: its untouched /workspace default inherits the E2B execution root. Calling setRoot("/workspace"), however, is an explicit choice and conflicts with a different explicit workspaceRoot(...).
Runtime image contract
You choose the sandbox image (Dockerimage, the Kubernetes agent-sandbox runtime image, etc.), but not every image works. Harness file tools (read_file / write_file / edit_file / grep_files / glob_files / list_files) and the snapshot machinery are all implemented by running POSIX shell commands inside the sandbox. The image must satisfy the contract below, or tools will fail in hard-to-diagnose ways.
Baseline contract (all backends)
The image must provide:
Images based on
ubuntu:24.04 or debian qualify out of the box; python3 is optional and only enables the native edit_file fast path. Without an executable python3, editing still works through a transfer fallback, but may be slower because the file is transferred through the host. alpine (BusyBox stat / grep behave differently) and distroless images do not satisfy the other baseline requirements.
Quick conformance check (run inside the image; all must succeed):
edit_file rejects a command whose complete encoded arguments and script exceed 112 KiB; split unusually large edits into smaller calls. The limit applies to the whole command, not independently to oldString and newString.
Additional contract for Kubernetes (agent-sandbox)
The agent-sandbox backend does notkubectl exec into the container; it talks to an HTTP API exposed by the runtime container (default port 8888). The runtime service in your image must implement:
timeout_seconds is a positive whole-second deadline covering queue wait, process creation and execution, and draining stdout/stderr. Reject invalid or unsupported values before starting the process. On expiry, terminate the command process group, bound cleanup by the advertised cleanup_timeout_seconds, and return exit code 124 with all output collected so far. The deadline must also cover inherited output pipes after the parent shell exits. Cancellation must clean up the process group. Detached processes that create a new session are outside the process-group guarantee.
The execute tool defaults to 30 seconds; direct SDK commands().run(command) defaults to 60 seconds. With a supported runtime, both send the effective timeout explicitly. The HTTP wait budget for each command POST is execution timeout + advertised cleanup allowance + connector perAttemptTimeout; concurrent requests do not change each other’s budgets. The capability preflight has its own connector perAttemptTimeout budget before the command POST; this formula does not include that preflight. Set every gateway/router request and idle timeout to cover the largest supported budget (or disable those limits deliberately). A fixed 180-second route still prevents a 300-second command from completing through that route.
When capabilities returns 404, the SDK logs a warning and sends the legacy {"command":"..."} request, with an HTTP wait budget of the requested timeout plus 30 seconds plus the connector perAttemptTimeout (the same transport allowance the modern budget adds). This bounds client waiting only; it cannot guarantee the remote command stops. A 404 can also indicate incorrect routing. Other capability errors and invalid responses reject execution. Do not advertise capabilities until timeout enforcement is implemented. HTTP errors, wait expiry, and invalid responses once command submission is attempted mean execution outcome unknown (Harness exit code null / outcome UNKNOWN), not confirmed process timeout. Transport submission failures are conservatively UNKNOWN because acceptance cannot be proven from the caller. Capability verification failures use SandboxRequestException.RuntimeCapabilityException (with the original HTTP status when available); a supported runtime rejecting an excessive requested timeout remains an ordinary 422 input error. Workspace directory creation checks the command exit code before setup, hydration, or upload can continue. Command POSTs are not retried or redirected; inspect command state before deciding to retry.
The file-API base directory must match the workspace root (recommended: /workspace for both). Harness transfers two kinds of content through /upload / /download: workspace snapshot tarballs (temp files under .agentscope-tmp/ inside that directory), and single-file bytes for write_file / file downloads when the path lies under the base directory (Linux caps a single command-line argument at about 128 KiB; native file-API transfer has no such limit). The base directory is configured via KubernetesSandboxClientOptions.fileApiBaseDir (default /workspace); setting it blank falls back to base64-over-exec transfer.
Note: the example runtime in the upstream agent-sandbox repo (examples/python-runtime-sandbox) runs commands viashlex.split+subprocess.runwithout a shell, and roots its file API at/app— it does not conform to this contract and should only be used as a reference for endpoint shapes. Upstream KEP-539.2 is standardizing the runtime interface (REST/gRPC spec + conformance tests); this contract can converge on the official spec once it lands.
Why it works this way
TheSandbox abstraction’s primary data-plane entry point is exec(command). This is deliberate — tool semantics like edit_file / grep_files (regex, string replacement, globbing) cannot be expressed through a small set of file-API endpoints; running shell scripts against a standard toolchain inside the image is the only portable answer. The file API (upload/download) handles pure byte transfer only: workspace snapshots and single-file upload/download go through it (backends declare the capability by implementing the optional SandboxFileTransfer interface); everything else goes through execute. Which also means: the image contract is part of the sandbox interface — run the conformance check above before switching images.
Crossing the boundary from the model’s point of view. The file API (upload/download) above is an internal mechanism — invisible to the LLM, and FilesystemTool exposes no transfer tools. The supported way for a sandboxed agent to hand an artifact it produced to a destination outside the sandbox is the generic deliver_artifact tool. It is registered only when you configure an ArtifactDeliveryTarget via HarnessAgent.builder().artifactDeliveryTarget(...). The SPI stays business-agnostic — deliver(RuntimeContext, ArtifactDeliveryRequest) -> ArtifactDeliveryResult — so destination logic (e.g. WebDAV upload) lives in your application. The tool downloads the file bytes from the sandbox workspace and delegates the transport to the target. Without a configured target, the sandbox workspace prompt states plainly that files cannot leave the container.
Kubernetes HTTP exchange timeouts
Kubernetes sandbox requests bound the wait for both response headers and the complete response body. A server that sends headers and then stalls cannot keep the caller waiting indefinitely. Command requests use the execution timeout plus the advertised cleanup allowance and connectorperAttemptTimeout, as described above. An expired command exchange reports an unknown execution outcome; only a valid runtime response confirms the exit code.
The runtime’s timed_out: true response confirms that the process was terminated by its execution limit and preserves its partial output. An HTTP exchange timeout leaves the execution outcome unknown; an exit code of 124 alone does not confirm a runtime timeout.
File uploads and downloads default to a 10-minute exchange budget. Directory listing and existence checks default to 60 seconds. The client methods accepting a Duration use that value as the exchange budget; retries and backoff share it.
These deadlines bound HTTP waits. Connection discovery and pod IP lookup retain their own timeout behavior. Cancelling the client exchange does not guarantee that the remote command has stopped; use runtime-side process limits when remote termination is required.
Kubernetes state persistence: PVC is the first layer
The Kubernetes store is fully based on agent-sandbox: sandbox pods are managed by the agent-sandbox controller, and image, resources, and storage are all declared cluster-side in aSandboxTemplate / SandboxWarmPool — the Java side only claims instances (SandboxClaim) and connects. This makes it different from other stores in one important way: workspace data persistence is primarily the PVC’s job, not the Harness snapshot’s. The two layers each own one thing:
The framework needs no special handling for PVCs: on resume, startup probes
test -d /workspace; when the PVC is there, the “workspace preserved” branch is taken and snapshot restore is skipped entirely.
Three things you must configure correctly:
- The PVC must be mounted at
workspaceRoot(default/workspace). Mount it elsewhere or useemptyDir, and the workspace is gone on every pod restart — each call degrades to snapshot restore or a cold start. Reference template (from the upstream agent-sandbox examples):
-
Mind the claim lifecycle boundary. Once
shutdownTime/shutdownPolicy: Deleteremoves the Sandbox, PVC-based warm recovery is over — the next resume won’t find the claim, and the framework creates a fresh sandbox (with an empty new PVC). Whether the workspace comes back depends on point 3. -
Pick
snapshotSpecaccordingly. PVC +NoopSnapshotSpec(default): skips the tar-and-upload at the end of every call, at the cost of a cold start once the claim is gone. PVC + OSS / Redis snapshot: belt and suspenders — claim expiry, PVC loss, and cross-cluster migration all recover from the snapshot, at the cost of a full archive per call.
SandboxState identity layer is never optional — in multi-replica deployments configure a distributed AgentStateStore, otherwise other replicas can’t learn the claimName and the data on the PVC is unreachable no matter how intact it is.
How the workspace maps into the sandbox
For a shared Kubernetes PVC, pass a stable directory from authenticated identity at Claim creation:KubernetesFilesystemSpec.claimEnv forwards these entries to SandboxClaim.spec.env. The cluster template must allow env injection and use subPathExpr: $(WORKSPACE_SUBPATH) for the external PVC mounted at /workspace. Provision storage permissions in the cluster. In agent-sandbox v0.5.4, a Claim with env bypasses prewarmed instances; the replacement sandbox can reuse the same directory on the retained PVC. Per-call null env inherits client defaults; an empty list clears them.
Host-side key files under workspace/ (AGENTS.md, skills/, subagents/, knowledge/) are synced into the sandbox at each start, content-hash-gated — unchanged content is skipped.
To bind a host directory into the sandbox (e.g. a code repo), use BindMountEntry (Docker only; for Kubernetes, declare mounts in the cluster-side SandboxTemplate pod template instead; managed sandboxes like Daytona / E2B run in the cloud and can’t mount your host paths).
File changes inside the sandbox don’t sync back to the host — to retrieve sandbox-produced artifacts, have the agent read_file them.
Implementing your own sandbox store
To integrate a non-Docker isolation environment (self-hosted remote executor, commercial sandbox API, local mock, etc.), no Harness source changes needed — implement a few contract interfaces and pass them tofilesystem(...). The InMemorySandbox family under agentscope-harness tests is the minimal skeleton to copy.
Related pages
- Filesystem — three declarative modes compared
- Workspace — which files under
workspace/sync into the sandbox - Architecture — where sandbox acquire / release sits in the call() timeline