Configuration Reference

AgentENV reads configuration from a TOML file. The default path is config/default.toml. Override it with:

export AENV_CONFIG_PATH=/path/to/config.toml
# or
cargo run --bin server -- --config /path/to/config.toml

Global Settings

Snapshots and paused sandboxes can only be restored in the mode in which they were created.

$AENV_HOME is a literal placeholder in state-path values, not a shell environment variable. AgentENV replaces it with the resolved home_path after applying AENV_HOME_PATH; ublk.daemon_socket_path additionally supports $AENV_RUNTIME, which resolves to runtime_path. Relative paths without these placeholders are resolved against the directory containing the configuration file.

Packaged runtime dependency versions and download URLs live in config/deps_manifest.toml. Only the dependencies for the selected mode are installed. User configuration should contain runtime behavior and explicit local path overrides, not the default dependency catalog.

[firecracker]

Firecracker VM binary and boot configuration.

[kernel]

Linux kernel image for microVMs.

[tools]

Tools drive image used to boot the AgentENV control plane inside each microVM. The default release is selected automatically; most deployments can leave this section unchanged.

Snapshots and paused sandboxes record only tools_drive_version. Launch and restore fetch that version from the configured URL template when it is missing locally. Releases must be immutable and retained for the lifetime of their snapshots; changing the default version affects new sandboxes and templates.

OCI tools rootfs images carry io.agentenv.tools-drive.format=oci-rootfs-v1 (the tools Dockerfile adds it automatically). They are downloaded and converted locally with a stable, pinned conversion contract. Converted layers live in the versioned dependency directory, independently of image-cache eviction. Explicit --setup-only prepares the default release for dependency bundles.

Native OverlayBD images keep lazy reads and automatically download tools layers in the background after envd is ready, independently of memory and user disks. Legacy images without the format label provide tools.ext4, which is downloaded and mounted read-only. Tools are not uploaded into snapshot storage. A local drive_path only identifies its declared version.

Template Rootfs Images

User-visible rootfs images are selected at the template API layer. POST /v2/templates/{templateID}/builds/{buildID} accepts an optional fromImage field:

• omitted: use [image.resolver].default_image
• full OCI reference: use the supplied image
• short name: normalize standard Docker Hub forms such as ubuntu:24.04 and node:20

[image.resolver]

How the three registry settings interact

Image resolution runs in two phases, and the three keys act at different points:

1. search_registries — completion. Only used for short / unqualified references (e.g. ubuntu). Each entry is prefixed to the name to build a list of fully-qualified candidates (docker.io/library/ubuntu:latest, ghcr.io/ubuntu:latest, …). Fully-qualified references skip this step.

2. allowed_registries — gating. Applied right after candidates are built, to both fully-qualified references and the candidates expanded from search_registries. Candidates whose registry host is not whitelisted are dropped; if none remain, the reference is rejected with a 4xx error. In effect the resolvable set of short-name hosts is the intersection of search_registries and allowed_registries — e.g. searching docker.io and ghcr.io while only allowing ghcr.io resolves short names to ghcr.io only.

3. try_referrers_overlaybd_prefixes — per-candidate optimization. Runs later, while resolving an already-permitted candidate: after its manifest is fetched, AgentENV may query the OCI Referrers API on the same registry/repository for an overlaybd-native artifact. Because referrer lookups never leave the source image’s own host, they are implicitly covered by allowed_registries — no separate whitelist entry is needed for referrers.

   Two referrer artifactTypes are recognized, in this order:

   artifactType	Produced by
   application/vnd.containerd.overlaybd.native.v1+json	accelerated-container-image (obdconv)
   application/vnd.azure.artifact.streaming.v1	Azure Container Registry artifact streaming (az acr artifact-streaming create)

   Both point at an overlaybd-native manifest; only the discovery label differs. The referrer manifest is re-validated after it is fetched, so a referrer that is not actually overlaybd-native is rejected rather than used. Turbo-OCI referrers (application/vnd.containerd.overlaybd.turbo.v1+json) are never selected — AgentENV’s overlaybd runtime does not implement the turbo read path.

   To stream from ACR, add your registry (with the trailing slash) to the list, e.g. try_referrers_overlaybd_prefixes = ["myregistry.azurecr.io/"].

[image.cache]

Node-local cache root for resolved and converted user images.

[image.cache.gc]

Background hard-commit garbage collection for the image cache. When enabled, each pass reconciles metadata from the on-disk source configs and then deletes hard-commit objects that are no longer rooted by source configs, held by image-cache leases, or referenced by the in-process running set. Committed snapshots are durable SnapshotRepository state and do not pin ImageCache commits. With capacity_gb set, GC first evicts least-recently-used source configs over the high watermark so hard-commit GC can reclaim what they unrooted.

Capacity-driven eviction runs only when [image.cache].capacity_gb is set; otherwise the GC still reclaims unreachable commits but performs no watermark eviction.

[image.cache.remote_blocks]

Overlaybd registryfs_v2 remote block cache settings. The directory is always <image.cache.root_dir>/remote-blocks.

Resolved image data is cached under:

<image.cache.root_dir>/
  commits/
    <sha256-commit-digest>/
      overlaybd.commit
                  # full overlaybd commit store shared by OCI conversion and download
  indexes/        # OCI layer + conversion context -> overlaybd commit descriptor
  remote-blocks/  # overlaybd-native remote block cache
  configs/         # resolved image configs
    <slug>-<hash>-image.json

[sandbox_proxy]

Optional host-based data-plane routing for sandbox services.

When domains is empty, the server still supports /proxy and routing-header proxy requests, but does not classify requests by Host. Domains are normalized to lowercase, deduplicated, and must be valid DNS names. The configured order is preserved because domains[0] is the advertised sandbox domain.

Environment variable override:

• AENV_SANDBOX_PROXY_DOMAINS

[network.egress]

Node-level sandbox egress guardrails. These rules are installed before per-sandbox allowOut / denyOut rules, so sandbox API requests cannot override them.

[network.internal]

AgentENV-internal sandbox address plan. Change these only when the defaults overlap with host or deployment network ranges.

The two configured CIDRs must not overlap each other or AgentENV’s fixed VM tap link 169.254.0.20/30. These networks are also treated as reserved sandbox egress destinations regardless of always_denied_cidrs.

[machine]

Default VM resources for sandboxes.

[envd]

In-guest envd daemon settings.

[sandbox]

Sandbox control communication settings.

The managed seed is node-local persistent state and must be included in backups of $AENV_HOME. AgentENV refuses to generate a replacement when persisted secure or private-ingress sandboxes exist. An explicit environment or TOML value takes precedence over the managed file; changing that effective value invalidates existing sandbox access tokens.

Configure AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED with the same value on every runtime node in a clustered deployment. Standalone runtime nodes use their managed seed when it is unset.

[volume]

[orchestrator]

Sandbox lifecycle management.

[pool]

Shared process-wide warm-pool defaults used by network slots, block devices, and pre-spawned Firecracker processes. Pools prewarm to the low watermark, then grow the refill target geometrically toward the high watermark when real acquisitions drain the pool.

Component sections:

Validation rules:

• low_watermark <= high_watermark
• [pool.firecracker].fill_concurrency > 0

[node_identity]

Stable identity fields for this node. These values appear in node API responses and scheduler heartbeats.

[observability]

Node-level observability and host metrics collection.

When observability is enabled, host CPU/memory/disk metrics are collected at request time. CPU percent is computed from two samples; the first node metrics request waits about 100ms to return a measured value.

[observability.scheduler_report]

Optional scheduler heartbeat reporting for multi-node control plane integration.

Environment variable overrides:

• AENV_OBSERVABILITY_SCHEDULER_REPORT_ENABLED
• AENV_OBSERVABILITY_SCHEDULER_ENDPOINT
• AENV_OBSERVABILITY_REPORT_INTERVAL_SECS

AENV_OBSERVABILITY_SCHEDULER_ENDPOINT overrides [cluster].scheduler_endpoint for the reporter process only.

[cluster]

Shared cluster-level service endpoints.

[p2p]

Experimental: P2P has not been tested in production. Keep it disabled in production unless the deployment accepts that operational risk.

Project-wide artifact transport configuration. The transport is disabled by default. When enabled, it is used by the overlaybd P2P HTTP facade and by snapshot publication/runtime resolution as an optional acceleration path.

[custom_extension]

Custom extension service configuration. When url is unset, the integration is fully disabled. See Custom Extension.

[snapshot]

Snapshot storage/build configuration.

Environment variable overrides:

• AENV_SNAPSHOT_LOCAL_CACHE_PATH

[snapshot.image_publish]

Source-registry image publication. Only takes effect when snapshot.repository_backend = "oss".

[snapshot.publish_compression]

Publish-time compression for snapshot layers uploaded to OSS/ACR. Local layers always stay raw, so local resume pays no decompression cost; enabled by default, memory layers and incremental read-write layers are compressed once as they are uploaded, cutting network bytes for cross-node resume. This is the only compression switch; the legacy capture-time knobs under [memory_snapshot] and [template_build] were removed from the configuration schema.

Known impact: compressed layers are recorded without a layer uuid (ZFile layers carry no LSMT uuid), so P2P uuid-keyed acceleration does not apply to them. Snapshot P2P publication also skips digest-keyed advertisements for local raw layers whose digest is absent from the committed record — the record names the compressed bytes, so the raw digest key would never be looked up by consumers.

[backend.posix_fs]

POSIX filesystem-backed snapshot repository configuration. This section is used when snapshot.repository_backend = "posix_fs".

Environment variable overrides:

• AENV_SNAPSHOT_STORE

[backend.oss]

OSS-backed snapshot repository configuration. This section is required when snapshot.repository_backend = "oss".

Notes:

• credential_process and static access key settings are mutually exclusive in practice; when credential_process is set, the backend ignores static credential fields.
• credential_process should be written as a portable argv-style command line. Avoid $VAR, backticks, $(...), pipes, and shell builtins.
• Although the config section is still named oss, the runtime path is implemented via a shared S3-compatible client, so region must be configured.
• Leave addressing_style unset when endpoint-based detection is correct. Set it to "virtual" or "path" when the provider’s required or preferred style differs from the detected default; for example, some Tigris or Cloudflare R2 deployments use virtual-host addressing.
• The setting covers both halves of the data path: the snapshot repository client (metadata and artifact upload/download) and the generated OverlayBD runtime config (ossConfig.defaultAddressingStyle), which the runtime uses when reading remote managed snapshot layers during sandbox restore.

For an S3-compatible provider where virtual-host addressing is required or preferred — for example Tigris — set addressing_style explicitly:

[backend.oss]
endpoint = "https://t3.storage.dev"
bucket = "agentenv-snapshots"
region = "auto"
addressing_style = "virtual"

Other path override:

• AENV_DEPS_PATH

Setup sysctl tuning is host-level setup. It is skipped before reading /proc/sys when the server detects that it is running inside a container. Set AENV_FORCE_SYSCTL_TUNING=1 only for a privileged container with writable host sysctls; otherwise configure these kernel parameters on the host.

[protoc]

Protobuf compiler metadata for code generation lives in config/deps_manifest.toml, not config.toml.

[ublk]

Userspace block device configuration. Rootfs is served through an OverlayBD-backed ublk device managed by uvm-ublk-daemon.

Environment variable override:

• AENV_UBLK_DAEMON_BINARY_PATH
• AENV_UBLK_DAEMON_METRICS_LISTEN_ADDR

[ublk.overlaybd]

OverlayBD configuration for ublk. Legacy enabled and device_type keys are ignored.

global_config_path and auto-generated config

The file at the configured default path $AENV_HOME/overlaybd/overlaybd-global.json is auto-generated by the server at startup. The generated JSON incorporates several TOML settings — [image.cache].root_dir, [image.cache.remote_blocks].max_size_gb, download_enable, [backend.oss] credentials, and Docker registry credentials detected from ~/.docker/config.json — into a single overlaybd runtime config file.

The server regenerates the file at global_config_path on every startup, so these TOML settings always take effect automatically — any manual edits to the generated file are overwritten on the next startup. To keep customizations, make them through the TOML settings, not by editing the generated JSON.

[memory_snapshot]

Memory snapshot overlaybd configuration. The server auto-generates the file at the default path on every startup.

[memory_snapshot.background_download]

Background download settings dedicated to remote memory-snapshot OverlayBD layers. They do not change the general rootfs or attached-drive defaults. All fields are serialized into the generated memory OverlayBD global config. Each remote layer is filled block by block into the node-local remote file cache: the cache-owned background-download scheduler registers one task per remote layer (deduplicated by blob, shared across sandboxes) and downloads only chunks still missing from the entry bitmap — each source request fetches block_size bytes (aligned to whole cache blocks) and publishes the chunk’s cache blocks as soon as they land. Submission is never rejected under load — tasks run as scheduler capacity allows, with at most maxConcurrentFiles layer tasks concurrently per file-cache backend (from the generated overlaybd download config, default 8) and at most concurrency chunk reads in parallel per layer, subject to the scheduler’s max_inflight_blocks cap. Downloads of a sandbox-bound device start only after envd is ready (plus delay), with a 20s fallback if the ready signal is lost; while foreground remote reads are in flight, background block reads yield to a small guaranteed floor instead of competing at full speed. The generated memory config leaves throttling off (maxMBps = 0); image configs that carry a positive maxMBps keep their historical shared rate limit across the block tasks. A completed cache block becomes visible to foreground reads as soon as it is committed to the cache bitmap; there is no staging file, no full-file digest check, and no switch-to-local, so a failed or canceled block simply stays uncached and is fetched on demand by foreground reads or a later retry. The cache is a bounded working set: blocks may be evicted under capacity pressure and are then re-fetched on demand.

[template_build]

Managed Dockerfile builder resources. The first Dockerfile build on a node prepares a reusable internal builder template. Each build mounts a separate clone of the repository’s shared cache seed; concurrent builds do not queue for cache ownership. The last successfully published cache becomes the next seed, with best-effort reuse of concurrent branches. Builder resources do not change the resulting template’s CPU or memory.