Docker Compose (Multi-Node Simulation)

Run a full multi-node stack on a single host using Docker Compose. This simulates a production-like topology with a gateway, scheduler, and multiple AgentENV backend nodes.

For a real multi-machine deployment without Kubernetes, see Static Multi-Node.

Prerequisites

Linux kernel 6.8+
/dev/kvm access (passed into the runtime containers)
Docker Engine with Docker Compose v2 (docker compose version)
curl for the verification commands

The checked-in Compose setup uses standard KVM. If the host does not support it, read PVM Deployment before adapting the runtime image and host configuration.

Clone the Repository

git clone https://github.com/kvcache-ai/AgentENV.git
cd AgentENV

Start the Cluster

sudo bash scripts/docker-setup.sh
make deploy-up

make deploy-up builds the runtime, gateway, and scheduler images with Docker Compose before starting the stack. The Rust and Go toolchains are installed in the image build stages, so they are not required on the host.

Runtime nodes mount the host’s upstream DNS configuration read-only for guest networks, including template builders. The make deploy-* targets select /run/systemd/resolve/resolv.conf when available, otherwise /etc/resolv.conf. Override HOST_RESOLV_CONF with an absolute path to a resolver file containing DNS servers reachable from guests; loopback addresses such as 127.0.0.53 and Docker’s 127.0.0.11 cannot be used by guests. Direct docker compose commands default to /run/systemd/resolve/resolv.conf; set HOST_RESOLV_CONF explicitly on hosts without that file. Containers retain Docker DNS for service discovery.

To build without starting, run make deploy-build. To start images that are already built, run make deploy-up-no-build:

# Build only
make deploy-build

# Start previously built images
make deploy-up-no-build

On first startup, the runtime nodes atomically generate one API key and sandbox access-token seed in the shared agentenv-auth volume. The gateway mounts that volume read-only and reads the API key; sandbox tokens are validated by the runtime nodes. Normal make deploy-down calls preserve both values.

To enable host-based sandbox data-plane URLs, set the shared sandbox proxy domain variable when starting the stack:

SANDBOX_PROXY_DOMAINS=sandbox.example.com \
make deploy-up

Compose passes this value to both the gateway routing allowlist and runtime nodes’ sandbox response metadata. The domain must resolve to the gateway, usually through wildcard DNS for *.sandbox.example.com.

Verify

# Health check via gateway
curl http://127.0.0.1:8000/health

# Authenticated cluster node snapshots via gateway
export AENV_API_KEY="$(docker compose -f deploy/docker-compose.yml exec -T agentenv-a \
  cat /workspace/env/secrets/api-key)"
curl -H "X-API-Key: ${AENV_API_KEY}" http://127.0.0.1:8000/nodes

# Health check from inside a backend container
docker compose -f deploy/docker-compose.yml exec -T agentenv-a \
  curl -fsS http://127.0.0.1:8000/health

Management Commands

make deploy-ps      # Show container status
make deploy-logs    # Stream logs from all services
make deploy-down    # Tear down the cluster

Removing Compose volumes with docker compose down -v also removes both secrets. The next startup generates new values, so existing clients and sandbox access tokens are invalidated.

To use an existing API key instead of the generated value, export AENV_API_KEY before starting the stack. Compose passes it to the gateway and both runtime nodes:

export AENV_API_KEY="e2b_..."
make deploy-up

Configuration

The Scheduler and Gateway use deploy/docker/config/default.json. See Gateway and Scheduler Configuration for JSON fields, environment overrides, HTTPS settings, and examples.

After changing only the JSON, restart the affected services to reload it:

docker compose -f deploy/docker-compose.yml restart scheduler gateway

After changing Compose environment variables, ports, or mounts, apply the new container configuration using the existing images:

make deploy-up-no-build

Keyboard shortcuts