Quick Start

Quick Start#

Get up and running with Mooncake in minutes.

This guide walks you through the entire flow of getting started with Mooncake:

Install Mooncake
Start the Store master and Send a request with the Python Store API

Serving-framework users can then connect SGLang, vLLM or other systems.

Prerequisites#

Python: 3.10 or later; a virtual environment is recommended.
RDMA: an RDMA driver and SDK (for example, Mellanox OFED), if you plan to use RDMA for data transfer. On ScaleFabric SHCA systems, install shca-tools and build with -DUSE_SHCA=ON.
CUDA: 12.1 or later. For most CUDA-enabled use cases, such as RDMA-based KV cache transfer between GPUs or between GPU and DRAM, NVIDIA GPUDirect support is also required. You may install CUDA from the NVIDIA downloads page.

Note

The default pip, build and Docker paths target NVIDIA CUDA. For other platforms, see Other Platforms below.

Installation#

The same package provides:

Mooncake Store Python bindings for vLLM and SGLang HiCache integrations.
Transfer Engine Python bindings and runtime components for direct mooncake.engine.TransferEngine usage.

pip / uv

We recommend using uv for faster installation:

pip install --upgrade pip
pip install uv
uv pip install mooncake-transfer-engine

Plain pip also works:

pip install mooncake-transfer-engine

Tip

The default wheel targets CUDA 12.1–12.9 and includes Mooncake-EP and GPU topology detection. For CUDA 13.0/13.1, install mooncake-transfer-engine-cuda13 instead.

From Source

Clone the repository and build the default configuration:

git clone https://github.com/kvcache-ai/Mooncake.git
cd Mooncake
sudo bash dependencies.sh

mkdir build
cd build
cmake ..
make -j
sudo make install

For CUDA, VRAM segments, NVMe-oF, and other backend flags, see the Build Guide.

Docker

Published images are available on Docker Hub at kvcacheai/mooncake:

docker run --net=host \
    --ipc=host \
    --ulimit memlock=-1 \
    kvcacheai/mooncake:latest \
    mooncake_master

The multi-architecture image publishes these CUDA flavor tags:

Tag	CUDA	Notes
`latest`	13	Default flavor
`0.3.14`	13	Default flavor for this version
`0.3.14-cuda12`	12	Fixed release, recommended for production
`0.3.14-cuda13`	13	Fixed release, recommended for production
`cuda12`	12	Floating alias for the newest CUDA 12 image
`cuda13`	13	Floating alias for the newest CUDA 13 image

For details, see Use Mooncake in Docker Containers.

Note

If users encounter problems such as missing lib*.so, first install the corresponding system runtime libraries. If the issue persists, uninstall the package and build the binaries manually.

Start Mooncake Store#

If you installed with pip or from source, start the master service:

mooncake_master

Wait until you see a line like this in the logs:

Master service started on port 50051, max_threads=4, ...

The default RPC port is 50051. Skip this step if the Docker command above is already running mooncake_master.

Send Your First Request#

Run this single-node put/get example after mooncake_master is running. This example uses P2PHANDSHAKE, so no separate Transfer Engine metadata service is required.

from mooncake.store import MooncakeDistributedStore

store = MooncakeDistributedStore()
store.setup(
    local_hostname="localhost",
    metadata_server="P2PHANDSHAKE",
    global_segment_size=512 * 1024 * 1024,
    local_buffer_size=128 * 1024 * 1024,
    protocol="tcp",
    rdma_devices="",
    master_server_addr="127.0.0.1:50051",
)

store.put("hello_key", b"Hello, Mooncake Store!")

data = store.get("hello_key")
print(data.decode())  # Output: Hello, Mooncake Store!

store.close()

Connect vLLM or SGLang#

Choose the integration path that matches your serving deployment.

PD Disaggregation#

PD disaggregation paths use Mooncake Transfer Engine for direct KV transfer between prefill and decode workers. Configure these paths through the serving framework guides, not by calling Transfer Engine APIs directly:

Distributed KV Cache Pooling#

Mooncake Store provides distributed KV cache storage for vLLM and SGLang HiCache:

Framework	Use case	Setup guide
SGLang	HiCache L3 storage backend with Mooncake Store	SGLang HiCache Quick Start
vLLM	KV cache storage and sharing with `MooncakeStoreConnector`	vLLM KV Cache Storage & Sharing

AI Coding Assistant Skills#

If you use Claude Code or another coding assistant that supports reusable skills, Mooncake provides built-in playbooks for common development tasks:

Skill	Use it for
`/mooncake-troubleshoot`	Diagnose services, RDMA, environment variables, and runtime logs.
`/mooncake-ci-local`	Run pre-PR local validation with Mooncake’s CI script.
`/mooncake-api`	Work with Mooncake Store, Transfer Engine, and EP/Backend Python APIs.

Install them from the Claude Code plugin marketplace without cloning the full repository:

/plugin marketplace add kvcache-ai/Mooncake --sparse .claude-plugin
/plugin install mooncake-troubleshoot@mooncake
/plugin install mooncake-ci-local@mooncake
/plugin install mooncake-api@mooncake

Next Steps#

For production deployment, standalone store services, high availability, allocation strategies, SSD offload, and runtime tuning, continue to the Mooncake Store Deployment & Tuning Guide.

For API details, see the Mooncake Store Python API and Mooncake Store design.