TENT Transport Selector#
Overview#
The Transport Selector is responsible for choosing the optimal transport and devices for each transfer request based on configuration policies.
Request Priority#
TENT uses a unified priority system across all components. See QoS.md for detailed description of priority levels and their usage throughout the system.
Quick reference:
"high"/0- High-priority requests (metadata, control, latency-sensitive)"medium"/1- Medium-priority requests (interactive queries, serving)"low"/2- Low-priority requests (bulk transfer, background jobs)
Configuration-Based Transport Selection#
Transport selection is driven by configuration with pattern-based rules.
Configuration Example#
{
"policy": [
{
"name": "foreground_get",
"segment_type": "memory",
"intent_type": "foreground_get",
"priority": "high",
"devices": ["mlx5_0", "mlx5_1", "mlx5_2"],
"transports": ["nvlink", "xpu", "rdma", "shm"]
},
{
"name": "low_prio_slow",
"segment_type": "memory",
"priority": "low",
"devices": ["mlx5_0"],
"transports": ["rdma", "tcp"]
},
{
"name": "file_storage",
"segment_type": "file",
"transports": ["gds", "io_uring", "rdma"]
}
]
}
Policy Fields#
Field |
Type |
Required |
Description |
|---|---|---|---|
|
string |
Yes |
Policy identifier (for logging) |
|
string |
Yes |
|
|
string or int |
No |
Match only requests with this priority: |
|
string or int |
No |
Match a standard transfer intent such as |
|
array[string] |
No |
List of allowed device names (empty = all devices) |
|
array[string] |
No |
Transport preference list (evaluated in order) |
A non-empty transports list replaces the transports registered on the
buffer, so it must name every transport the policy should use. In particular,
same-host Intel XPU peer-to-peer copies (Level Zero IPC between processes)
are only used when the list contains "xpu"; put it ahead of "rdma", as in
the example above. A policy that omits it keeps such requests on the network
path (RDMA dma-buf or host staging).
Intent-Based Policy Binding#
Request::intent_type can select an intent-specific policy before transport,
device, QP-pool, and SL/TC resolution:
{
"policy": [
{
"name": "foreground-kv",
"segment_type": "memory",
"intent_type": "foreground_get",
"qp_pool": "foreground",
"service_level": 3,
"traffic_class": 96,
"transports": ["rdma"]
},
{
"name": "memory-fallback",
"segment_type": "memory",
"transports": ["rdma", "tcp"]
}
]
}
Policies are evaluated in JSON order, so intent-specific entries should appear
before a catch-all entry. A policy without intent_type retains the historical
behavior and matches any intent. INTENT_UNSPEC therefore behaves exactly as
before with existing configurations.
An explicit Request::policy_name remains the strongest per-request override:
it selects the named policy by segment type and bypasses the policy’s other
match filters, including intent_type. Invalid intent values cause that policy
entry to be skipped rather than silently converted into a catch-all rule.
Memory Type Filters#
For memory segments, you can filter by source/destination memory type:
Pattern |
Matches |
|---|---|
|
CUDA GPU memory |
|
CPU/host memory |
|
ROCm/HIP GPU memory |
|
Ascend NPU memory |
|
Intel GPU device memory (host/shared USM matches |
|
Any memory type |
For standard tcp, the local memory type always describes the initiating
process’s buffer, for both READ and WRITE. A CPU-only build can transfer between
its host buffer and a peer’s registered GPU buffer when the peer exports that
buffer over TCP. The peer’s device-enabled RPC handler performs the GPU copy
through host staging; the initiator does not need a local GPU runtime. Local GPU
buffers still require a device-enabled build. This applies to standard tcp;
hp_tcp currently supports host memory only.
Size Filters#
Restrict policies to specific transfer sizes:
{
"name": "small_transfers",
"segment_type": "memory",
"min_size": 0,
"max_size": 1048576,
"transports": ["shm", "nvlink"]
}
Device Mask#
The devices field in a policy creates a bitmask that restricts which NICs can be used:
{
"name": "use_nic_0_only",
"segment_type": "memory",
"devices": ["mlx5_0"],
"transports": ["rdma"]
}
This is translated internally to a 64-bit bitmask where each bit represents one device:
devices: ["mlx5_0"]→device_mask = 0x0001(bit 0 set)devices: ["mlx5_1", "mlx5_2"]→device_mask = 0x0006(bits 1 and 2 set)devices: [](empty) →device_mask = ~0ULL(all devices)
Transport Fallback#
When multiple transports are listed in the transports array, they act as fallback options:
{
"transports": ["rdma", "tcp"]
}
This means:
Try RDMA first
If RDMA is unavailable or fails, fall back to TCP
Fallback with transport_index#
For programmatic control, the transport_index parameter selects which transport to use:
// transport_index = 0 → First transport (rdma)
// transport_index = 1 → Second transport (tcp)
// transport_index = 2 → Third transport (if exists)
auto result = selector.select(context, transports, transport_index);
Per-request override (transport_hint)#
Request::transport_hint lets a caller bypass the policy lookup for one request at a time. It’s the per-request analogue of transport_index and sits on top of the configured policies, so callers can keep the global policy config as the default while still pinning specific requests.
Request r{};
r.opcode = Request::WRITE;
r.source = local_ptr;
r.target_id = seg;
r.target_offset = 0;
r.length = 4096;
r.transport_hint = TransportType::TCP; // UNSPEC (default) defers to policy
engine.submitTransfer(batch_id, {r});
Semantics#
UNSPEC(default): unchanged —TransportSelector::select(...)runs as documented above.Pinned: the request’s first attempt uses the hinted transport. On failover (when enabled), the engine asks the selector for the next candidate with the hint excluded, so failover never loops back onto the transport that already failed.
Default Behavior#
If no policy is configured, TENT falls back to original behavior:
Segment Type |
Default Transport Order |
|---|---|
File |
GDS → IOURING → RDMA |
Memory |
Uses |
UB as a selectable transport#
UB (TransportType::UB) takes part in the same policy machinery as every other
transport: it can be listed in a policy’s transports array, pinned with
transport_hint, or reached through transport_index. Two places name it
explicitly instead:
Local memory allocation. When the requested transport is
UNSPEC, allocation walksMNNVL → RDMA → UB → FABRIC → TCP → HP_TCP(withSHMtaking the host-memory slot ahead of RDMA for CPU and wildcard locations), so an installed UB transport is preferred over the generic TCP paths.TPU staging. The TPU staging policy needs a host-DRAM network transport for the host↔host hop and gates on
RDMA,UB,FABRIC,TCPorHP_TCP.
Once UB is the selected transport, the rail for each slice is chosen inside
UbTransport rather than by this selector; see TENT Failover for UB rail health, path ranking and endpoint rebuild
behaviour.
Complete Example#
{
"policy": [
{
"name": "high_priority_local",
"segment_type": "memory",
"same_machine": true,
"priority": "high",
"transports": ["nvlink", "shm"]
},
{
"name": "high_priority_remote",
"segment_type": "memory",
"same_machine": false,
"priority": "high",
"local_memory": "cuda",
"remote_memory": "cuda",
"devices": ["mlx5_0", "mlx5_1", "mlx5_2"],
"transports": ["rdma"]
},
{
"name": "medium_priority",
"segment_type": "memory",
"priority": "medium",
"transports": ["rdma"]
},
{
"name": "bulk_transfer",
"segment_type": "memory",
"priority": "low",
"min_size": 104857600,
"transports": ["rdma", "tcp"]
},
{
"name": "file_ops",
"segment_type": "file",
"transports": ["gds", "io_uring"]
}
]
}
Unified Priority System#
The priority value propagates through multiple layers:
Request.priority (application layer)
↓
TransportSelector policy matching
↓
DeviceSelector allocation (QoS filtering)
↓
Worker thread scheduling queues
Layer interactions:
TransportSelector: Policy’s
priorityfield filters which requests matchDeviceSelector:
priorityparameter controls device eligibility (QoS mode)Workers: Separate queues per priority level with strict draining order
See QoS.md for details on worker scheduling and global slot coordination.
Data Flow#
Request with priority
↓
TransportSelector.select(context, transports, transport_index)
↓
Match policy by:
- segment_type (file/memory)
- intent_type (exact match if specified in policy)
- priority (exact match if specified in policy)
- location constraints
- size constraints
↓
Build device_mask from policy.devices
↓
Select transport from policy.transports[transport_index]
↓
Return SelectionResult { transport, device_mask }
↓
RdmaTransport.submitTransferTasks(batch, requests)
↓
DeviceSelector.allocate(..., request.priority, batch.device_mask)
↓
Worker scheduling (separate queues per priority)