For contributors — how the device runtime works internally. It documents an internal xlog-cuda component, not a feature you enable from the CLI.
Every CudaKernelProvider owns a device runtime inside the xlog-cuda crate. It sits between the code that runs CUDA kernels and the GPU’s memory allocator. It gives every provider four things: a stream-aware allocator (one that knows which CUDA stream each allocation belongs to), a process-wide device-memory budget, block-use tracking, and deferred free/reap semantics for work that runs on a non-default stream. A CUDA stream is an ordered queue of GPU operations; “non-default” means work that runs on a stream other than the implicit default one, so ordering between streams has to be tracked explicitly. There is no alternate provider construction path with a separate allocator. CudaProviderBuilder constructs and validates the complete ownership graph: one device handle, one stream pool, one asynchronous resource stack, one runtime, and one memory manager. Those exact shared handles are then used by ordinary, recorded, solver, and resident execution.

​
Resource Stack

The core trait is DeviceMemoryResource. Implementations compose as decorators — each layer wraps the one below it and adds one responsibility: XlogDeviceRuntime is provider-owned, not a process singleton. A process may create more than one provider, and each provider has its own coherent runtime graph. The builder rejects mismatched device or allocator components instead of letting callers assemble them independently.

​
Allocation Lifecycle

Each allocated block records:
  • an allocation tag describing the caller;
  • the stream that allocated it;
  • a monotonic generation counter used to reject stale handles;
  • the last writer event;
  • outstanding read events;
  • live and pending byte accounting.
The lifecycle enforces safe ordering across streams. Before a kernel uses a block on another stream, prepare_block_use waits on the events needed for safe ordering. After the kernel records a read or write, finish_block_use installs the event that future users must respect. A free can be deferred until prior stream work is complete; reap_pending then retires it. The generation counter matters for correctness. If a pointer address is freed and later reused for a new allocation, an old DeviceBlock handle will not silently mutate the new allocation — its stale generation no longer matches.

​
Budget Behavior

GlobalDeviceBudget wraps an inner resource and refuses allocations that would exceed the configured limit. It tracks three quantities separately:
  • bytes reserved by live allocations;
  • bytes pending a deferred free;
  • bytes still available for new work.
The runtime uses this budget to make memory pressure explicit. A byte-addressed allocation that cannot fit returns ResourceExhausted. A bounded row, slot, candidate, or launch-element limit returns CapacityExceeded with its unit. An optimized route may record a typed decline before the existing ordinary GPU route runs; it never changes to host execution implicitly.

​
Recorded Launches

“Recorded” kernel paths ask the runtime to preserve ordering across streams. Each launch follows five steps:
  1. Acquire or reuse a stream.
  2. Prepare all input and output blocks for that stream.
  3. Launch the kernel through the provider.
  4. Record the block uses.
  5. Reap pending frees once their stream work is complete.
This is the mechanism behind the runtime-backed WCOJ (worst-case-optimal join), groupby, join, and solver paths — the execution routes that need reliable non-default stream behavior.

​
CUDA Version

XLOG’s public release process targets NVIDIA CUDA Toolkit 13.x. The workspace currently uses the cudarc crate’s cuda-12040 feature as a driver API binding level. That binding level is not the same thing as the toolkit requirement.

​
Failure Modes

When the device runtime cannot proceed, it surfaces the problem as an explicit resource error rather than failing silently. The cases are:
  • allocation over budget;
  • stale block use after free;
  • invalid stream or generation state;
  • CUDA allocation/free failure;
  • inability to satisfy a launch dependency.
These are runtime diagnostics. A clean run of the device runtime does not prove that a query result is correct or that an optimized route fired. To check those, pair these diagnostics with route counters and validation gates.