Struct GlobalDeviceBudget 

pub struct GlobalDeviceBudget { /* private fields */ }

Per-runtime byte-limit decorator.

Implementations

impl GlobalDeviceBudget

pub fn new( inner: Box<dyn DeviceMemoryResource + Send + Sync>, limit: usize, ) -> Self

Wrap inner with a hard limit in bytes. The initial reserved tally is sampled from inner.bytes_outstanding() so callers may compose around an inner that already has live allocations — though in practice the decorator is installed before any allocation flows through it.

pub fn limit(&self) -> usize

Hard byte limit. Set at construction; not adjustable.

pub fn reserved_bytes(&self) -> usize

Bytes currently reserved against the budget (live + pending async free). Matches inner.bytes_outstanding() at every quiescent moment.

pub fn remaining(&self) -> usize

Headroom in bytes for the next allocation. Equal to limit - reserved_bytes, saturating at zero.

Trait Implementations

impl DeviceMemoryResource for GlobalDeviceBudget

fn allocate( &self, bytes: usize, stream: StreamId, tag: AllocTag, ) -> ResourceResult<DeviceBlock>

Allocate bytes bytes on the resource’s device, ordered on stream. The returned block is in BlockState::Live.

fn deallocate(&self, block: DeviceBlock) -> ResourceResult<()>

Return block to the resource. After this call the block’s state is BlockState::Retired (or BlockState::Quarantined for debug-guard resources). Reuse of the underlying memory is resource-specific but must respect the stream-ordered contract.

fn device_ordinal(&self) -> u32

CUDA device ordinal this resource serves. Resources are pinned to a single device.

fn bytes_outstanding(&self) -> usize

Bytes currently outstanding (live + retired-but-not-yet-freed). Used by tests and by the global budget adaptor.

fn reap_pending(&self) -> ResourceResult<()>

Drain any retired-but-not-yet-freed bytes whose underlying CUDA work has completed. For synchronous backends this is a no-op. For stream-ordered async backends this synchronizes the streams that have queued cuMemFreeAsync calls and re-counts bytes_outstanding accordingly.

fn record_block_use( &self, block: &DeviceBlock, use_stream: StreamId, ) -> ResourceResult<()>

Record that work has been (or is being) submitted on use_stream that touches block’s bytes. Resources that participate in cross-stream lifetime tracking (notably the stream-ordered async backend) MUST attach a CUDA event from use_stream to the block; on deallocate(block), the block’s alloc_stream will wait on every recorded event before queueing the underlying free.

fn supports_block_use_tracking(&self) -> bool

Whether this resource (and any inner resources it composes) actually tracks cross-stream uses via record_block_use. Used by the launch recorder’s preflight to fail BEFORE queueing CUDA work, rather than after. The default returns false to match the trait’s default record_block_use behavior; resources that override record_block_use to track events MUST override this to return true. Decorators forward to inner.

fn prepare_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Pre-launch / pre-copy hook: queue any cross-stream waits required for use_stream to safely access block with access semantics. MUST be called BEFORE the GPU work is enqueued on use_stream.

fn finish_block_use( &self, block: BlockId, use_stream: StreamId, access: Access, ) -> ResourceResult<()>

Post-launch / post-copy hook: record an event on use_stream capturing the work just enqueued and update block’s dependency state.