Realm

oa::ml::nn::AttentionCache

Caller-owned incremental-decode key/value cache for MultiHeadAttention. Each MultiHeadAttention::step call appends one [B*H, 1, D/H] key/value pair and attends over every cached position. The cache binds to the constructing module instance; presenting it to another attention module fails closed.

StRust public API

Declaration

pub struct AttentionCache { /* private fields */ }
Public Rust declaration.

Public Methods

pub const fn batch_size(&self) -> usize
pub const fn len(&self) -> usize
pub const fn is_empty(&self) -> bool

Public Method Documentation

pub const fn batch_size( &self ) -> usize
No public source comment is attached to this declaration.
pub const fn len( &self ) -> usize
No public source comment is attached to this declaration.
pub const fn is_empty( &self ) -> bool
No public source comment is attached to this declaration.