oa::ml::nn::AttentionCache
Caller-owned incremental-decode key/value cache for MultiHeadAttention. Each MultiHeadAttention::step call appends one [B*H, 1, D/H] key/value pair and attends over every cached position. The cache binds to the constructing module instance; presenting it to another attention module fails closed.
Declaration
pub struct AttentionCache { /* private fields */ }Public Rust declaration.