Realm

oa::ml::nn::EmpyrealmConfig

Layer schedule and widths for the hybrid Empyrealm stack. attention_every interleaves one causal self-attention block after every attention_every - 1 Mamba-3 blocks, at positions k * attention_every - 1; zero produces a pure Mamba-3 stack. Every block owns a pre-normalized residual mixer plus a dense or routed-expert feed- forward tail, and the stack ends in a final RMSNorm.

StRust public API

Declaration

pub struct EmpyrealmConfig { pub layers: usize, pub attention_every: usize, pub attention_heads: usize, pub sequence_length: usize, pub ffn_hidden: usize, pub moe_experts: usize, pub moe_experts_per_token: usize, pub epsilon: f32, pub mamba: Mamba3Config, }
Public Rust declaration.

Public Methods

fn clone(&self) -> EmpyrealmConfig
fn fmt(&self, f: &mut Formatter<'_>) -> Result
fn eq(&self, other: &EmpyrealmConfig) -> bool

Associated Functions

pub fn new(model_width: usize, sequence_length: usize) -> Self

Public Data Members

layersusizefield
attention_everyusizefield
attention_headsusizefield
sequence_lengthusizefield
ffn_hiddenusizefield
moe_expertsusizefield
moe_experts_per_tokenusizefield
epsilonf32field
mambaMamba3Configfield

Public Method Documentation

fn clone( &self ) -> EmpyrealmConfig
No public source comment is attached to this declaration.
fn fmt( &self, f: &mut Formatter<'_> ) -> Result
No public source comment is attached to this declaration.
fn eq( &self, other: &EmpyrealmConfig ) -> bool
No public source comment is attached to this declaration.
pub fn new( model_width: usize, sequence_length: usize ) -> Self
No public source comment is attached to this declaration.