Realm

tu_nlp_byte_empyrealm — Byte × Empyrealm hybrid

Source-owned OA v0.12.0 SDK tutorial. Recorded results qualify the named device and driver.

Experimental

The byte-tokenizer × Empyrealm hybrid cell of the — same 576-character corpus, same 300-step recipe and captured training lifecycle. This entry selects the tokenizer and model shown below. See for the full walkthrough.

This cell

Tokenizerthe 256-token packed-U8 byte vocabulary (ByteSampler) — one token per source byte
ModelEmpyrealm(Byte Embedding + Mamba-3/attention hybrid stack) → Linear(32→256)
Parameters73,848
OptimizerAdamW(lr=0.003) — lower rate for the Mamba-3/Empyrealm family

The loop is a prepare/record pair per step — GradientTape → cross_entropy → backward → AdamW — driven by oa::ml::ItTraining, so the whole run is one captured GPU program replayed 299 times (184 nodes for this model).

Measured — Surface Laptop 5 (ADL GT2)

Byte × Empyrealm hybrid training loss

Canonical 300-step training loss, recorded by this tutorial.

Loss init→finalAccuracyProgram nodesWall ms/stepToken/s
5.667→0.18292.9%18440.2025.5K

Reference generation on the default recipe (asserted by the tutorial):

1prompt: "to be"
2generated: "to be that is the question whether tis nobler in the mind to suffer the slings and ar"
3

Run

cargo run --release -p sdk --example tu_nlp_byte_empyrealm

Regenerate the training-loss image from the repository root:

cargo run --release -p sdk --example tu_nlp_byte_empyrealm -- --plot sdk/rs/tutorials/ml/nlp/tu_nlp_byte_empyrealm/asset/training-curve.png

On the default recipe the tutorial self-tests: loss regime, accuracy floor, exact reference generation, and .oam checkpoint round-trip.

Terminal output

Recorded canonical 300-step run on Intel Iris Xe (ADL GT2), Mesa 26.2.7. Startup logs, intermediate progress updates and the plot-save path are omitted. Timings vary by run; these are not new benchmark baselines.

╔══════════════════════════════════════════════════════════════════╗
║ OA Tutorial — Empyrealm Core · autograd fidelity ║
╚══════════════════════════════════════════════════════════════════╝
tokenizer: raw bytes · vocab=256
Task: dense next-byte prediction at every position
Model: Empyrealm(Byte Embedding + Mamba-3/attention hybrid stack) → Linear(32→256)
params: 73848 Optimizer: AdamW(lr=0.003)
300/300 |██████████| 12.83s · 42.78 ms/step · 1.50K sample/s · 23.94K byte token/s · 23.94K byte/s · loss: 0.1824
Summary:
loss: initial 5.667074 · final 0.182391 · mean 0.568823
Wall: 42.78 ms/step · 1.50K sample/s · 23.94K byte token/s · 23.94K byte/s
GPU (byte_empyrealm_training_step): mean 41.226 ms/step · p50 39.541 · p95 49.846 · min 25.428 · max 57.788 · 1.55K sample/s · 24.84K byte token/s · 24.84K byte/s · wall-GPU gap 4%
run: 12.83s · 300 steps · batch 64 · sequence 16 byte token/sample
Evaluation:
Random-loss baseline ln(256) = 5.5452
byte-token accuracy: 92.9%
post-update evaluation loss: 0.179192
Generation:
prompt: "to be"
generated: "to be that is the question whether tis nobler in the mind to suffer the slings and ar"
program: 184 nodes · 1 recording · 299 cache hits · 0 uploads
checkpoint: optimizer step, accuracy, and exact generation restored

Source