Audio

Waveforms and features through the same matrix runtime.

OA Audio is a compact offline, batched, capture, playback, and media layer. It uses explicit Float32 channel-major buffers and composes with the Core, Vision, and ML modules.

15-operation verified sliceWAV · FLAC · MP3 decodeFloat32-first

Audio

Input and output

File decode

WAV, FLAC, and MP3 decode to planar Float32 OaMatrix data with explicit rate, channels, and sample count.

Capture

A bounded real-time device ring publishes timestamped Float32 chunks without blocking the audio callback.

Playback

OaAudioStream supports incremental decode, play, pause, seek, loop, and monotonic position tracking.

Encoding

Lossless WAV-F32 output is public; native AAC packetization is used by the media recorder. FLAC and Opus encoding remain roadmap work.

Verified operations

Group	Operations
I/O	Load file or memory; save WAV-F32
Signal	Gain, mix, fade, clip, pre-emphasis, mono, normalization, resampling
Features	STFT magnitude, mel spectrogram, MFCC, amplitude to dB
Interop	Zero-copy raw-audio matrix view

Audio to model

STFT, mel, and MFCC outputs are ordinary OaMatrix values. The feature path can therefore feed a model without a second tensor type or an adapter framework.

Audio_features.cpp

auto decoded = OaAudioDecoder::LoadFile("speech.flac");
if (not decoded.IsOk()) return decoded.GetStatus();

OaMatrix waveform = OaFnAudio::Resample(
    decoded->Buffer,
    OaResampleConfig{.InRate = decoded->SampleRate, .OutRate = 16000});
waveform = OaFnAudio::Normalize(waveform, -6.0F);

OaMelConfig melConfig{.FftSize = 512, .HopSize = 160, .NumMels = 80};
OaMatrix mel = OaFnAudio::MelSpectrogram(waveform, 16000, melConfig);

Media integration

Video sessions can open the first audio track from the same URI and preserve video-only operation when no compatible track exists. Capture and recording share timestamps with Vision; entropy codecs remain explicit CPU codec boundaries around Vulkan DSP.