Vision
Images, video, capture, and models on one GPU-first path.
OA Vision combines 50 schema-owned image operations, Vulkan Video decode and encode, timestamped capture, media containers, and direct OaMatrix interoperability.
50 image operationsH.264 · H.265 · AV1 · VP9 decodeCapability-gated codecs
Vision
The frame contract
OaVideoFrame carries the image or buffer resource, format, dimensions, presentation timestamp, color metadata, and GPU completion token. Decode, capture, processing, display, and encode use that same contract instead of copying through an intermediate CPU frame abstraction.
Image operations
The public inventory is generated from Vision schemas; complex graph lowering remains handwritten and tested.
Vulkan Video
Codec support and resource paths are selected from device capabilities, queue topology, formats, and profile limits.
Media sessions
OaVideo provides open, next, seek, flush, playback state, frame reordering, and optional synchronized audio.
ML interop
CurrentFrameToMatrix and OaFnImage preserve the direct path from decoded or captured imagery to model input.
Operation families
| Family | Coverage |
|---|---|
Pixel | Thresholding, range tests, clamp, inversion, tone controls, compositing, channel operations, and noise. |
Geometric | Resize, crop, flip, rotate, pad, remap, affine warp, and perspective warp. |
Filter | Convolution, blur, Sobel, Scharr, Laplacian, morphology, bilateral, sharpening, and adaptive thresholding. |
Color | Per-channel normalization plus semantic image color conversion. |
Matrix to image operation
The examples use only calls present in the current C++ surface and oa.vision bindings.
Vision_ops.cpp
OaMatrix image = OaFnMatrix::Rand(OaMatrixShape{1, 3, 1080, 1920});OaMatrix resized = OaFnImage::Resize(image, 640, 640, OaInterpolationMode::Bilinear);OaMatrix edges = OaFnImage::Sobel(resized, 1, 0, OaBorderMode::Reflect);OaContext::GetDefault().Execute();OaContext::GetDefault().Sync();
Codec boundary
| Format | Route | Output contract |
|---|---|---|
| H.264 / AVC | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| H.265 / HEVC | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| AV1 | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| VP9 | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| JPEG / PNG | File decode and GPU upload | Texture, image planes, or OaMatrix |
Support is queried at runtime. A listed codec is part of the OA implementation surface; it is not a claim that every Vulkan device exposes every profile, format, or queue route.