O OVIR

Opportunistic Verifiable Inference Runtime

Content-addressed inference across a swarm of machines.

OVIR is a distributed runtime for neural network execution. It splits inference into reproducible, cacheable work units, dispatches them across many VMs, and avoids recomputation through content-addressed results.

View GitHub Repo Gensyn REE Examples

Architecture

No code generation. No agentic detours. OVIR focuses on execution: graph partitioning, cache reuse, pipeline parallelism, reproducible workers, and verifiable outputs.

Hash the requestModel ID, weights digest, input tensors, decoding parameters, execution mode.
Build an execution DAGLayers, tensor shards, KV-cache segments, microbatches, and pipeline stages.
Check the swarm cacheSkip any unit whose content-addressed result already exists.
Dispatch actorsElixir-style processes supervise work across VMs, GPUs, and backends.
Run reproducible kernelsREE-like execution where determinism or reproducibility is required.
Return receiptsArtifact hashes, worker metadata, validator traces, latency, and cache-hit evidence.

Runtime components

The first prototype can stay small: an Elixir control plane, Nx-backed numerical workers, content-addressed storage, and a deterministic execution boundary inspired by REE.

01

Actor scheduler

Supervises millions of small work units with Erlang/Elixir-style isolation, retries, backpressure, and failure recovery.

02

Tensor cache

Stores content-addressed inputs, intermediate tensors, KV-cache blocks, logits, and final outputs.

03

Graph partitioner

Divides the model execution graph across layer, tensor, batch, token, and pipeline dimensions.

04

Nx worker layer

Uses Elixir numerical definitions and tensor backends for concurrent distributed execution.

05

REE boundary

Wraps model calls or subgraph calls in reproducible containers when bitwise replay matters.

06

Verifier

Compares hashes, receipts, output equivalence, timing telemetry, and cache-hit claims.

Helpful references

Core materials for the placeholder page, proposal, and first runtime prototype.

OVIR repositoryProject home↗ Gensyn REE examplesReproducible execution examples↗ Gensyn REE overviewMachine-agnostic model inference↗ REE product pageReceipts and validation context↗ Elixir NxTensors and numerical definitions↗ Nx documentationTensor programming in Elixir↗ AxonNx-powered neural networks↗ BumblebeePre-trained neural networks in Elixir↗ OrtexONNX Runtime bindings for Elixir↗ STRIDE AI Efficiency ChallengeProgram overview↗