O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

Omnimodal AI models — systems that understand both video and audio together — represent the current frontier of video understanding. But they come with a steep cost: processing long, joint token sequences makes inference slow and memory-intensive, and existing benchmarks do not fully verify that models are actually using both modalities. O-MARC addresses both problems. It introduces a new public benchmark that demands genuine audio-visual reasoning, and a compression distillation framework that makes omnimodal inference 34.6% faster and 34.7% more memory-efficient — while improving accuracy over full token inference.

What Problem Does O-MARC Solve?

Running AI on continuous video and audio is expensive. Each second of video generates a large number of tokens that a model must process. At scale — multiple cameras, continuous feeds, real-time requirements — this becomes the central deployment barrier for AI in physical environments.

The problem is compounded by a measurement gap: existing audio-visual benchmarks do not confirm that models are genuinely using acoustic signals. A model can score well on many benchmarks using visual information alone, meaning apparent audio-visual capability may be visual reasoning in disguise.

O-MARC resolves both: it measures real audio-visual association, and it makes compliant models efficient enough to deploy.

UGC-AVQA: A New Benchmark for Genuine Audio-Visual Reasoning

A core contribution of this research is UGC-AVQA — a new public benchmark containing 1,000 user-generated videos and 4,816 question-answer pairs designed to ensure that correct answers require both acoustic and visual evidence.

The benchmark's audio removal test is the distinguishing feature: each question is tested against a version of the video with audio stripped out. Questions that can be answered without audio are excluded. The result is a benchmark that isolates genuine multimodal reasoning — the specific capability that matters for real-world deployment in environments where audio signals carry operational significance.

How O-MARC Compresses and Distills

O-MARC introduces two components that work together.

OMAC — Training-Free Plug-In Compression: OMAC is a compression method that operates without retraining. It identifies salient visual memory frames and temporally grounded audio anchors — the moments in a video that carry the most information — and discards the rest. Unlike compression methods that reduce tokens uniformly, OMAC selects intelligently, preserving what a reasoning model will need.

O-MARC Distillation Framework: To ensure compact models remain robust when given compressed inputs, O-MARC adds a distillation stage. A larger teacher model — operating on full, uncompressed context — supervises the compact student model during training. The student learns to reason as well as the teacher, but from a leaner input. This is the key insight: compression and distillation together produce a model that is faster and more accurate, not one that trades speed for capability.

Benchmark Results

Evaluated on Qwen2.5-Omni-3B across four benchmarks:

O-MARC improves over both full token inference and the leading alternative compression method. It does not trade accuracy for speed — it improves both simultaneously.

Inference efficiency gains (OMAC compression alone):

• Latency reduced by 34.6% (1.53× speedup)
• Memory usage reduced by 34.7%

Key Insights

• Compression and accuracy are not in opposition: O-MARC demonstrates that intelligently compressing video and audio tokens improves accuracy over processing everything, because it removes noise and preserves signal.
• Most existing benchmarks overstate audio-visual capability: The UGC-AVQA audio removal test reveals that many apparent audio-visual models are solving problems through vision alone. Real audio-visual association is rarer and harder than benchmarks suggest.
• Training-free compression is practically valuable: OMAC works as a plug-in without retraining, meaning it can be applied to existing deployed models — lowering the barrier to efficient inference in production environments.
• 34.6% latency reduction enables real-time deployment: At this efficiency level, omnimodal inference becomes practical for continuous camera feeds in operational environments where cloud latency and compute cost are real constraints.

Why This Matters for Visual Memory Infrastructure

Efficiency is the unlocking problem for AI in physical environments. A system that requires cloud-scale compute to process a single camera feed is not a deployable product. O-MARC demonstrates a path to on-device omnimodal inference — the prerequisite for AI agents that can reason about what cameras see and hear, continuously, at scale.

Conclusion

O-MARC makes omnimodal video understanding significantly more efficient without sacrificing — and in fact improving — reasoning accuracy. It contributes a new public benchmark (UGC-AVQA) that closes a measurement gap in the field, and a compression-distillation framework (OMAC + O-MARC) that closes the gap between lab performance and production deployment.