Closing the SOP Gap: Using Video AI to Audit Frontline Operations

A note from Ryan Gaertner, COO and founding member of Memories.ai

Hey, I'm Ryan. Today I'll be showing you how we're closing the SOP with real use case close to my heart: fried chicken.

‍From spot-checks to full coverage: using video to audit how frontline SOPs actually run

If you're anything like us, you're a visual learner. So instead of another wall of text about how our video data lake and fine-tuned VLMs get deployed in the real world, I want to walk through one concrete problem we're solving for customers, with a real example you can watch.

SOPs are the executable code of frontline operations

Standard operating procedures define "the right way" to do repeatable work, so quality, safety, and speed stay consistent at scale. They are, quite literally, the executable code of frontline operations.

If you've ever eaten at a McDonald's (or, if you're like me, a Chick-fil-A), you've felt what good SOPs do without ever seeing one. At those chains, SOPs define exactly how every sandwich, fry, and shake is made, how orders are taken, and how the store is cleaned, so the experience is uniform across tens of thousands of outlets worldwide. That is how iconic chains deliver consistent quality, speed, and food-safety compliance at global scale.

Here is the catch. The SOP lives on paper, or in a training video. The actual work happens on the floor, thousands of times a day, at a single store. And almost nobody can see the gap between the two.

‍The problem: thousands of actions a day, and you can only spot-check a few

Take one store making fried chicken cutlets. A single location runs thousands of individual prep actions every day. The SOP for one cutlet alone has nine steps: thaw, trim, bread, start fry, end fry, cut, rest, dust, serve. Every one of those should be executed the same way, every time.

So how do operators check today? They spot-check. A manager samples a few moments, or reviews a slice of camera footage, and extrapolates from it. The problem is obvious the moment you say it out loud: spot-checks see a tiny fraction of the day, miss most of what happens, and can't be traced back to specific evidence. Ninety-plus percent of the day is a blind spot.

The cameras are already recording all of it. The footage just sits there, because no human can watch 24 hours of every camera in every store.

What we did: turn the surveillance feed into a full SOP audit

This is where the Memories.ai video data lake and a fine-tuned VLM come in. We pointed it at a real fried-chicken chain's in-store kitchen footage and had it do the thing a human never could: watch all of it.

‍The method is simple to describe:

1. Read the long video. Ingest the full surveillance feed, not a sample of it.

2. Locate the action areas. Find where in the frame the relevant work is happening.

3. Identify action boundaries. Detect where each action starts and ends.

4. Generate SOP events. Turn each one into a timestamped, labeled event, tied back to the exact video segment that proves it.

The output is the raw footage sliced automatically into the SOP itself: thaw, bread, cut, fry, dust, serve, each occurrence found, counted, and evidenced. Every step gets a running tally, and every tally links to the original clip a manager can watch to verify.

‍The result: full coverage, and things a spot-check would never catch

Run against real store data, the model mapped all nine SOP steps end to end. Full coverage of the day instead of a handful of samples.

The interesting part is what full coverage surfaces. Thawing is the step that gets missed most often. On this footage, the store's own self-check had logged thawing at 2 of 18 (about 11%). Watching the entire feed, our model found 14 of 18, roughly 78%. That is twelve executions the store's own process never recorded, and about a 67-percentage-point jump in what actually got seen. On several other steps, the model and the store's records lined up at 100%.

And where the store's own labels are still more complete, in this run that was dusting and serving, the video showed that too. We are not claiming the model is already perfect. We are claiming it sees the whole day, is honest about the gap, and closes it a little more every cycle. That is the payoff of building on a data lake and fine-tuning: it gets sharper on your specific operation the more it runs.

What this unlocks

Once every SOP action across every store is an evidenced, searchable event, three things become possible that weren't before:

- Know what actually happens vs. what's on paper. Not a sample, the whole day.

- Be alerted about where and when behavior drifts from the SOP, with the exact clip attached.

- Quantify business impact: which deviations correlate with incidents, waste, or slower throughput.

Consistency is what these brands are actually selling. For the first time they can measure it directly, instead of inferring it from a spot-check.

We're just getting started

We deliver the best fine tuned video understanding model in your scenarios and we're genuinely excited about where it goes. So if you work in large-scale frontline operations – whether that's kitchens, retail, manufacturing, or logistics – we'd love to hear your thoughts, questions, and feedback.

Follow along. There's a lot more to come.