Introducing Meta-Merge: Fan-Out/Fan-In for Savant Video Analytics Pipelines

· Savant Team
Introducing Meta-Merge: Fan-Out/Fan-In for Savant Video Analytics Pipelines

We are excited to announce the availability of Meta-Merge, a new service in the Savant ecosystem that enables Fan-Out/Fan-In (also known as scatter-gather) pipeline architectures for real-time video analytics.

Meta-Merge documentation is available here: https://insight-platform.github.io/savant-rs/services/meta_merge/index.html

Completing the Fan-Out/Fan-In Pattern

Earlier this year we released the Router service, which acts as a Fan-Out (ventilator) — distributing a single video stream to multiple downstream consumers based on content, labels, or unconditionally. The Router solved the first half of the puzzle: getting frames to parallel processing branches.

Meta-Merge solves the second half. It is the Fan-In counterpart that collects independently enriched frames from multiple parallel branches and reunites their object metadata onto a single canonical frame. Together, Router and Meta-Merge form a complete Fan-Out/Fan-In architecture:

                        ┌─ Pipeline A ─┐
Video Source → Router                   ├→ Meta-Merge → Downstream
                        └─ Pipeline B ─┘

The Router fans out; Meta-Merge fans in. Every service in between runs independently, potentially on different hardware, with different models, at different speeds — and the merged output carries all of their annotations combined.

Why Fan-Out/Fan-In Matters

Linear video analytics pipelines — ingest, infer, output — work well for simple use cases. But real-world deployments quickly outgrow this pattern:

  • Multiple perception tasks on the same stream. A single camera feed may need person detection, vehicle detection, and license plate recognition simultaneously. Each task benefits from a dedicated model, often with different architectures and hardware requirements.

  • Horizontal scaling. Heavy inference workloads can be distributed across multiple GPUs or machines, with each branch processing a subset of the work.

  • Fault isolation. When one inference branch fails or slows down, it should not block or crash the others. Independent branches with a merging point provide natural fault boundaries.

  • Modular development. Teams can develop, test, and deploy each inference branch independently, composing them into larger pipelines through configuration rather than code changes.

The Fan-Out/Fan-In pattern addresses all of these needs by decoupling parallel processing from result aggregation.

How Meta-Merge Works

Meta-Merge is a generic, Python-extendable merging framework implemented in Rust. It does not contain domain-specific merge logic — all decisions are delegated to user-registered Python callbacks:

  • on_merge — Called when a frame arrives from any ingress. This is where you import object trees from incoming frames into the current merged state. Return True when all expected branches have reported in.

  • on_head_ready — Called when the merge is complete. Converts the merged frame to a message and forwards it downstream.

  • on_head_expire — Called when the configurable timeout elapses before all branches have reported. Forwards the partial result, ensuring the pipeline never stalls indefinitely.

  • on_late_arrival — Called when a frame arrives after its slot has already been sent. Allows you to log, discard, or handle stragglers.

This callback-driven design means Meta-Merge adapts to any merge strategy: combining different object namespaces, resolving conflicts between overlapping detections, selecting the highest-confidence result, or any custom logic your application requires.

Key Technical Characteristics

  • Rust core for low-latency, multi-core performance — matching the Router’s architecture.
  • Python callbacks for flexible, domain-specific merge logic without recompiling.
  • Configurable timeout (max_duration) ensures frames are never stuck in the merge queue indefinitely.
  • EOS policy control — exactly one ingress forwards End-of-Stream, preventing duplicate delivery.
  • ZeroMQ DEALER/ROUTER transport with backpressure, ensuring reliable frame delivery across all branches.
  • Docker-based deployment with JSON configuration and environment variable substitution.
  • Cross-platform — supports x86_64 and ARM64 (NVIDIA Jetson).

Real-World Applications

The Fan-Out/Fan-In pattern with Router and Meta-Merge enables architectures that were previously complex to build and maintain:

Application Fan-Out (Router) Fan-In (Meta-Merge)
Multi-model inference Duplicate the stream to N model-specific pipelines Merge all detection types onto one frame
GPU load balancing Distribute streams across GPUs Recombine results from all GPUs
Edge-cloud hybrid Route to local fast model + cloud heavy model Merge edge and cloud results
A/B model testing Send the same stream to model A and model B Compare outputs side by side
Selective deep analysis Route flagged frames to a slow, high-accuracy model Merge enriched metadata back into the main stream

Getting Started

Meta-Merge is available now as a Docker image for both x86 and ARM64:

ghcr.io/insight-platform/savant-meta-merge-x86:savant-latest
ghcr.io/insight-platform/savant-meta-merge-arm64:savant-latest

For a complete working example — including a Router, two parallel detector modules, Meta-Merge, declarative visualization, and an automated end-to-end test — see our in-depth tutorial: Building a Fan-Out/Fan-In Video Analytics Pipeline with Savant.

The full sample source code is available on GitHub: samples/meta_merge/.

A Growing Service Ecosystem

Meta-Merge joins Router, Retina RTSP, and Replay in the growing lineup of Savant infrastructure services. Each service is a focused, independently deployable microservice that plugs into Savant’s ZeroMQ-based protocol — no monolithic binaries, no tight coupling, just composable building blocks for real-time video analytics at scale.

Service Role
Retina RTSP Stream ingestion from IP cameras
Router Fan-Out — stream distribution and routing
Meta-Merge Fan-In — parallel result aggregation
Replay Video archiving and replay

For teams building scalable, distributed inference pipelines, the Router + Meta-Merge combination provides a production-ready Fan-Out/Fan-In pattern that has been missing from the open-source video analytics toolkit.