Metadata-Aware Output Converters in Savant
Model output converters are one of the most useful extension points in Savant. A converter turns a model’s raw output tensors into Savant’s representation — bounding boxes for detectors, attribute values for classifiers, or both for complex models. Until now, a converter saw only the tensors, the model definition, and the ROI it ran on. It had no idea which stream the frame came from or what else was on that frame.
That limitation is gone. Savant output converters can now optionally receive the frame’s metadata — the same NvDsFrameMeta wrapper that PyFuncs work with, exposing source_id, pts, the VideoFrame, existing objects, and tags. This turns post-processing from a stateless tensor transform into something that can adapt to the context of each frame.
Compatibility Note: this feature is available in Savant develop and ships in the upcoming release.
Opt-In and Fully Backward Compatible
The best part is that nothing breaks. The new metadata argument is opt-in: a converter receives it only if its __call__ method declares a metadata parameter. Converters written for previous versions of Savant — including all the built-in ones — keep working unchanged.
The base converter signature now looks like this:
def __call__(
self,
*output_layers: np.ndarray,
model: ObjectModel,
roi: Tuple[float, float, float, float],
metadata: Optional[NvDsFrameMeta] = None, # new, optional
) -> Any:
...
Under the hood, Savant inspects each converter’s signature once (the result is cached per converter class, so there is no per-frame reflection cost) and only builds and passes the metadata wrapper when it is actually wanted. When metadata is used, the output conversion also gets its own convert-output telemetry span nested under the frame’s span, so it remains fully traceable.
Why This Matters
Because the converter now knows the source_id, a single pipeline instance serving many streams can apply different post-processing to each stream — a common need that previously required workarounds. A few examples:
- Per-source detection thresholds. Busy scenes and quiet scenes rarely want the same confidence or NMS settings. Now the converter can pick them per stream.
- Per-frame adaptation. Because the converter also sees the frame’s objects, tags, and PTS, it can adjust decoding based on what is already known about the frame — for instance, tightening thresholds when an upstream element has flagged the frame.
- Context-aware decoding. Any decision that depends on which camera or what state a frame belongs to can now live in the converter, rather than being pushed downstream into a separate PyFunc.
The Sample: Per-Source Thresholds from Etcd, Live
The PR ships a runnable output_converter_metadata sample that combines this feature with Savant’s dynamic Etcd reconfiguration. Two RTSP streams — city-traffic and town-centre — are ingested by one module and detected with a YOLO11n model. The output converter is a small subclass of the built-in YOLO converter:
class EtcdConfigurableConverter(TensorToBBoxConverter):
"""YOLO bbox converter whose thresholds are overridden per source_id from Etcd."""
def __call__(self, *output_layers, model, roi, metadata=None):
config = {}
if metadata is not None:
config = self._load_source_config(metadata.source_id)
# per-source override with fallback to construction-time defaults
self.confidence_threshold = config.get(
'confidence_threshold', self._default_confidence_threshold
)
self.nms_iou_threshold = config.get(
'nms_iou_threshold', self._default_nms_iou_threshold
)
# reuse the parent's YOLO tensor decoding / NMS / coordinate transform
return super().__call__(*output_layers, model=model, roi=roi)
For each frame, the converter reads metadata.source_id, looks up a per-source configuration object in Etcd (cached for a few seconds to keep the hot path fast), and overrides its thresholds accordingly — falling back to the defaults declared in module.yml when a source has no config yet.
The thresholds can be changed live, with no pipeline restart, using the provided helper:
# you are expected to be in Savant/samples/output_converter_metadata/ directory
# keep low-confidence detections on city-traffic (more boxes)
./set-config.sh city-traffic '{"confidence_threshold": 0.2}'
# require high confidence on town-centre (fewer boxes)
./set-config.sh town-centre '{"confidence_threshold": 0.7}'
Run it and watch the two streams react independently:
# you are expected to be in Savant/ directory
# if x86
docker compose -f samples/output_converter_metadata/docker-compose.x86.yml up
# if Jetson
docker compose -f samples/output_converter_metadata/docker-compose.l4t.yml up
# open 'rtsp://127.0.0.1:554/stream/city-traffic' and
# 'rtsp://127.0.0.1:554/stream/town-centre' in your player,
# or visit 'http://127.0.0.1:888/stream/city-traffic' (LL-HLS)
Wrapping Up
Metadata-aware converters are a small, backward-compatible signature change with a big payoff: post-processing that can finally see the context of the frame it is working on. Combine it with Etcd and you get per-source, live-reconfigurable detection behavior from a single pipeline — no restarts, no duplicated pipelines. Grab the sample and try wiring your own per-source logic into a converter.
As always, join our Discord for help and Savant news, and star the project on GitHub.