PyTorch Model Serving

  • Description
    PyTorch is one of the most known DeepLearning solutions in the world, alongside Tensorflow and Keras. Its functionality is way broader than that provided by TensorRT and NVInfer. With PyTorch, you can infer and train models and use advanced algorithms implemented for data processing. Also, it does not focus on computer vision but can serve models to predict data in other modalities; thus, you can use it to handle auxiliary data delivered with video data in attributes.

    PyTorch wins from using Savant by accessing raw video frames most efficiently – in the GPU RAM without downloading them to the CPU RAM. The OpenCV-based reading or GPU-accelerated decoding implemented in the Torchaudio extension does not provide comparable performance and resource utilization.

  • Key Features
    — PyTorch CUDA inference
    — CUDA-based Tensor processing
  • End-To-End Performance
    — 76 FPS on Nvidia A4000