Video semantic segmentation with low latency

Abstract

Recent advances in computer vision and deep learning algorithms have yielded intriguing results. It can perform tasks previously requiring human eyes and brains. Semantic video segmentation for autonomous cars is difficult due to the high cost, low latency, and performance requirements of convolutional neural networks (CNNs). Deep learning architectures like SegNet and FlowNet 2.0 on the Cambridge-driving labeled video database (CamVid) dataset enable low-latency pixel-wise semantic segmentation of video features. Because it uses SegNet and FlowNet topologies, it is ideal for practical applications. The decision network chooses an optical flow or segmentation network for an image frame based on the expected confidence score. Combining this decision-making method with adaptive scheduling of the key frame approach can speed up the process. ResNet50 SegNet has a “54.27%” mean intersection over union (MIoU) and a “19.57” average FPS. In addition to decision network and adaptive key frame sequencing, FlowNet2.0 increased graphics processing unit (GPU) frame processing per second to “30.19” with a MIoU of “47.65%”. The GPU is used “47.65%” of the time. This performance gain illustrates that the video semantic segmentation network is faster without sacrificing quality.