Sliding window support for image processing in autonomous vehicles

Abstract

Camera-based autonomous driving extensively ma-nipulates images for object detection, object tracking, or camera-based localization tasks. Therefore, efficient and fast image processing is crucial in those systems. Unfortunately, current solutions either do not meet AD’s constraints for real-time performance and energy efficiency or are domain-specific and, thus, not general [14]. In this work, we introduce Sliding Window Processing (SWP), a SIMD execution model that natively operates on sliding windows of image pixels. We illustrate the benefits of SWP through a novel ISA extension called SLIDEX that achieves high performance and energy efficiency while maintaining pro-grammability. We demonstrate the benefits of SLIDEX for the image processing tasks of ORB-SLAM [17] [18], a state-of-the-art camera-based localization system. SLIDEX achieves an average end-to-end speedup of ~1.65× and ~1.2× compared to equivalent scalar and vector baselines respectively. Compared with the vector implementation, our solution reduces the end-to-end energy consumption a 22% on average.