Multi-level Fusion Network for 3D Object Detection from Camera and LiDAR Data

Abstract

In 3D Object Detection (3DOD), the fusion of point cloud data and image data is of vital importance, because this can maximize the use of the high resolution information of RGB image and the rich 3D information of point cloud data. This thesis proposes a two-stage 3D object detection system, which takes input from the camera and LiDAR data, and outputs the localization and category of the 3D bounding box. The system uses a novel feature extractor to learn the full-resolution scale features while keeping the computation speed coupled with a multimodal fusion Region Proposal Network (RPN) architecture. The second stage detection network regresses the offsets between the 3D proposals generated by the RPN and the ground truth boxes using a 6-dimension encoding technique. Experiments conducted on the Kitti dataset showed the performance boost of the proposed algorithm over the state-of-the-arts on the 3D Object Detection tasks.