Multispectral Deep Neural Network for Low Light Object Detection

Abstract

In recent years, multi-modal object detection has garnered attention in the research community for automotive and surveillance applications. Visual and infrared image fusion has demonstrated promising results for object detection in adverse weather and lighting conditions due to infrared cameras being robust against illumination challenges. However, there is still a lack of studies on effectively fusing two modalities for optimal object detection performance. This thesis presents a novel approach to fuse visual and infrared images using Faster R-CNN with Feature Pyramid Network. The proposed network fuses visual and infrared channel features using concatenation operation. In addition to our proposal, we conduct comprehensive ablation experiments on KAIST and FLIR datasets. Our ablation experiments include fusion analysis using addition and concatenation operator at varying stages of the network. Our proposal and ablation experiments are evaluated on mean Average Precision (mAP), and Log-average miss rate (MR) evaluation metrics. Our extensive evaluation of the proposed framework demonstrates that our framework outperforms the current state-of-the-art benchmarks.