Advancing Neuromorphic Event-Based Vision Methods for Robotic Perception Tasks
El Shair, Zaid A.
2024-04-27
View/ Open
Abstract
This dissertation explores the emerging field of event-based vision, a significant innovation in visual sensing technology that represents a marked departure from traditional frame-based imaging. Inspired by the biological processes of the human retina, event-based sensors operate asynchronously at the pixel level. They are characterized by their ability to capture data with high temporal resolution and exceptional dynamic range, detecting and recording changes in light intensity independently. This unique capability allows for the continuous and selective monitoring of a scene, dynamically capturing information only as necessary.The core focus of this research is to harness the potential of neuromorphic event-based vision for advancing object detection and tracking methodologies. Despite the promising attributes of event-based sensors, their integration into conventional Computer Vision (CV) architectures poses substantial challenges, primarily due to the asynchronous and sparse nature of their output. This dissertation aims to address these challenges by developing novel methodologies that leverage the unique strengths of event-based vision while overcoming its inherent limitations.A key contribution of this work is the introduction of the Multi-Modal Event-Based Vehicle Detection and Tracking (MEVDT) dataset. This pivotal resource, comprising synchronized streams of event data and grayscale images, facilitates the development and evaluation of novel event-based algorithms, particularly in automotive contexts. Building on this foundation, the dissertation presents a hybrid approach that integrates state-of-the-art frame-based detectors with novel event-based methods, achieving high temporal resolution in object detection and tracking. This approach is further refined with advanced techniques to enhance both detection accuracy and tracking robustness.A central element of this research is the Compact Spatio-Temporal Representation (CSTR). This novel representation effectively encodes event data into a format that is directly compatible with modern computer vision architectures, integrating spatial, temporal, and polarity information. The CSTR, in conjunction with a specially designed augmentation framework, significantly improves the performance of various recognition tasks.The culmination of this dissertation is a comprehensive analysis of the CSTR and other image-like event representations in the context of event-based and multi-modal object detection. Rigorous testing on two event-based multi-modal datasets demonstrates the effectiveness of these methods, offering insights into their comparative performances and the synergies between event-based and frame-based sensors. Through these comprehensive evaluations, this work underscores the importance of optimal spatio-temporal representations for event-based vision tasks. Ultimately, this dissertation represents a step towards the practical application of event-based vision, contributing to the ongoing evolution in the field of CV.Deep Blue DOI
Subjects
Event-based Vision Computer Vision Deep Learning Object Detection and Tracking Multi-Modal Perception Spatio-Temporal Representation Image Processing
Types
Thesis
Metadata
Show full item recordCollections
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.