Date: 21 June, 2024

Dataset Title: MEVDT: Multi-Modal Event-Based Vehicle Detection and Tracking Dataset

Dataset Creators: Z.A. EL Shair & S.A. Rawashdeh

Dataset Contact: Zaid A. El Shair zelshair@umich.edu

Funding: N/A

Key Points:
- The MEVDT dataset addresses the critical need for annotated datasets in the domain of event-based computer vision, particularly for automotive applications.
- The dataset comprises 63 multi-modal traffic sequences featuring approximately 13k images, 5M of high-temporal resolution events, 10k manually-annotated object labels, and 85 unique object tracking trajectories.
- Labels for object detection and tracking, alongside multiple data formats, are included to support the development of advanced computer vision algorithms.

Research Overview:
The MEVDT dataset was developed to support advancements in event-based computer vision by providing a rich source of annotated visual data specifically targeting the automotive domain. Its primary goal is to enable the development of event-based and multi-modal algorithms focused on object detection and tracking in dynamic traffic environments, leveraging the unique capabilities of event-based sensors. This dataset includes a significant volume of labeled data essential for advancing research in a novel field where such resources are scarce.

Methodology:
Data was collected using the hybrid sensor DAVIS 240c, which combines an Active Pixel Sensor (APS) and a Dynamic Vision Sensor (DVS) within the same pixel array. The APS captures grayscale images at 24 FPS, while the DVS records pixel brightness changes (i.e., events) at microsecond resolution. The collection process took place at the University of Michigan-Dearborn campus, in two scenes under clear daylight conditions. Data recording was managed using the Robot Operating System (ROS) DVS package running on a laptop. The camera was fixed capturing moving vehicles which were manually labeled with 2D bounding boxes and unique IDs.

Files contained here:
The MEVDT dataset is organized into four directories, each with training and testing splits:
- sequences/: Holds grayscale images and sequence-long event streams. Within each scene, sequences are titled by the first sample's timestamp, including image files (.png) and events stream lists (.csv).
- labels/: Contains ground truth data for object detection and tracking computer vision tasks. The tracking_labels/ includes annotations in formats like COCO JSON, MOT Challenge, and a custom format for sequence-long object tracking; while detection_labels/ provides per-image object class and bounding box coordinates in x_min, y_min, x_max, y_max format for each event sample file in event_samples/.
- event_samples/: Consists of fixed-duration batch-sampled event files in AEDAT format, organized by sampling interval, enabling research at various temporal resolutions.
- data_splits/: Features CSV files that index the dataset splits for object detection, guiding users through the existing training and testing structure for different sampling durations.

Note: each line in the sequence-long event stream CSV files corresponds to a single event in comma-separated in the following format: ts, x, y, p.
In this format, ts denotes the event's timestamp in nanoseconds, x and y correspond to its two-dimensional pixel coordinate, and p indicates its polarity as either positive (p=1) or negative (p=0). 

For more information about the dataset's structure please see Dataset_Structure.txt. Additionally, refer to the dataset manuscript for full details about this dataset.


Related publication(s):
El Shair, Z. and Rawashdeh, S., 2024. MEVDT: Multi-Modal Event-Based Vehicle Detection and Tracking Dataset. Data In Brief (under review).
El Shair, Z. and Rawashdeh, S.A., 2022. High-temporal-resolution object detection and tracking using images and events. Journal of Imaging, 8(8), p.210.
El Shair, Z. and Rawashdeh, S., 2023. High-temporal-resolution event-based vehicle detection and tracking. Optical Engineering, 62(3), pp.031209-031209.
El Shair, Z.A., 2024. Advancing Neuromorphic Event-Based Vision Methods for Robotic Perception Tasks (Doctoral dissertation, University of Michigan-Dearborn).

Use and Access:
The MEVDT dataset is made available under Creative Commons Attribution 4.0 International (CC BY 4.0) and is freely available for research usage.

To Cite Data:
El Shair, Z.A. (2024). MEVDT: Multi-Modal Event-Based Vehicle Detection and Tracking Dataset [Data set]. University of Michigan - Deep Blue. https://doi.org/10.7302/d5k3-9150