Work Description

Date: 3 July, 2024

Dataset Title: SID: Stereo Image Dataset for Autonomous Driving in Adverse Conditions

Dataset Creators: Z.A. El-Shair, A. Abu-raddaha, A. Cofield, H. Alawneh, M. Aladem, Y. Hamzeh, & S.A. Rawashdeh

Dataset Contact: Zaid A. El-Shair zelshair@umich.edu

Funding: This project was supported by a research initiation and development grant from the office of research and sponsored programs at the University of Michigan-Dearborn.

Key Points:
- Large-scale stereo image dataset tailored for autonomous driving perception algorithms.
- Captures a wide range of adverse weather and complex lighting conditions.
- Offers 27 sequences totaling over 178k stereo image pairs, each accompanied by comprehensive sequence-level that include weather conditions, time of day, road conditions, camera soiling, and lens obstructions.

Research Overview:
SID was created to facilitate the research and development of robust perception algorithms for autonomous driving under challenging conditions. The ZED stereo camera mounted on a vehicle captured diverse real-world scenarios, including snow, rain, fog, and nighttime driving across different urban and residential locations. Proper annotations provide metadata, such as weather conditions, time of day, and road conditions, to accompany the high-resolution stereo images. This dataset addresses the need for advanced models to operate consistently and safely across varied geographic and environmental settings.

Methodology:
Data were collected using a ZED stereo camera at a 20 Hz frame rate with 720p resolution. Various routing environments, such as university campuses, residential areas, and city streets, were included in the collection. The camera system produced stereo pairs with factory calibration for depth estimation purposes. Images were initially stored as BMP files before being converted to PNG format for improved storage efficiency. Collected during differing weather conditions and times of day, the dataset includes sequence-level annotations to augment model training and testing effectiveness.

Files contained here:
- The dataset is organized into 5 directories: Clear/, Cloudy/, Overcast/, Rain/, Snow/. These correspond to unique weather conditions under which the data were captured.

- Within these directories, subdirectories reflect individual sequences following the naming convention:
Location_Weather_TimeOfDay_SequenceNo/

- Each sequence folder includes two subfolders for the respective camera images:
- Left/: Contains images captured by the left camera.
- Right/: Contains images captured by the right camera.

Image Naming Format:
- Each image in the dataset has a unique filename that allows users to easily differentiate between left and right camera images and their associated sequences, formatted as follows:
SequenceNo_Camera{L/R}_FrameNumber.png

Data Categorization:
- Weather Conditions: Clear, Cloudy, Overcast, Rain, Snow.
- Times of Day: Day, Night, Dusk.
- Road Conditions: Dry, Wet, Snowy, and others.
- Locations: University campus drives (clockwise and counter-clockwise), residential areas, and city settings.

Training/Testing Categorization:
- Sequences are designated as 'Train' or 'Test' to facilitate standardized machine learning development and evaluation. The dataset is divided as follows:
- Train (20 sequences): 1, 3, 4, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17, 18, 19, 21, 23, 25, 26, 27
- Test (7 sequences): 2, 5, 11, 12, 20, 22, 24

Note: Detailed statistical information for each sequence of the Stereo Image Dataset (SID), such as weather conditions, time of day, sequence location, road conditions, number of image pairs, and lens soiling annotations, is compiled in the "SID_Sequence_Metadata.csv" metadata file. Additionally, the file includes the categorization of sequences into 'Training' and 'Testing' sets, aiding in the dataset's practical application for developing and benchmarking perception algorithms. For comprehensive context and an in-depth understanding of the dataset, please refer to the related publication manuscript outlined below.

Related publication(s):
- El-Shair, Z.A., Abu-raddaha, A., Cofield, A., Alawneh, H., Aladem, M., Hamzeh, Y., and Rawashdeh, S.A., 2024. SID: Stereo Image Dataset for Autonomous Driving in Adverse Conditions. In 2024 IEEE National Aerospace and Electronics Conference (NAECON). IEEE. In press.

Use and Access:
This dataset is made available under Creative Commons Attribution 4.0 International (CC BY 4.0) and is freely available for research usage.

To Cite Data:
- El-Shair, Z.A., Abu-raddaha, A., Cofield, A., Alawneh, H., Aladem, M., Hamzeh, Y., and Rawashdeh, S.A. (2024). SID: Stereo Image Dataset for Autonomous Driving in Adverse Conditions [Data set]. University of Michigan - Deep Blue.
- El-Shair, Z.A., Abu-raddaha, A., Cofield, A., Alawneh, H., Aladem, M., Hamzeh, Y., and Rawashdeh, S.A., 2024. SID: Stereo Image Dataset for Autonomous Driving in Adverse Conditions. In 2024 IEEE National Aerospace and Electronics Conference (NAECON). IEEE. In press.