Work Description

Title: Data for Predicting Driver Takeover Performance in Conditional Automation (Level 3) through Physiological Sensing Open Access Deposited

h
Attribute Value
Methodology
  • This data is collected to develop a comprehensive analysis of the effects of takeover behaviors on the commonly collected physiological data. A program for conditional automation was developed based on a game engine and applied to a driving simulator. The experiment incorporated three types of secondary tasks (observing, 1-back task, and 2-back), three takeover events, and two traffic densities. A low traffic density scenario contains 40 vehicles per mile while a high traffic density scenario has 80 vehicles per mile. The three takeover events include: (1) an obstacle in front of the lane, (2) a police car on the right side, and (3) a fake alert. Brain signals, Skin Conductance Level (SCL), and Heart Rate (HR) of the participants were collected while they were performing the driving simulations. In this study, a 14-channel EEG headset, Emotiv EPOC X, was applied. It allows quick setup using saline solution. The real-time brain signals at different channels could be visualized by the Bluetooth connection on the computer. The sampling frequency of the headset was set to128Hz. The raw data collected from the EEG headset was floating-point values with a DC offset of approximately 4200 μV. In addition, it contained a lot of noise which was categorized into extrinsic and intrinsic artifacts. The extrinsic artifacts were removed by a band-pass filter with a frequency from 0.5 Hz to 65 Hz [64]. The intrinsic artifacts were caused by the eye blinking and muscle movements of the participants, which needed to be identified and removed manually. In this study, EEGLAB was applied to help with the removal of the intrinsic artifacts. The Optical Pulse Ear-Clip was used to collect the Photoplethysmography (PPG) of the participants which were further converted to HR. The Shimmer3 GSR+ Unit was used to record the GSR signals, which allowed real-time visualization of the data through the Bluetooth connection to the software named ConsensysPro. The sampling frequency of the GSR signal was 128 Hz.
Description
  • The National Highway Traffic Safety Administration (NHTSA) calls for fundamental research on “the driver performance profile over time in sustained and short-cycle automation … and driver-vehicle interface to allow safe operation and transition between automated and non-automated vehicle operation.” The emerging level 3 autonomous vehicle (AV) has the potential to transform driving because it can perform all aspects of the driving task and allow for complete disengagement of drivers (e.g., sit back and relax) under certain driving scenarios. The vehicle can handle situations that require an immediate response, such as emergency braking. However, this is not fully autonomous, and still requires the driver to be prepared for takeover at all times with a few seconds of warning. Being able to measure and predict the takeover performance (TOP) ahead of time and issue adequate warnings is thus critical to ensure driver comfort, trust, and safety in the system and ultimately acceptance of the technology by different stakeholders. This has not been explored to the extent of establishing complete and irrefutable trust in the autonomous vehicle system and its ability to engage the driver in safe and effective takeover under certain driving scenarios. Therefore, the objective of this project is to perform fundamental research to understand drivers’ capabilities of taking over the vehicle safely and promptly at any time in level 3 automation. This project advances fundamental research in human factors in level 3 AVs. This is achieved through an integrated treatment of the drivers’ TOP measured and predicted through physiological features and driving environment data in level 3 AVs. Thus, the main objective of this research will be to investigate the feasibility of using multimodal physiological features collected from drivers in level 3 AVs under different driving and disengagement scenarios (secondary tasks) to develop a personalized and real-time prediction of TOP. The project will engage a diverse group of students and faculty and develop a research program in an unexplored area of level 3 AVs, leading to substantial advances in how human physiological sensing can be used to understand the driver’s TOP, especially in a personalized manner. Such an understanding can eventually lead to the design of adaptive and personalized alerts that can be integrated in level 3 AVs.
Creator
Creator ORCID iD
Depositor
  • menassa@umich.edu
Contact information
Discipline
Funding agency
  • Department of Transportation (DOT)
ORSP grant number
  • CCAT grant number: 69A3551747105
Keyword
Date coverage
  • 2022-01-02 to 2022-03-30
Citations to related material
  • Deng, M., Gluck, A., Zhao, Y., Li, D., Menassa, C., Kamat, V. and Brinkley, J. (2022). “A Systematic Analysis of Physiological Responses as Indicators of Driver Takeover Readiness in Conditionally Automated Driving.” Accident Analysis and Prevention. Elsevier. In Review.
  • Gluck. A., Deng. M., Zhao. Y., Menassa. C., Li. D., Brinkley. J. and Kamat. V. (2022). “Exploring Driver Physiological Response During Level 3 Conditional Driving Automation.” In the proceedings of the 2022 International Conference on Human Machine Systems (ICHMS). Orlando, FL.
Resource type
Last modified
  • 10/09/2023
Published
  • 10/09/2023
Language
DOI
  • https://doi.org/10.7302/b312-3t56
License
To Cite this Work:
Deng, M., Gluck, A., Zhao, Y., Menassa, C., Kamat, V., Li, D., Brinkley, J. (2023). Data for Predicting Driver Takeover Performance in Conditional Automation (Level 3) through Physiological Sensing [Data set], University of Michigan - Deep Blue Data. https://doi.org/10.7302/b312-3t56

Relationships

This work is not a member of any user collections.

Files (Count: 3; Size: 1.39 GB)

Thumbnailthumbnail-column Title Original Upload Last Modified File Size Access Actions
TOP_L3_DriverPerformanceData.zip 2023-09-27 2023-10-09 1.38 GB Open Access
Deep_Blue_Data_Readme.txt 2023-10-07 2023-10-07 7.53 KB Open Access
Informed_Consent_HUM00206075.docx 2023-10-09 2023-10-09 976 KB Open Access

Date: 28 September, 2023

Dataset Title: Data for Predicting Driver Takeover Performance in Conditional Automation (Level 3) through Physiological Sensing

Dataset Creators: Deng, Min; Gluck, Aaron; Zhao, Yijin; Menassa, Carol; Kamat, Vineet; Li, Da; and Brinkley, Julian

Dataset Contact: Carol Menassa menassa@umich.edu

Funding: CCAT grant number: 69A3551747105 (DOT)

Key Points:
- The main objective of this research will be to investigate the feasibility of using multimodal physiological features collected from drivers in level 3 Automated Vehicles (AVs) under different driving and disengagement scenarios (secondary tasks) to develop a personalized and real-time prediction of Take Over Performance
- An experiment was designed using a driving simulator
- Data was collected from 20 participants.

Research Overview:
This study investigated the effects of takeover activities on the physiological responses of the drivers in conditional L3 automation. An experiment was designed using a driving simulator. Different takeover scenarios (i.e., two traffic densities and three takeover events) were incorporated to diversify the driving simulation. The vehicle data and the physiological responses of the participants were collected while they were performing the driving simulation (both before and during a takeover event). The experiment incorporated three types of secondary tasks (observing, 1-back task, and 2-back), three takeover events, and two traffic densities. A low traffic density scenario contains 40 vehicles per mile while a high traffic density scenario has 80 vehicles per mile. The three takeover events are shown in Figure 1 and include: (1) an obstacle in front of the lane, (2) a police car on the right side, and (3) a fake alert. Brain signals, Skin Conductance Level (SCL), and Heart Rate (HR) of the participants were collected while they were performing the driving simulations.

Methodology:
This data is collected to develop a comprehensive analysis of the effects of takeover behaviors on the commonly collected physiological data. A program for conditional automation was developed based on a game engine and applied to a driving simulator. The experiment incorporated three types of secondary tasks (observing, 1-back task, and 2-back), three takeover events, and two traffic densities. A low traffic density scenario contains 40 vehicles per mile while a high traffic density scenario has 80 vehicles per mile. The three takeover events include: (1) an obstacle in front of the lane, (2) a police car on the right side, and (3) a fake alert. Brain signals, Skin Conductance Level (SCL), and Heart Rate (HR) of the participants were collected while they were performing the driving simulations.

In this study, a 14-channel EEG headset, Emotiv EPOC X, was applied. It allows quick setup using saline solution. The real-time brain signals at different channels could be visualized by the Bluetooth connection on the computer. The sampling frequency of the headset was set to128Hz. The raw data collected from the EEG headset was floating-point values with a DC offset of approximately 4200 μV. In addition, it contained a lot of noise which was categorized into extrinsic and intrinsic artifacts. The extrinsic artifacts were removed by a band-pass filter with a frequency from 0.5 Hz to 65 Hz [64]. The intrinsic artifacts were caused by the eye blinking and muscle movements of the participants, which needed to be identified and removed manually. In this study, EEGLAB was applied to help with the removal of the intrinsic artifacts.

The Optical Pulse Ear-Clip was used to collect the Photoplethysmography (PPG) of the participants which were further converted to HR.
The Shimmer3 GSR+ Unit was used to record the GSR signals, which allowed real-time visualization of the data through the Bluetooth connection to the software named ConsensysPro. The sampling frequency of the GSR signal was 128 Hz.

The simulation of the conditional automation was developed using the Unity game engine, and the driving simulator ProSimu T5 Pro was used to incorporate the program. The simulator was equipped with three Samsung 55’’ 4K QLED HDR Monitors to display the driving scenarios. Gas and brake pedals were provided to simulate the real driving experience. The participants were allowed to adjust the seat, steering wheel, and pedals to maintain their own preferred driving postures.

Instrument and/or Software specifications: NA

Files contained here:
The folders show the physiological data collected from different subjects. Each folder contains the data collected in 18 different driving scenarios as well as the practice sessions (containing three scenarios). Each folder contains the following types of data:

- Electroencephalography (EEG) data
- Galvanic Skin Conductance (GSR) data collected from fingers
- Galvanic Skin Conductance (GSR) data collected from back
- Heart rate data

Taking Subject 15 “S15” as an example,

“Ttain_9” is the folder containing the collected physiological from three practice scenarios and the first 9 experimental scenarios, and the folder “10_18” indicates the data collected from the 10th to the 18th scenarios.

The .CSV file with the name containing “denoised” refers to the EEG while others are for GSR.

The EEG data contains 14 columns corresponding to 14 EEG electrodes. In each .CSV file collected from GSR, there are many columns, and please find the reference below for the important ones:

“Shimmer_XXXX_GSR_Skin_Conductance_CAL” – conductance of the skin

“Shimmer_XXXX_GSR_Skin_Resistance_CAL” – resistance of the skin

“Shimmer_XXXX_Temperature_BMP280_CAL” – skin temperature

“Shimmer_XXXX_Event_Marker_CAL” – markers that are important for data processing

“Shimmer_XXXX_PPGtoHR_CAL” – heart rate

(Note: “XXXX” can be either “95A4” or “9593”, if some of the columns contain “-1”, it was due to poor connection of the devices)

The folder named “Vehicle data” contains the vehicle data (e.g., speed, wheel angles, acceleration, time to collision, etc.) collected in different scenarios (18 in total for each subject), which are in .txt format.

There is also a .docx file named “Experimental Design”, which is the reference of the exact events, takeover scenarios, and traffic density that happen in each scenario.

The “Results” folder contains the results of questionnaires collected from the subjects, which indicate their experience regarding the driving simulation for each scenario.

Related publication(s):

Deng, M., Gluck, A., Zhao, Y., Li, D., Menassa, C., Kamat, V. and Brinkley, J. (2023). “A Systematic Analysis of Physiological Responses as Indicators of Driver Takeover Readiness in Conditionally Automated Driving.” Accident Analysis and Prevention. Elsevier. In Review.

Gluck. A., Deng. M., Zhao. Y., Menassa. C., Li. D., Brinkley. J. and Kamat. V. (2022). “Exploring Driver Physiological Response During Level 3 Conditional Driving Automation.” In the proceedings of the 2022 International Conference on Human Machine Systems (ICHMS). Orlando, FL. https://ieeexplore.ieee.org/abstract/document/9980597

Use and Access:
This data set is made available under license: http://creativecommons.org/licenses/by-nc/4.0/.

To Cite Data:

Deng, M., Gluck, A., Zhao, Y., Menassa, C., Kamat, V., Li, D., Brinkley, J. Data for Predicting Driver Takeover Performance in Conditional Automation (Level 3) through Physiological Sensing [Data set], University of Michigan - Deep Blue Data. https://doi.org/10.7302/b312-3t56

Download All Files (To download individual files, select them in the “Files” panel above)

Total work file size of 1.39 GB may be too large to download directly. Consider using Globus (see below).

Files are ready   Download Data from Globus
Best for data sets > 3 GB. Globus is the platform Deep Blue Data uses to make large data sets available.   More about Globus

Remediation of Harmful Language

The University of Michigan Library aims to describe its collections in a way that respects the people and communities who create, use, and are represented in them. We encourage you to contact us anonymously if you encounter harmful or problematic language in catalog records or finding aids. More information about our policies and practices is available at Remediation of Harmful Language.