DMD - Driving Monitoring Dataset

A multi-modal dataset for different

driver monitoring scenarios.

Find out more

Today's driving accidents are mainly due to human error. To make roads safer, we present: DMD, a multi-modal dataset that can contribute to the development of  Driver Monitoring Systems; this way, we can rely on technology and put a brake on these undesirable statistics.
23
Times
Is the increase in accident risk for Distracted Driving.
-WHO-
About.

Driver Monitoring Dataset

The DMD was devised to address different scenarios in which driver monitoring must be essential in the context of automated vehicles SAE L2-3

A lot of data

The DMD dataset was designed specifically to cover the lack of data for a fully operational DMS.
37
Participants
40:45
Hours of video

Lights, Camera, Action!

Variation in the material was one of our priorities:
  • 27% were women and 73% were men.
  • The average age of the participants is 30 years (all over 18 years old).
  • 10 of the participants wore glasses, some were recorded with and without wearing them.
  • The recordings present variation in lightning angle conditions according to the time the session was recorded (morning or afternoon)
  • Some activities were recorded with the car moving, the car stopped and in a second simulation environment.

Material X-Rays

These are a few facts about our dataset:
  • There is about 25TB of raw data!
  • All recordings were grouped by sessions, each one determined by a protocol of activities, a participant, an environment and a lighting condition.
  • In each session there were 3 cameras recording simultaneously: Body, Face and Hands camera.
  • Each camera captures 3 channels of information: RBG, Infrared and Depth.
  • All material comes with VCD annotations of different modalities.
Features.

Activities Included

The dataset consists of videos of drivers performing actions related to different driving scenarios in which it is intended to add monitoring systems, so driver state can be identified and later be able to estimate its risk on the road
Distraction-Related:
  • Talking on the phone (L/R)
  • Texting (L/R)
  • Drinking
  • Talking to passenger
  • Reaching behind/side
  • Operating the radio
  • Hair and MakeUp
Gaze-Related:
Stare at a defined region inside the car
  • Front & left front
  • Left/right window
  • Rear-view mirrow
  • Left/rigth side-view mirror
  • Radio zone
  • Lap zone
Wheel-Related:
  • Hands on the wheel (quiet/moving)
  • Hands off the wheel
  • Give control to the vehicle
Fatigue-Related:
  • Yawning with/out hand
  • Microsleep
  • Sleepy driving
Features.

Annotations

DMD Annotations are in VCD (Video Content Description). It supports the description of scenes with spatio-temporal object annotations, and semantics with actions, events and relations between them.
Here there are a few of those annotations available in the DMD.
Get to know VCD!
Spatial annotations:
  • Body landmarks
  • Face landmarks
  • Eyes landmarks
  • Objects Location
  • Hands Location
Temporal annotations:
  • Driver Actions
  • Camera Occlusions
  • Talking
  • Hands using wheel
  • Objects in scene
Features.

Channels & Scenarios

We recognize the need to include an infrared channel to the dataset for circumstances in which the lighting is not in favor. Also, our data set also contains depth information of the scene, opening possibilities towards a new analysis approach.We bring to you the data you needed in 3 different forms!
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Setup.

Recordings Set-Ups

For the recordings, 3 of Intel® RealSense™ Depth Camera D400-Series were installed in the car and simulator, and placed correctly to capture images of the driver’s face, body and hands as shown in the figure.

1 Microphone

A microphone was included in the recordings, so audio file can be found in some sessions too.

3 Cameras

For the face and the hands, 2x D415 cameras were used respectively, and 1x D435 for the body since it has a wider field of view. All cameras captured the 3 channels in high resolution and with an established fps without sacrificing the exposure level.

2 Scenarios

The recordings took place in Vicomtech's car. The activities that are considered as illegal/dangerous to perform while real driving, were made with the car stopped. Also a driving simulation set-up was prepared recreating as much as possible the conditions of the car; this way, the participant could perform these activities having normal reactions and behaviours as he/she was indeed driving.
Applications.

Potential Uses of DMD

The DMD Annotations allow using the material for different research purposes and fields.

Body Pose Estimation

This dataset can be used by algorithms for body pose estimation. This is possible with the annotations of body landmarks available. Also, depth information analysis could bring interesting results on this matter.

Action Recognition

With all images/videos, you can easily train an algorithm that classifies the action the driver is performing. Timestamps of time intervals of each action are included.

Hands-on-wheel Detection

You can create your own hands-on-wheel detector with our dataset. Body camera and hands camera material can be useful for this task. Using the timestamps of time intervals of the different hands position classification you can train an algorithm, also you could use de body landmarks to have a different approach.

Gaze Direction Estimation

With the timestamps of time intervarls in which the driver is looking at a defined region, you could be able to develop a gaze direction estimator. Also from the face camera, the face landmarks might be an important input to your algorithm.
Tools.

TaTo and DEx

Our team had developed a couple of open-source tools dedicated to DMD annotation (TaTo) and exploration (DEx).
With TaTo, you can edit, adapt o create new temporal annotations of the DMD. And, if you want to access the annotations and prepare material for training ML algorithms, DEx will do it for you.
Find out more!
Reference.

Cite us!

We have presented DMD on this paper.
It would be awesome if you reference this dataset on your work.
The DMD Team thank you.
APA:


Ortega, J., Kose, N., Cañas, P., Chao, M.a., Unnervik, A., Nieto, M., Otaegui, O., & Salgado, L. (2020). DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops (Accepted).


BibTex:

@inproceedings{Ortega2020,
author = {Ortega, Juan Diego and Kose, Neslihan and Cañas, Paola and Chao, Min-an and Unnervik, Alexander and Nieto, Marcos and Otaegui, Oihana and Salgado, Luis},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops (Accepted)},
title = {{DMD : A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis}},
year = {2020}
}

Download.

I Need the DMD!

If your research needs this juicy dataset, feel free to get in touch with us to download it or to get more info.
The dataset is still being processed... Please, come back in a couple of weeks and hopefully a lite version of the dataset will be available for you to download!
Thanks for understanding :)

You must agree to the following terms:

Dataset use

You must be 18 or older to download. This dataset can only be used for academic purposes. Any other use beyond that is prohibited. To ensure the continuous development of this project, we demand a responsible use of the data you are about to acquire.

This dataset is published under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Privacy policy

We care about your data and privacy, that is why your information requested on this form will be treated in a way that complies with the GDPR policy. Your data will only be used to have a registry of the dataset users and eventually communicate updates with you.

You may request access to and correction of, amendment to, or deletion of any information provided to us at any time by contacting us at info-dmd@vicomtech.org.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.

Thank you for Downloading!
You should get a Box notification in your email, in which you are invited to collaborate with a folder that contains the dataset.
Please, check spam in case you can't see our emails on your inbox.
Oops! Something went wrong while submitting the form. Try again, please!