A multi-modal approach for detecting drivers' distraction using bio-signal and vision sensor fusion in driver monitoring systems

초록

According to a report by the World Health Organization (WHO), approximately 1.3 million people lose their lives annually owing to traffic accidents. The majority of road traffic accidents stem from driver negligence. Recently, there has been a growing interest in utilizing deep learning and machine learning technologies to enhance the safety and efficiency of road traffic, with the aim of addressing issues arising from driver inattentiveness. Most studies focus on detecting abnormal driver behavior using driving sensors or driver images; however, they often overlook physiological factors such as the driver's bio-signals. Considering that the driver's state, including fatigue, stress, and concentration, can significantly affect driving safety, it is crucial to build models that consider biometric information. Therefore, this study proposes a multi-modal transformer model called Bio-Vision Transformer (BiViT) that comprehensively considers both driver bio-signals and images. The BiViT model uses a vision transformer to extract features from driver images and employs a time-series transformer to capture features from the driver's bio-signals. In addition, the interactions between the extracted features are modeled, and the joint fusion method is employed as the feature-fusion approach. To validate the proposed model, performance comparisons and analyses were conducted using commonly used models in image analysis. The experimental results demonstrated that the proposed BiViT model exhibited high performance, with an accuracy of 0.91 and a harmonic mean of precision and recall (F1-score) of 0.91, surpassing the performance of the comparison models.

키워드