Assessment of mental fatigue using multi-modal feature fusion

Fatigue can result in serious short- and long-term health/safety issues. In the short term, fatigue prevents people from functioning safely and within normal boundaries which results in injuries and accidents. In the longer-term, fatigue may also result in body stress, diabetes, gastrointestinal disorders, mental health issues, cardiovascular disease and work-related musculoskeletal disorders which account for more than 70% of workers’ compensation claims. The emerging field of objective fatigue monitoring aims to use physiological sensing and computer vision to monitor fatigue continuously and address the issue of bias associated with the traditional fatigue self-reporting processes. Despite significant innovations in objective fatigue monitoring, there are significant challenges for implementing the technology in the real work environment that need to be addressed. This project aims to propose a multi-modal feature fusion approach with the capability of measuring mental fatigue for both workers and operators. The proposed method will combine facial parameters, physiological, and contextual information (e.g., age, experience, light) to assess the level of mental fatigue more accurately and has the potential to be used in different sectors such as transport and construction in real-world environments. As part of the project, a situational awareness system will also be developed to give real-time feedback on the level of fatigue to make farsighted and informed decisions in ongoing operations. The outcomes of this project will provide an efficient tool for real-time monitoring of mental fatigue in different sectors such as construction, transport, and mining and will aid in minimizing the injuries and accidents arising from fatigue.