Driver and Occupant Monitoring System
Monitoring the Driver
HARMAN’s advanced driver monitoring solution uses camera sensors to capture the driver’s most important first-order biometric features, such as gaze, head position and pupil diameter, among many other key facial features. It detects minute fluctuations in pupil diameter and calculates brain activity level, especially high cognitive load.
Workload & Emotion
Besides supporting NCAP driver monitoring safety requirements, the breakthrough for Driver Monitoring Systems (DMS) was to add emotion to cognitive workload, resulting in a combined system that is better than either of the parts in terms of performance and value criteria. It measures facial and voice cues to calculate emotional status, in particular high emotional load (in addition to fatigue, drowsiness, gaze, head orientation, etc.) and combines this input to generate highly reliable signals relating to engagement and attention.
- Senses brain activity, emotion, and facial features, and combines this to provide highly reliable signals relating to engagement and attention
- Measures minute fluctuations in pupil diameter and calculate cognitive load
- Measures facial and voice cues to calculate emotional status, in particular, high emotional load
- For AV hand-off events, dynamic ADAS parameters (increase warning times), adaptive/personalized HMI, etc
Advanced DSP and AI Usage
The HARMAN system can also analyze the auditory content and heart/breathing rate of the occupant. It then uses proprietary and patented algorithms that combine the most cutting-edge DSP and AI to provide insights into second-order biometric signals such as emotional activity and cognitive load. The system is currently under further development to provide truly novel third-order signals, such as driver engagement and attention, critical to paving a successful pathway for the Level 3 and autonomous handoff of the future. The HARMAN system is extended by the DMS/OMS documentation to also support additional in-cabin monitoring features, including detection of driver identity, gender, age or number of child seats.
Solve the Driver Understanding Gap
AI means highly accurate, fast and personalized results. It takes advantage of modern compute platforms coming to today’s vehicles. The goal is to ensure driver engagement at all times. DMS responses are emotion-based playlists, customized alerts or cognition-based lighting. For future scenarios, the system facilitates the difficult hand-over process for autonomous driving at Level 3 and above.