Using Expert Knowledge and Rules for Driver Monitoring: an Alternative Approach
E Bekiaris, S Nikolaou, Hellenic Institute of Transport, GR; W Janssen, R Brouwer, TNO-TM, NL
Driver hypovigilance (?falling asleep at the wheel?) is a major cause of road accidents, accounting for up to 20% of serious accidents on motorways and monotonous roads in Great Britain, whereas 56,000 crashes are stated by the US police annually, having as primary cause the driver hypovigilance, according to NHTSA studies. The European Commission Road Safety strategy identifies driver hypovigilance as an important factor in road fatalities and supports the application of driver monitoring systems to achieve its goal to reduce road fatalities by 50% by the year 2010.
AWAKE is a European project, co-funded by the European Commission under the IST initiative of the 5th Framework programme. The objective of AWAKE is to increase traffic safety by reducing the number and the consequences of traffic accidents caused by driver hypovigilance. In order to achieve this objective, AWAKE intends to develop an unobtrusive, reliable system, which will monitor the driver and the environment and will detect in real time hypovigilance, based on multiple parameters.
In order to enhance the reliability of the detection system, AWAKE uses both a stochastic Hypovigilance Diagnosis Module (HDM) and a deterministic filter (DHDM).
More specifically, in parallel to the stochastic approach followed by HDM development, and based on results of previous studies and projects, both absolute and relative criteria will be defined to be fed into an expert knowledge system, in an attempt to filter the sensor data input to HDM, as well as to develop a deterministic type of HDM (DHDM), as a fall-back position. This minimal HDM will be compared also to the overall stochastic HDM in terms of efficiency, computation time, etc.
This paper deals with the design of the deterministic hypovigilance diagnosis module, the expert tools and rules to be used within it, as well as on the use cases definition.
System Design The Deterministic Hypovigilance Detection Module (DHDM) has the following specifications:
* A simple (deterministic) system (DHDM) that predicts the vigilance state based on a given set of sensor measurements, such that driver?s hypovigilance can be diagnosed.
* Potentially, the DHDM must be robust (adaptive) with respect to inter-subject variability (adjusted to specific driver), intra-subject variability (adjusted to specific driver state) and specific traffic conditions (hypovigilance must be predicted for all specified traffic conditions).
* The DHDM must have a diagnostic performance level comparable to the (stochastic) HDM. The trade-off between system simplicity and diagnostic performance is a key issue to be investigated.
System components DHDM consists of the following components:
* Sensor data (measurements);
* Derived measures;
* Integration of measures (data fusion);
* Hypovigilance state criteria;
* Diagnosis of hypovigilance.
h4. Sensor data
The selected sensors for the DHDM are: eyelid closure, hand pressure on steering wheel, steering wheel movement, lateral vehicle position, front obstacle distance, TTC (Time To Collision) and TLC (Time to Line Crossing) and emerge also widely in literature.
Other measures in the category of psychophysiological measures are: EEG, heart rate, GSR, EMG, etc., and in the category of task performance measures, such as rate of turn and driving speed.
Derived measures The foregoing raw sensor data (whether or not filtered, etc.) have to be processed in order to be useful as indicators of the corresponding notion. For example, the measurement of eyelid closure can be related to loss (or a reduced state) of alertness if the derived measure indicates that the eyes are closed for some time. The most well known derived measure in literature is the PERCLOS measure, which is defined as (typically) at least 80 % of the time that the eyelid is closed. This reflects a sustained (slow) eyelid closure rather than a blink. Another example is the steering wheel movement. In order to reflect an increase in driver drowsiness, the derived measure should reflect a deviation of ?normal? driving, e.g. in terms of a minimum or a maximum amount of control activity, e.g. in terms of the standard deviation of the steering wheel angle. Finally, the lateral vehicle position can be related to degraded driving performance due to hypovigilance, in terms of a given lane exceeding, or in terms of a given standard deviation of the lateral position, etc.
h4. Integration of measures
There is a general agreement that more than one of the foregoing measures should be used to obtain a reliable detection of hypovigilance. Typically, one or more driver behavioural measures (e.g. slow eyelid closure, handgrip pressure and EEG) are combined with task performance measures (e.g. steering wheel movement, lateral acceleration and lateral position). This integration of the various measures can be based on a variety of algorithms. In Wierwille et al. 120 different algorithms were tested. The best prediction results to be implemented in algorithms were in fact obtained by using 4 to 7 different variables together.
Another simple way to combine various measures is to consider these different measures as follows: each measure is tested against some criterion value (to be discussed in the following) and the decision that hypovigilance has been occurred is made if all the separate measures have exceeded the corresponding criterion value. This approach is indicated within this study as an ?engineering? approach.
More practical is the use of a Neural Network to combine the various measures as inputs to a network, resulting in a prediction of hypovigilance. This approach will is also analysed in this study and is compared with the ?engineering? one.
h4. Hypovigilance state criteria
Given the information of the separate or combined measures, it is the question how this information can be used to decide about hypovigilance. For example, the standard deviation of the lateral position exceeding a certain value possibly reflects an incipient hypovigilance. However this value is varying per subject, but also depends on the specific traffic situation. Driver behavioural measures (e.g. EEG) also vary in time and therefore also the criterion value cannot be time-invariant.
The goal of the present phase of this study is to design a deterministic hypovigilance detection system, which means that the effect of the traffic situation will not be considered.
h4. Diagnosis of hypovigilance
Based on the (separate or combined) measures and the corresponding criterion value(s) the decision has to be made that the hypovigilance state has been occurred. The general issue of any decision is the decision performance in terms of the percentage of correct decisions (recognition rate) and the decision errors. The two types of errors are the miss and the false alarm. Typically, decision-making performance is specified in terms of the recognition rate and the false alarm rate.
Because no definitive (hypo)vigilance measure and specific sensor measurements were available, the design had to be tentatively and exploratory, i.e. based on a hypothetical vigilance ?data? and considering possible, alternative sensory inputs. Nevertheless, much insight could be obtained in the effect of a variety of system aspects (sensory inputs, signal processing characteristics, decision making aspects, etc.) on the hypovigilance prediction performance. In addition, a system design procedure could be developed that can be used for the definitive design process, based on the definitive measure of (hypo)vigilance and the selected sensory inputs, using the models and computer programs that are developed within this study.
AWAKE (IST-2000-28062) Project, ?Description of Work?, August 2001.
Clark, J. and Yuille, A.L. (1990). Data fusion for sensory information processing systems. Kluwer Acad. Publishers.
Dillies-Peltier, M. A. (1997). Driver vigilance decrease detection: A real-time, driver adaptive on-board system. Proc. of the 4th World Congress on Intelligent Transport Systems, Berlin, Germany.
Hall, D. (1992). Mathematical techniques in multisensor data fusion. Boston Artech House.
ITS ?E-safety? Lyon congress proceedings, ERTICO, September 2002. Kircher, A., Uddman, M. & Sandin, J. (2002). Vehicle control and drowsiness (VTI-Report).
Martindale, C. (1992). Cognitive Psychology, A Neural-Network Approach, Brooks/Cole Publishing Company.
NCSDR/NHTSA Expert Panel on driver fatigue and sleepiness, ?Drowsy Driving and Automobile Crashes?, NCSDR/NHTSA report HS 808 707, 1998.
Onken, R. & Feraric, J.P. (1997). Adaptation to the driver as part of a driver monitoring and warning system. Accid. Anal. And Prev., vol. 29, no.4, pp. 507-513. Pergamon.
Russell, S.J. & Norvig, P. (1995). Artificial Intelligence ? A Modern Approach. Prentice-Hall Inc., USA.
Sánchez-Sinencio, E. & Lau, C. (1992). Artificial Neural Networks: Paradigms, Applications, and Hardware Implementations. IEEE Press.
Simpson, P.K. (1990). Artificial Neural Systems: foundations, paradigms, applications and implementations. Pergamon Press, New York.
The Royal Society for the prevention of accidents, ?Driver fatigue and road accident: A literature review and position paper?, February 2001.
Wewerinke P.H., Janssen W.H., Brouwer R.F.T., ?DESIGN OF A SIMPLE HYPOVIGILANCE DIAGNOSTIC SYSTEM?, AWAKE project report, 2002.
Wewerinke, P.H., Hogema, J.H. & Verschuren, R.M.A.F. (2001). Modelling lateral vehicle control to describe handling. TNO report, TM-01-D018.
Wierwille, W.W. Ellsworth, L.A., Wreggit, S.S., Fairbanks, R.J. & Kim, C.L. (1994). Research on vehicle-based driver status/performance monitoring: Development, validation and refinement of algorithms for detection of driver drowsiness. NHTSA, Final report: DOT HS 808 247.
Association for European Transport