Evaluation of IVIS/ADAS Using Driving Simulators. Comparing Performance Measures in Different Environments
T Engen, L-E Lervåg, T Moen, SINTEF Transport Research, SINTEF Technology and Society, NO
Evaluation of traffic safety measures has traditionally been conducted in terms of observational studies as before and after studies or comparative studies. Experimental studies have traditionally been seen as somewhat difficult to conduct, however they are now becoming more readily available.
In this paper we demonstrate how to measure change in performance due to the use of ADAS/IVIS and unveil how we can represent real life through experiments. Results from experiments are compared to observational studies as well as evaluation and comparison of experiments conducted in different environments.
MEASURING THE INFLUENCE OF IVIS/ADAS
There are several different methods to measure the influence of IVIS/ADAS. Theories both of workload influence and of measuring primary and secondary tasks have been developed. All of these theories rely on the possibility to measure different variables. We have conducted studies were we can compare the results of Lateral control, Longitudinal control and Interaction with other vehicles.
THEORITCAL PROPERTIES OF DRIVING SIMULATORS, TEST TRACKS AND REAL TRAFFIC EVALUATION
When doing evaluation one must be aware that the validity of the results can not be absolute, but will always have some uncertainty to them. Through a literature review we have found research projects that involve validation of driving simulators:
? Direct comparison with real life data
? A comparison of the driving simulator with physiological tests and a questionnaire
? Expert testing
? Validation compared to specific driver characteristics
? Stability over time and driver characteristics
? Driving training
The different methods and environments have their specific drawbacks:
? Driving simulator ? lacks the possibility to produce real danger feeling, Realism might be missing, due to low resolution of devices (Screens, audio?)
? Test track ? Lack danger of interaction with other vehicles.
? Real traffic ? Extreme and dangerous situation can not be tested
Each IVIS/ADAS system has its own characteristics and thus need specific characteristics of the experiments and measuring variable.
COMPARISON OF RESULT BETWEEN DRIVING SIMUALTORS AND REAL TRAFFIC
REACTION TIME IN DRIVING SIMULATORS AND REAL TRAFFIC
The reaction time studies conducted in the driving simulator were compared to real life measurements, previous research, and measurements of reaction time in a video-based simulator. The reaction time found in the driving simulator varied a great deal in different situations, this was, however, reasonable and comparable to the results from all the other measurement methods.
SPEED AND LATERAL POSITION IN DRIVING SIMULATOR AND REAL TRAFFIC
Speed and lateral position measurements of related conditions were conducted through observational studies in real traffic and experiments in the driving simulator. The results were similar in both situations.
TIME GAP IN DRIVING SIMULATOR AND REAL TRAFFIC
Measurements of time gap were done both in the driving simulator and the instrumented vehicle. This case study was meant specially for testing the method and not for finding the precise time gap.
PERFORMANCE WHEN INFLUENCED BY ALCOHOL IN DRIVING SIMULATOR AND ON TEST TRACK
Comparing driving performance when influenced by alcohol in driving simulator and on test track revealed that values of most traffic behaviour variables in simulator do not differ considerably from corresponding values on test track. The variables tested where: response time, speed, steering wheel reversals, steering wheel movement speed, number of cones knocked down during serpentine driving, stopping, distance to tracking line and self reported experience.
DETERMINING DISTANCE TO OBJECT IN THE DRIVING SIMULATOR
Compared to the real world the ability for the driver to calculate the distance of an object in a simulator is quite different. This is because the simulator image is 2D compared to the real 3D world. During a serpentine exercise conducted both on a test track and a corresponding simulated test track, the subject knocked down or touched more cones in the simulator than compared to the real world test track. It was also easier to assess the distance to a stop line on the test track than in the simulator.
Experimental studies are important tools for evaluation of IVIS/ADAS, but at the same time it demands special expertise to create good experimental design. For example an important strength of driving simulator experiments lies in controlling the confounding variables, but at the same time this creates results with less variance than experiments in real traffic.
Association for European Transport