The Influence of SP Design on the Incentive to Bias in Responses
Hui Lu, Anthony Fowkes. Mark Wardman, Institute for Transport Studies, University of Leeds, UK
This paper investigates some sources of bias in Stated Preference experiments and the influence of SP designs on biases in responses, illustrated with data from rail commuters in North West England.
Stated Preference (SP) methods have been regarded as a promising approach for better understanding of individual behaviour. In recent years, SP methods have seen widespread application in the transport field (see Louviere, 2000) such as route choice, value of time analysis, evaluation of traffic noise etc. From SP surveys, transport planners and policy makers hope to discover the probable effects of a change to the transport system and how the users might respond to this change.
Some voices of caution regarding SP methods have been raised, for example, Arentze et al.(2003) stated that 'their gradual acceptance in the transportation research community has not taken place without criticism'. It has been known that biases are found in the SP experiments due to the 'irrational' individuals and complex tasks. By knowing the direction and magnitude, such systematic biases can be reduced and compensated for. Many studies have been conducted on the effects and reduction of task complexity (Swait and Adamowicz, 2001; DeShazo, 2002). However, few studies have been carried out on incentives for individual to bias their answers.
This paper reviews and explores incentives for respondents to bias their answers, such as strategic bias, affirmation bias, and status quo bias etc. Through investigating the effects of biases, the paper aims to find the influence of different design on responses, and try to find methods to reduce biases caused by the design and practice of SP experiments. In an empirical demonstration using data obtained from 1221 commuters on the valuation of new rolling stock in Greater Manchester, UK, this paper presents results for different designs. The aim is to help SP designer and practitioner to reduce biases, and improve the validity and reliability of SP results.
In previous studies, SP was found to be the most reliable method to investigate the service quality improvements (MVA, 1993). Some reviews on the SP application to rolling stock studies found the monetary values for time and frequency derived from SP methods have been remarkably consistent among studies. However, the valuation of soft variables such as service quality (rolling stock, punctuality, crowding and so on) is rather less convincing. This can be explained as the bias by which respondents overestimate the value of those characteristics to improve their well-being (Wardman, 1998) or by packaging effects. As a result, SP models have been observed to produce demand forecasts or values which are too high, and bias has been found (Wardman and Whelan, 2001; ATOC, 2002; Wardman and Bristow, 2003).
After a review of studies on rolling stock in recent years, an SP experiment has been designed to investigate the effects of different designs on responses. Two factors were introduced into the experiment, 'Cheap Talk' and 'Masking the Aim of Experiment', to amend the incentive to bias. Cheap Talk is a term used in the Contingent Valuation Methods where a proper designed Cheap Talk has been shown to reduce the strategic bias. Simply put, Cheap Talk (CT) is a warning message before the survey that tells respondents what caused biases in previous experiments. Wardman and Bristow (2003) found that adding more attributes to mask the aim of research could be shown to amend incentives for respondents to bias their answers, although this is also impact on task complexity.
A pilot survey was conducted in August 2005 followed by the main survey from November to December 2005 in Greater Manchester. Four groups of questionnaires which include the above two factors separately and together. A total of 2768 paper-based questionnaires were sent out. 1321 commuters mailed back, of which there were 1221 usable questionnaires. For these 1221 commuters, both 'Cheap Talk'and 'Masking the Aim'are seen to have some significant effects on the SP responses and the results estimated to them. The effects are varied via different circumstances such as different area or different social economic characteristics. Analysis is on going.
Association for European Transport