Dealing with Repeated Choices in Stated Preference Data: an Empirical Analysis

Dealing with Repeated Choices in Stated Preference Data: an Empirical Analysis


C Choudhury, RAND Europe, UK and Massachusetts Institute of Technology, US; A Sivakumar, RAND Europe and Imperial College London, UK; C Rohr, P Burge, RAND Europe, UK; A Daly, RAND Europe and ITS, University of Leeds, US


We use real-world transport data to evaluate different approaches used to account for the correlation among repeated choices. The practical implications of using jack-knife, bootstrap and restricted random-coefficients are compared in this regard.


Dealing with Repeated Choices in Stated Preference Data:
An Empirical Analysis

An important advantage of stated preference (SP) discrete choice experiments is that several responses can be collected from each individual. This reduces substantially the cost of data collection and allows for more advanced experimental designs. However, the collection of multiple responses means that each respondent?s basic preferences apply to the series of responses that he or she has given: those responses are therefore interdependent (serially correlated). Naïve analysis methods that assume the independence of observations are therefore, in principle, invalid.

A number of methods can be used to correct for the interdependence of SP observations, including methods that capture these effects through sub-sampling procedures such as the jack-knife (Bissell and Miller 1974, Ferguson 1975) or the boot-strap (Efron 1979), and models with specifications that explicitly estimate terms that capture the serial correlation among observations. Another possible approach can be to use an imploded sample where all responses from an individual are regarded as a single observation with multiple choices and each set of possible choice combinations is regarded as an alternative in the estimation. Each approach has benefits and draw-backs, both theoretical and practical (Ouwersloot and Rietveld 1996, Cirillo et al. 1998, Ortuzar et al. 2000). However, to date little research has been published comparing the results from these different procedures. This paper therefore presents an empirical analysis to evaluate the performance of each of the methods, using a real-world transport application to compare and contrast the benefits and disbenefits of each approach.

Comparisons are made between simple models with generic coefficients only and models with explicit consideration of observed taste variation. The effects of serial correlations are then accounted for separately through each of the competing methods: implosion, jack-knifing, bootstrapping and incorporating individual-specific error terms in the model structures (restricted random coefficient models). The latter tests have been undertaken using several commercially available software packages, including Alogit ( and Biogeme (Bierlaire 2003).

For model evaluation the estimated coefficients of the jack-knifed, bootstrapped and restricted random coefficient models are compared with the coefficients of the base (naïve) models. Ease of use and estimation time are also discussed.


Bierlaire, M. (2003), BIOGEME: a free package for the estimation of discrete
choice models, Proceedings of the 3rd Swiss Transport Research Conference,
Monte Verità, Ascona.

Bissell, A. and Ferguson, R. (1975), The Jackknife: Toy, Tool or Two-Edged Weapon?, Statistician. V. 24. pp79-100

Cirillo, C., Daly, A. and Lindveld, K. (1998), Eliminating Bias due to the Repeated Measurements Problem in SP , in J. de D. Ort´uzar (ed.), Stated Preference Modelling Techniques: PTRC Perspectives 4, PTRC Education and Research Services Ltd, London.

Efron, B. (1979), Bootstrap Methods: Another Look at the Jackknife.. The Annals of Statistics,V. 7., No. 1., pp1-26

Miller, R. (1974), The Jackknife: A Review?. Biometrica, V. 61. pp. 1-14.

Ortúzar, J. de D., D. A. Roncagliolo and U. C. Velarde (2000) Interactions and independence in stated preference modelling, in J. de D. Ort´uzar (ed.), Stated Preference Modelling Techniques: PTRC Perspectives 4, PTRC Education and Research Services Ltd, London.

Ouwersloot, H. and P. Rietveld (1996) Stated choice experiments with repeated observations, Journal of Transportation Economics and Policy, 30, 203?212.


Association for European Transport