Examination of the Respondent Reported Attribute Processing Strategies in Stated Choice Experiments Constructed from Revealed Preference Data
John Rose, David Hensher, Stephane Hess, ITLS - University of Sydney, AU
This paper shows how the risk of biased results is significantly reduced by taking into account respondents' attribute processing strategies in stated preference studies
The area of discrete choice modelling has seen a hype of activity over recent years, with the development of ever more flexible model structures that allow for a representation of complex error structures, in terms of substitution patterns between alternatives as well as random taste heterogeneity across individuals.
However, while these new model structures represent mathematical advances, they do not per se lead to a deeper understanding of the actual choice processes undertaken by decision makers. In fact, it can be argued that by allowing for a more flexible error structure, we simply make up for our relative lack of knowledge as to the true nature of choice behaviour represented in the data at hand. This is particularly true in the context of how and if individuals respond to changes in individual attributes. As such, not only do the sensitivities to such changes potentially vary across respondents, but it is entirely reasonable to expect that some individuals do not respond at all to certain attributes, or only evaluate certain attributes in conjunction with others. Additionally, some individuals may ignore certain attributes as long as their value remains below a certain threshold. Finally, there may be individuals who ignore certain attributes in situations where some other, higher-priority attribute attains or exceeds a certain threshold value.
While variations in sensitivities are now routinely dealt with through the use of mixture models, the possibility of some individuals not responding to certain attributes (under any or certain conditions) is not generally dealt with in existing studies. This paper aims to give an in depth discussion of the potential effects of ignoring respondents? attribute processing strategies, using a number of case studies to support the theoretical claims.
The effects of ignoring respondents? attribute processing strategies in model estimation are potentially very significant. The most basic scenario is one in which the population is split into two groups, of which one part is indifferent to changes in a certain attribute (either always or below a certain threshold value), where there is potentially additional variation in the sensitivity in the other (non-zero sensitivity) group. Depending on the weight of the zero-sensitivity group, the population-level estimates will be affected significantly by the presence of this group, in terms of parameter estimates as well as standard errors. While in the most basic models, this will lead to problems in the fixed-point estimates, additional issues arise in the case of mixture models, where the estimated range will be affected in addition to the mean value. As such, the presence of a large number of respondents with a zero-sensitivity to a certain attribute could lead to a significant probability of a positive coefficient even in the case where the coefficient is negative for the remainder of the population. This is especially likely when additionally making use of inflexible distributions such as the Normal. Similar issues clearly arise in the case where some individuals treat attributes A and B separately while the remainder of the population reacts only to changes in the combined value of A and B.
The applied part of this paper presents a number of case studies making use of stated preference (SP) data collected by the Institute of Transport and Logistics Studies at the University of Sydney. In this dataset, car drivers were given a choice between two routes, described by five attributes, namely free flow and slowed down travel time, travel time variability, running costs and toll costs. In addition to making a choice between the different alternatives (variably including and excluding the reference trip), respondents were asked to state if they had systematically excluded any attributes in the evaluation of the alternatives, and were also asked to rank the five attributes in order of importance. Finally, respondents were asked if they had considered the travel time attributes separately from each other or if they had added them up.
In the empirical experiments, a number of different approaches were taken to account for the attribute processing strategies of respondents. In the most basic approach, a flag was used in the utility specification such that, for respondents who indicated that they had systemically ignored a certain attribute, this attribute was excluded from the utility specification. The next approach extends on this by also allowing for the fact that some respondents treat free flow and slowed down travel time jointly, while others treat them separately. With both approaches, Multinomial (MNL) as well as Mixed Multinomial Logit (MMNL) models were estimated.
These basic approaches are then extended to account for respondents? ranking of the attributes in order of importance. Here, a number of offset parameters were estimated in addition to the overall coefficients, to allow for greater sensitivity with higher-ranked attributes. Finally, attempts are currently underway to incorporate respondents? attribute processing strategies in a parameterisation of the distributions used for the different coefficients in the mixture models.
In each case, the models allowing for individuals? attribute processing strategies were compared to a base model (either MNL or MMNL). Preliminary results show that allowing for these strategies can have a strong effect on model estimates. As such, we have observed significant changes in the parameter estimates and standard errors in MNL models. Furthermore, and of high interest in the context of recent discussions on the specification of random taste heterogeneity, the MMNL results show that, by accounting for the presence of respondents who ignore changes in certain attributes, we not only obtain a much reduced spread in the associated coefficient values, but also observe a significant reduction in the incidence of sign violations, even with the use of unbounded and inflexible distributions such as the Normal. This supports the above claims that so called evidence showing a large share of respondents with a counter-intuitively signed coefficient can simply be a reflection of the presence of individuals with a zero-sensitivity.
Association for European Transport