Confounding Between Taste Heterogeneity and Error Structure in Discrete Choice Models
Stephane Hess, John Polak, CTS-Imperial College, UK; Michel Bierlaire, EPFL , CH
This paper discusses the issue of confounding between taste heterogeneity and error structure in discrete choice models
The past decade has seen substantial advances in the specification flexibility and estimation technology associated with discrete choice. In particular, it is now possible to accommodate both flexible patterns of inter-alternative correlation and random taste heterogeneity amongst agents. Although methods now exist to accommodate these two phenomena simultaneously (e.g. GEV mixture structures), it is still commonplace for these two phenomena to be considered separately. At the conceptual level, such a separation is entirely reasonable, however, in practice the distinction is by no means so clear-cut, and there is a significant risk of confounding. As an example, the random variations in the sensitivity to public transport fares will lead to correlation between the utility functions of various public transport alternatives, such that, in a misspecified model, this taste heterogeneity may be misrepresented as simple inter-alternative correlation such as that caused by the presence of shared unobserved attributes.
From this point of view, advanced mixture structures are not only a tool allowing for the joint representation of the two phenomena, but potentially also a means of avoiding misleading results caused by confounding in models allowing only for either of the two phenomena to have an effect. The aim of this paper is therefore to explore the issue of confounding between inter-alternative correlation and random taste heterogeneity and to illustrate how the use of advanced mixture structures can reduce the risk of biased results.
While the issue of confounding has been discussed by a number of authors, the results cannot easily be generalised, given the overwhelming reliance on real-world data, where the true error structure is not known. What is needed in this case is a systematic comparison using synthetic data, across a range of different scenarios. This is the approach taken in this paper, which however also looks at the issue on a theoretical level and aims to develop at least a basic specification test.
The theoretical part of the paper demonstrates how confounding between inter-alternative correlation and random inter-agent taste variations can arise, showing how allowing for only one of the two phenomena can produce results that are biased by the presence of the unmodelled effect. Although, in some cases, it is possible for the wrongly specified model to attain similar model fit, the risk of misinterpretation of findings persists. This relates to the implied cross-elasticities as well as to implications in terms of behaviour in the tails of the population.
It can be seen that two possible scenarios arise in which the issue of confounding can play a major role. The first case is one where, in the true model, only one of the two phenomena plays a role, but where the estimated model allows only for the other phenomenon to have an effect. Here, the effects of the unexplained phenomenon can lead to erroneous results showing an effect of the other phenomenon. In the second scenario, both phenomena play a role, but the model employed in estimation allows only for the presence of either of the two phenomena. Here, the presence of the second, unexplained phenomenon, can lead to biased estimates in relation to the other phenomenon.
The applied part of the paper presents six separate case studies using simulated data, representing the various possible situations in which confounding can play a role. The first two case studies illustrate how the presence of unexplained inter-alternative correlation can lead to erroneous results with regards to the prevalence of random taste heterogeneity. The following three case studies illustrate the converse, showing how the presence of unexplained random taste heterogeneity can lead to erroneous results with regards to the presence of inter-alternative correlation. The final case study shows that, in the case where both phenomena play a role, not accounting for the effect of one of the two phenomena can lead to biased results in relation to the second phenomenon. Each of the six case studies also shows how, by using models allowing jointly for the effects of the two phenomena, the risk of confounding is much reduced, although, in some cases, minor issues with confounding can still exist even with the use of such models.
The next part of the paper illustrates the impact of the biased results in terms of model forecasts. As such, the use of models affected by confounding can lead to biased forecasts of market shares, which can in turn lead to misguided policy decisions. While the issue with misleading forecasts arises especially in the case of incorrect results in relation to inter-alternative correlation, problems are also caused in the case of incorrect results in terms of random taste heterogeneity, for example by giving an inadequate account of variations in willingness-to-pay measures across individuals, which can lead to major problems in cost-benefit analyses.
The results discussed in this paper offer strong evidence that modellers should acknowledge the potential risk of confounding, especially given the lack of a priori knowledge as to the true nature of the error structure. While testing separately for the two phenomena can alert the modeller to the relative performance of the two approaches, it does not remove the risk of biased findings. As such, the findings from this paper suggest that modellers should always allow for the effects of both phenomena in a joint fashion, either in a GEV mixture structure, or with the help of a combined ECL-RCL formulation. In this case, some guidance is however still required to help modellers make an informed choice of specification. For this purpose, the final part of the paper proposes a simple test based on this combined formulation. Rather than just comparing the combined model to the two individual single-effect models, the test is based on estimating a model structure that, depending on circumstances can yield a final combined model or reduce to either of the two single-effect models. We conclude the paper by analysing the power of the test in various situations using the synthetic data sets discussed above.
Association for European Transport