Evaluation of Transfer Methods for Spatial Travel Demand Models
N Karasmaa, Helsinki University of Technology, FI
The main goal of this research was to compare alternative methods of spatial transfer as a function of sample size. The methods examined were the Bayesian updating, combined transfer estimation, transfer scaling, and joint context estimation.
The idea of model transferability is to use previously estimated model parameters from a different area for model estimation. The transfer of previously estimated model to a new application context can reduce or eliminate the need for a large data collection and model development effort in the application context. However, the usefulness of a transferred model depends on the degree to which it can provide useful information about the behaviour or phenomenon of interest in the application context.
The main goal of this research was to compare alternative methods of spatial transfer of discrete choice models as a function of sample size. In addition, different test measures for studying model transferability were compared and the applicability of the traditional statistical tests, with respect to those based on the prediction accuracy of sample enumeration tests and forecasts, were assessed. The paper presents the transfer of the four-step travel demand model system of internal trips in the Helsinki Metropolitan Area (HMA), Finland. The HMA data base is used to estimate the models that are to be transferred. The data collected in the Turku region in 1997 represents the application context to which the estimated HMA models are transferred. The transfer procedures examined were the Bayesian updating, combined transfer estimation, transfer scaling, and joint context estimation. To explore the impact of sample size on transferring performance, model transferability was tested using six different sample sizes. The resampling was performed by using bootstrap, which made it possible to study the variance of the coefficients for the full sample models as well. All transferability tests were carried out by using 100 samples for each trip group, transfer method and sample size category.
There are two main factors, which affect model transferability. The first one is transfer bias, that is the difference between the coefficients estimated from the estimation and application contexts data. The other is the statistical reliability of model parameters. Thus the final results depends on both the quality of the models in estimation and application contexts, and the differences in the traffic system, and travel behaviour pattern.
Earlier transferability studies have mainly focused on the study of model transferability by using only one method and one sample size. The assumption, in many cases, has been that model transfer is only possible if the coefficients used in the estimation and application contexts are quite similar. As a large part of the difference in coefficients and predictions is due to the discrepancies in the formulation of the initial data, or random variation, and only partly due to real differences in the estimation and application context, the main emphasis in this study was to investigate the relationship between the transfer bias and impreciseness caused by the sample size. Different transfer methods were compared to each other, as well as the amounts of data needed to estimate especially mode and destination choice models. The model transferability was tested by comparing the transferred models to the models estimated using the entire data set which can be regarded as the best estimate representing ?the real situation? in the application context.
The results indicated that joint context estimation gives the best prediction performance in almost all cases. In particular, the method is useful if the transfer bias is large or only some of the model coefficients are precise. The applicability of joint context estimation can be improved by viewing the coefficients as variable-oriented and emphasizing precise and imprecise coefficients differently.
The use of empirical data highlighted problems which are not connected to the transfer method itself, but greatly affect the model transfer. Of these, the most important was the repeated measurement issue. Repeated measurements imply that there will be a correlation between the answers provided by the same individual, and therefore in the error terms in the utility function. Consequently, the estimated standard deviations are underestimated. Repeated measurements do not affect the parameter estimates of new sample models, the transfer scaling or joint context estimation, but they do affect the results based on the Bayesian method and combined transfer estimation. The correlation between the observations means that when using the Bayesian method in particular, the results may be strongly biased.
Model transferability has traditionally been evaluated on the basis of how well transferred models reproduce existing behaviour rather than on their ability to adequately forecast changes in travel demand. The comparison of value of time and elasticities to the corresponding TTS (Transferability Test Statistic) values indicated, however, that statistical tests are not able to evaluate the goodness of transferred models with a high enough degree of versatility. For example, two models that have totally different values for their coefficients may have the same likelihood value and hence the same TTS. Consequently, their ability to predict the effect of changes in a transportation system may differ greatly.
Another related problem both in calculating the TTS as well as in calculating elasticities is that in modelling the scale parameter in the choice probability function is fixed to one. This means that the absolute level of the coefficients is actually dependent on the real value of the scale parameter. Consequently, the ratio of the model parameters to each other can be estimated reliably but the absolute level of the parameters is unknown. Thus, the comparison of model parameters on an equal basis, as has been the case in many transferability studies, is not justified. Another problem is that in spite of the fact that the absolute level of the model parameters is unknown, the elasticities are highly dependent on the absolute level of the coefficients, which can be regarded as a weakness of the logit model formulation.
Association for European Transport