Uncertainty in Traffic Models
O Anker Nielsen & M Aagaard Knudsen, Technical University of Denmark, DK
The paper suggests an approach to assess the uncertainty of transport models, and exemplify this on an applied model.
Transport modellers are sometimes asked the question, ?What is the uncertainty of the model?. This is often answered by pure guesses in vaguely defined terms, e.g. ?20%?. The truth is that the modeller may know the uncertainty of certain sub-models or elements of the model, e.g. parameter estimates indicated by e.g. t-statistics, but not the uncertainty of the model system as a whole.
We presented a paper at ETC 2005 on the uncertainty of the solution algorithm of a typical road transport assignment model based on the principle of stochastic user equilibrium. The discussion raised, made us interested in a more general analyses of model uncertainty.
During the last year there have been a debate in Denmark and abroad on forecasting failures, including international publications by Bent Flyvbjerg, e.g. in the Journal of the American Planning Association: This led to a recommendation from this organisation to account for uncertainty by reducing all transport forecasts by a fixed relative factor following recommendations from Flyvbjerg.
The truth is that different models may have very different uncertainties. It is therefore very simplified to reduce all forecasts by the same factor.
Furthermore, there is a tendency that projects where the traffic ? maybe just due to the nature of uncertainty ? are over estimated are decided, while projects where the model underestimates the traffic may be rejected. The pure process of decision making will therefore bias conclusions on forecast bias based on before-after studies only.
However, it doesn?t help the reputations of the art of transport modelling, that the modellers are not able to give an answer on a question concerning the uncertainty of the model. This is maybe why decision makers also tend to disbelieve in models.
The paper begins with a literature review of the ? surprisingly limited ? quantitative oriented literature on uncertainties of transport models systems as a whole.
Based on this review, we suggest a structure that categorise different sources of uncertainty, e.g. uncertainty on explanatory variables, data used for the model estimation, the statistical estimates for each sub model (which is often the only uncertainty presented in model documentation), parameters (e.g. in speed-flow curves, link type dependent capacity definitions, etc. which are often just taken from hand books rather than estimated for the specific model), forecast variables, and solution algorithms (e.g. when models are solved by simulation procedures, or with iterative approaches).
The sources of uncertainties are then arranged in order to describe the four most commonly used model components, i.e. 1) trip generation, trip attraction and balancing, 2) trip distribution, 3) mode choice, and 4) assignment. These may occur in various nesting structures and step orders.
Once the structure of sources of uncertainty have been outlined, it is fairly easy to estimate the statistical distribution of each source, if this is indeed based on a new study and statistical estimation (which is e.g. often the case for mode choice models). It is on the other hand much more difficult to assess the uncertainty on figures that are just based on guidelines or pure tradition, such as e.g. standard parameters in the US BPR (Bureau of Public Road) curve for speed-flow relationships. Many assignment models use e.g. standards software with standard values that are not estimated for the specific case.
If ? however ? uncertainties have been assessed by estimation, judgements or pure guesses, then the overall uncertainty of a model system may be assessed by Monte Carlo simulation techniques. The paper illustrates how to do this.
Finally, we discuss how to model the interaction between model components, i.e. how uncertainty propagates through the model steps, and then how capacity restrictions in the assignment may ? or may not ? reduce or boost the uncertainty of the whole model system. This clearly depends upon how the feedback cycle of road congestion is implemented in e.g. the traditional 4-step model, and of the congestion level.
We used a fairly small transport model for the town of Næstved as a ?test laboratory? for evaluation of model uncertainty. It is a traditional 4-step model of trip production (regression model), trip distribution (gravity model), mode choice (logit model) and assignment models (stochastic user equilibrium for road, schedule-based assignment for bus). The model has 97 zones and about 3,000 links in the road network. Passenger transport is split into commuters, business and leisure trips, and vans and trucks are added for the road assignment. The model system was implemented in ArcGIS by Model Builder, which made it easy to add different assumptions of uncertainty and simulate this.
The paper presents first results that exemplify the influence from different sources of uncertainty on each model component, as well as on the whole model system, with or without use of the feedback cycle. Then various sources of uncertainty are combined in an overall analysis. As the city of Næstved is fairly uncontested, we also made some hypothetical tests of a future year with much higher traffic level, in order to evaluate how uncertainty propagates in a low congested versus highly congested situation.
We also tested it on a specific new road project, and showed how the uncertainty distribution of the results could be revealed.
Although difficult to generalise from one case-study only, the work shows that it is possible to reveal the uncertainty distributions of a whole transport model system. This reveal naturally only the ?internal? uncertainty of the given model, but not the influence of simplifications or omission in the model, e.g. using a linear utility function instead of an unlinear, income dependent VoT versus not, etc. Our approach will therefore only indicate a lower bound of the true uncertainty. But this will still provide more insight into the uncertainty of the model, than only e.g. parameter estimates of some of the sub models.
Some conclusions are made on how some sources of uncertainty may propagate in a linear way, while other may influence the final result in a non-linear way. It was also revealed, that the uncertainty of some components in some model steps may have much higher influence on the outcome than other.
Association for European Transport