Are CBA Results Robust? Experiences from the Swedish Transport Investment Plan, 2010-2021
M Lundberg, M Borjesson, J Eliasson, KTH, SE
We investigate how sensitive ranking of investments is to scenario assumptions and benefit valuations. We find that ranking is surprisingly robust. Decision makers can trust that the CBA methodology will lead to sound investments being prioritized.
It has recently been showed that the CBA outcome influenced the transport investment decisions in the latest Swedish transport investment plan (Eliasson and Lundberg, 2010). This finding is contrary to previous international studies and emphasizes that reliable CBA results are of practical importance.
However, in Sweden and elsewhere using the CBA result as a tool for prioritisation is often questioned. Common views in the public debate are that:
1. The CBA results are very sensitive to the choice of assumptions and valuations,
2. Assumptions are often chosen by the analyst in order to get wanted results and
3. When results are not as expected, the forecast assumptions are wrong.
Even among planners and other professionals a widespread belief is that certain scenario assumptions and valuations are very important for the optimal mix of investments. The present study aims at investigating if ranking of investments is sensitive to scenario assumptions and benefit valuations and thus if decision makers can trust the resulting rankings. We explore how different scenario assumptions and valuations affect the CBA outcome and ranking. The analysis is focused on scenario assumptions such as oil price, policy measures to reduce green-house gases, technological development of cars and car ownership. Examples of valuations that are altered are value of trip time differentiated by length of trip and mode. The database used consists of CBA results for the 479 investments that were studied in the latest Swedish transport investment plan.
We find that CBA rankings are surprisingly robust. Doubling the oil-price, the freight benefits or the business time valuation hardly changes the ranking. As a rule of thumb, doubling the weight of a certain type of benefit only changes around a tenth of a given top ranking list. Some explanations are that existing demand constitutes a large share of future demand, that changes often have a rebound effect (one example being that increased fuel price results in increased demand for more fuel efficient cars) and that most investments are used by many different categories of traffic. The most important uncertainties in a Swedish context are due to missing methods for forecasting effects such as increased reliability for trains, reduced road congestion or capacity improvements for freight railways. Another important deficit is that improvements of urban environments can't be valued.
Overall, our conclusion is that decision makers can indeed feel secure that following the CBA methodology will lead to sound investments being prioritized. The top-ranked investments are more or less the same in all sensitivity analyses. However, the CBA results should not be the sole decision criteria. One reason is that some types of benefits are systematically underestimated; another is that the method, per definition, does not take effects on equity into consideration.
Association for European Transport