Ex Post Appraisal: What Lessons Can Be Learnt from EU Cohesion Funded Transport Projects
J Laird, C Kelly, J Nellthorp, ITS, University of Leeds; S Costantini, J Carbajo, X Mas, J E Leon, Frontier Economics, UK
The results of an ex post evaluation of ten large transport projects in EU countries is presented. The ex post and ex ante impacts and the respective CBAs are compared. Differences between the two are identified and recommendations made.
Investment decisions are rarely made with future impacts known for certain. Minimising this uncertainty plays a large part in any case for funding. A number of authors have considered the role that bias plays in influencing ex ante cost benefit results - e.g. Mackie and Preston (1998) and Flyveberg (2007). Completing an ex post appraisal is one method that can be used to determine whether in reality the impacts of an investment or part of it are anyway similar to that expected/ forecast in the ex ante CBA submitted (e.g. are the expected time savings in reality of the same magnitude as those forecast) and to investigate the causes of any differences.
This paper describes the results of a research project (completed in 2011) funded by DG REGIO, which focused on the ex post evaluation of ten large transport projects in EU countries benefiting from EU Cohesion and ISPA funding during the period 2000-2006. The aim was to evaluate the ex post impacts of these projects and in comparing an ex post Cost Benefit Analysis (CBA) appraisal carried out as part of the research with the original ex ante CBA submitted to the EC determine whether any differences exist the impacts were realised and consider what recommendation could be made for improving the ex ante guidelines. The paper reviews the ex post literature and describes the key findings and recommendations.
In terms of comparing ex ante to ex post CBA the key findings were that:
- Optimism bias was not as large as expected a priori. Capital cost over runs were on average 13.5%. Possibly this arises as the ex ante studies against which the ex post studies were compared were the latest in a series of ex ante studies ? and therefore included the most up to date capital cost estimates and demand forecasts.
- In the main the projects examined represented good value for money to the European tax payer. Net present values and benefit cost ratios were in the main strong, though in seven of the ten cases NPVs were lower ex post than they expected to be ex ante.
- Whilst the CBA framework adopted in each of the ex ante studies was consistent with good practice it was generally found that the ex ante studies were too narrowly focused (e.g. excluded environmental effects and induced traffic) and the counterfactuals could have been better defined.
- The main sources of the differences between the ex ante and ex post evaluation were variances in expected and realised travel demand and capital costs. Realised and expected travel demands varied due to differences in realised and expected economic growth, development impacts and whether or not other transport projects in the vicinity had been constructed.
- In almost all instances the risk analysis did not identify or examine the sensitivity of the ex ante appraisal to these key variances in inputs to the modelling process.
In terms of the role of ex ante and ex post CBA in the decision making and appraisal process the key findings were that:
- For many of the projects we reviewed, we found that CBA had been used mainly to confirm that the project offered value for money and to support the application to access EU funding. Whilst in most countries planning processes have developed since the ex ante analysis was undertaken our discussions with stakeholders identified this limited role for CBA still exists in many countries. Transport planning in these countries is motivated by regional development and environmental objectives, which are more difficult to include in a CBA.
- Ex post CBA offers transparency to the decision making process and improves accountability for decision-makers by casting a light on the decisions that lead to construction of projects with a poor realised return.
- Ex post CBA also helps improve ex ante methods. The key messages from this research are that the demand forecasting and the risk analyses are in need of improvement.
The research identifies a clear need to improve the quality and consistency of ex ante CBA. This can be done in three ways. Firstly by updating current regulations regarding the need and quality of ex ante CBAs, secondly through enhancing the ex post regulations to improve the evaluability of infrastructure projects and bringing those evaluations together in a program to identify patterns and finally though the provision of further training and guidance.
Association for European Transport