Sustainable, Participatory and Practical: the NISTO Evaluation Framework for Urban and Regional Mobility Projects

Sustainable, Participatory and Practical: the NISTO Evaluation Framework for Urban and Regional Mobility Projects


Imre Keseru, Vrije Universiteit Brussel, Jeroen Bulckaen, Vrije Universiteit Brussel, Cathy Macharis, Vrije Universiteit Brussel


An innovative evaluation framework is presented for small-scale mobility projects. The framework combines the assessment of sustainability through multi-criteria analysis and stakeholder participation through the multi-actor multi-criteria analysis.


Several methods for the evaluation of mobility projects are used by transport planners and authorities across Europe. The existing tools demonstrate a wide variation in approaches, coverage of transport modes, comprehensiveness, ease-of-use and flexibility. Our review of current practice revealed that relatively small-scale mobility projects (budget up to EUR 200.000) that aim to change travel behaviour are often not or only partially appraised due to the lack of simple evaluation tools or budgetary constraints. A full-scale cost-benefit analysis is often not an option due to the cost and time needed.
In the framework of the NISTO project (New Integrated Smart Transport Options), which is co-financed by the Interreg IVB programme, a new easy-to-use, flexible and integrated evaluation toolkit has been developed specifically for smaller scale mobility projects to overcome these deficiencies.

The NISTO evaluation toolkit is based on the assessment of sustainability of projects considering a balance between the three pillars of sustainability (economy, society, environment) and a close integration of stakeholders into the evaluation process. In addition it provides an easy-to-use online tool for practitioners.

The sustainability of projects is assessed through 16 criteria grouped under the three pillars of sustainability that were selected by an in-depth analysis of current practice in North-West Europe. The performance of these criteria is measured through a set of core and optional indicators and the evaluation is carried out with Multi-Criteria Analysis.

Stakeholders are involved in the appraisal through the multi-actor multi-criteria analysis (MAMCA), which allows the consideration of conflicting stakeholder objectives and helps to identify synergies and disagreement between different stakeholder groups. The methodology is based on assessing the evaluation criteria of the different stakeholder groups rather than appraising the project based on a set of criteria agreed on with all stakeholders at the beginning of the process. Therefore the evaluation shows which implementation alternatives or scenarios each group would prefer and allows for a straightforward comparison of preferences across all stakeholder groups. The MAMCA provides a structured way of involving the stakeholders in the decision-making process.

The appraisal and/or ex-post evaluation is assisted with online software for both the sustainability assessment and MAMCA to make it easily accessible to practitioners.

The toolkit is tested through five innovative urban mobility demonstration projects, which vary in theme, geographical coverage and objectives: intelligent information provision for transport users in Birmingham (UK), a bicycle rental scheme in Boulogne-sur-Mer (France), improving cycling connections in the Netherlands, mapping and influencing travel behaviour through a smartphone app in Leuven (Belgium) and an integrated tourist ticket in Kassel (Germany). Data to evaluate the performance of these demonstration projects has been collected and monitored using the NISTO framework. The final framework and toolkit is being created based on experience from the demonstrations and feedback from stakeholders (future users) from across North-West Europe.


Association for European Transport