Theory-based evaluation in complex environments (EN)
We are pleased to present you a publication devoted to the theory and practice of evaluation of public interventions – Theory-based evaluation in complex environments.
The subject of the publication was inspired by challenges related to evaluating public initiatives, in a complex and ever-changing environment. An example here may be inter alia the challenges in implementing innovation-oriented programmes, in the face of various economic crises and changes in attitudes of consumers as well as in organizations and business models. Taking into account this type of determinants, both the programme theory and the logic of carried out evaluation activities should be continually adjusted. This is perfectly highlighted by Prof. Michael Quinn Patton in the text entitled “Developmental evaluation”, opening the publication.
Development of evaluation activities is also stimulated by the European Commission. In the present market reality the EC indicates the need for the more rigorous methodological approach in evaluation (including an innovative and experimental approach). It would allow to explain the actual mechanisms of intervention and to obtain reliable information about the real effects of policies. One of the approaches recommended for the evaluation of Cohesion Policy of the European Union (EU) for 2014-2020 (and for the ex post evaluation of support within the perspective of 2007-2013) is the theory-based evaluation (TBE). Its application should allow to explain the changes taking place as a result of intervention, but also contribute to improving the quality and usefulness of evaluation itself.
This direction of the development of evaluation entails a necessity to adapt the existing principles of implementing the programmes by establishing a stronger link between evaluation processes and the programming and implementation of interventions. It enforces cultural changes in the public sector. This issue is referred to in the text by Maciej Gajewski and Andrzej Regulski presenting an example of evaluating the systems used for selecting innovative projects. Linking evaluation of interventions to programming and implementation processes remains the challenge at the level of nationwide activities, but particularly in the evaluation of regional programmes (as it is well illustrated by the text by Anna Bruska and Katarzyna Lotko-Czech on experiences of using TBE at the regional level). It should be stressed that the approach based on the theory of change is not only useful for the evaluation itself but, most of all, may be helpful in implementing development policies, and particularly their continuous adaptation to the ever-changing context
I. Developmental Evaluation
II. A comprehensive review of the theory-based evaluation concepts
III. Qualitative comparative analysis in theory-based impact evaluation
IV. Testing intervention theory using Bayesian Network Analysis: evidence from a pilot exercise on SMEs support
V. The use of contribution analysis as a theory-based evaluation approach in the ex post evaluation of support to large enterprises under Cohesion Policy programmes 2007-2013
VI. How to better understand the results of counterfactual analyses? Selection of beneficiaries for qualitative researchusing statistical matching, illustrated by an example of the impact evaluation of the Operational Programme Innovative Economy 2007-2013
VII. Stratified Propensity Score Matching – quasi-experimental research design for theory-based impact evaluation with lacking dependent variable
VIII. Theory-based evaluation in studies of the innovative project selection system in Operational Programmes for 2014-2020
IX. Experience in the application of theory-based evaluation at regional level