How to Make Evaluations of EU Cohesion Policy More Credible

The EU’s Cohesion Policy (CP) intends to promote European economic, social, and territorial cohesion. The policy occupies about a third of the EU budget and it is the most evaluated of all EU policies. With the support of the German Ministry of Finance, two in-depth ZEW research papers by international author teams have assessed this evaluation system by analyzing its institutions and by applying AI-powered textual analysis of about 2,500 Member State evaluations. The results indicate that the evaluation system lacks important elements in producing fully credible evaluations. A fundamental problem stems from the design of the CP itself: An increasing number of CP objectives blur the precision of the policy and lead to a loss of a well-defined yardstick against which policy success can be judged. Our survey of evaluators confirms this with more than 60 percent of respondents regarding unclear policy objectives as a bottleneck for the evaluation system. Our results also point to limits in evaluation culture and methods where evaluations are sometimes just seen as a formalistic obligation. Another crucial limitation is a lack of full and effective impartiality. This is evidenced by the fact that 71 percent of survey respondents report at least somewhat intense involvement by the managing authorities in their work. Moreover, evaluation teams lack internationality, while the national markets are typically dominated by a few groups. In the end, data using measurements from our textual analysis points to a suspicious inconsistency between the findings of the academic literature on the impacts of Cohesion Policy and the results of evaluations.

Asatryan, Zareh, Carlo Moana Birkholz and Friedrich Heinemann (2024), How to Make Evaluations of EU Cohesion Policy More Credible, ZEW policy brief No. 24-08, Mannheim