ZEW President Franz on Evaluation Mania

This piece appeared in the October 2004 edition of the ZEWnews.

We’re evaluating ourselves to death. For once it’s not active labour market policy that’s up for discussion, but rather economics in the more general sense. A once completely justified concern is gradually developing into a nuisance, some might even say an obsession.

When I peruse the messages in my inbox from the previous month, I find, among other things, project proposals from renowned research foundations awaiting evaluation, scientific papers that the publisher of a well-respected journal wants checked, requests for my participation on external appointment or structuring committees, regular invitations to sit on some or other evaluation commission, cheery letters from (corporate) foundations announcing the establishment of a new scientific advisory council along with a respectful invitation to become a founding member, as well as various messages from students asking for an appraisal for their university, internship and job applications both in Germany and abroad. The list goes on.

For those subject to evaluation the list requires just as much effort to sift through. Based on my own experience, I can say that the projects carried out an economic research institute and carefully evaluated first and foremost by commissioning parties and third-party donors. This could be the relevant authority with the European Union, one of the German federal ministries or private institutions and companies or even research foundations such as the German Research Foundation. Furthermore, we are very grateful for the work of ZEW’s scientific advisory council, who have been carrying out intensive assessments of the quality of research conducted at the institute for a number of years. From now on, the council also has the responsibility of presenting a comprehensive evaluation every two years to the Leibniz Association, which in turn carries out a main evaluation of the institute every seven years. In addition, the scientific advisory council reports regularly to the supervisory board, who are legally obligated – but thankfully also motivated by their own interest in scientific research – to assess the scientific advisory council and provide us with constructive criticism and a good deal of encouragement. And this leaves barely any time to mention the almost constant monitoring of the institute on the part of accountants, pricing audits, health insurance companies, the tax office and the Court of Auditors. In short, we have absolutely no reason to complain about a lack of attention from various evaluators.

Can we have too much of a good thing? Shouldn’t previous high-quality performance justify greater trust in an institution rather than almost permanent monitoring? Are we forcing our researchers to sacrifice too much of their time to evaluations to the detriment of their own research? It’s understandable to want to only entrust the evaluation to recognised experts, but in practical terms this means that, depending on the field, it is often the “usual suspects” who get called in. Is there not a danger of these experts growing weary of all this evaluating, especially since their careers are supposedly dependent on the number of papers they have published in well-respected journals? Is it fair to say that research institutions are spending a little bit too much of their time preparing for evaluations? Because naturally, every evaluation has to be taken seriously and great effort must go into the preparations.

Let me make myself clear once again: Critical evaluations are without doubt an absolute necessity and have in the past been wantonly neglected. But we shouldn’t throw the baby out with the bathwater.