From output to impact

Last modified: May 22, 2017

by Rinze Benedictus

There’s an international debate going on about how evaluation practices informed by bibliometric indicators shape biomedical research and distance it from its societal mission. Waste in biomedical research [1] has been linked to the current rewards and incentives, which are increasingly under critique [2,3]. There is an urge towards ‘responsible metrics’ [4,5].

At the University Medical Center Utrecht, The Netherlands, we have taken up the gauntlet of steering towards responsible metrics [6]. Starting this year, all candidate professors have to submit a portfolio encompassing five domains. We ask for a narrative about current scientific activities and a description of the most important ten publications (not an extensive list). The applicants provide proof of teaching skills and describe educational products, for example study guides or study books. If relevant, applicants describe their clinical tasks, including efforts resulting in improved clinical care. The portfolio also covers leadership and management responsibilities and asks for valorisation activities.
The portfolio provides the selection committee with a broad view and arguments to promote or hire someone that may not have the perfect ‘excellent’ scientific profile in bibliometric terms, but someone who is excellent because of qualities that may be harder to quantify, but can very well be talked about, evaluated and judged.
We started using the portfolio in the appointment procedures for professors this year. Approximately 20 new professors are appointed each year. In 2017 we will extend the approach to associate professors.

Preparing and reading the portfolio implies a slight but undeniable increase in preparation time for both the applicants and the committee members. But, based on our experience so far, we feel this is easily justified by the rich and broad description of the candidates and the fruitful discussion it inspires during the interviews.

We complement this approach by evaluating our research programs based on self-developed ‘indicators for impact’ that ask research groups to describe the processes and structures underlying their scientific output. Instead of quantifying past performance this forward-looking view shows us how research programs are geared towards creating societal impact. Indicators for scientific quality should reflect our mission, not shape it.

Rinze Benedictus is staff advisor at the University Medical Center Utrecht, Utrecht, the Netherlands, and a PhD candidate at the Centre for Science and Technology Studies, Leiden University, Leiden, the Netherlands.

NRIN devotes a great deal of attention to the website’s content and would greatly appreciate your suggestions of documents or links you believe to belong on this website.

This selection is an incomplete convenience sample, and does not reflect NRIN’s vision or opinion. Including an item does not necessarily mean that we agree with the authors nor does it imply we think unmentioned items are of poorer quality.

Please report any suggestions, inaccuracies or nonfunctioning hyperlinks etc. that you discover. Thanks in advance!