Life in corrupt science: when metrics become targets

Publish or perish’. The overused phrase describing career aspects in today’s academia. Whereas day-to-day work is rarely quite as grim, we all learn early on that our performance as scientists is very much measured by the number of publications, citations, and successful funding applications we produce.

Perverse_incentives

For our latest Reading Group meeting, Heini Kujala had selected a paper by Edwards and Roy (2017) on the importance of maintaining scientific integrity in the 21st century climate of perverse incentives and hyper-competition. It is a depressing but engaging read addressing the current metrics-based evaluation of scientists, which, coupled with an increasing number of academics and stagnant funding budgets, has created a perverse incentive system that is pushing academia to the ‘normalisation of corruption’. In their paper Edwards and Roy introduce a conceptual model that describes how the true scientific progress changes along the quality-quantity axis: too much focus on quality slows down productivity; too much focus on quantity increases risks of both unintentional and fraudulent errors. The authors suggest that the overemphasis on quantity is far more susceptible to unethical behaviour and may therefore have an even greater impact on both scientific productivity and reliability. They back-up their theory with a compelling set of evidence of the ongoing productivity loss due to reduced ethical standards. Quantity-based metrics coupled with hyper-competition has the risk of favoring unethical practices such as data modification, p-hacking, overselling of results, substandard review and even outright fabrication. Over time this may lead to a ‘perversion of natural selection’ for scientists that are more comfortable and responsive to such incentives while pushing out actors with more ethical and altruistic work motives. This is hardly the first paper to raise the issue, but Edwards and Roy succeed in identifying the roots, extent, and wider ramifications of the phenomena beyond incidental reports of misconduct all the way to world politics.

“…if normally ethical actors feel a need to engage in unethical behavior to maintain academic careers, they may become complicit as per Granovetters’s well-established Threshold Model of Collective Behavior (1978). At that point, unethical actions have become ‘embedded in the structures and processes’ of a professional culture, and nearly everyone has been ‘induced to view corruption as permissible’.” (Edwards and Roy, 2017)

Many of us felt the paper very much hit the mark, resonating with some of our experiences as actors and observers in the academic game and in interacting with a public that has an ever-declining view of the accuracy and relevance of scientific research. We discussed the many ways in which metric-based incentives drive our own work routines, or even the way we talk about our work: the subtle difference between praising a new publication in a high impact factor journal and praising the scientific findings in that paper. It is painful to think about the consequences of corrupt science. The discoveries that are never made because today’s scientists do not have the time to focus on their work, and the mental health issues sparked by constant feelings of underperformance, competition and lack of job security, just to name a few.

While Edwards and Roy shine a bright light on the problems, we found their recommendations for actions toothless. Mapping the extent of the issue, writing guidelines for best practices and training PhD students to better identify and handle unethical conduct seemed to us as too little too late. As long as incentives that promote quantity over quality exists, it will be difficult to change the system from bottom-up. But not all is as bleak as it might seem. Calls to stop using simple metrics to evaluate scientists and institutions are slowly but surely becoming louder. In response, the Australian Research Council will be conducting a pilot study this year to test a range of quantitative and qualitative measures of engagement and research impact, with the aim to direct more of funding to research that translates into real-world benefits. How well these metrics will work, and whether they will succeed in pushing the research culture to the right direction, remains to be seen.

Australia is one of the few Western countries that does not have an office of research integrity. This seemed as one obvious gap that could and should be filled. An independent body could better help in handling with allegations of misconduct, assisting whistle-blowers and overseeing the development and trends in research integrity. In addition, since ‘pretending research misconduct does not happen is no longer an option’, why not start rewarding individual scientists and institutions for ethical behaviour and scientific rigour? Approaches as simple as giving out badges for good scientific practice could be a start, as shown by the recent advancements in improving the reproducibility of scientific work. In our own lab, we have committed to continue this conversation and related ethical and mental health issues in future group discussions, and to highlight and applaud the varied ways each of us contribute to the group, to the wider ecological research community and to the public.

Edwards M.A. and Roy S. (2017) Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science, 34(1): 51-61. doi:10.1089/ees.2016.0223.

Advertisements
This entry was posted in Reading Group and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s