Do we need to redefine our concept of questionable research practices?

Last modified: May 24, 2018

The second plenary discussion was hosted by dr. Stephanie Meirmans and dr. Gerben ter Riet from the AMC in Amsterdam. They are both involved in the research project: ‘Follow the money: Does competitive research funding contribute to Questionable Research Practices?’ An internal discussion in their project group arose concerning the definition of QRP’s. For dr. Meirmans, with a background in Evolutionary Biology and Ecology & Philosophy, her concept of what is meant by QRP’s differed from the viewpoint of dr. ter Riet, who has a background as a Clinical Epidemiologist. They wanted to use the opportunity to discuss this issue with the conference audience, who have varying backgrounds but a shared interest in research integrity:
‘Are the classical QRPs in need of a redefinition for the humanities, or other fields?’ and ‘At which level do we distinguish QRPs?’
Overall, it became clear that the disciplinary field was very important for how the audience felt QRP’s should be defined.

Introduction by Dr. ter Riet: Classical QRP’s

Photo: Fenneke Blom

First Dr. Gerben ter Riet gave a short talk about his viewpoint on QRP’s. His motivation for investigating QRP’s originated from the issue of avoidable waste. He defines three major stages in the process of research waste, based on a paper by Chalmers and Glasziou (2009). The first is relevance: This waste is generated by not asking relevant questions. For instance, did you perform a systematic review to identify the lack of knowledge and to define your research question? The second stage is methodology: With a poorly designed method, the experiment is doomed to fail. This can be solved by involving other researchers so that you can come up with the correct questions and consequently use a proper and rigorous methodology. The third stage is publication selection waste since not all performed studies are published. He concludes that the mechanism, in which QRP’s often come about, is a result of conflict in interests from different parties. Dr. ter Riet shows a list of the 60 classical QRP’s and research misconducts, taken from Bouter et al. (2016). They are defined on an individual researcher’s level and are ranked from most to least prevalent. The highest ranking theme in this list is about reporting, followed by data handling, study design and collaboration with other people. Fabrication of data is not ranked very high, because – although it is a serious research misconduct – it is not very prevalent.

Introduction by Dr. Meirmans: Novel QRP’s

Photo: Fenneke Blom

Then dr. Stephanie Meirmans elucidated her viewpoint on QRP’s. Her critique on the classical QRP’s is, that they are not applicable to the Humanities and other research fields that perform exploratory or qualitative research. Classical QRP’s are mainly about research design, data and statistics. She sees the relevance of these QRP’s for disciplines such as biomedical research where one is supposed to stick to the initial research plan, because there is pressure for a specific outcome.
In the Humanities, however, this format does not really apply because researchers are sometimes supposed to change the plan according to what is found during the research. She shares some QRP’s of which she believes do not apply to all sciences. First, protocols and reports: For some studies it is not always possible to be completely transparent about sensitive data. Second, research findings must be replicable. However, this is often not possible in the Humanities. For instance, some of the most robust insights from evolutionary biology originate from studies which took decades to perform and are not easily repeated.
Dr. Meirmans proposes novel and general QRP’s by first looking at principles of performing good science in general: Being aware of which method and tool you are using and what it means for your outcome, reflecting on your questions and approach, looking at neighbouring fields, seeing science as a balanced ecosystem instead of having all the incentives on one field, and lastly, performing slow science: which entails carrying out more experiments to obtain a higher power for one study instead of spending this time on five small studies, yielding a low power. Based on those, she suggests general QRP’s: Overinterpreting research results, neglecting research fields and approaches, only monodisciplinary or only interdisciplinary studies, ignoring older research or tools, creating only specialist scientists, and finally, fast mindless knowledge production.

Plenary discussion

Following the introductions by these two researchers with different views on QRP’s, the attendees were asked if the science community as a whole should update their QRP’s and if we should move towards a more macro level description of QRP’s, by taking into account the research community and the funders and not only the individual researchers (micro-level). An attendee agrees with dr. Meirmans that QRP’s should be defined based on what good research practices are. However, he prefers if each research community does this separately since communities agree on different research practices. Someone adds that it would actually be useful to combine these different perspectives to learn from each other. A contrary viewpoint from another participant is to have only one model for all the sciences. With his experience in varying research fields, he sees the same work sequence: generating a hypothesis, test the hypothesis, etcetera. He suggests that we stick to a simple approach and then make the QRP model more complex for each field. However, someone responds that not all disciplines actually use hypotheses, and therefore this model is not possible. Next, the discussion became a bit more philosophical in nature. One of the attendees believes that if we change ‘hypothesis’ to ‘question’, the QRP model would actually fit to all sciences. However, in the field of mathematics and engineering, a hypothesis or research question is not necessarily needed. Lastly, someone pondered whether we should change the definition of science. And if we could agree that science is just one type of research, but not all research is scientific. This ultimately led to a discussion on which fields can actually be called science. Would mathematics have to be excluded then? We conclude that agreeing on general QRP’s across fields can be a challenge.

By Sanne Joon

Photo: Fenneke Blom

References

  1. Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. Obstetrics & Gynecology, 114(6), 1341-1345.
  2. Bouter, L. M., Tijdink, J., Axelsen, N., Martinson, B. C., & ter Riet, G. (2016). Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Research Integrity and Peer Review, 1(1), 17.

Do you want to read more?

Keynote lecture by Prof. Wicherts
Discussion: Future or scholarly communication
Closing remarks
Taxonomy of non-integrity – a reflection on the conference

NRIN devotes a great deal of attention to the website’s content and would greatly appreciate your suggestions of documents or links you believe to belong on this website.

This selection is an incomplete convenience sample, and does not reflect NRIN’s vision or opinion. Including an item does not necessarily mean that we agree with the authors nor does it imply we think unmentioned items are of poorer quality.

Please report any suggestions, inaccuracies or nonfunctioning hyperlinks etc. that you discover. Thanks in advance!

Contact
Icon