Fake news and research is a story of the moment. Many professions self-regulate and academics are no exception. The system we use is peer-review, which governs the publication of research and in many countries the availability of competitive research funding. It can make or break research careers and is there in part to safe-guard against ‘fake’ research, but to what extent does it hold back creativity and innovation?
While there is a growing proliferation of journals across a range of disciplines, careers and reputations are made by publication in the most elite publications, such as Nature or Science, for which competition is fierce. But whatever the journal, getting controversial or truly innovative ideas published can be a challenge and is at least in my experience limited by the very self-regulatory system we as academics uphold. Peer-review aims to uphold academic standards of scholarship and should protect decision-makers and the public from bogus claims (fake news), but it can and sadly does become a form of censorship in some cases. Now in my experience reviewers often stray from a being a constructive and critical friend focused on issues of quality, presentation and logic, to expressions of opinion and reaction to a new idea and the boundaries here are unclear.
As a former journal editor I know just how hard it is to secure reviews from busy researchers. As a reviewer I know that such tasks can become squeezed into bad tempered and stolen moments at the end of a busy day. As an author, and a dyslexic one at that, I have experienced many painful reviews over the last 25 years, some deserved others not, and many have verged from professional to the personal. Challenging convention is what researchers should be trained to do, but many don’t choosing to replicate research and innovation slows. To do otherwise incurs pain and disappointment as peer review system can allow vested interests to stifle true innovation. I am reminded of a piece I wrote as a young researcher about R.G. Carruthers a British geologist so infuriated by the interference of reviewers he took the unusual step of publishing a private pamphlet in 1953. Carruthers was realistic about its success: ‘One has to recognize that the independent issue of scientific pamphlets is rarely a success. The life of such things, like that of the medieval peasant, is apt to be “nasty, brutish, and short”. Still there are times – and this is one of them – when there is no other way, if one’s work is to be presented as written, and free from the interference of others. Whether it be accepted now, or later – perhaps much later – is no great matter. But it will be . . .’ (Carruthers, 1953: p. iii).
A year or so ago I got a paper to review from an elite journal with some challenging findings and a long history of rejections, ill-tempered reviews and downright unprofessional behaviours on the part of some reviewers including breaches of confidentiality. I reviewed it fairly, the first fair review the author’s had received in several attempts to get the material published, but the paper was still rejected by the editor. One lone voice is not enough. The authors were so shocked to receive a kind review given their past experience that I was invited as an expert in the field to help them re-shape their work. I did so, and the new manuscript was tamer, better formed with more data and more considered, but it still ruffled too many feathers and was rejected yet again. Some of the author’s felt as if they were being accused of participating in a scientific forgery (fake research!) from the tone of the reviews. It has recently been published finally after six years.
This story, and the one about Carruthers, illustrates how difficult it is to get innovative ideas published and discussed openly, particularly when they challenge established paradigms and figures. It is after all for the research community as a whole to adjudicate their value, not just a couple of reviewers, acting as representatives of that community. I almost feel angry at the thought of how much innovative and provocative research might have been rejected in the name of the academic community, in my name and in yours (!), without us even being aware. Surely, open discussion of all ideas, however unusual, is essential for innovation and progression?
Peer-review revolves around well-established academics and experts in the field; the very people who often have most to lose by the publication of new models and ideas. Prejudice of all sorts is rife; institutional and national prejudices are often to the fore and are subconsciously applied without thought by many academics. Yet unconscious bias training as part of recruitment processes is now common in most of the UK’ universities. One of the people I consulted about this piece said bluntly ‘it is about academic morals and they are not as white as one would hope’. Treat as one would wish to be treated is a good adage but is easy to forget in a world dominated by competition. Academic competition, through such things as the research enhancement exercise in the UK and for scarce funding, coupled with the competition between journals for the most cited research breeds competitive behaviours. They are easy to measure and monitored by metrics, but there are no easy metrics for compassion, for mentoring and coaching of talent and innovation beyond your immediate research team. One academic I spoke to said ‘what do you expect? You go to see a lawyer and they charge a fortune per hour, or go to a private clinic and you pay handsomely to see a specialist, but you ask an academic expert with over 30 years of experience for their considered view and expect it for free!’ Most academic journal editors are not paid for their time. In the digital age it can take a matter of days to get a paper formatted, proofed and online once accepted, as productions time fall editors are under pressure to cut editorial processing times and consequently many editors (as I was) are encouraged to simply reject rather than nurture papers that need a lot of re-working.
Now to be clear I am not suggesting that we abandon peer-review, it plays an important role in ensuring that what is published is at least intelligible and meets some basic standards of ethical research. The case here when dealing with medical or drug trials for example is clear, but the need to nurture, debate and support the publication of innovative ideas needs greater thought. It is something that is under even greater threat with the current focus on fake news. There have been a number of experiments and new approaches over the years to try and make the process more transparent and less open to personality and abuse. Some journals now offer blind-blind reviewing, others publish the reviewer’s comments and the author’s responses, and there are a number of journals that now allow reviewers to debate the decision letter between them. Blind-blind has its advocates, but any form of anonymity allows abuse in my experience. It is the reviewer’s anonymity not the authors that is the problem and the lack of redress permitted by authors when treated unfairly.
For innovation in self-regulation we have perhaps to go back to the early origins of the peer-review system itself. A learned researcher would present their work via a formal lecture and the audience would discuss and question the author in person effectively providing peer review through active rather than passive debate. Those comments would be minuted and published along with the original lecture. Now we can’t restrict publications to oral presentations and conference invitations are far from unbiased now as they were in the past; you don’t invite the opposition to your own jamboree that often! My point here however is that a more open and transparent debate is needed, not one simply limited to a select, and often self-nominating few acting on behalf of a wider, and usually oblivious, community. The Arxiv (https://arxiv.org/) project is one example. Here articles can be uploaded and receive online comments, it also acts as a digital archive for more conventionally published works. Forums and publications that allow more open peer debate and active, rather than passive and hidden debate, is perhaps closer to the true spirit of peer review? Further experiments are much needed to protect society against fake news and research yet create the innovative, free thinking research talent that our society so desperately needs.
This post is based on a presentation given by Professor Bennett at Bournemouth University in 2018 entitled ‘The Dark-side of Peer Review’.
Professor Sandcastle 