Data Leakage Via Covid-19 Journal Preprints -- How Serious?
Back in pre-internet days it was common for academic researchers to distribute “preprints” of research articles to interested colleagues even before the article had been formally reviewed and accepted by refereed journals.
This preliminary status of the preprint article was common knowledge. It was generally understood by members of the academic community that the final journal article when published might be different, but the value of getting even preliminary research findings distributed was seen by many as outweighing the possibility that the findings might undergo significant revision if and when the article was published.
Fast forward to today. Pre-publication “preprints” are still used for early dissemination of research findings although distribution is now electronic. Journals, also published and distributed electronically, still rely on peer review to provide an assessment of submitted articles which can then be accepted as is, accepted with revisions, or rejected.
One major difference in the current process is that the system now “leaks” more than when exchange was dependent on paper and snail mail. Information now is easy to distribute via email, personal web sites, blogs, shared archives, cloud based repositories, and other methods. These can easily be used to bypass the professional imprimatur of the refereed journal.
One consequence of this “leakage” is that recipients of preprint-based information distributed through popular media may have no way of knowing whether the research has been “vetted” through the more traditional journal peer review channels. Catherine O’Grady writes about this situation in the January 21 Science in the article Media outlets inconsistently mention uncertain status of COVID-19 preprints:
Within a few months of the first known cases of COVID-19, scientists had published thousands of preprints on the mysterious outbreak that was becoming a raging pandemic, sparking a flood of news stories from media outlets accustomed to the more stately pace of peer-reviewed publications. Now, an analysis of some of the most widely covered COVID-19 preprints has found that these outlets vary widely in the way they refer to preprints, with about half of their stories failing to mention that the research was unreviewed or otherwise unverified.
O’Grady was reporting on the research paper by Alice Fleerackers and others in Taylor & Francis Online on January 3, 2021 titled Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets.
Given the ease with which information can be distributed online in bits and pieces, contextual information essential to accurately interpret that information is easily left behind. In the case of health related research such as that related to Covid-19 an incorrect interpretation of research findings can have significant negative consequences. Also, flawed research findings released too early or distributed without appropriate explanation or vetting can also lead to incorrect decisions.
As the research cited above suggests, even “mainstream media” channels can leave out contextual information.
This problem will not disappear anytime soon even when professional journalists make strenuous efforts to include context when preliminary research findings cross over from academia to public media. Professional journalists cannot possibly control 100% of the information that is so easily publicized and spread via social media.
These days there are just too many links in the disjointed chain of information channels now available to anyone with internet access. The question is one of costs and benefits. Those who favor the “free and open flow of information” may need to seriously consider how far they are willing to promote that concept given possible misinterpretation or misuse of incomplete or partial research findings.
My own preference is for scientists and researchers themselves to make the determination of how and when to release information. They always need to couch that information with whatever caveats or cautions they believe appropriate. Beyond that they may need to be more aggressive about addressing how that information is used, distributed, or in some cases, how distributed research information is misinterpreted or even abused.
We may all need to acknowledge that, in the complex information economy we now operate, traditional roles, responsibilities, and gatekeeping controls over what can be distributed online are increasingly difficult to enforce, especially when the free exchange of information is highly valued.
Copyright (c) 2021 by Dennis D. McDonald
