Six months after COVID-19 shut down world economies, closed borders and unleashed sickness and death around the globe, the pandemic has helped to break open the cloistered world of medical research.
“Open science” is a movement predating the pandemic, largely led by younger researchers eager to cast aside the arcane rules of academic publishing and often secretive, slow peer reviews in favor of posting findings quickly, freely and subject to public critique on the internet.
It was clear in January 2020 that the alarming new virus emerging from Wuhan, China would be a test case for open science, and it has been. The urgency to understand SARS-CoV-2 — the virus that causes COVID-19 — and the desperate search for vaccines and treatments triggered an explosion of preliminary research released for open vetting and discussion.
At Seattle’s Fred Hutchinson Cancer Research Center, evolutionary biologist Dr. Trevor Bedford is known as an expert in tracking the spread of diseases like influenza and Ebola by tracing small genetic mutations from viral samples taken from patients. He has also been an advocate for open science and the quick release of vital information from research on epidemics.
Jolted by early data in January suggesting that there was sustained human-to-human spread of the yet-to-be named virus in China, he began posting his observations publicly from his Nextstrain website, on his Twitter feed and on his laboratory blog. Based on genome sequences from just 24 patient samples, he and his team deduced that a dire situation was at hand, and he began sounding alarm bells to his public health contacts in the U.S. and abroad.
“Generally, I have been willing to kind of go out on a limb a bit more than many of my colleagues, and rather than treading more slowly, change and update things until they are correct,” Bedford said.
In the ensuing months, he became one of the most sought-after scientists in the world. His Twitter followers ballooned to 264,000 from 3,000. His team’s genomic analysis of the first cases in Washington state, which concluded the virus had been spreading silently for weeks, primed public health and civic leaders for a speedy shutdown of major businesses and conventions in Seattle in the first week of March. That prompt response is credited by many for averting a catastrophic spread such as occurred only weeks later in New York City, where nearly 24,000 later died from COVID-19.
In fact, one of Bedford’s early conclusions was eventually shown likely to have been wrong. The spread of cases in Washington state appears now to have come from a separate introduction event from China, rather than from the first known case from a visitor to Wuhan, as Bedford first suspected. It was a correction prompted by new data, a real-time response to public peer review he was happy to make. The fine technical adjustment did not change the urgency or wisdom of his initial finding.
“You want to strike a balance,” Bedford said. “On the other hand, the World Health Organization has really erred on the side of certainty. So, they’ve been slow on a bunch of things they should not have been slow on, like transmission by people who are asymptomatic, or the degree of airborne transmission. They’ve been really focused on waiting until it is absolutely certain the evidence is fully supportive of whatever hypothesis there is, and that’s slower than it should be.”
A primary vehicle of open science is the so-called “preprint server,” a place on the internet where research teams download their fresh papers to be viewed, publicly, free of charge and without private peer review. That process takes place in the open, involving sometimes blistering comments from other scientists. Researchers call these open critiques “crowd-sourced” peer review.
Scientists from China began a trickle of postings about the new coronavirus in January on the preprint server bioRxiv (pronounced "bio-archive"), which was founded in 2013 as a service of Cold Spring Harbor Laboratory, in New York. To date, more than 8,500 research articles related to COVID-19 have been posted on bioRxiv and its newer sister publication medRxiv (pronounced "med-archive"). The latter preprint server was launched in 2019, but still had a low profile in December. It quickly became the leading site for COVID-19 papers after Cold Spring Harbor divided topics, leaving primary laboratory biology studies to bioRxiv, and assigning epidemiological and clinical studies involving humans to medRxiv, which accounts for 6,700 of the total.
Whether the flowering of preprints in COVID-19 research will spill over to other fields of science is unclear. Dr. Eleonora Presani, an astroparticle physicist, is executive director for arXiv, the original preprint server founded in 1991 at Los Alamos National Laboratory and now operated by Cornell University. It was set up before the internet boom to connect the high-energy physics community, which is accustomed to working in large, international collaborations.
Presani said growth in typical postings has remained at 20% per year, except for an explosion in COVID-19 science, where researchers have already posted 2,204 coronavirus papers to date — more than in bioRxiv. Presani said that arXiv collaborates with bioRxiv and medRxiv, and COVID-19 has deepened those connections.
“We don’t see ourselves as competitors,” she said. “Our communities are closer than ever.”
The arXiv COVID-19 posts are mostly by scientists using mathematical modeling, machine learning and computer science applied to biology. Because of concerns that the general public may not understand the preliminary nature of findings posted on the preprint server, arXiv runs a red banner above each post stressing that these COVID-19 studies are not peer reviewed, should not be relied upon to guide health-related behavior, and should not be reported in the news media without consulting “multiple experts in the field.”
Dr. Sue Biggins, senior vice president and director of the Basic Sciences Division of Fred Hutch, said that one of the most significant changes wrought by open science in the COVID-19 era is in how science journalists treat preprints. The non-peer-reviewed posts in bioRxiv and other preprint servers until recently were largely ignored. The results were rightly seen as preliminary, and scientists were reluctant to comment on their posts, lest they run afoul of journal editors who might reject a paper if findings were made public ahead of their schedules.
Journalists preferred to wait until research had been blessed by the rigorous process of peer review — and selected by prestigious science journals — before considering it worthy of coverage in general-interest media.
“I think, and others would agree, that the media are now promoting non-peer-reviewed science, which is a huge change,” she said.
That places a burden on journalists to stress the preliminary nature of preprint findings in their reporting, and to go the extra mile to seek feedback from experts who can evaluate the work. And it still leaves an opening for misleading or inaccurate science, even outright fraud, to proliferate and find a following — especially because these preprints are free and can be accessed and shared by anyone.
As an advocate for open science, however, Biggins said the public nature of preliminary findings and the rigor of public critiques can be a valuable process for quicker acceptance of important results.
“Some people say that Trevor Bedford putting all this stuff up on Twitter that is not peer reviewed, or even written up as a paper, is not the way you want to do science. I would counter that Trevor is putting it out there for 200,000 reviewers. He’s actually getting more scrutiny for his research,” she said.
“The other part that I love is being able to see the feedback. I’m always curious about who is agreeing with him, and why they agree, and who is pushing back. Science is fluid. It changes. In social media, he’s got this interesting way of continuing to update his work in effective ways.”
Open science advocates including Bedford and Biggins note that top-tier publications with peer review are not immune to charlatans. On June 4, the New England Journal of Medicine and The Lancet each retracted — at the request of the authors — separate papers regarding potential health hazards of proposed COVID-19 treatments.
It turns out that both research teams relied on false or faulty information based on electronic medical records provided by a Chicago-based company, Surgisphere, which then refused to cooperate with subsequent investigations.
That debacle only underscores the potential problem of open science research being taken as gospel by a public or political leaders who do not have the insider’s perspective or skills to know how far the trust these studies.
“I think the open science movement has just accelerated what we know about this virus. I think it has done so in a way where, otherwise, we would still be waiting for the first papers to hit the press,” said Dr. Harmit Malik, an evolutionary biologist at Fred Hutch who is not directly involved in COVID-19 research.
On the other hand, Malik is a heavy reader of scientific papers, and he believes the vast majority of COVID-19 research on preprints is of poor quality. “All this substandard work is now available to everybody, including journalists or politicians who have their own kind of axes to grind,” he said.
Well before COVID-19 entered the picture, however, many scientists were growing frustrated with the rituals and delays associated with publication in peer-reviewed journals, as well as the high cost of subscriptions, and the open science movement sprang from that discontent.
Hutch evolutionary biologist Dr. Jesse Bloom, who like both Malik and Biggins is also a Howard Hughes Medical Institute Investigator, was drawn to open science in 2014 because he saw it as a way to devote more time to the lab bench.
“Peer review can be very slow, and it is nice to have a paper done, out there where other people can see it, and feel like you can move on to the next thing,” he said. “I still think we should have peer review as a quality control, but most people are also aware of their reputations as scientists, so they are careful what they post.”
Bloom has also taken to Twitter, as many of his colleagues have done, to expand the network of people who are aware of his recent COVID-19 research. He recently used a Twitter thread to explain how work on re-infection by influenza and measles puts into context recent reports of re-infection by SARS-CoV-2. His thread offered a reassuring conclusion that re-infection with SARS-CoV-2 is not concerning unless it turns out to be a frequent occurrence.
Bloom’s lab posted its own bioRxiv preprint in June describing their technique of “deep mutational scanning,” which they used to test the effects of every conceivable mutation of a key structure on the spike of the coronavirus, which it uses to bind with the ACE2 receptor found on the surface of many human cells — including those deep in the lung. The same research was subsequently peer-reviewed and published on Aug. 11 in Cell, which, as most top scientific journals have been doing with COVID-19 papers, made the article available on open access — free to anyone.
That sequence of discussing work publicly via social media, publishing research in preprints for crowd-sourced discussion and peer review, and then publication in open access, peer-reviewed, high-impact journals may be a rough cut of how science of all kinds is carried out in the future.
Hutch physician-researcher Dr. Joshua Hill and his colleagues recently participated with University of Washington researchers in a study of whether a drug that dampens immune responses, tocilizumab, might tone down the extreme immune reaction known as a cytokine storm that is responsible for the deaths of many COVID-19 patients. Their study found that the costly drug did not improve patient outcomes.
As the results became apparent, he and co-principal investigator Dr. Guang-Shing Cheng discussed posting the article on a preprint server, but thought it better to wait for publication in a peer-reviewed journal. Shortly after they submitted it for publication, editors of the peer-reviewed journal reached out to them to suggest they post it first on a preprint server and to share the findings with the World Health Organization. On Aug. 11, the team posted their work on medRxiv, and their manuscript in a peer-reviewed journal is still under review.
But in the COVID-19 era, traditional routes from the lab bench to the bedside are changing.
On Aug. 27, the National Institutes of Health COVID-19 Treatment Guidelines Panel, which previously was neutral on use of tocilizumab, switched to recommend against it for treatment of the disease. It cited not a paper, but a July 28 press release from the drug’s maker, Genentech, which announced that the drug had not met its goals in a large clinical trial. Results of that trial, which have yet to be published in a peer-reviewed journal, were posted on medRxiv on Sept. 1, a week after the NIH decision.
In medical research labs around the country, scientists are participating in a dramatic change in how their work is carried out and how the results of their experiments are revealed. This movement toward open science predates COVID-19, but the mortal and economic threat of this virus has accelerated that trend among those most intimately engaged in the scientific struggle to stop it.
Whether that trend spills over to other realms of science in the U.S. or abroad remains to be seen, but should that happen, chalk it up to another way COVID-19 has changed the world.
Sabin Russell is a former staff writer at Fred Hutchinson Cancer Center. For two decades he covered medical science, global health and health care economics for the San Francisco Chronicle, and he wrote extensively about infectious diseases, including HIV/AIDS. He was a Knight Science Journalism Fellow at MIT and a freelance writer for the New York Times and Health Affairs.
Are you interested in reprinting or republishing this story? Be our guest! We want to help connect people with the information they need. We just ask that you link back to the original article, preserve the author’s byline and refrain from making edits that alter the original context. Questions? Email us at communications@fredhutch.org
Are you interested in reprinting or republishing this story? Be our guest! We want to help connect people with the information they need. We just ask that you link back to the original article, preserve the author’s byline and refrain from making edits that alter the original context. Questions? Email us at communications@fredhutch.org
Every dollar counts. Please support lifesaving research today.
For the Media