Last month, residents of Santa Clara County, California, stuck at home and newly reacquainted with the Facebook scroll, may have noticed an unusual proposition pop up in their feed: a targeted ad from researchers at Stanford’s medical school offering blood antibody tests for Covid-19. The curious who clicked through were offered a plan to arrive at a parking lot somewhere in the county, where a volunteer would give them a quick finger prick through their car window.
So-called serological tests work differently from Covid-19 diagnostic tests, which require a nose or throat swab and look for viral RNA. Instead, they check a person’s blood for evidence of an immune response to the virus, which can be found even in people with no symptoms or those who have already recovered from the disease. Researchers have eagerly awaited those kinds of tests to gain insight into the true infection and fatality rates across age groups, and to help answer questions about things like asymptomatic spread and how long antibodies last—a key part of understanding how Covid-19 immunity might work. In early April, more than 3,000 people took the Stanford researchers up on their offer.
The results, posted Friday by the Stanford researchers as a preprint, haven’t been peer-reviewed. But they have gotten a lot of attention. And they’ve quickly become emblematic of this age of rapid-fire scientific communications: Surprising results are widely shared before they’re published in an academic journal, followed by an attempt at peer review by Twitter thread.
First, the results: The Stanford researchers calculated that between 2.5 percent and 4.2 percent of the county’s residents were infected as of early April. That sounds like a reasonably small number, but if true, it would mean Covid-19 is drastically more widespread than local swab testing suggests: 50- to 85-fold, the researchers calculated. The math that follows from there is even more significant. Assuming a higher infection rate consequently lowers the disease’s estimated fatality rate, driving it from around 1 percent to just 0.12 to 0.2 percent. For the record, the death rate from the flu is about 0.1 percent.
“The comparison with the flu can be polarizing. I hope that’s not the headline,” Eran Bendavid, an infectious disease professor at Stanford and a coleader of the study, said at a press conference Friday, where he stressed that the seriousness of the pandemic shouldn’t be understated. Of course, for many—especially those who think the pandemic is overblown—that was the headline. The comparison even made it into a Wall Street Journal editorial published Friday by Andrew Bogan, a local biotech investor and one of the coauthors of the study (though he wasn’t identified as such in the editorial). Had policy officials held a comparison with influenza in mind, “would they have risked tens of millions of jobs and livelihoods?” he wondered.
Then on Monday, more fuel was added to the fire. A group at the University of Southern California, sharing a subset of authors with the Stanford group, as well as the same tests and similar methodology, released the conclusions of a serosurvey of residents in Los Angeles County. The exact figures were different—they calculated 28 to 55 times more cases than previously caught via swab testing—but the gist of their conclusions was the same.
Skeptics have noted that the conclusions seem at odds with some basic math. In New York City, where more than 10,000 people, or about 0.1 percent of the population, have already died from Covid-19, this estimated fatality rate would mean nearly everyone in the city has already been infected. That’s unlikely, since the number of new cases, and deaths, is still mounting, fast. Others pointed to the Stanford group’s unusual use of Facebook recruitment, which may have drawn in people who were sick in February or March and couldn’t get a swab test to confirm it—the situation for a lot of people in Santa Clara County, an early Covid-19 hot zone where tests were initially scarce. That could have led to an oversampling of people who had antibodies to the virus. Others, noting that only 50 out of the 3,330 people tested, or 1.5 percent, actually tested positive, quibbled with the methods used to weight the sample, which skewed heavily white and female. (The USC researchers say their sample is more representative, albeit smaller, and was chosen by gathering participants through a market research firm, not Facebook.)
All of that might be moot if the testing devices used in both surveys turn out to be flawed. The results come amid widespread concerns about the accuracy of blood antibody tests—especially the rapid lateral flow tests like the ones used in this study. The Stanford preprint referred to a test from Premier Biotech, based in Minneapolis, but that company is only a distributor. The firm that makes the test, Hangzhou Biotest Biotech, was previously identified by NBC as among those recently banned from exporting Covid-19 tests because its product hasn’t been vetted by China’s equivalent of the FDA. A representative for Premier Biotech confirmed to WIRED that the same test was used by the Stanford and USC researchers. (On Monday, a USC spokesperson emailed WIRED a statement from Neeraj Sood, the lead researcher, acknowledging the test’s origins and noting they were exported legally, prior to the ban.)
At Stanford, the researchers performed their own validation of the tests and found only false negatives, not false positives. But, as a chorus of statisticians have noted (including one who concluded his analysis by demanding an apology from the researchers for wasting everyone’s time), even a low false positive rate would wipe out the significance of their results. Representatives for Stanford declined to make the researchers available for follow-up questions, including whether they planned to retest the positive individuals.
Reached by email, Sood wrote that his USC team did not plan to confirm the positive results with additional blood tests, but had offered genetic swab tests to those who had tested positive and might have current infections. Speaking at the press conference Monday, Sood acknowledged the results were “preliminary,” but said they fell in line with his expectations about the scale of untested infections.
The Stanford researchers said at their press conference on Friday that they’ve accounted for the potential biases in who volunteered and that they stood by their validation methods. They also pointed out that outcomes might be different in Santa Clara County than in overwhelmed New York City. “It seems to me that when the hospitals are overrun, that can make this epidemic a lot worse,” Bendavid said. The researchers do not intend to replicate the tests in Santa Clara, though they note others are conducting serosurveys in the Bay Area. In Los Angeles, they plan to do another round of tests in coming weeks.
That, at least, is one point everyone in this debate agrees on: More serosurveys are a good thing, and will ideally involve properly validated tests and representative populations. “We should be rolling out these tests to everybody,” said Martin Hibberd, an infectious disease researcher at the London School of Hygiene and Tropical Medicine.
While Hibberd called the methods used in the Santa Clara study “dubious,” he also said the group’s conclusions do likely point in the right direction—that the fatality rate will come down as more undetected cases are uncovered. That’s true of any disease outbreak. Other initial serosurvey results trickling in over recent weeks, from places like Denmark, Germany, the Netherlands, and the UK, as well as China, seem to back that up. While the rates of seroprevalence vary—from 1 percent among Scottish blood donors tested in mid-March to 15 percent in one hard-hit German town—they all generally point to lots of untested people. But it’s still a matter of how many. “It’s been guesswork really,” Hibberd added. “Just to see the first bit of data is quite exciting.”
Even assuming solid tests and methods, this first round of serosurvey results only gives us a snapshot of the disease’s progress in each place. That means uncertainties abound, Hibberd noted. Antibodies take time to develop, and people also take time to recover—or die—from the disease. We’re still at a relatively early stage of data collection, when the actual number of deaths due to Covid-19 remains uncertain. New York, for example, recently revised its death toll upward to include “likely cases” of Covid-19.
A better use for these very early results is the simple one: a rough estimate of much further we have to go in this pandemic. One hope among researchers is that we’ll get to a point of herd immunity, when enough people have antibodies to slow or stop the disease’s transmission. (That’s still just a hope, given uncertainties about whether antibodies correspond to actual immunity.) But so far, all the serosurveys show results in the single digits, which means the virus remains dangerous to most of the population.
In the Netherlands, for example, the seroprevalence figure of 3 percent released Thursday by Sanquin, the national blood bank, wasn’t exactly cause for celebration. “I hoped for a few percent more,” Hans Zaaijer, a microbiologist at Sanquin who is leading the testing effort, wrote in an email to WIRED. He noted false positives are an issue with the test they used as well. (More accurate lab-based blood tests are currently being reserved for health care workers; Sanquin’s study anonymously tests the blood of anyone who donates to the bank.) But for Zaaijer, the takeaway is clear: With only 3 percent of the population showing the antibody, many more people could still be infected by Covid-19, and herd immunity—if it exists—is a far-off vision. The plan in Holland is to keep on testing the blood bank supply once a month.
That kind of repetition will be important if we want to answer more complicated questions about the disease, according to Michael Busch, director of the Vitalant Research Institute, which is partnering with the National Institutes of Health to test antibody prevalence in blood donations in six cities across the US. Those tests will also happen monthly, which is useful not only for tracking the spread of the disease over time, but for validating results and ironing out potential biases. (Blood donation tests aim to be more representative than Facebook targeting, but they also skew toward healthy people who are eligible to give blood.)
Their method involves lab-based blood tests from Abbott and Ortho, which were validated in the past week by the FDA and are generally held to be more accurate than the finger prick tests. They will do additional validation on all results that come back positive. “That’s what we do, always,” he said. “We do repeat testing and confirmatory testing so we’re absolutely sure it is a positive for antibodies.” A separate study is also planned to track individuals with confirmed infections to see whether these people’s antibody levels wane over time, and use that data to help calibrate the population-based results. And there are plans too to expand to more cities over time to get a better sense of the national picture.
In short, it’s likely we’ll have to wait a little longer before we have clear numbers about how many people have been exposed to this virus and how many have died from it. And when making important policy decisions that affect life and livelihood, that’s a good thing, Busch said: “It’s a big deal to do it right.”
- What if it returns every year, like the common cold?
- “Here in spirit”: an oral history of faith amid the pandemic
- We need a vaccine—let’s get it right the first time
- Un-miracle drugs could help tame the pandemic
- WIRED Q&A: We are in the midst of the outbreak. Now what?
- Read all of our coronavirus coverage here
