What does "95% effective" mean when it comes to the new vaccines being produced to combat the novel coronavirus? NCSE Executive Director Ann Reid sheds some light.
Photo by Christian Wiediger on Unsplash.
Check out our entire series explaining the science involved in the coronavirus pandemic. Sign up to receive our coronavirus update each week.
If you ever want to start a lively discussion in your science classroom—or even around your kitchen table—you could do worse than to pose this question: “What does it mean when the weather forecast says that there’s a 50% chance of rain?” That it will rain half the day? That it will rain in half the area? Nope. It turns out that there is an official definition (charmingly, the official term is Probability of Precipitation, or PoP), according to which it means that there’s a 50% chance that it will rain (measurably: 0.01 inches or more) somewhere in the area.
But believe you me, that’s not the end of the story. It turns out that the chance that your local weather forecaster is using that precise definition is lower than the chance that Punxsutawny Phil can predict the end of winter. You don’t have to take my word for it—there’s actual science about it. This study found that meteorologists understood PoP in five different ways. Unsurprisingly, only 8% of them believed that the definition has been used consistently. And on average, the meteorologists felt that only about 23% of the public understands what PoP means. The paper’s abstract ends with this confidence-inspiring sentence: “until meteorologists develop a consistent conception of the PoP and disseminate it, the public’s understanding of PoP-based forecasts may remain fuzzy.”
What does this have to do with coronavirus vaccine efficacy? Good news! In distinct contrast to weather forecasts, calculations of vaccine effectiveness are pretty straightforward and easy to understand. When you read that a vaccine has proven to be 95% effective, it means pretty much what you might guess—it’s really good at keeping people from getting sick. So, as we read the wonderful news about clinical trials showing various vaccines to be 70%, 90%, or even 95% effective, we can allow ourselves some optimism that vaccines will be able to make a huge contribution to getting life back to normal.
Not that there’s not some room for confusion. In fact, you might ask your class to come up with some possible interpretations. For example, if you receive a 95% effective vaccine, does it mean that you have a 5% chance of contracting the virus? (No.) If you receive a 95% effective vaccine, does it mean you will get 95% less sick than if you didn’t get the vaccine? (No again.) Neither of these interpretations is correct because that 95% effectiveness number isn’t meant to reflect your personal risk upon getting the vaccine. It simply reflects what happened when tens of thousands of people were given either the vaccine or a placebo.
Here’s the math (from this study of a vaccine against pneumococcal disease):
That may look a little intimidating at first glance but if you plug in the reported numbers from the recent coronavirus vaccine trials, you’ll see that it’s pretty straightforward. For example, in the trial of Moderna’s vaccine, there were 30,000 patients, half of which received two doses of the vaccine and half a placebo. When the initial results were reviewed (by an independent group of reviewers, who were the only people who were able to “unblind” the data—in other words, to see which participants got the vaccine and which got the placebo), it turned out that there had been 95 cases of COVID-19, and 90 of those cases had occurred in the placebo group.
So here’s the equation with those numbers plugged in:
That’s it. Easy-peasy, right?
So you can see that the effectiveness number really doesn’t have anything to do with how well the vaccine works in an individual; it’s a measure of how well the vaccine protects the population. But when it comes to infectious diseases, that metric makes sense; if the number of cases is reduced by 95%, the human and social costs of the pandemic are vastly reduced. Especially if the vaccine not only reduces symptomatic illness by 95% but also reduces infection and transmission rates. But that is something these particular trials did not measure.
Here’s the distinction: the “endpoints” used in these trials were symptomatic cases. If anyone in the trial developed a fever, cough, or any other symptoms of COVID-19, they were tested for the virus. As the results show, very few people in the vaccinated arm of the trial (see what I did there?) tested positive. Furthermore, none of the cases in vaccinated individuals was severe, suggesting that the vaccine might shut the virus down quickly. What we don’t know is whether some, or even many, of the vaccinated participants developed asymptomatic infections, possibly generating high enough viral loads to transmit the virus to others.
Whether vaccinated individuals can still spread the virus will be a subject of intense scrutiny as vaccination efforts get underway. It will take many, many months to vaccinate the whole population, and if vaccinated people can still infect others, it will take longer to get the pandemic under control. Most experts, however, seem to think it likely that vaccination will at the very least reduce the odds of transmission. For more on the actual vaccines under development, the differences between them, and how they will be distributed, you might want to have your students check out the December 3, 2020, NovaNow podcast, “Covid vaccines are coming: What’s inside, and how and when you’ll get one.”
It certainly looks like the vaccines from Moderna and Pfizer will play a huge role in blowing away the huge, dark clouds that we’ve all been living under since March 2020. However, it’s not clear whether they’ll mean sunny skies and a 0% chance of rain. More likely, there will be some, not very precisely known, continued risk of spread of the virus until some 60–70% of the population is vaccinated. So, until we know for sure what will be the “Probability of Passing” it on, however, getting vaccinated will not be a “get out of social distancing free” card. Just as with weather forecasts, we’ll have to live with some uncertainty and not throw away our umbrellas and galoshes just yet.
