COVID Vaccines: Efficacy vs Effectiveness
On Friday afternoon I got my second dose of the Moderna COVID vaccine. The rest of Friday I was totally fine, but then I had an awful night’s sleep as I alternated between feeling feverish and feeling chilled. I woke up with a 100-degree fever and spent most of yesterday pretty blah. I was lethargic, feverish, and didn’t have much of an appetite. (My main source of sustenance was pretzels.) I finally took some ibuprofen around 7pm, and once that kicked in I was feeling MUCH better. I had a great night’s sleep and woke up feeling refreshed and back to normal. Now I just have to wait two weeks, and I’ll be fully protected.
So yes, the horror stories of the second dose being worse than the first are definitely true, albeit maybe exaggerated. After the first dose my only side effect was a very sore arm. (Interestingly, this time around I barely had any soreness.) The second dose does pack a punch, and I’ve heard the side effects are worse for women and younger people. Still, while it was far from pleasant, it wasn’t exactly bad. And as one of my coworkers pointed out, it was nice knowing why she was feeling sick. It was a predictable response to the vaccine, and one that passed fairly quickly.
The nurse who administered my second shot told me that other patients had heard the strangest misinformation about the second dose: that the concentration was stronger, or that the needle was bigger. “People need to get off the internet,” she said, which is good advice in general, not just about vaccines. I had also seen examples of right-wing misinformation about vaccines thanks to the one conservative person I follow on Twitter: anything from “why should we get vaccinated if we still have to wear masks in public?” to “95% efficacy isn’t the same as 95% effectiveness.” I want to address both of these points today, because it’s important to clarify what these terms mean, and why everyone should get the vaccine.
Let’s start with the second argument first: “95% efficacy isn’t the same as 95% effectiveness.” The point this person was trying to make is that the vaccine isn’t really as effective as we think it is, therefore it’s not worth getting. Here’s a link to a super helpful article from The New York Times that helped me understand this concept better. Very broadly, efficacy is a measurement made during a clinical trial, while effectiveness is how well the vaccine works in the real world. Thus, one important difference between efficacy and effectiveness is the way in which each measurement is captured: the former in a controlled setting (a clinical trial), the latter in a much less controlled setting (the real world). Another important difference is timing: we can capture a vaccine’s efficacy during the span of a clinical trial, but we won’t know a vaccine’s effectiveness until much later, once more and more people become vaccinated. Thus, we can measure efficacy in a finite amount of time, but effectiveness is an ongoing measurement.
So, how exactly do scientists calculate efficacy? This is kind of confusing and not super intuitive. 95% efficacy does not mean that 95 out of 100 people will be protected. Instead, we have to think about this in terms of the clinical trial itself. Every clinical trial takes a large group of people and divides them roughly in half: Group A is the experimental group (meaning they’re the ones who get the vaccine) and Group B is the control group (meaning they get a placebo). In the case of Pfizer’s clinical trial, they had 43,661 participants. This means there were roughly 21,830 people in each group. Thus, somewhere around 21,800 people received the Pfizer vaccine and 21,800 did not. Then the researchers waited to see who tested positive for COVID-19. In the Pfizer study, 170 people (out of the total 43,661 participants) received a positive test result. Out of those 170 people, 162 of them were from Group B — the placebo, non-vaccinated group — while only 8 of the vaccinated people tested positive.
As you can see, both groups had pretty low numbers of positive test results (keeping in mind that anyone who was asymptomatic wouldn’t take a COVID test), but the number of people who did test positive were overwhelmingly in the unvaccinated group. In fact, of the 170 people to test positive in the study, only 5% of them were vaccinated. (If you’re wondering how I got this number, just take the number of vaccinated people who received a positive test result [8] and divide that by the total number of people who received a positive test result [170], then multiply by 100. The actual number is 4.7%, but I rounded up.) Thus, Pfizer’s vaccine has a 95% efficacy rate.
(Note: This is a rather crude way of arriving at the efficacy rate. While the numbers match up, the actual technical way to calculate efficacy has to do with comparing two ratios: the number of unvaccinated people who tested positive and the number of vaccinated people who tested positive. The difference between those two fractions is 95%, which is the same as we found above, but since I don’t have exact numbers from the Pfizer trial I can’t replicate their math.)
So, to recap: 95% efficacy does not mean 95 out of 100 people will be protected, simply that, out of everyone in Pfizer’s trial who did test positive, only 5% of them were vaccinated. More importantly, it also doesn’t mean that 5% of Group A (the vaccinated group) tested positive, since that would have meant close to 1,090 infections. This is all really good news, because assuming there were about 21,800 people who were vaccinated, 8 out of 21,800 means that the vaccine’s effectiveness in the trial (note the intentional use of “effectiveness” here) is 99.96%! That’s amazing!
Now, sure, it’s very possible that the vaccine’s effectiveness in the real world won’t be 99.96%. Partly this due to the fact that many million more people are receiving a vaccine compared to Pfizer’s small group of 43,661. More people means more chances that people will get infected, thus potentially dropping the effectiveness. Also, people in clinical studies are often healthy and don’t have any other chronic health problems. This is of course not true in the real world. Still, there’s no reason to think that the vaccine’s effectiveness will plummet once we apply it outside of a controlled environment. In fact, we very well could see that the vaccine’s effectiveness nearly matches (if not surpasses) the 95% efficacy rate. This is why it’s important to correct misinformation when people claim that 95% efficacy does not mean 95% effectiveness. While that is true, the good news is that the effectiveness very well could exceed 95%, which we’re already seeing. (Stay tuned!)
I can hear some of you thinking: “Okay, Brian, that all sounds well and good. But these trials were conducted earlier in 2020, back before the new variants. Are the vaccines effective at all against these newer, stronger strains?” That’s a great question! Here’s another article from The New York Times that helped answer that one. Basically, the takeaway here is that vaccines have different rates of success against different variants, but they are still highly effective, especially at preventing serious illness or hospitalization. Thus, even if these newer strains cause more infections (even among the vaccinated), the likelihood of that infection becoming life-threatening is greatly reduced in those who have been vaccinated.
These instances of a vaccinated person becoming infected are known as breakthrough cases. Pfizer and Moderna are both still studying the number of breakthrough cases among trial participants, so we don’t have exact numbers aside from what we know from their clinical trials. That said, we have enough data to know that they are very rare, even as variants spread across the country. One particularly useful group to track regarding breakthrough cases are vaccinated health care workers since they have a much higher risk of exposure to the virus. According to that New York Times article I linked to above, there are two studies that have been conducted, both published in The New England Journal of Medicine. The first was at the University of Texas Southwestern Medical Center in Dallas. Out of 8,121 fully vaccinated employees, only four were infected. And at UC San Diego Health and the David Geffen School of Medicine at the University of California in Los Angeles, out of 14,990 workers, only seven were infected. These numbers are incredibly impressive at showing the real-world effectiveness of these vaccines. Not to put too fine a point on it, but in both of these studies the vaccine effectiveness was 99.95%. We’re talking a one one-hundredth of a percentage point difference between the clinical trial effectiveness and the real-world effectiveness. This is incredible.
But I’m not done yet. The most recent CDC report (using data as of April 20, 2021) found that out of 87 million Americans who had been fully vaccinated, there were only 7,157 breakthrough cases, of which there were only 88 deaths. That means the vaccine was 99.99% effective for 87 million people. This is an example where the effectiveness of the vaccine might very well surpass the efficacy calculated during the clinical trial. All of which is to say, based off of the data we have so far, it looks like our current vaccines are quite good at preventing infection, even with the newer strains. I definitely think it’s worth continuing to monitor these numbers to make sure there’s no surge of breakthrough cases, but all signs are looking good. And remember, even if infections go up among the vaccinated, your chance of getting a severe case of COVID is minimal.
Here again from the vaccine skeptics: “But Brian, why do we still have to wear masks in public even after being vaccinated? If I still have to wear a mask, why even bother getting a vaccine in the first place?”
These are good questions! The reason why you should continue to wear a mask in public even after getting vaccinated is two-fold: 1) because you are not 100% protected from getting infected, and 2) because we still don’t know how much a vaccinated person can infect unvaccinated people. Thus, mask-wearing among a large group of people is still important, not just for yourself, but mostly for the people around you. This is why everyone who’s tired of wearing masks should be clamoring to get a vaccine. The more people who get vaccinated, the sooner we can stop wearing masks. Because that’s the thing: Yes, you might have to wear a mask in public for now, but this is a temporary measure until enough people have been vaccinated so that public mask-wearing will no longer be necessary.
As for your second question, I hope the past 1,800 words have helped to convince you that a day or two of mild side effects are well worth it for the protection that comes from the vaccine. And remember: Once you’re two weeks out from your second dose, you’re fully protected. That means no more masks if you’re hanging out with other vaccinated people. So please, if you can, get vaccinated. Do it for yourself. Do it for the benefit of others. And, as we’re seeing in Michigan, where more and more young people are getting hospitalized, do it for the health care workers who have had an incredibly long and hard year. They deserve a break. They’ve saved so many lives. This is the least we can do to pay them back.
