Thoughts on the AAFP's newest COVID-19 "Rapid Evidence Review"
These recommendations are disappointingly easy to rip apart
Yesterday, I received my April copy of American Family Physician, a peer-reviewed journal from the AAFP. Each month, the magazine summarizes clinical practice guidelines for 5-6 conditions. Now in my second year of residency, I often refer to these monthly magazines, taking notes on unfamiliar topics. This time, I was excited to see an article on COVID-19 management, a topic which I exerted myself to understand more than perhaps any other subject. October 2020 was the last time COVID-19 practice guidelines were published in American Family Physician. So, I was curious to see how recommendations changed.
Like many publications, American Family Physician uses an evidence rating system to communicate the strength of notable recommendations. The one and only guideline that earns an “A” rating is the following:
Encourage all patients to receive a COVID-19 vaccination to reduce hospitalization and death associated with the disease
They go on to assert that “numerous randomized trials and a meta-analysis” support this conclusion. An unscrupulous reader might miss that the referenced meta-analysis only refers to data collected in the first half of 2021, immediately after the first vaccine was released. During this time, the efficacy of vaccination was blatant. Vaccination likely saved millions of lives. But the authors go ahead and cite this source while also claiming that vaccination “continues to be effective at preventing severe illness, hospitalization, and death”.
The authors apparently recognize the need for newer evidence because they cite an MMWR study that claims a mind-blowing “90% reduction in risk for COVID-19-associated invasive mechanical ventilation or death” with 2-3 vaccines in January 2022. But the quality of this evidence is poor. It is observational data, which is not replicated by any randomized controlled trials. No trials are cited to provide confidence that in 2023 vaccination achieves any clinically relevant endpoint.
The authors further erode the validity of their universal vaccination recommendation by correctly admitting that “the clinical effectiveness of the bivalent mRNA vaccine is not yet known”. No attempt is made to help readers make sense of the FDA’s approval of a 4th, 5th, 6th, or an imminent 7th vaccine. I wonder if the authors find it odd that randomized data was rapidly produced with the first vaccines, never again to be replicated. And I wonder if the authors find it relevant that the CDC has demonstrated that past infection provides stronger protection than vaccination, given most people are now seropositive. And lastly, I wonder why the authors didn’t seem to find it worthwhile to discuss the potential harms associated with COVID vaccination. There is a broad perception in family medicine (at least where I practice) that these vaccines are harmless. Yet, multiple studies have demonstrated myocarditis risk, particularly among men under 40 years old. The risk seems as high as 1:3000-1:7000 per vaccination. Meanwhile, there seems to be no meaningful benefit in boosting individuals younger than 40 years old. Given the questionable risk:benefit of boosting, many European countries have restricted mRNA vaccines among individuals younger than 30-65yo.
When patients ask me about the COVID-19 vaccine, they want to know if continued boosting produces a net health benefit. They want to know if it is similar to the flu shot. And they want my honest opinion. In the effort to empower physicians to have shared decision-making conversations, perhaps the authors could have also mentioned that:
COVID vaccination protects against strains that have passed, it does not predict future strains like the flu shot.
Vaccination and boosting do not reliably prevent COVID infection.
COVID is now equally or less lethal than the flu for most adults. This has always been the case for children.
COVID vaccination causes more fever and days of work missed than flu shots.
mRNA is a novel platform. Even though the FDA is approving booster after booster, unknown and incompletely understood risks may outweigh any possible benefits.
Rather than mention any of the nuances, the authors barrel ahead with even weaker recommendations…
They speak positively of mask mandates in the community and in schools, again citing two observational studies. They claim these recommendations are current as of March, 6, 2023. Though, I have a hard time believing they missed the Cochrane meta-analysis on masking published on January 30th. It produced a lot of controversy and headlines. In case you missed it, Cochrane concluded that community mask mandates “probably [make] little to no difference in how many people …catch a flu-like illness, or respiratory illness”. 11 RCTs were cited. Meanwhile, hospitals around the country have begun to drop their own mandates.
Lastly, I will mention the recommendation to “consider drug therapy for all patients diagnosed with COVID-19”. Another mind-blowing universal recommendation. In this case, at least, they acknowledge weak evidence (C rating). They offer nirmatrelvir/ritonavir (Paxlovid), injectable remdesivir, and molnupiravir as first-line treatments. But nirmatrelvir/ritonavir is the only one I have seen prescribed. In support of this therapy, they cite the EPIC-HR study. This trial was conducted July 2021 - December 2021, when the delta variant predominated. The number needed to treat was 17 to prevent hospitalization or death. However, this trial excluded people who were already vaccinated or had a prior COVID-19 infection. In 2023, this trial is irrelevant, given most people have some form of immunity and given the low morbidity of the Omicron variant. To date, no RCT has replicated any benefit of nirmatrelvir/ritonavir.
This American Family Physician article is notable because the authors have assumed the responsibility of interpreting data on behalf of all family physicians, the most populous medical specialty. Many of my colleagues that read this publication will likely take it at face value.
It seems the current medical consensus on COVID-19 is based on bad science. I only began to recognize this when a Stanford professor privately disclosed to me in 2022 that he feared retaliation if he were to publicly express his thoughts on COVID interventions. The United States stands out on a global stage for endorsing aggressive and universal vaccination, masking, and treatment. This deviation should raise a red flag for physicians in the United States. At the very least, we owe patients honest conversations about what is known and unknown about these interventions.
Where do we go from here? I think physicians need to decide whether to be scientists or guideline-followers. If we indeed believe ourselves to be scientists or interpreters of science, we ought to make an effort to (1) understand what science even is and (2) why randomized controlled trials are a superior investigational method. These linked podcast episodes from Plenary Session are essential listening, whether you find yourself working in a community health clinic or the CDC.