Correcting Berenson's optimally wrong take on vaccination and hospitalization risk from JAMA study

Alex Berenson is again misinterpreting/misrepresenting scientific data, this time saying this JAMA study shows vaccines “don’t stop Covid hospitalizations and deaths” in a recent substack post

He is talking about this very nice cohort study published this week in JAMA Internal Medicine from the N3C network looking at ~665k patients given at least one dose of SARS-CoV-2 vaccine between December 2020 and September 2021 based on electronic medical record (EMR) data from a large health system network.

The focus of the study was effect of immunocompromise on breakthrough infection rate and severity. They found that indeed immunocompromise led to higher rate of breakthrough infection. Here is the key figure showing that the rate of breakthrough infections was higher in those with various immune dysfunction conditions:

They also also looked at proportion of COVID-19 patients that were outpatient, inpatient, and severe and found significantly higher proportion of immunocompromised were inpatient or severe. You can see the definitions of severity in the paper. They did this comparison both for the 17,262 breakthrough infections (1837 in immunocompromised) among the 665k vaccinated, as well as for 2,737,963 prevaccination COVID-19 cases (71,220 in immunocompromised) in the N3C network.

Here is a plot of the proportion of hospitalizations that were outpatient/inpatient/severe in each vaccination/immunocompromise subgroup:

The data corresponding to these plots are given in the supplementary table 4:

The intended purpose of this table is to show that in both vaccinated and unvaccinated COVID-19 patients, immunocompromised had 1.5x higher risk of admission (inpatient, 20.7% vs. 14.8%, and 35.1% vs. 21.1%, respectively) and 3.0x higher risk of severe disease (2.1% vs. 0.7%, and 6.0% vs. 1.9%, respectively).

But Berenson reads something else into this table. He reads into this table that the vaccines don’t “keep hospitals from being overrun” and on the basis of this, he produces his headlines "Vaccines don't stop COVID hospitalizations or deaths"

I'm not sure how he concludes that here. This table doesn’t tell you anything about the relative rate of hospitalized infections between vaccinated and unvaccinated.

It only tells you the relative distribution of outpatient/inpatient/severe in COVID-19 patients who were unvaccinated (revaccinated infections) or vaccinated (breakthrough infections).

And looking at the numbers, we see that vaccinated COVID-19 patients are 1.5-1.75x less likely than unvaccinated COVID-19 patients to require hospital admission (inpatient, 14.8% vs. 21.1% for immunocompromised, and 20.7% vs. 35.1% for non-immunocompromised ) and 2.7-3.0x less likely to have severe disease (0.7% vs. 1.9% for immunocompromised, and 2.1% vs. 6.0% for non-immunocompromised).

Note that these ratios are the same magnitude as seen for immunocompromised vs. non-immunocompromised:

• Immunocompromised COVID-19 patients had a 1.5x higher risk of admission and 3.0x higher risk of severe disease than non-immunocompromised COVID-19 patients.

• Unvaccinated COVID-19 patients had ~1.5x higher risk of admission and ~3.0x higher risk of severe disease than vaccinated COVID-19 patients.

We have to be careful to not overinterpret the comparison of pre-vaccinated vs. post-vaccinated infections, since this was not a primary comparison of the study and the pre-vaccination infections tended to occur during an earlier time period than the post-vaccination infections.

However, for the sake of argument in showing how far off Berenson's take on these data are, if these results held up causally, this would suggest the benefit of vaccination in reducing the risk of a COVID-19 patient requiring admission or having severe illness is of the same order of magnitude as curing someone's medical condition leading to immune dysfunction (if that were possible). It seems to me that these data suggest that the vaccines are doing something clinically beneficial. And this doesn't even account for the fact that a much lower proportion of post-vaccination infections end up as "patients" in a hospital at all. The data in this paper condition on hospitalization, i.e. based on the fact they come from a hospital network, the data include only patients, i.e. individuals that in fact DO end up in the hospital.

if we wanted to calculated the vaccine effectiveness vs severe disease, we’d have to compare, between vaccinated and unvaccinated, the probability of a severe hospitalized infection, which would be obtained by multiplying together 3 components:

Pr(severe hospitalized infection) = Pr(infection) x Pr(hospitalization | infection) x Pr(severe disease | hospitalized infection)

This paper only deals with the 3rd component. In numerous studies the vaccines also show some benefit for the first two components as well, so that combining all 3 components the vaccines demonstrate 75-90% effectiveness in protecting against severe infections.

So once again, Alex Berenson's take is about as wrong as wrong can be. This very nice paper that demonstrates immunocompromised individuals have higher risk of breakthrough infections, and these infections tend to be more severe. Incidentally, it also suggests the risk of a COVID-19 patient requiring admission or having severe illness is substantially lower in vaccinated than unvaccinated patients.