are covid vaccines working? (take 2, corrected data)

a look at the UK society level data for delta variant

this is an updated version of the piece i posted earlier. the prior piece had a fundamental error in the calculations that led to inaccurate takes on vaccine efficacy. the picture is a bit different than i previously laid out.

mea culpa, my bad. i got it wrong, so please re-read and get what i believe to be the correct picture.

key variations: i did not adjust for risk cohort comparison correctly.

this understated VE for medical care and death prevention.

however, it also understated the extent of negative VE for cases. i now believe the higher case count in vaccinated to be material.

apologies for the confusion.

cliff notes:

• UK data on delta and vaccines is one of the best datasets available

• cutting it to 2 week intervals (from one report to the next) lets us normalize for vaccination rates

• this lets us look at vaccine efficacy in mitigating cases, ER visits, overnight hospitalization, and deaths

• the results make vaccines look far less effective than has been claimed.

• mitigation is negative in terms of cases, around -16%

• mitigation of need for med care/death is 23-52% in under 50’s

• mitigation of need for med care/death is 52-59% in over 50’s

• these outcomes provide poor pretext for mandates and vaccine passports, but likely make vaccines an effective mitigation for risk groups where these gains outweigh vaccine risk.

discussion:

whether and to what extent vaccines are stopping spread of covid and/or mitigating severe covid and death has been a (justifiably) hot topic of late. the data is difficult to use, normalize, and risk adjust. it would have been great to still have control groups in the actual randomized controlled studies from Pfizer etc, but we do not. all those control groups have long since been vaccinated. this likely means that we will never be able to get real RCT data here. this will be a deeply limiting issue to getting a strong understanding, but we must do the best we can with what we have.

the UK has been providing some of the better data here and has been doing so fortnightly in their variants of concern technical briefings. all can be found HERE and all data in this analysis comes from briefings 21 and 22.

one of the challenges in using this data is normalizing for vaccination levels. the UK report aggregate data from 1 Feb 2021 until present and vaccination rates varied massively over that period as can be seen below. this makes analyzing that aggregate data essentially impossible as there is no way to normalize for “vaccinated person days” and map to counts.

in order to mitigate this issue, i have chosen to look only at the most recent two week period and have generated a dataset by taking the delta variant counts from technical briefing 22 (aug 29) and subtracted from them the counts of technical briefing 21 (aug 15). presuming there is not too much backfill and catchup being played (and given how assiduously this is reported, this seems a reasonable assumption) this should give us pretty good insight into what has happened over the last 2 weeks of available data.

it also means that the % vaccinated line looks like this and that we can, therefore, get a good estimate on cohort size. note that i have moved here to “at least one dose.” let me explain why.

choosing to use only “fully vaccinated” counts with people 7 or 14 days post dose 2 strikes me as a dishonest accounting. there is a fair bit of evidence pointing to negative protection for vaccines post dose 1 before full immune gain a week or 2 after dose 2. whether this is neutropenia or some other issue remains an open question, but a danish study showed -100% VE in HCW’s in the 2 weeks post dose one and numerous others have shown similar effects.

this seems to me to be an issue that must be included in “vaccinated” and calling it unvaccinated seems akin to saying “you’ll be safe from enemy bullets if you get into the bunker” but ignoring that you must run across a field of concentrated fire to get there.

no one gets to “fully vaxxed” without passing though “partially vaxxed.” thus, i see no other fair accounting than to include partially vaxxed in the vaccinated counts. fortunately, unlike most places, the UK provides this data and so i was able to perform this aggregation.

before we wade into this data, a couple of additional caveats:

this is NOT RCT data. the cohorts are not risk balanced or randomized. this may inject all manner of systemic bias into the data. perhaps those who do not get vaccinated tend to be healthier or more likely to have already had covid. perhaps they include, especially in older groups, the VERY sick who were simply too high risk to vaccinate. under 50 contains a large number of children none of whom were vaccinated and this will skew numbers.

the 70% societal average is polyglot by age group. i will use 90% for over 50’s and 50% for under 50’s (as the under 16’s are zero). these are approximations for now but look close. i hope to tighten this in later analyses.

OK, so there are the caveats. clearly, we have some error bars here, but OTOH if vaccines truly show the 90%+ efficacy claimed, they still ought to really pop in terms of societal impact regardless.

unfortunately, it does not appear that they do. efficacy seems present, but muted.

let’s look:

data from UK is split into 4 buckets.

• <21 days post dose 1

• >21 days post dose 1

• >14 days post dose 2

• unvaccinated

for the reasons described above, i aggregated the first 3 groups. that aggregate can then be compared to the unvaccinated (control) arm. data is split by age group over and under 50.

to avoid simpson’s paradoxes, i’m not going to aggregate this data by age group. i fear that can be misleading and (unfairly) make vaccines look like they are accelerating disease.

recall that the bogey for <50’s is about 50/50 (based on 50% vaccinated estimate) for “no effect.” to the extent that vaccinated is below 50% and unvaccinated exceed it, we see vaccine effectiveness (VE)

likewise, this bogey is 90/10 for over 50’s. to the extent that vaccinated is below 90% and unvaccinated exceed it, we see vaccine effectiveness (VE)

we can risk adjust the populations by size in this way.

based on this, we see VE emerge in terms of risk of severe events, but we see meaningful additional case risk emerge as well.

on cases, we see no protection at all.

in both cases, VE is double digit negative. so, we can say “vaccines show no evidence of protection vs cases” with confidence. whether they intensify risk (possible) or this is risk stratification would take more granular data to assess.

on emergency care, the data is consistent with protection though the benefit to the young is not terribly high.

• VE is 23% for under 50’s

• VE is 53% for over 50’s

for overnight hospital stay, the data is consistent with protection, this is high enough to be more meaningful even for the young.

• VE is 40% for under 50’s

• VE is 56% for over 50’s

for deaths, the data is consistent with protection, and is high enough to be more meaningful in both cases

• VE is 52% for under 50’s

• VE is 59% for over 50’s

overall, this paints a mixed picture for VE assessment.

there is some apparent protection from severe outcomes, but not from cases.

vaccines are clearly non-sterilizing and may worsen spread.

this would seem to imply that they offer personal, but not societal protection

if we pile this all together, we see significant amplification in overall cases over this 2 week period driven by vaccinated arm.

• we see perhaps a 1114 patient reduction in ER visits.

• we see 495 patient reduction in overnight hospital admission

this is in a county of of 67 million people with about 170,000 hospital beds. to describe these effects as a non material reduction of strain on the hospital system seems an understatement. you could up this effect by 10X and still not make a meaningful difference. (about 3% of hospital beds)

i think we all really wanted to see vaccines work as advertised. this implies that they do seem to be working in terms of personal protection (though at 50-55%, not the 97-99 promised) but it’s looking less and less like that are anything like the mitigation that was promised to stop the spread. they may well be making it worse.

this seems to invalidate the “your vaxx protects me” argument and perhaps invert it.

they have a far more aggressive and dangerous side effect profile than any other vaccine in widespread use. measles mumps rubella (MMR) is a literal wonder of modern medicine. it has saved a staggering number of lives, is incredibly safe, and amazingly effective. it also provides sterilizing immunity and thus contributes to herd immunity.

it’s clear the covid vaccines do not. they are extremely “leaky” and those who are vaccinated clearly can and do get the virus, contract symptomatic covid, act as carriers, and spread disease. they also appear to be rapidly fading in terms of efficacy there and this data makes me suspect that this is true in terms of severity mitigation as well (or that the original studies on efficacy were rigged or salted). frankly, both seem like strong cases can be made.

framing this debate as some sort of anti-vaxxer crusade is both disingenuous and unhelpful. this is mRNA, not MMR. it’s a false equivalence. you might as well argue that because some one will not take fentanyl for a hangover that they are anti-medicine because advil was shown to work well.

all medicine is a cost/benefit analysis. you balance risk vs reward. the reward here is looking far lower than promised and the risk looks much higher than other vaccines and will multiply every time you need a booster.

these vaccines do not look to stop spread and may be causing nasty adverse selection in virus evolution as some other leaky vaccines have.

there is no case, either moral or medical to mandate them and push one size fits few coercive medicine.

do these vaccines make sense for some? yes, they likely do. perhaps even many.

are they bad risk reward for others? yes, they seem to be.

do they stop overall spread or spare hospitals meaningfully? it does not look like it, though perhaps if cases were 100X this, they would.

this should be a personal choice, not a governmental diktat.

i also note that this data (which aligns with other high vaccine uptake places like israel and scotland) completely fails to line up with US data and claims about an “epidemic of the unvaxxed” and 99% of hospitalizations being unvaccinated etc. honestly, i think the US data on that is being used selectively, manipulated, and is generally of such poor quality and integrity as to be useless. there is just no way that 80% of ER and hospital overnights in the UK are vaccinated and that it’s 1% in the US. and the UK is clearly better and more honest data.

i caution against accepting US data on vaccinated hospitalizations etc at anything like face value as pretty much no place else looks like that. it does not pass the smell test.

i will try (as time permits) to expand this analysis and look at other past periods to see if they look similar and perhaps (tantalizingly) if we can find evidence of vaccine fade.

stay tuned.