purr review: wesselink study on covid vaccine effects on fertility

fact checking the fact checkers

THIS STUDY has been making the rounds and has been cited by a number of people as evidence that covid vaccines are not inhibiting pregnancy.

this is certainly the claim made by the abstract of the study that viki, “Immunologist working on pregnancy at @ImperialCollege equality and diversity in STEM | she/her” cites and the methodology was interesting and seemed to separate pregnancy rates by vaccine status by using data from a self reporting study. i’m normally a bit suspect of self-reporting, but for something like “am/am not jabbed” and “had menstrual cycle” and “got pregnant” i suspect we’re on pretty safe ground. nothing terribly subjective there.

the base claims are these:

as they ran counter to a lot of society scale data, this seemed interesting so i dug in a bit hoping that we could perhaps put this issue to bed and all sleep a bit easier.

unfortunately,

it looks to me like either there were some very significant mathematic errors made with this study or

the entire conclusion set of the study comes from “adjusting” raw data in a fashion that outright inverts the signal.

let’s take a look at table 2 that shows the data and results.

they define FR (fecundity ratio) as

Fecundability is the per-cycle probability of conception. FRs >1 indicate an exposure associated with improved fecundability (or shorter time to pregnancy), whereas FRs <1 indicate an exposure associated with reduced fecundability (or longer time to pregnancy)

ok, this seems both straightforward and sound.

but then we run into what looks to be some trouble.

FR for females for second dose is described as 1.05. this would imply that double jabbed women would be 5% more likely to conceive per cycle than the unjabbed. but simple algebra shows that this is not the case. they are, in fact, 5.3% LESS likely to conceive. by my math, unadjusted FR should be 0.947

the rest of the adjustments seem suspect (and somewhat opaque) and are likely based off “unadjusted” anyhow, so let’s set them aside for the time being and just focus on “unadjusted” which ought to be the raw data.

there is no group in which post vaxx is more likely to conceive than pre in the raw data. (note that 1 dose appears to be anyone with one or more doses and 2 dose means anyone with 2 or more, so 1 dose includes all of 2 dose which is why it always has more cycles counted.)

none of this data seems to align with their own conclusions or “unadjusted” FR ratios. having had covid as a male seems to actually increase the rate of conception. (perhaps because they did not vaccinate?)

obviously, this would seem to pose a significant problem for their claims as unless i’m misreading something, it looks like they got their "unadjusted" signal backwards.

the cohorts are well balanced by age and race and the unvaxxed are actually fatter and more likely to smoke, so the idea that we need large "adjustments" that would invert this raw data from, say -5.3% to +5% looks fairly implausible and trusting studies whose whole signal is "adjustment" is always fraught.

i’m just not seeing how to get to the key abstract conclusions from this raw data without using some form to highly impactful adjustment.

either, the “unadjusted” figures have some sort of serious math error (unlikely) or they are not, in fact, unadjusted and have been modified using some sort of propensity or balancing model whose effects are so pronounced as to take data that says X and make it say “not X” which is pretty much the definition of a study you cannot trust.

it’s very hard to take initial imbalances out of a study after the fact.

if your initial cohorts are so unbalanced (which does not seem to be the case here) that they would require “adjustment” that full on reverses the raw results, then your study was already doomed. it’s very likely junk and really just expressing the base assumptions used to bend it 180 degrees.

so, i suppose one could try to claim that this study is just non-useful because the two groups were not similar enough to be well compared and it was too underpowered to keep confidence intervals from crossing unity, but honestly, it looks to me like the cohorts here are well enough balanced to give a decent raw comparison and when we do so, the outcome seems to align with other issues we have studied.

(it makes me wonder about the confidence intervals as well)

in search of clarification, i reached out to some folks who had been sharing the study.

the answer that i received from viki male, who was kind enough to respond to my inquiry is that these FR’s are, in fact, adjusted using a form of cox regression called gill-andersen and are not, in fact, “unadjusted.”

one needs to be very careful about applying such techniques as they are highly situational and what works well for one kind of data will become a GIGO engine with others.

this is GA:

the challenge here is that this sort of model to “adjust” for left truncation is extremely sensitive to assumptions baked into the proportional hazards assumption of the underlying cox math and it appears to me that this sort of model is poorly suited to analyze this situation.

(survival in this case can mean anything that implies “continuing in the study” such as “not get getting pregnant.”)

the issue here is that the effects from vaccination on fertility (as clearly seen in the israeli data on sperm motility) are neither uniform over time nor additive but rather seem to affects a small group strongly and others little or transiently.

this is exactly what you do NOT want if a cox regression is to have validity and in a case like this one, it seems poised to inject an inversion bias into the data by taking the success stories and projecting them forward.

this would seem to be how they are getting to these FR’s that fail to align with the raw data and given this adjustment modality why i think the raw data of simple cycles of exposure vs births is actually going to be the more accurate look.

(and also why the rest of the “adjustments” cannot be relied upon as the base data to which they were applied was modified by cox/GA)

when one looks at the raw data alone:

provocatively, this drop in male and female conception compounds to 0.921 X 0.948 = 0.85 = 15% drop in expected births

this aligns with pretty good gross precision to the drops seen in many countries, esp if we adjust for <100% vaxx rates.

all these are showing drops around 8-11% (SOURCE) which given inherent signal noise, drop timing, and vaxx rates in the 70’s maybe 80’s for cohort, this aligns about as well as one could expect in such a polyglot analysis.

something very odd has been going on in global natality (like 4 std dev odd) and contrary to the seeming headline, the raw data from this study that actually compares vaxxed to unvaxxed on a per cycle basis aligns with it quite well (and refutes its own abstract).

maybe i’m missing something here, but this looks to me like this data corresponds quite well to exactly the drops we’re seeing and using to to claim no drop seems to lack foundation and the this study is reaching its conclusions entirely by rejecting that raw data and simple approach and transforming it via proportional hazards model’s seeming inapt assumptions such that negative performance is reported as positive.

ok, so that’s my take. did i miss or misunderstand something?

let’s get some purr-review going here.