stats with cats: the summer of the air-shark?

are we really seeing a rise in issues with airlines?

every time you turn around these days, united airlines seems to be in the news for some sort of incident or accident.

this was the latest:

obviously, this is not great. on the other hand, the plane landed safely and no one even noticed this until it pulled up to the gate which is, i suppose, either terrifying or reassuring depending on how one looks at it.

there is no question this story and others like it are being widely carried. the question is are we seeing any sort of actual rise in events and are they actually scary or is this just another “summer of the shark” or “train derailment panic” where the incidence of reporting rose but incidence of outcomes did not?

is the current kerfuffle based in fact or in base rate fallacy?

let’s look:

“are you that crazy shark?”

this is one of those issues you really need to be careful with because many, many things are barely ever reported and so even a small rise in reporting rate can make trends look extreme. if something happens 100 times a month but only gets reported in 5 of them, even if it drops in frequency by half, suddenly reporting every single one as breathless clickbait takes the story count from 5 to 50 and everyone gets all worked up over a trend that’s actually in huge decline.

this was the precise case in “the summer of the shark.” the whole country was fixated on shark attack stories like jaws had decided to come for us all. meanwhile, the actual data was a below average summer for hammerhead hijinks. the whole thing was a media panic. this tends to crop up whenever the news cycle is a bit dull. gotta sell papers somehow…

so what about airplanes?

let me start by saying that the first thing you always need to do is check your priors. united has been in the news for DEI and other wokery. so has boeing. memes abound and lots of people are (quite possibly quite rightly) concerned about what happens if we hire the people who keep airplanes from falling out of the sky for demographics instead of competence.

but, the facts care about no one’s feelings and when one wants to accept something because it aligns with their world view, it’s easy to gloss over critical thinking and actual data analysis. so, let’s get rigorous and see where the data leads us.

let me preface this by saying: i am a tourist in this data and am using what looks sound and reputable to me but lack the deep grounding in it to know for sure so i’m just going to lay out all my sources and methods and if folks have better data, i’d love to see it. it’s possible this is incomplete or inapt.

i started HERE at the aviation herald website. they list, by occurrence date once you hit the toggle, aircraft incidents and accidents.

i then used THIS data on the flights per day of the 3 US majors.

i then loaded the data into excel and used the COUNTIF function to aggregate incidents and accidents and then generated a ratio of outcomes per 100,000 flights.

you can see the results here:

from this, it does, indeed, seem like united has been having some issues of late, especially in march. they have consistently had higher incident rates than others and march has been a real surge.

the spike in accidents, however, is new. united had nothing odd in jan or feb, but the first 2 weeks of march have been a big upside excursion.

so, there may actually be a bit of shark here for united all of a sudden. they seem to be having, relative to their own track record and to those of peers, a lot of issues in the last 15 days and reporting here is pretty statutory, so i doubt that there is much variance in outcome capture. they all pretty much track everything.

however, this is a tricky thing to which to ascribe meaning.

perhaps something meaningful just changed.

or, perhaps these are just small numbers and tiny occurrence rates and so the randomness of clustering can sometimes lead to big excursions that don’t really mean much of anything.

i’m not really sure we can tell yet from this data, but it does bear watching.

there are also some alternate explanations that may or may not dovetail with DEI. i was speaking to a united pilot on twitter. he agreed to let me post this:

Some additional food for thought on the topic...

While the data you listed is normalized per 100K and accounts for increased flying over previous years, it isn't accounting for the record hiring occurring at the various airlines over the last few years. Ours, for example, has been hiring near 150 pilots a month for several years. Just over 1/5 of our current pilots are in their first year.

Upgrades from First Officer to Captain have been massive as well leading to shorter time frames in the First Officer-to-Captain pipeline. While I don't have the data for rampers and maintenance, I know they've been hiring significantly as well. I would suspect that this level of hiring would lead to increase in incidents, so perhaps the data is pointing that way now based on what you listed above.

Most of the incidents being reported in the news currently are either exagerated and incorrect or just a part of the daily operation of thousands of flights. Not something I would attribute to new folks on the line, but that is a real risk lurking under the surface I think.

Next time I have an opportunity to chat with maintenance folks, I'll pick their brains on the topic. Unlike my military days where I had my hand in the maintenance efforts daily, outside of writing up a gripe on the plane and having maintenance meet us at the jet, that's about my only interaction these days.

I wouldn’t hesitate flying or putting my family on a commercial flight.

that last bit is some important perspective.

fear and rage farming make for great click rates, but it’s a lousy way to assess risks.

this is not some outrageously dangerous new threat to us all because base rates matter.

doubling your risk of being hit by a meteorite is pretty negligible in terms of actual danger. i would not spend a nickel to halve my risk of being hit by space junk. nor should pretty much anyone who does not have “spacewalk” in their job description.

still a friendly dolphin

commercial air travel remains quite probably the safest means of transport ever devised. actual deaths from flight mishaps are so rare that they are news.

keep in mind that yes, a rise from 7 to 14 incidents per 100,000 flights is a doubling, but it’s still just a risk increase of 7 1/1000th’s of 1 percent.

it’s 0.00007 rising to 0.00014.

and “incidents” are a pretty low bar that include things like “injuries from turbulence,” “odor in cabin,” and “tire pressure indicator.”

actual accidents are an order of magnitude less frequent.

0.000003 to 0.000013.

among the 3 US majors, there were no crashes nor airline caused fatalities that i’m aware of in this period. the only actual crash listed was a JS32 (jetstream) charter with 5 passengers (this is a small commuter turboprop).

meanwhile, ~43,000 americans were killed in car accidents in 2022, ~117 per day.

so i would not panic just yet about air travel.

that said, i would also keep an eye on united over the next couple months and see if this is a blip or a trend and if i were the united management team, i’d be taking a marked interest in recent outcomes.

but as it stands, i don’t think we can conclusively say much about chance vs failed process at this point. we just don’t have sufficient data.

but you know who’ll keep scratching…