Saturday, January 29, 2011

Airline Safety Rankings Like Baloney All Flavor - No Substance

My middle-school teacher Paul Wesche once told me that the word assume could be broken down to into three parts ass – u – me, as in -  to assume is to make an ass out of you and me. The truth behind this clever wordplay was made abundantly clear in the article just published by U.S. News and World Report, America’s Safest Airlines

Look, I know that everything associated with the word “airline” is out-of-control sexy, and that the fact that US airlines have completed a year of fatality-free flying is frustrating for journalists who see the airlines as a source of juicy news when they are

a) crashing airplanes
b) annoying passengers and
c) embarrassed by the antics of pilots and/or flight attendants.

I could rest my case by writing that the article’s author, Hamooda Shami’s last venture into the world of air travel journalism resulted in  America’s Meanest Airlines, but that would be too easy.

Anyway, in this latest endeavor to capitalize on the fact that airline articles are a search engine optimization jackpot on par with Bradgelina, U.S. News has produced an article based on the premise that by counting up airline safety incidents and dividing by the number of flights, the consumer can accurately gauge how safe is a specific airline. That this is nonsense is even implied in the article itself when Mr. Shami writes the incidents used in calculating this safety ranking “should be taken with a serious grain of salt.” Fourteen hundreds words on the subject is enough salt to season  a lot of baloney, (Coincidence? I think not!) Nevertheless, as an aviation writer seeking to uphold the reputation of aviation writers everywhere, I feel compelled to point out that this article has holes of logic through which we could route the entire fleet of Airbus A380s.
Photo courtesy of Airbus
Mr. Shami limited his mathematical analysis to major US carriers, excluding regional airlines from his calculations even though they are responsible for moving one in four air travelers in the United States and responsible for 100% of the eleven airline crashes since 2000.

If this weren’t caveat enough, the article then goes on to say that in doing the math Mr. Shami is going to include only those incidents in which the “airlines were at fault.” Say what? The National Transportation Safety Board employs upwards of 90 scientists and engineers to determine probable cause, but one reporter has read, digested and analyzed a year’s worth of airline incident reports and come away with 305 incidents in which he has determined fault lies with the airline? Fast work Mr. Shami! I’m exhausted at the very thought.

Even having excluded the “airline-blame-free incidents” from the calculus, he can’t resist the urge to tell us the details of some of them. Two American Airlines turbulence events and a Continental near-collision are described as Mr. Shami justifies American’s (7) and Continental’s (4) position on his list.

There, there, there, airline executives, I know you're grimacing at my even mentioning these numbers but don’t get your shorts in a bunch. The safety ranking collapses under the weight of its own  moronic conclusions.  Have no fear for your reputations. If there’s anyone who should be worried, its the editors at U.S. News. They've risked the magazine's well-established reputation for careful list making by attaching its name to this dreck.


doug morey said...

Dear Christine,

I would like to make a slight correction to your blog concerning baloney at US News and World Report. Hmooda Shami's article had so many holes of logic and lack of knowledge that the entire fleet of Airbus 380s could have flown through it in side by side formation. It appears the author has no knowledge of 'reporting cultures' in any basic safety management system and the "Heinrich Accident Triangle." I won't go into details, it all over the web. When Shami finds an airline with no incidents, have him contact me and I will buy him a one way ticket to any of its destinations.

doug morey
nairobi, kenya

Grumpy said...

Dear Christine,
Let's not confuse an uncritical data collector with a journalist. Shami's source for his statement that major airlines are "very, very safe" is David Castelveter, spokesman for the Air Transport Association -- which may also be why Shami ignored the record of the regionals. The dead and injured don't consider the distinction relevant.
Shami apparently isn't familiar with the NTSB identification code either. "Incidents" are coded "IA." For example, the AAL Flt. 20 NTSB Report to which he referred is coded "LA", which is a "Limited" accident code.
In 2010 the NTSB issued only 68 air carrier IA (Incident) reports. (Data from Air Data Research)

Jay Donoghue said...

These stories are like the tides, rolling in with great regularity regardless of any rational justification. Actually, tides have much more justification, with the moon's gravity and all that.
However, you're right on the money. Several of these sorts of stories ago a good friend pointed out that immediately before the American Airlines accident in South America, the Cali CFIT, anyone looking at the numbers would have said such an accident was impossible, given the safety records of both American and the 757. You'd have better luck predicting earthquakes and volcanic eruptions.

Patrick Smith said...

When it comes to which is "safest," the entire enterprise of comparing airline to airline is more or less pointless. It's an exercise in statistical hairsplitting that drives me crazy.

Is airline A really "less safe" than airline B by virtue of a single accident spread over many years and millions of departures? American, United, Delta... is there any meaningful difference? No.

Moreover, whether we're talking US or globally,commercial aviation has never been safer than it is right now. There hasn't been a large scale crash in North America involving a major carrier since November, 2001. That sets a record going back at least to the dawn of the Jet Age. Despite crushing fiscal woes, the airlines have managed to maintain an astonishing safety streak.

Where is the acknowledgment of THAT? Instead the author is coming up with academic comparisons that serve no purpose other than to stoke the public's anxieties about flying, most of which are unwarranted to begin with.

Patrick Smith

Jim Blaszczak said...

Although one could say that a fatal or hull loss accident is an indicator of an airline's level of safety, that would be far too simplistic. That reasoning is similar to simply assigning a student's test failure to the teacher. The teacher has a significant role, but individual performance is the deciding factor. It's all about the relationship.

In the test analogy, teacher effectiveness can have a profound effect on student performance, but is rarely taken into account. Airline managers, like teachers have a tremendous impact on the individual performance of their employees and students. We now know ,however, that the accounting of test scores and standardized test results does a very poor job of evaluating overall teacher effectiveness. The same line of thinking is used to assign a "safety rating" to an airline from statistics generated by it's individual employees. To carry this analogy a bit further, I believe the practice of "teaching the test " and calling it learning is much like the development and training of procedures and calling that safety. It is the effective application of information and knowledge that will allow students to be successful and airline employees to be safe.

There is one significant difference between the airline manager and teacher. The exam is not academic. There are no retakes. The only passing grade is 100%.

Flying the Backside