Saturday, December 29, 2012

Fatality Statistics Meaningless as Safety Measure

As certain as the ball will drop in New York's Times Square on Monday midnight, there will be a round of news stories summing up 2012 in terms of aviation safety. Oh wait a minute, it's already started. 

"Overall, it was certainly the safest year ever," Paul Hayes told UPI.com. Hayes is the director of safety at Ascend, a consulting firm, that specializes in the commercial and financial aspects of aviation. 

I've got no problem with good news, or with the numbers that Hayes and Ascend present. The problem is the meaninglessness of the particular criteria used. Ascend is restricting its safety data to accidents with fatalities.  

In the vast majority of aviation accidents, nobody dies. When the National Transportation Safety Board did an in depth statistical study 10 years ago, the survival rate in reportable accidents was 95%.  If that surprises most people, and I suspect it does, Arnold Barnett, a professor of statistics at the Massachsetts Institute of Technology can explain why. "The events that make the greatest impression are the ones that involve fatalities," he said. 

The disconnect between perception and reality - (which has as its spokesman, Ryanair boss, Michael O'Leary) may be confusion over the term, “accident”. 

To the general public, airplane accident brings to mind the scene from Denzel Washington's new movie Flight, in which a plane smashes to earth trailing tongues of fire.  But an accident is also an aborted takeoff and a runway overrun, an in-flight upset and a structural failure. Turbulence and plane to plane collisions are also accidents and while people can be injured seriously, all this mayhem is not in the statistics Ascend is feeding to reporters as it raves enthusiastically about the safest year ever in aviation. 

When safety is measured by an absence of fatalities, the significance of non lethal events is diminished even though most safety specialists think these percursors are the most indicative of an overall safety culture. A preventive approach to reducing risk is so important commercial operators in most parts of the world are required to have safety management systems

Several years ago, at the International Society of Air Safety Investigators annual seminar, the entire three day program was dedicated not to presentations on the big accident investigations, but on how to better probe the little ones.  "Our biggest opportunity to make things better is before someone dies," one participant told me.

Robert Sumwalt, a member of the NTSB is fond of telling people that culture of safety, is one where people do the right thing - the safe thing - even when no one else is watching. That's not the spectacular crash, everybody is has an eye on that. 

Nope, it's a loose row of airliner seats and the ruptured fuselage at thirty-four thousand feet.  It's the engine failure on an Airbus A380 and the flight attendant's severed finger. When we see a reduction in these events we really will have something to celebrate.





18 comments:

Oussama said...

I suppose accidents are like delays; the longer the delay the easier the resolution. It is the small delays, the five or ten minutes delays that are the hardest to resolve, they are the ones that depend on individual actions that can make a difference.
Safety is a culture, it is something we should do instinctively without hesitation.

Malcolm Cox said...

Christine

Whilst I agree with the thrust of your piece, the non-aviation community does equate accident to a "smoking hole", the public at large use the same definition for all other forms of transport,cars, boats and trains. In their eyes, if nobody dies it is fine, the public do not care and are not interested in statistics.

However, I must disagree with your use of the term accident for all events. To my mind some of your examples are what should be called an incident, context is everything. I find myself in the Ryanair camp, not something I thought I would say; but the overuse of the term accident does cloud the picture of flight safety.

Malcolm

ThinkingMan said...

I believe they are called "incidents"...

Anonymous said...

Without getting into the technical definition of accidents, incidents, events… Christine’s point is accurate.

In a macro sense, we have a tendency to gauge safety on a body count. When we bring it closer to home, there is a tendency to gauge safety on occurrence (use whatever definition you like). How many operators measure (or even know about), “holy crap, that was close.”

For the most part, the difference between accidents, incidents, events, occurrences, or whatever definition you apply, is more a matter of planetary alignment than it is about a purposeful safety management strategy.

Good article Christine.

Paulo M said...

Agreed, (some) non-fatal incidents which could very nearly be fatal do not galvanize the relevant industry minds as do major accidents -- (some) non-fatal incidents cannot attract the kind of resources and government action (and funding) as per major incident at times when government budgets are squeezed, or at any time at all. It's up to industry stakeholders to better profile and eliminate risks in aviation.

Grumpy said...

I agree with Anonymous that Safety Management Systems tout their success with specious metrics. If you measure safety by the absence of something, then your conclusion is meaningless; i.e., not having one accident counts as much as not having a hundred.
If we evaluate each undesired event as a unique entity from which we can extract meaningful information that can be applied to learning processes toward prevention, we will have accomplished much.

Andries said...

I can not agree more, Christin. The death rate spin is so yesteryear, long before IATA and ICAO had arrived at real measurement controls.

It simply is a spin airline executives and aviation authorities use to not go to the real facts of safety measure and the success or failure thereof.

AND: There simply is no such thing as an aviation "accident" no matter how authorities try and distinguish between severity levels.

In fact, the higher the severity level the less of an "accident" it was...

An "accident" implies something having occurred "accidently" - like the earth accidently rose up to smite an aircraft in perfectly normal flight.

In my 36 years as a mishap investigator in aviation I yet have to give expert evidence to a judge in the high courts internationally that the causes of a crash was an "accident".

In each and every instance there was a human being who had somewhere negligently done something he should have known or did know to be wrong, or neglected to do something he should have done.

Christine Negroni said...

In writing this post I relied on the ICAO defination of accident - any event in which an air traveler is injured or an airplane substantially damaged - The events cited all fall into the accident category with the exception of the loose seat episodes on American this past October.

Mike Miller said...

Well written Christine. After 22 years in aviation, I find myself working with reporters constantly to help them understand what an accident is, and what really is serious (e.g. an engine problem happens every day, but is it serious or just an early landing out of caution?) Since the major media amplifies small things 100x over, it's good to have perspective. Please keep providing it!

Jim Blaszczak said...

Although "Andries" is absolutely right, his last paragraph is simply a statement of the obvious.

Aviation (as well as medicine, and other endeavors with a human/technical interface) safety is the acknowledgement that human error cannot be eliminated and development of systems that address that truth. Human error cannot be eliminated, it can only be managed. Managing error means identifying errors and/or the conditions that promote them and employ mitigation strategies before there is a negative result.

Christine, to your very well articulated point, there is no difference in the factors that lead to a simple mistake, an incident or fatal accident. Take the case of an approach that would result in a landing well outside the touchdown zone. The conditions are set up for an incident or accident. Most likely a mistake has been made, or at least allowed to occur. At this point the outcome is dependent on error mitigation. If the pilot identifies the error and performs a go around safety is maintained. If the pilot continues the landing which results in a runway excursion (potentially fatal) a statistical data point has been generated. Same error two results. We can't always know the potential outcome so we have to be ever vigilant. An error managed to no consequence has the same result as no error. The only difference is error management is possible, elimination of error is not.

The further improvements in safety, aviation and otherwise, statistics will come from improvements in the area of threat and error management including decision making processes. It all begins with a frank and open discussion of these topics

What a blessing it is to share these thoughts with others in an attempt to raise awareness to safety issues.

Christine Negroni said...

To my well articulated point? Jim, you made my post 100% more comprehensible with your well written addition of the threat and error management component. Yes indeed, what a blessing it is to share thoughts. Best of the New Year to you.
Christine

Wale Adebisi said...

Very interesting topic. This is another way of reopening a topic I started ealier (difference between accident and incident in aviation). I think we should ask ourselves the very objective of SMS in aviation. Is it to avoid air crash (especially those that can lead to fatalities) or to prevent unplanned events which are accident precursors. I have seen many air operators claiming to have operated for several years without a single accident. Even when it had recorded several emergency landings, air returns, bird strikes, equipment failure, occupational injuries and aircraft damage due to ground operations. Yet required attention was not given to all these occurrences because they are considered as ordinary incidents.

I asked these questions; what is the aim of aircraft engineer when releasing an aircraft for service? is he certifying the aircraft airworthy because he knows it can operate the flight without a fatal crash? even when such aircraft has many unclared snags, operating under MEL, one or two concessions/extensions.
secondly, is the PIC accepting that aircraft because he knows he will not have a fatal crash even though he might have emergency landing or one engine short down?
Christine thank you for that educative article.

Wale Adebisi

Christine Negroni said...

Wale,
This is a very important question you post. Errors are managed until they are not. If you change the focus as suggested to eliminate all the "acceptable" potential error-makers you would go much farther in eliminating incidents and therefore, accidents. The first thing that came to mind when I read your question was the minimum equipment list. Is the MEL not an acceptance that some level below perfect airworthiness is okay when balanced against some other non-safety measure like schedule or budget? Safety is always on one side of such a measurement which is why your question is so provocative. Thank you for weighing in. Christine

Ron Kuhlmann said...

Aviation is just applying the same measures as are used in the rest of society. Traffic reporters differentiate between accidents and fatal accidents and stories of fires are considered to be less worrisome when the article includes a comment that says all occupants were accounted for.

It is also noteworthy that unlike other dangers in society, there were no major injuries in the incidents that occurred--certainly not true for auto statistics where even a non-fatal accident can maim and destroy lives.

And as we have all been reminded by events of late, we strive for greater safety in many fields while allowing the risk of gun violence to go unchecked.

I get your point and agree that any laxity needs to be addressed, but as a person who has logged over 3 million air miles, I take this as a singular triumph, unrivaled amongst other forms of transport as well as many other activities in which we engage.

We use "at least nobody died" as a positive comment in many situations. Yea for the IATA carriers!

Marc Master said...

I agree with most. Those accidents that are not fatal are not to be reported as a "FAA incident". This includes "accidents" with no loss of life and include a person who died due to health concerns. Those "incidents" that do not cause death/injury should be measured but in a means not to scare the public.

For instance, the AC went out on my car during the summer. If it were my airplane (which did happen during flight -- not pleasant). Should not be reported as an "accident". Although, the safely masks dropped and we descended quickly to reach a level of safe oxygen levels.

Considering the fact, I fly for my career that "incident" was 1/thousands of flights I have taken successfully without any incident (in other words, I survived the flight). Furthermore, I left the flight as an inconvenience. The other "non-native" passengers thought we were going to die as the fuel was discharged from the plane to prepare for an emergency landing.

If reporting statistics are changed with full unnecessary reports of "incidents" the media would go crazy sensationalize the safety of travel and no one would fly.

In conclusion, air travel is the safest means of transportation. An engine falling off an airplane sounds much worse than it really is. Changing the way FAA reports to the public air safety can jeopardize travel.

I personally do not look up the age of an airplane (one public information) because I don't want to subconsciously know I am on a 40year old machine :).

Marc Master
www.MasterREStravel.com

Christine Negroni said...

Marc,
Thanks for your contribution. So that I am clear, accident is not a subjective term. There is a strict ICAO definition for it and when that definition is applied the survivability rate for air accidents is around 95%. This is based on an NTSB study, that admittedly is a decade old. If one filters out all accidents that do not include fatalities, one gets a distorted picture of air safety.

Jim Blaszczak said...

As far as threat and error management is concerned, it doesn't matter whether it is an deviation, incident, accident or fatality. The objective is to avoid negative consequences. Is the theft of a penny any less unethical than the theft of one hundred dollars.

It is a mindset not a graduated scale. If you would not steal a penny you have the same mindset you would for a hundred dollar bill.

There is not enough time here to get into this, but the next step in this discussion is acceptable risk and unacceptable risk. Engine failure on takeoff is an acceptable risk because we are trained and plan for it. Landing on a short slippery runway with insufficient braking action is an unacceptable risk. The difference is in the first instance we can manage the threats in the second we cannot.

This is a great discussion. Thanks to all who are participating and to you Christine for hosting!!

Well done.

Anonymous said...

Incidents or accidents they are both something to look at. I have been working in aviation maintenence for 21 years have seen bird strikes, aircraft to aircraft damage, damage to seats from passengers, hard landings. In my point of view the two go hand and hand, because an incident is an event and an accident is the result of an event.