What Are The Odds Safety Statistics Are Tarnished?

What Are The Odds Safety Statistics Are Tarnished?

Guest post by Ken Roberts – HSE Specialist – LinkedIn Profile: Ken R. Roberts

Did you know one of the smallest risks that statisticians have ever measured is—dying as a result of cancer caused by the release of plutonium from a deep space probe that loses control during its swing around the Earth to gain velocity and burns up in the atmosphere. It measures in at three-millionths of one percent. [1] It goes to show you how far we’ll go to measure something.

But I’m wondering if determining behavioural safety policy or developing company HSE strategy using statistical data is such a good idea. I can’t count how many times I’ve been tasked with sifting through company records searching for trends or information that might explain poor performance or support an inferred hypothesis. Always the request arrives with a hint of an agenda and an excuse why the data is required. Tainting somewhat the independence statistics are touted to provide.

More so, I suggest statistical information can never be used for the purposes of predictive data. Certainly never used as Leading Indicators. Statistics by their very nature can only be Lagging Indicators in their most purest form. Data is collated after the fact and create statistics mostly manipulated to make a point or prove a supposition. When we extrapolate historical data for predictive purposes, any trends they suggest are mostly at the behest of management or for the benefit of our superiors. Allow me to elucidate.

Switch or stay?

Suppose you’re a game show contestant, and you’re at the pointy end of the show. You’ve been given a choice of three doors. Behind one door is a new car, behind the other two are goats. You pick say door No. 1. The host, who is a master at building tension, knows what’s behind the doors so he opens say, door No. 3 which reveals one of the goats. He then says to you, “Do you still want door No. 1 or you can change to door No. 2?” [2]

Question: Is it to your advantage to switch your original choice? [3]

Since you don’t know how the car is hidden, nor why the host is offering an opportunity to switch, you could make use of your first opportunity, neutralise any hidden agenda and stick with No. 1. Or maybe they really want you to win the car. You’re a nice enough person and the station’s popularity has increased since you came onto the show. So why not give the audience their ultimate feel-good experience and leave them with a popular winner. The car must be behind door No.2. On the other hand, maybe your popularity is exactly why they want you to miss out on the big prize, and have you back on the show next week to maintain ratings. Stick with No. 1 right?

Given these parameters, what’s your answer. Switch or stay? Write that down or make a mental note of your decision before you read on.

Now try this. Imagine you’ve received a mass of Lost Time Injury Frequency Rate (LTIFR) data from your manager and he’s given you the task of benchmarking it against two other sets of data; LTIFR’s for the Industry as a whole and LTIFR’s from one of your major competitors. Your conclusion must include a recommendation to maintain company status or embark upon a new safety direction and strategy.

It turns out that your company’s LTIFR is quite acceptable. It’s also above industry standards. Initially you sense your recommendation will be to continue with current company policy. What with that newly won contract and all, starting up a new safety program now would need a lot of resources. And you’re already the industry leader.

However, you soon discover that your competitor’s LTIFR trumps both your performance and industry’s. Now, do you still recommend maintaining status quo and be satisfied that you’ve exceeded industry standards but sit 2nd, or embark upon a new strategy to equal or outperform your competitor? Over lunch you learn that the new CEO is offering a bonus to anyone who successfully implements an innovation that markedly improves any kind of efficiency over your competitors. How might that also affect your decision?

All the while your workforce is going about their business chasing contracts, production quotas and meeting corporate deadlines. LTIFR’s have little to do with Jimmy and his family’s needs or Johnny’s issues with his bank. But this doesn’t diminish your responsibility. You’ve been charged with a task that will affect, or not, company strategy and policy.

Now let’s go back to the original game show thought experiment. You chose door No. 1 for a new car. You know door No. 3 has a goat. Given the choice to swap do you switch to door No. 2 or stay with door No. 1? What will you rely on – your gut or your interpretation of the math? The odds were 1/3 when you started and now you have a 50/50 chance right. Os so it seems.

The actual math proves you should always switch!!? Yep that’s right. A player who stays with the initial choice wins in only one out of three times, whereas the player who switches wins in two out of three. The host opens a door, therefore the odds for the two sets don’t change, but the odds move to 0 for the open door and 2/3 the closed doors. For a Full Mathematical Explanation Go Here

What if I told you that in a study of 228 subjects, only 13% chose to switch!!? [4] An overwhelming majority of people assumed that because each door had an equal probability in the beginning they concluded that switching does not matter. What was your choice?

In reality this whole experiment suggests how counter-intuition gravitates us away from true data and towards decision-making bias. Initially, the chance aspects of how the car is hidden and how an unchosen door is opened were unknown. When you have two opportunities to make a choice, which door to choose; and secondly, whether or not to make the switch, it’s suggesting we use our gut-feeling more than empirical fact.

This thought experiment is called the Monty Hall problem. The really interesting thing is that in 1990 when the experiment became popular, many people still did not accept that switching is the best strategy. Even when given explanations, simulations, and formal mathematical proof. Paul Erdős, one of the most prolific mathematicians in history, remained unconvinced until he was shown a computer simulation confirming the predicted result. [5]

So is statistical information relevant or do we rely more on gut feel and following our sometimes blind intuition? Do we misunderstand the data and simply make our own choices regardless. With regards to safety, are we trying to use numbers to support our suppositions when really, safety is about individual values, human interpretation, manual dexterity and collaboration/interaction with others? Do we try to dumb down complexity (human behavioural psychology) into a language we think we understand (numbers)?

Do we try to dumb down complexity into a language we think we understand?

Going back to our other thought experiment using LTIFR data. Originally we thought recommending our company to maintain the status quo and continue on its current path sounded good. Then we learnt our competitors have an edge over us. But that’s OK. We’re still ahead of the industry right? But then the CEO’s bonus announcement introduces another equation. My, how things have changed. What’s your recommendation?

Trying to simplify a complex issue by plying data and statistics in support of our concepts is an illusion. Even if we wanted to avoid it, our human nature will always manipulate the results and always follows its own agenda. Our gut feelings and intuition compel us to do the things we do. Also packed away back there in the Neo-Cortex, behind our mid-brain next to the medial temporal lobe, are our herding instincts. This ensures we will always conform to public opinion and follow authority. Statistics are the norm so statistics we shall produce. Regardless of the truth. Regardless of the consequences to people or industry. Counter-intuitive or not, our biology ain’t gonna let go so easily.

So the bigger question is still at large. Does statistical data and information shape our decisions or do decisions shape our statistical data and information? Behind which door do you think has the answer?

Your comments are welcome!

References:

[1] Library of Economics and Liberty (Risk and Safety by Aaron Wildavsky and Adam Wildavsky)

[2] The Monty Hall Problem

[3] These assumptions apply: • The host must always open a door that was not picked by the contestant. • The host must always open a door to reveal a goat and never the car. • The host must always offer the chance to switch between the originally chosen door and the remaining closed door.

[4] “Ask Marilyn”, Parade Magazine

[5] “Which Door Has the Cadillac?” (PDF).