• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

SafetyRisk.net

Humanising Safety and Embracing Real Risk

  • Home
    • About
      • Privacy Policy
      • Contact
  • FREE
    • Slogans
      • Researchers Reveal the Top 10 Most Effective Safety Slogans Of All Time
      • When Slogans Don’t Work
      • CLASSIC, FAMOUS and INFAMOUS SAFETY QUOTES
      • 500 OF THE BEST WORKPLACE HEALTH and SAFETY SLOGANS 2023
      • CATCHY and FUNNY SAFETY SLOGANS FOR THE WORKPLACE
      • COVID-19 (Coronavirus, Omicron) Health and Safety Slogans and Quotes for the Workplace
      • Safety Acronyms
      • You know Where You Can Stick Your Safety Slogans
      • Sayings, Slogans, Aphorisms and the Discourse of Simple
      • Spanish Safety Slogans – Consignas de seguridad
      • Safety Slogans List
      • Road Safety Slogans 2023
      • How to write your own safety slogans
      • Why Are Safety Slogans Important
      • Safety Slogans Don’t Save Lives
      • 40 Free Safety Slogans For the Workplace
      • Safety Slogans for Work
    • FREE SAFETY eBOOKS
    • Free Hotel and Resort Risk Management Checklist
    • FREE DOWNLOADS
    • TOP 50
    • FREE RISK ASSESSMENT FORMS
    • Find a Safety Consultant
    • Free Safety Program Documents
    • Psychology Of Safety
    • Safety Ideas That Work
    • HEALTH and SAFETY MANUALS
    • FREE SAFE WORK METHOD STATEMENT RESOURCES
    • Whats New In Safety
    • FUN SAFETY STUFF
    • Health and Safety Training
    • SAFETY COURSES
    • Safety Training Needs Analysis and Matrix
    • Top 20 Safety Books
    • This Toaster Is Hot
    • Free Covid-19 Toolbox Talks
    • Download Page – Please Be Patient With Larger Files…….
    • SAFETY IMAGES, Photos, Unsafe Pictures and Funny Fails
    • How to Calculate TRIFR, LTIFR and Other Health and Safety Indicators
    • Download Safety Moments from Human Resources Secretariat
  • Social Psychology Of Risk
    • What is Psychological Health and Safety at Work?
    • Safety Psychology Terminology
    • Some Basics on Social Psychology & Risk
    • Understanding The Social Psychology of Risk – Prof Karl E. Weick
    • The Psychology of Leadership in Risk
    • Conducting a Psychology and Culture Safety Walk
    • The Psychology of Conversion – 20 Tips to get Started
    • Understanding The Social Psychology of Risk And Safety
    • Psychology and safety
    • The Psychology of Safety
    • Hot Toaster
    • TALKING RISK VIDEOS
    • WHAT IS SAFETY
    • THE HOT TOASTER
    • THE ZERO HARM DEBATE
    • SEMIOTICS
    • LEADERSHIP
  • Dr Long Posts
    • ALL POSTS
    • Learning Styles Matter
    • There is no Hierarchy of Controls
    • Scaffolding, Readiness and ZPD in Learning
    • What Can Safety Learn From Playschool?
    • Presentation Tips for Safety People
    • Dialogue Do’s and Don’ts
    • It’s Only a Symbol
    • Ten Cautions About Safety Checklists
    • Zero is Unethical
    • First Report on Zero Survey
    • There is No Objectivity, Deal With it!
  • THEMES
    • Risk Myths
    • Safety Myths
    • Safety Culture Silences
    • Safety Culture
    • Psychological Health and Safety
    • Zero Harm
    • Due Diligence
  • Free Learning
    • Introduction to SPoR – Free
    • FREE RISK and SAFETY EBOOKS
    • FREE ebook – Guidance for the beginning OHS professional
    • Free EBook – Effective Safety Management Systems
    • Free EBook – Lessons I Have Learnt
  • Psychological Safety
    • What is Psychological Health and Safety at Work?
    • Managing psychosocial hazards at work
    • Psychological Safety – has it become the next Maslow’s hammer?
    • What is Psychosocial Safety
    • Psychological Safety Slogans and Quotes
    • What is Psychological Safety?
    • Understanding Psychological Terminology
    • Psycho-Social and Socio-Psychological, What’s the Difference?
    • Build a Psychologically Safe Workplace by Taking Risks and Analysing Failures
    • It’s not weird – it’s a psychological safety initiative!
You are here: Home / Lead and Lag Indicators / The Madness Of Measuring Nothing

The Madness Of Measuring Nothing

February 19, 2014 by Phil LaDuke Leave a Comment

The Madness Of Measuring Nothing

First published Phil La Duke on his blog See the whole article here

Phil says: In a business climate where every company seems to live and die by its metrics one important business element is often over looked, or worse yet is misleading business leaders. Safety has been poorly measured for almost 100 years, and in this week’s post I offer better ways to measure worker safety. I hope you will give it a read and tell me what you think.

cg_measuring-cupThese days’ organizations live and die by measurements. It seems that no matter where we work we are confronted with the dreaded balanced score card and so we are tasked with measuring “safety”. I’ve said for a long time, “the absence of injuries does not denote the presence of safety” and zero injuries doesn’t tell us a heck of a lot about the risk of injuries within a given population.

The traditional measures of safety, i.e. injuries or days away or restricted time, don’t help us predict the likelihood of future performance, and yet that is the best we seem able to come up with. We play at leading indicators like near miss reporting (as somehow indicating the level of participation of workers in safety) but even these measurements are fraught with statistical noise that can lead us to conclusions not in evidence; so many of our indicators mislead us that one has to wonder if there is any value in them at all.

Recently I was asked to address a meeting of the senior leaders of a multi-national manufacturer and I was asked what some good predictive measures would be for safety. I was pressed for time and I’m afraid I didn’t have the luxury of a prolonged discussion on metrics (my topic was Creating A Culture Of Safety Excellence). So given the fervour set in motion from my last three posts I thought I would add a bit of metaphorical fuel to the fire and lay out for professional debate, what I see as some good ways for correlating business measures to future performance in safety.

Risk Factor #1: Worker Stress and Distraction

Worker stress has a profound impact not only on human error, but on risk taking, and worker’s health as well.  Highly stressed workers are distracted and distraction leads to mistakes which lead to injuries.  Some measures that I think directly correlate to worker stress are:

  • Worker absenteeism.  Absenteeism rates are indicators of both worker stress and worker competence. Research has shown that stressed workers tend to miss more work, and when a worker misses work, his or her job is done by someone less skilled, less practiced at the job, and therefore more likely to deviate from the standard.  In other words, the worker stuck doing the job is at greater risk of injury than the worker whose muscle memory is completing many of the tasks by rote.  Of course this isn’t universally the case, but it is true often enough to correlate, and when it comes to prediction, correlation is the best we’ve got.
  • Number of calls to employee assistance programs.  When we talk about worker distraction, we tend to think in terms of distractions borne in the workplace.  Workers who are worried about financial problems, divorce, or other “off-hours” problems while working face the same dangers as those distracted by work issues.  The number of calls to EAP lines can provide a good idea of how much distraction is in the workplace which correlates to human error, behavioral drift, lapses in judgement and ultimately  workplace injuries.
  • Worker turnover.  Employee turn over creates risk in much the same way absenteeism does: it introduces greater variation into our work processes which in increases the risk of injuries.  The greater the worker turnover rates the higher the risk of injuries as newer, less competent and skilled workers replace higher performing, more experienced workers.
  • Engagement survey scores. Engaged workers tend to do things because these things are the right thing to do.  The lower the level of employee engagement the higher the risk of worker injuries.

In all these cases we have to remember that we seldom have a perfect correlation (a case where every time factor A is true factor B is also true) and even in those rare cases where there is a perfect correlation such a condition does not mean that there is a cause and effect relationship between two factors.  But since we are looking at the measurement’s predictive value there is always a margin for error, statistical anomalies and statistical outliers.  If we had a perfect way of predicting exactly where and when an injury would occur we would be using it.

Risk Factor #2: Worker Incompetence

When we talk about worker incompetence, we’re not talking about the nincompoop  who doesn’t seem able to do even the most rudimentary task without screwing things up, rather, we are talking about the skill level at which a worker is able to perform his or her job.

There is a strong correlation between level of mastery at which a worker performs the tasks associated with his or her job and the risk of injuries.  To that end these measurements are appropriate and predictive:

  • Required training % complete. Assuming that we require training because it is necessary to do one’s job, the lack of this training would indicate process variability.  Tracking the percentage of training provides us with a glimpse of how much risk a worker faces of being injured because he or she performed a task improperly.  The greater the percentage of people who have completed training the lower the risk of injury because of a gap in essential skills.
  • % of licenses and certificate expired. Just as the percentage of required training that is complete provides us with an understanding of approximately how many people are likely working out of process (it’s tough to do the job right simply by guessing) so too does the percentage of workers who are working despite having expired licenses and certificates.
  • Time to complete required training.  The longer it takes to complete required training the longer a worker is exposed to workplace risk associated with a skills gape.
  • Worker performance appraisal scores. This particular measure is tricky—it assumes that the worker appraisals are fair assessments of the worker’s ability to accurately complete tasks and do the job. Assuming that there is a robust worker performance appraisal assessment the lower scored individuals should be at greater risk that those who are performing at higher levels.

Risk Factor #3: Leader Incompetence.

Workers generally perform in ways for which they are rewarded and eschew behaviors for which they are punished. Low-performing leaders often exacerbate safety issues by behaving inappropriately in their interactions with workers. Some measures that I think directly correlate to leadership competency are:

  • 360 Reviews. 360 Reviews, that is, reviews where a leader’s team members, boss, and peers all contribute to the review, are often excellent indicators of how well a leader interacts with his or her team. The weaker the leader the higher the risk of process variation and hence a rise in the risk of injuries.
  • Leader performance review.  Leaders who perform poorly are generally allowing more variation into the work area the higher the performance of the leader the less likely workers will be harmed on his or her watch. It’s important to note that the leader’s performance review will most likely include things like the productivity of his or her team, general performance in things like cost, quality, and efficiency, in other words, things that will either directly or indirectly impact the risk of injuries.
  • Worker morale.  Of course worker morale can be effected by a host of things unrelated to the leader, but worker morale is heavily influenced by the performance of the leader.  Workers suffering from poor morale generally perform at lower levels which fall outside the processes control limits.  The worse the morale the higher the risk of variation and ultimately injuries.
  • % of safety reviews completed on time. I am not a fan of “behavioral observations”; I’ve always felt the time watching someone work could be better spent taking a more holistic view of worker safety by reviewing the risk conditions (procedural, physical, or behavioral).  That having been said, it is important that leaders conduct routine and repeated inspections of the workplace to identify hazards.  The percent of safety reviews/tours/inspections/observations completed on time is a, at least ostensibly, an indicator of the time to which workers are exposed to hazards.
  • % or performance reviews completed on time.  Completing performance reviews on time isn’t just about making employees feel good, it is also about assessing competency.  The more reviews that are completed on time, the more skills and performance gaps are identified in a timely manner.
  • % Attendance at safety meetings.  The percentage of safety meetings that a leader attends provides a good insight into the level of priority on which the leader places on safety.

Risk Factor #5: Process Capability

Process variability creates risk; to the product, to the equipment, and to the workers.  The frequency and duration of non-standard or out of process work is a good predictive indicator of risk of injury.  Good measures of process capability (relative to safety) are:

  • % of nonstandard work.  Statistically speaking nonstandard work tends to be more dangerous and the injuries associated with nonstandard work tend to be more lethal than its standard counterpart.  The percentage of work that is nonstandard can indicate a substantial bump in risk associated with any operation.
  • % of jobs with completed JSAs.  A complete and current Job Safety Analysis (JSA) is crucial for the safe execution of work, yet I don’t know any company that has 100% of it’s jobs with JSAs, and many companies don’t have a good track record of keeping the JSA’s current with the standard operating procedure. Understanding the percentage of your tasks have good and current JSAs is a good predictor of future risk (the higher the percentage the lower the risk).
  • % of jobs with Standard Work Instructions. Personally, I prefer Standard Work Instructions (SWI) to JSAs (a good SWI should address all the safety concerns of a job), but SWIs suffer from the same problems that I discussed regarding JSAs above.
  • % behind in production.  I still have nightmares about my days working an assembly line and falling “in the hole” screams of “man in the hole” booming above the cacophony of hand tools, presses, and industrial vehicles still give me chills.  Whenever ever workers are struggling to catch up because they are behind in production the risk of injuries rises.
  • % parts shortages.  When there are part shortages (or tools shortages, or materials shortages, or labor shortages for those of you who work outside manufacturing) workers are forced to work outside the standard process.  This is incredibly dangerous because the standard process is designed with protections against injuries embedded in the tasks.  When a worker is working outside the process the organization is relying on luck to protect them.

Risk Factor #6: Worker Engagement In Safety

We’ve discussed worker engagement in a broad sense, but I think it is important enough to look at worker engagement specific to safety.  Engaged workers will work safely for no more reward than because working safely is the right thing to do.  Worker engagement in safety can be measured by:

  • Number of reported near misses.  Some will argue, correctly, that near misses are lagging indicators, but whether or not a worker choses to report a near miss correlates to the level of worker engagement in the safety process. This measurement, admittedly, is difficult to get accurately.  Since we don’t know the total actual number of near misses we can’t say with certainty whether the current level of reporting is a high or low percentage.  Even so, the number of workers who report, even more so than the raw numbers of near misses, can provide a good glimpse into the level of importance workers place on safety.
  • Number improvement suggestions.  Workers who take an interest in improving the organization are generally interested in finding and eliminating failure modes, which will include those failure modes that will ultimately place workers at risk of injury.  The greater the number of suggestions the lower the risk.
  • Participation in continuous improvement workshops.  Eliminating variation, risk, and hazards are part and parcel of the continuous improvement process so it should surprise no one that the level of participation in these activities correlate to the level of risk.
  • Number of worker grievances.  Worker grievances shed valuable light in to many of the other risk factors identified here and generally the greater the number of grievances the higher the level of risk of injuries.
  • Number of disciplinary actions for safety violations. The number of disciplinary actions for safety violations are indicative of two things: the number of unsafe acts being committed and the extent to which these incidents are taken seriously.

Of course one has to be careful in designing and managing these measurements to avoid unintended consequences (for example, one could easily reduce the number of disciplinary actions by not applying appropriate discipline, or one could raise worker performance evaluation simply through “score inflation” but the risk of these unintended consequences can be reduced by solid management practices and random sampling audits.

The Imperfection Of Predictive Measures

To some extent we can never have a perfect set of measures.  In many ways it’s like predicting the weather, since we are talking about probability there is always a chance that the organization will beat the odds.  In fact, there isn’t one of these measures that I couldn’t construct a convincing argument against.  What’s important is to use those of these measures that make sense and use them in conjunction with each other.  One correlation does not a pattern make, but when we look at multiple areas of risk and analyze them in a holistic context we can find a more useful way to measure safety than counting bodies and broken bones.

  • Bio
  • Latest Posts
  • More about Phil
Phil LaDuke

Phil LaDuke

Principle and Partner at ERM
Phil LaDuke

Latest posts by Phil LaDuke (see all)

  • Seven Ways To Tell If BBS Is Right For You - October 17, 2022
  • 80% of Safety Practitioners Are Idiots - July 28, 2022
  • Where is the Value in Safety Day? - June 1, 2022
  • Professional Conferences Are A Sleazy Con - March 29, 2021
  • Hey Idiots, You’re Worried About the Wrong Things - March 7, 2020
Phil LaDuke

Please share our posts

  • Click to print (Opens in new window)
  • Click to email a link to a friend (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)

Related

Filed Under: Lead and Lag Indicators, Phil LaDuke, Zero Harm Tagged With: measuring safety, Zero Harm

Reader Interactions

Do you have any thoughts? Please share them below Cancel reply

Primary Sidebar

Search and Discover More on this Site

Never miss a post - Subscribe via Email

Enter your email address and join other discerning risk and safety people who receive notifications of new posts by email

Join 7,495 other subscribers

Introduction to SPOR – FREE!!

SAFETY MYTHS SERIES

The Mythic Symbology of Safety

Posture Myths and Holistic Ergonomics

Safety Mythbusters

Don’t Be Emotional! Another Safety Myth

Tackling the Challenge of Heuristics in Safety

The Myth of Normal

NEW! Free Download

Please take our 2 minute zero survey

Recent Comments

  • Rob long on Welcome to the Nightmare, Safety Creates its Own Minefield (as usual)
  • Matt Thorne on Welcome to the Nightmare, Safety Creates its Own Minefield (as usual)
  • Anonymous on Welcome to the Nightmare, Safety Creates its Own Minefield (as usual)
  • Jason on How Bias Inhibits Learning in Safety
  • Rob Long on How Bias Inhibits Learning in Safety
  • Admin on How Bias Inhibits Learning in Safety
  • Rob Long on 400,000 Free Downloads
  • Gustavo Saralegui on 400,000 Free Downloads
  • Rob long on To Err is Human, You Better Believe It
  • Wynand on To Err is Human, You Better Believe It
  • Rob Long on To Err is Human, You Better Believe It
  • simon cassin on To Err is Human, You Better Believe It
  • Rob Long on Records of safety activities: evidence of safety or non-compliance?
  • Matt Thorne on Free Online Workshops
  • Rob long on No Good Reason to Follow Reason
  • Brian Edwin Darlington on No Good Reason to Follow Reason
  • Risk Diversity on Book Launch – For the Love of Zero – in Portuguese
  • Rob Long on No Good Reason to Follow Reason
  • Risk Culture Builder on No Good Reason to Follow Reason
  • Mark Taylor on All Things Must Pass in Risk

FREE eBOOK DOWNLOADS

Footer

VIRAL POST – The Risk Matrix Myth

Top Posts & Pages. Sad that most are so dumb but this is what safety luves

  • Free Safety Moments and Toolbox Talk Examples, Tips and Resources
  • Road Safety Slogans 2023
  • 500 OF THE BEST WORKPLACE HEALTH and SAFETY SLOGANS 2023
  • Download Safety Moments from Human Resources Secretariat
  • Welcome to the Nightmare, Safety Creates its Own Minefield (as usual)
  • How Bias Inhibits Learning in Safety
  • What is SPoR?
  • Safety Acronyms
  • 15 Safety Precautions When Working With Electricity
  • How to Calculate TRIFR, LTIFR and Other Health and Safety Indicators

Recent Posts

  • Welcome to the Nightmare, Safety Creates its Own Minefield (as usual)
  • Getting the Balance Right in Tackling Risk
  • What is SPoR?
  • How Bias Inhibits Learning in Safety
  • Afraid to Let Go of What Doesn’t Work in Safety
  • When You Don’t Know What to do in Safety, Have Another Blitz!!!
  • Gloves and Glasses Compliance
  • A Case of Desensitisation – What Would You Do?
  • How to Leave the Safety Industry
  • The Mythic Symbology of Safety
  • Dark Waters, The True Story of DuPont and Zero
  • 400,000 Free Downloads
  • Am I stupid? I didn’t think of that…
  • Don’t Look Now Safety, Your Metaphor is Showing
  • Ratio Delusions and Heinrich’s Hoax
  • To Err is Human, You Better Believe It
  • Culture as a Wicked Problem, for Safety
  • Safety Leadership Training
  • Cultural Orientation in Risk
  • The Stanford Experiment and The Social Psychology of Risk
  • Objectivity, Audits and Attribution When Calculating Risk
  • Records of safety activities: evidence of safety or non-compliance?
  • Zero, The Seeking of Infinity
  • Safety Leadership Essentials
  • What Can Indiana Jones Tell Us About Culture
  • Safety as a Worldview
  • The Loathing of Limits
  • Culture Cannot be Framed Through Safety
  • Free Online Workshops
  • Safety Culture–Hudson’s Model
  • Book Launch – For the Love of Zero – in Portuguese
  • Advancing Backwards in Safety
  • The ‘Noise’ of Safety, Silence and Practicing of Mindfulness
  • All Things Must Pass in Risk
  • I’m just not that into safety anymore
  • Sticks and Stones and the Nonsense of Zero Harm
  • Courting Infallibility in Safety
  • Indicators of Risk
  • What Can Safety Learn From Playschool?
  • No Good Reason to Follow Reason
  • Just as Well Culture Doesn’t Listen to Safety
  • What Are the Benefits Of Social Psychology of Risk?
  • Short-Sighted Lenses by Safety
  • Is Safety the Empire of Non-Sense?
  • No Wonder Safety is Confused About Culture
  • Building High Performance Safety Cultures
  • Understanding iCue, a Visual, Verbal, Semiotic Method for Tackling Risk
  • On Culture and Safety
  • Focus on ‘Meeting’ people, not legislation – a path to risk maturity
  • The Moral Harm of the Zero Cult

VIRAL POST!!! HOW TO QUIT THE SAFETY INDUSTRY

FEATURED POSTS

Zero ‘Arm

Social Psychology of Risk Doability

The Common Sense Fallacy

No Soft Skills in Safety

SAFETY IS A MYTH, LONG LIVE SAFETY

Introduction to The Social Psychology of Risk – Free Online Module

I am a Spreadsheet King

Semiotics and Safety

What is a Safety and Risk ‘Thinking Group’?

The Worm at the Core

Human Factors Factors

The Curse of Cognitivism

Why is Safety an Easy Target?

Developing Our Inner Introversion

The 5 Ways We Identify Hazards

Zero Accident Vision Non-Sense

Failure Must be an Option

Numerology and Psychic Numbing

Even Safety Is Fallible

10 Reasons Why Safety Can Never Make You Happy

non-Leadership in Risk

EGO is not a dirty word

Safety Utopia as Abuse

The Immediacy of Zero

Focus on ‘Meeting’ people, not legislation – a path to risk maturity

SPoR and Myth

Surfacing – Making the Unconscious Conscious

Who Gives a Toss?

More Realistic Swiss Cheese Symbol

The Trajectory of Zero Vision

King of the World – Why is Sociopathy and Psychopathy so prevalent ‘at the top’?

Consciously Safe, Unconsciously Unsafe or Head in the Sand Safety

Why is fallibility so challenging in the workplace?

The Essentialist Pleasure Of Safety

Cultivating Resilience

Desensitisation–the by-product of ill-conceived safety initiatives

Social Psychology of Risk Challenge

Safety is the Wrong Anchor

When the Target Drives the Method

Zero Discourse as Gobbledygook

More Posts from this Category

Subscribe to Blog via Email

Enter your email address and join other discerning risk and safety people who receive notifications of new posts by email

Join 7,495 other subscribers

How we pay for the high cost of running of this site – try it for free on your site

WHAT IS PSYCHOLOGICAL SAFETY?

What is Psychological Safety at Work?