The Seduction of Measurement in Risk and Safety

The Seduction of Measurement in Risk and Safety

Rather than measure what we value, we tend to value what we can measure

Many safety mottos and mantras for organisations are sent to me and I recently saw one for a mantra and motto of ‘safety beyond measure’. The strange thing about the presentation of the mantra was that it was all about measurement – TIFR, MTIs, LTIs and LTIFRs. Rather than going beyond measure, the organisation was clearly stuck in measurement, a clear indicator of a calculative organisation (Hudson). The website argued that injury data and measurement are safety culture measures. The WHS policy of the company is also wrapped in the discourse of zero harm and claims that both measurement and the zero goal have changed the culture. At no time in all the reporting or discussion on safety culture in the measurement discourse is culture defined. The discourse on the website essentially equates culture with systems and mindsets, mindsets about data and data reduction. The same company was implicated in three fatalities in 2013.

Drucker was partially right when he stated: ‘what gets measured gets managed’ but the opposite is certainly not true: ‘if it can’t be measured it’s not worth managing’. So much of what comprises culture is difficult to measure especially when most organisations neither define it or, define it simplistically. The trouble is, the more one fixates on measurement the more the dynamics of measurement dictates practice. I find it interesting that many organisations speak about going to the ‘next level’ in safety but really mean greater vigilance in marking time on the one spot.

All assessment contains an embedded methodology that invisibly drives goal setting, work validation, anthropocentric value, understanding of learning, discipline, risk perception, cultural norms and self-confirming assumptions. Measurement discourse tends to create its own trends, dependencies and blindness to the validity of qualitative data and most data associated with culture is qualitative not quantitative. For example, measurement associated with the zero ideology drives: under reporting (necessary for learning), fixation on low level risk, punitive response and exhausting investment of time and energy into data interpretation and reporting. We now have people in the safety trade spending most of their time in unthinking processes such as: checking checklists, making graphs, counting backwards and excessive bureaucratic attention on minor events.

Culture is best defined as:

… value-based interpretations; artefacts; shared experiences; interaction, adaptation, and survival; social customs and social norms; the expressive forms of social and material life; a distinctive ‘way of life’ of a group or class; historically transmitted ensembles of symbols; ‘maps of meanings’ that make social life intelligible to its members; systems of knowledge shared by large groups of people; the quotidian, self-interpreted conduct of particular groups and communities; historically shaped forms of consciousness; contradictory forms of ‘common sense’ that shape public and popular life; everyday activities and patterns of actions; an evolving totality of meanings; a living tradition; socially transmitted patterns of behaviour; meanings alive in institutional life as well as in ordinary behaviour; socially embodied differences and ‘performed’ at the level of everyday life; the symbolic production of material structures; a conception of the world or worldview (Mclaren, 1996) in summary, culture is a social psychological activity.

The idea that an LTI or LTIFR number in some way represent or are connected to any of these cultural indicators is as best fanciful and at worst dangerous. The idea of measuring LTIs as a predictor of safety culture is a delusion of the calculative mindset. For example, BP Deepwater Horizon One claimed to have millions of LTI free hours before it killed 11 people and poured billions of tonnes of oil into the Mexico Gulf. The rig owner, Transocean, was said to have had a strong safety record with no major incidents for 7 years. The reality was that a culture of denial, hubris and a fixation on measurement blinded leaders to ‘warning signs’ that generated the explosion. (https://s3.amazonaws.com/pdf_final/DEEPWATER_ReporttothePresident_FINAL.pdf). What the fixation on measurement did at Deepwater Horizon One was create the delusion that they were measuring culture whilst all the time leaving most cultural indicators untouched. The seduction of measurement, particularly measurement of injury rates attributes a fictional connection to culture and creates blindness to qualitative indicators of culture.

In contrast to the preoccupation with measurement, Wagner’s research (SIA, 2010) states:

‘Most CEOs no longer relied on the Lost Time Injury Frequency Rate (LTIFR) and OHS audits as primary performance measures. All reported some difficulty in measuring the effectiveness of their programs and most were exploring lead measures’. This is indeed encouraging but to truly move beyond an injury data (calculative) focus requires courage and the humanization of systems. If organisations want to go to the ‘next level’ in safety, they must let go of the anchor of measurement of injury data and mathematical security blankets.

The idea that lag indicators are a measure of performance is based upon the ideas of Herbert Heinrich (1931) and similar approaches to incident prediction, causation and mechanistic approaches to understanding risk. Heinrich was an insurance salesman and sought to impose a ‘scientific’ approach to understanding of risk. Heinrich’s Safety Pyramid whilst popular
in the safety industry, has no validity either as a predictive or explanatory tool of how humans
and organisations manage, or make decisions about risk. Heinrich’s approach is Tayloristic and in the genre of ‘scientific management’. There is no evidence to show that the ideas of Heinrich or the later Behavioural-Based Safety (BBS) discipline, equate to reality or explain social-psychological or neuropsychological evidence about human judgment and decision making. Heinrich’s Pyramid is present in the discourse of most calculative organisations and remains in Certificate IV and Diploma studies in WHS qualification (Stoll, McGill and Ritchie, 2014).

The seduction of measurement promotes the idea of predictability and certainty and, has a strong appeal to those with a binary opposition mindset. Binary opposition uses the classic language of entrapment, ‘how many people do you want killed at work today?’ Binary opposition thinking neither critiques its own assumptions or logic nor applies it to any other part of life and living. The mathematical thinking embedded in binary opposition often fails to understand that its own mindset makes it blind to the priming of its own language. It fails to see that language primes culture and that binary language creates blindness to issues outside of mathematical logic especially, cultural space and issues. Measurement helps those fixated in binary opposition to prove their own assumptions (https://safetyrisk.net/binary-opposites-and-safety-goal-strategy/). There is nothing more appealing to the binary opposition mindset than promises of having ‘everything in control’. There is nothing so cozy as having the solution to every threat ‘in hand’, to have ‘black and white’ answers. Attributions of cultural meaning to mathematical data provides the promise of certainty about the uncertain (risk) but unfortunately enables a sense of hubris to flourish. Mathematical thinking enables binary language gymnastics about zero to make sense, creates selectivity about data (eg. FIFO, DIDO harm, mental health harm) and ignores a host of cultural indicators that fly under the radar. Then when a severe event happens, whilst the LTIFR rate is low, zero organisations don’t understand the miscorrelations or attributions with their data and so create more vigilance and systems compliance as a solution.

Mechanistic, systems and mathematical thinking requires balance if issues of culture are to be known and influenced. Qualitative knowledge can be known but is not understood in the same way as mathematics, this is the challenge for the engineering and science mindsets that dominate the risk and safety industry. The social and psychological understandings and methodologies required to understand and influence culture are not foundational to the disciplines of science, engineering or safety. To step to the ‘next level’ requires some un-learning in mathematical dependence and some new learning in knowledge validation and thinking. Without some movement up the steps to risk maturity, the safety industry will continue to mark time on the calculative step. This is illustrated below.

Figure 1. Human Dymensions Risk and Safety Maturity Matrix ©

The Human Dymensions Risk and Safety Maturity Matrix shows what kinds of foci are required to go to the ‘next level’. Hudson’s model is overlayed on the left alongside the model, indicating the calculative mindset that understands culture as systems and measurement. The words inside each step represent programs delivered by human dymensions to help organisations unlearn, learn and go to the ‘next level’ in the journey to risk and safety maturity.

See it explained on Youtube: