Formal Safety Assessments

In this episode of Risk! Engineers Talk Governance, due diligence engineers Richard Robinson and Gaye Francis discuss Formal Safety Assessments. 

Key highlights include:

• A Formal Safety Assessment (FSA) is meant to provide a logical and reasoned argument that can withstand legal scrutiny.
• Formal Safety Assessments should identify the critical issues of concern, the current controls in place, and the further practical controls that could be considered, as well as the reasoning for implementing or not implementing those further controls. 
• Many FSAs lack this logical reasoning and instead just list risks and general controls, without clearly connecting them to the specific hazards. 
• Threat barrier diagrams can help provide a logical structure by clearly showing the issues of concern, the controls, and the consequences. 
• Formal Safety Assessments should be concise and focused, not hundreds of pages long, as the key is to present a clear, robust argument. 
• Formal Safety Assessments should also be regularly reviewed and updated to reflect changing context and availability of new controls, rather than just being recycled from previous versions.

For further information on Richard and Gaye’s consulting work with R2A, head to https://www.r2a.com.au, where you’ll also find their booklets (store) and a sign-up for their quarterly newsletter to keep informed of their latest news and events. 

Gaye is also founder of Australian women’s safety workwear company Apto PPE https://www.aptoppe.com.au.

Transcript

Megan (Producer) (00:00):

Welcome to Risk! Engineers Talk Governance. In this episode, due diligence engineers Richard Robinson and Gaye Francis discuss formal safety assessments.

(00:12):

We hope you enjoy the chat. If you do, please give us a rating and subscribe on your favourite podcast platform. If you'd like more information on R2A's work or have any feedback or topic ideas, you can head to the website www.r2a.com.au.

Gaye Francis (00:29):

Hello Richard.

Richard Robinson (00:30):

Welcome back Gaye.

Gaye Francis (00:32):

Welcome to another podcast session. We're going to talk today about formal safety assessments and what they are and what they mean. I think this is sometimes the bane of our consulting work that we do.

Richard Robinson (00:46):

This certainly is.

Gaye Francis (00:47):

And I guess articulating what we think that a formal safety assessment is may help others when they're putting it together to think through some of these elements.

Richard Robinson (01:01):

Yeah, because they pop up in different places and a lot of people don't call it that, but some legislation, for example, the ESMS (Electricity Safety Management Schemes) in Victoria, called up formal safety assessments, and you might recall...

Gaye Francis (01:10):

So that's the electrical industry.

Richard Robinson (01:12):

Well, you might recall that when we did the functional safety assessment of the trains, that was a formal safety assessment too. So it pops up in a number of places and we do rather find that people, let's just say there's a difference of opinion what constitutes a formal safety assessment. It's an aspect of a safety case, but it's meant to be the core aspect. Now, I dunno how you wanted to actually do this, but I was going to explain what formal actually means by jumping back to a bit of 19th century philosophy.

Gaye Francis (01:38):

You can do that, Richard. You don't get many opportunities.

Richard Robinson (01:40):

In the 19th century, the philosophers used to break philosophy to three parts. They had formal philosophy, natural philosophy, and moral philosophy. Formal philosophy is what these days we call reason or well, it's reason, logic, I guess. Natural philosophy, that was to do with what we these days call science. What we in R2A tend to call the laws of nature to try and distinguish it from the legal concept of natural law. But it's how the world, the natural material x space/universe actually operates. So gravity of 9.8 meters per second square, and other things that Gaye perhaps doesn't wish to recall. And then there's moral philosophy, which is the way the world is supposed to be. And that's normally split into two parts practically. And when we looked at, we've been looking at the way -- we sponsored David Howarth, the professor of public policy from Cambridge -- it's the lawyers who spell out socially how it should be, and the engineers tend to spell out technically in terms of managing the laws of nature how it's supposed to be. Now, if you don't have this understanding of what the formal actually means, just the logical and reasoned argument, which you would think would be self-evident, but you review a lot of formal safety assessments, and they're not logical and reasoned, are they?

Gaye Francis (02:55):

No. And they can be not so concise either.

Richard Robinson (02:59):

Yes. So did you want to summarise that?

Gaye Francis (03:02):

I think when we are looking for that formal safety assessment, that logical and reasoned argument, we want to have those core, credible, critical issues of concern. So in safety terms, it's the things that can kill and maim, which we've always said. What are the current controls in place? And if we go down the SFAIRP option, which is what the WHS legislation/OHS legislation requires, what are the further controls that could be put in place? And then what is your reasoning for doing or not doing those particular further practical precautions?

Richard Robinson (03:37):

And that's the reasoning part.

Gaye Francis (03:38):

And that's the reasoning part. So often when we get an assessment, a Formal Safety Assessment or FSA, often it's almost just their "Risk Register". So it's got what the issue is, what some of the mechanisms are, and then a whole lot of, well...

Richard Robinson (03:56):

They tend to just characterise it by risk, not looking at criticality, which just drives us crackers. The logical argument sort of says, how bad can it be? Not what is the risk, simultaneous appreciation of likelihood and consequence. That'll be totally misleading in safety terms.

Gaye Francis (04:11):

And then they list a whole lot of what they say are controls. But when you actually look through the controls, many of them are procedures, policies...

Richard Robinson (04:22):

Not actually what you're going to do to actually stop this particular nasty thing from actually occurring.

Gaye Francis (04:27):

Correct.

Richard Robinson (04:28):

And then they haven't therefore logically connected the quality system to make sure that particular control is sustained. And sometimes they do because they talk about it as a critical control to deal with a critical hazard and then they're heading in the right place.

Gaye Francis (04:40):

Correct. And I think that's why we favour the use of the threat barrier diagrams, isn't it? You can spell out what your issue of concern is. The barriers or controls are easy to see in a document or in a diagram, and then you've got the consequences at the end. And so then if you list those quality control elements at the bottom, then you've got that logical reasoned argument.

Richard Robinson (05:03):

Well, you can spell out what the current controls are, what the further possible controls are and where they would act in terms of the hierarchy of control that a court will find logical not what our regulators are doing. And we've been writing separately about that to the <inaudible> and we'll see if that has any effect. But then it's actually a reasoned logical argument. I suppose it always mystifies us because you've probably noticed that when we write a report, what surprises most people is it's brevity. Well, possibly its pithiness is perhaps a more accurate statement.

Gaye Francis (05:34):

Concise. Can we use concise?

Richard Robinson (05:36):

Yeah, we can use concise. But normally dealing with a particular issue does not require scads and scads of paper. If you've got scads and scads of paper, you're probably confused and missing the point. And the one thing you can't do is use it to a target level of risk and safety, which again, somebody putting a formal safety case argument together or a formal safety assessment using a target level of risk and safety -- you're kidding yourself, it means you really haven't thought it through.

Gaye Francis (06:02):

Yeah. It's really that being able to present a clearly thought through argument, isn't it? And you're right, it doesn't need to be long. We get these safety cases that are hundreds of pages long and the bulk of it is almost summarised in 10 to 20 pages.

Richard Robinson (06:19):

Or even less. I mean, remember what you're trying to do. Because if that bad thing happens, it will be subject to legal scrutiny. So it's got to make sense to a court. And that court won't be trained in the laws of nature. It'll almost certainly only be trained in the laws of man. And so that mean that argument has to survive that kind of scrutiny. And it's that robustness of the court process. I mean, I can have a lot of opinions about the court process. And we were talking before about the law is much too important just to be left up to lawyers, which I have to say we keep hitting that point and the lawyers keep making that point to us. But if the lawyers don't understand the laws of nature and really are just relying on the experts, technical experts, to explain the laws of nature, that really does mean that there's a bit of a gap in the way in which human beings and our society actually make decisions.

Gaye Francis (07:11):

I think also coming back to the formal safety assessment is you really want to see what controls they're testing for or precautions.

Richard Robinson (07:19):

What further controls they're testing for.

Gaye Francis (07:21):

And that's often the element that is missing in the FSA that we have seen.

Richard Robinson (07:26):

Correct. And that's the thing the courts will test for.

Gaye Francis (07:28):

So especially when it goes to a regulator or it's part of a bigger safety case, they're asking to see: we understand that you've got these controls in place.

Richard Robinson (07:37):

Typically the ones that are Standards, most of them, they've complied with the regulation.

Gaye Francis (07:43):

Yes.

Richard Robinson (07:43):

But the question was: Have you tried to do more than just the regulation or standard? And that's the bit they're not testing for properly.

Gaye Francis (07:51):

And it's okay not to do some of those things, but you've got to have an argument as to why you haven't done it. And that's where you would expect to see it would be in the FSA, Formal Safety Assessment.

Richard Robinson (08:00):

The one that always puzzles us, I suppose, is the REFCL (Rapid Earth Fault Current Limiter), the ground fault neutralises for the high voltage, 22 KV circuits in South East Australia and so forth. And the only jurisdiction that has actually adopted it is so far is Victoria. And the reason is because the AEER, the Australian Energy Economic Regulator, basically said, no, you can't do it, we won't let you have the money to do it.

Gaye Francis (08:22):

Well, it won't be funded through the regulator. Now they could decide that in some locations they really do need to do it for bushfire.

Richard Robinson (08:29):

Well, that's correct, but they won't get extra funding for that purpose. That will be their decision, so therefore their loss of profit to achieve that outcome.

Gaye Francis (08:38):

But as a board, they could make that decision, but they need to document that that's the reason that they're not going to do it. And they have considered it in all of their circumstances because all of this is contextual, remember? So everyone's FSA is not the same thing.

Richard Robinson (08:54):

Well, even Victoria, I mean the reference got put into the high bushfire prone areas, but that means somebody decided which was high and which was low bushfire. And that's not actually a binary distinction. That's obviously a graded thing. And therefore you'd probably find the fact that you can have an argument in some areas where that's exactly correct, and that would depend on the actual circuit layout and all sorts of things.

Gaye Francis (09:15):

But then there were other organisations that said, we've only got a small high bushfire consequence area, we're going to underground ours. Because that was better for them or that was their reasoned logical argument going forward.

Richard Robinson (09:29):

Well, particularly they got an isolated high bushfire consequence area, just undergrounding. You might just say, well enough, we'll just underground those ones and not worry about the REFCL. That's correct.

Gaye Francis (09:36):

And I think that feeds into some of the questions we're being asked around these formal safety assessment is what is the context? And I think that context is changing. So this is not just put on your shelf, do it once, put it on the shelf and forget about it. This is something that you've got to keep revisiting and whether it's on milestone occasions or when things change or whether it's on an annual or every couple of years basis. But these things have to keep evolving and we're certainly not seeing that in some of the formal safety assessments that come through to us.

Richard Robinson (10:06):

Yeah, it's usually the previous one recycled,

Gaye Francis (10:08):

Correct? Correct. And can we update it accordingly?

Richard Robinson (10:12):

Yep.

Gaye Francis (10:13):

So just be careful of formal safety assessments. For us, it's that logical reasoned argument.

Richard Robinson (10:18):

That would survive post-event legal scrutiny.

Gaye Francis (10:21):

And I think the key element that we see missing is that consideration of what the further practical controls are or possible controls are, and the argument as to why you are or aren't going to do something. I always say that I think it's more important to document why you're not going to do something compared to why you are going to do something, and then say when you're going to revisit that, because technology becomes more available, more robust, more reliable.

Richard Robinson (10:47):

Yep.

Gaye Francis (10:48):

So thanks for joining us, Richard. Hope everyone found that interesting.

Richard Robinson (10:51):

Thanks Gaye.