Season 7 Wrap: The Growing Divide of Moral Imperative & Commercial Reality in WHS

In this season finale of Risk! Engineers Talk Governance, due diligence engineers Richard Robinson and Gaye Francis reflect on Season 7's central theme of SFAIRP and the growing divide between moral imperative of work health and safety legislation and the commercial pressures organisations face in practice.

After their recap of topics covered, the conversation focuses on AI's growing role in decision, using marine pilotage as their example. Richard outlines ways AI could be applied, such as AI as the pilot, with crew simply responding to its instructions; a smarter Personal Pilotage Unit (PPU) that draws on historical passage data of what previous pilots did under similar conditions, while leaving the final call to the human pilot; and TCAS (Traffic Collision Avoidance System) where the system directs action in an emergency. 

The finish the episodes with their concerns of how commercial forces are pushing AI tools as replacements for human judgement, while the safety case for that level of trust hasn't been made, and the instinct to treat AI output as gospel needs to be actively resisted.

Thanks for all your support in Season 7! Watch out for Season 8.

If you’d like us to cover a specific topic or have any feedback we’d love to hear from you. Email admin@r2a.com.au.

For further information on Richard and Gaye’s consulting work with R2A, head to https://www.r2a.com.au, where you’ll also find their booklets (store) and a sign-up for their quarterly newsletter to keep informed of their latest news and events.

Gaye is also founder of Australian women’s safety workwear company Apto PPE https://www.aptoppe.com.au.

Transcript

Megan (Producer) (00:00):

Welcome to Risk! Engineers Talk Governance. In this episode, due diligence engineers, Richard Robinson and Gaye Francis wrap up the season and the theme of the growing divide of moral imperative versus commercial reality.

(00:16):

We hope you enjoy the chat. If you do, please support our work by giving us a rating and subscribing on your favourite podcast platform. And if you'd like more information on R2A, our newsletter and resources, or have any feedback or topic ideas, head to the website www.r2a.com.au and we look forward to seeing you in Season 8!

Gaye Francis (00:40):

Hi, Richard. Welcome to the season wrap for season seven. SFAIRP, the moral imperative versus the commercial reality. It's been an interesting season and I think we've covered quite a lot of topics.

Richard Robinson (00:53):

Somewhat to your surprise, I've decided.

Gaye Francis (00:56):

Very much to my surprise. So (episodes): To Grock or Not. Resilient Infrastructure. Risk and Adaptation Strategies. Delayed Decisions to Avoid SFAIRP - I think that's been really well listened to. We did a special on the New Zealand amendment to their Health and Safety at Work legislation to focus on criticality, which we really applauded, and we're hoping that Australia goes the same way. Ignorance in Health and Safety. Cunning versus Smart in Leadership. How Information Sharing has Changed with SFAIRP in the Internet and the Public Sphere. And then Risk Curves was our last podcast for this season before this one.

(01:40):

We were having a discussion before we started recording this about what we were actually going to cover in the wrap and the little joke we may have to contain, Richard. There's some hot topics here that he's got some passion about. But I think one of the key things was that we're seeing that divide between what the moral imperative of as the work health and safety legislation and safety legislation versus the commercial reality. And I think the divide's getting bigger.

Richard Robinson (02:09):

Yes.

Gaye Francis (02:12):

Also, it's the way that people are now assessing what to do and the initiatives that are being taken and the innovations that are needed. And we are relying heavily on the ... And so this is now your opening, Richard. We're now relying heavily on the internet and AI in particular to help us inform those decisions, although more people are starting to rely on AI to potentially make decisions.

Richard Robinson (02:44):

Yes. Well, your children, for example, tend to regard it as gospel.

Gaye Francis (02:47):

They do. We've had the discussion and our producer did say that my girls are used as examples in these podcasts, but often that's the first thing that comes up on the internet when they do a search is the AI summary of whatever they've put in to ask a question or something. And I had to say to them, where did the source come from? And they didn't (know). They looked at me as if I was stupid and didn't really understand that.

Richard Robinson (03:14):

I don't know if you've tried it a couple of times, but I did a couple of searches on due diligence and safety due diligence and so forth and the reference is R2A.

Gaye Francis (03:21):

Oh, there you go.

Richard Robinson (03:23):

And it's the core reference. Now that was a bit of a surprise to me because I don't even know which AI I was using because Apple I think has gone with Google, I think.

Gaye Francis (03:33):

Right.

Richard Robinson (03:34):

So it's a bit hard to tell sometimes what's actually going on out there. But anyway, you want me to say my piece now?

Gaye Francis (03:40):

Yes, you can say your piece now.

Richard Robinson (03:41):

Well, I've been invited to give a presentation for the Marine Pilots in Darwin in the next month or something like that. So I was trying to work out what to say and talking about it. And one of the things that sort of pops up obviously is what's going to be the role of AI for marine pilotage. Now the whole point about marine pilots is that they're there because they're dealing with critical things because collisions or groundings with ships is bad news. And if you have a major environmental contamination or what happened in the Jolly Nero in Genoa and you knock the control tower down or something like that and a lot of people get killed, all hell breaks loose.

Gaye Francis (04:12):

High consequence, low likelihood.

Richard Robinson (04:15):

And that's the reason why marine pilots exist and why they norminally get civil immunity from all these sorts of things as long as they're acting in good faith. Now the trick is of course, because the SFAIRP business and the WHS legislation, you've got to do all reasonably practical things. And one of the things that pops up, obviously, what's the role of AI and what benefits can it provide you and ought you to be considering this. Now we've sort of looked at over time different places where things could act. I mean, we sort of commented when we're doing the grounding in Tai Ping in Southport, New Zealand all those years ago now.

Gaye Francis (04:45):

Almost 20 years ago, I think it was.

Richard Robinson (04:46):

Closer to 25 Gaye.

Gaye Francis (04:48):

Okay.

Richard Robinson (04:50):

And so forth. But one of the things we noticed, there were three pilots on the water, one on having the conduct of the vessel and one on the tug tied past the stern, and one on the pilot at the other end of the channel. And because it's very short, quick passage, as they approach their abort point, the point of no return, the pilot on the bridge asks the other two pilots, "Oh, look." But that means they've effectively got three pilots - first, second and third pilot, which is obviously very rare. But because it's such a short passage and it's convenient to do and you have the resources, it was reasonable in the circumstances and because the passage was a particularly awkward and difficult passage. Now obviously the question of a second pilot...

Gaye Francis (05:27):

Is coming up more and more, especially as the operating envelope of the pilotage...

Richard Robinson (05:35):

Bigger vessels, port size not changing, wider conditions, bring it in no matter come hell or high water, commercial pressures.

Gaye Francis (05:42):

The constraints are becoming more restrictive, aren't they?

Richard Robinson (05:46):

But if it's a two and a half hour pilotage coming down the Brisbane river or something like that, second pilots are expensive.

(05:52):

And so that obviously begs the question, what could an AI do for you? And it actually can do a number of things. One of the things that's popped up in the last 30 years obviously is the personal pilotage unit, which is a backup independent NAVAID, which these days looks at six or seven satellite geosatellites up there, but you could actually, at the highest level, actually have the AI as the pilot in effect and the ship crew just responding to the AI. Now that's a pretty scary idea, particularly since AI seems to hallucinate a fair bit of the time and tells you what you think they want to hear, not what is actually the case. We did do assessment on large language models and we won't talk about them again now, but they're just for the most part, giant inference engines. And so the more data they've got and the more times the pilot has been done, the better information you'd have.

(06:43):

But the other way to do it, for example, the PPU could actually be upgraded to in effect what would be a watchdog and it would say, over the last 2000 passages and this thing under these conditions of this size ship, here's the recommended path that you should probably follow. And if the wind or something suddenly changes, here's what the other pilots and the other times did, what would you like to do? And you have to still make the decision.

Gaye Francis (07:05):

But then you're using it in two different ways. The second way that you're using it is to improve situational awareness and to give you as much information, but the pilot is still making the decision of what they do in the circumstances.

Richard Robinson (07:18):

I suspect that's not going to change. I reckon the pilot's still got to be in charge.

Gaye Francis (07:21):

The other one is almost going to the pilot model of the TCAS system that we've talked about where it's advising or instructing you what to do.

Richard Robinson (07:35):

TCAS (traffic collision avoidance system) get a resolutionary advisor for two approaching aircraft, go up, go up, go up, electronic voice to one pilot, go down, go down, go down, and electronic voice, the other pilot. And if you get that message, that's what you do.

Gaye Francis (07:48):

But I don't think AI is at a robust enough state to be able to do that.

Richard Robinson (07:54):

Well see, it depends what you mean by AI. And here's where it's starting to get messy because the current investment is for giant data centers. There's one popping up in East Fooscray, apparently which is driving the locals nuts. Huge power consumption, lots of heat and it becomes obsolete within two years and everything goes to the data center and the data comes back and all these people are trying to figure out how to make money out of you, but you still got to rely on internet communication. And in real time, are you going to rely on something that relies on internet communication even if Apple has done a deal with Starlink? I don't think so. But then you've got the other model, which is what Apple's trying to do. They're trying to drive AI down to the local device and you're looking at the two commercial of those two, I reckon Apple's got the edge. I think they're more likely because their chips now are so far ahead, it's getting embarrassing. Their latest, if you buy a MacBook, it's just a big iPhone. It's not actually...

Gaye Francis (08:54):

A computer.

Richard Robinson (08:54):

Well, it is a computer, but it's a giant iPhone in effect. And if that's what a PPU turns into with massive computing power, the M5 chip and it's got the AI on board with all the passage plans that have been used on that passage for the last 20 or 30 years into it and it's telling you this starts to happen, this is what the other pilots did.

Gaye Francis (09:17):

But you would want to think that through as you're doing it though. I mean, that's some of these things that we've talked about in this podcast about the ignorance and delaying decisions and all of that sort of thing. If you go back to just relying on an AI computer model to tell you...

Richard Robinson (09:33):

If the ship blacks out at this point unexpectedly. Under these weather conditions. This is what the last guy did.

Gaye Francis (09:39):

Okay. But then you're using it as by exception, whereas I think they want us to use it as the tool to do all of these things.

Richard Robinson (09:48):

Yes, that's right.

Gaye Francis (09:50):

So from a commercial imperative or from a commercial reality viewpoint, they're pushing these tools toward you, but from a safety and a moral imperative, we really have to treat some of these tools with a little bit of caution, maybe?

Richard Robinson (10:05):

Skepticism perhaps.

Gaye Francis (10:07):

Maybe I've got a bit too much, but I think that balance hasn't been put in place yet, especially around the safety aspects.

Richard Robinson (10:17):

And the commercial imperative, because I don't know how much money and how many billions and billions these people are spending on the giant centers, but they want to make a pile of money out of everybody and tie you into it. And I don't think as a society, I mean, you've heard me saying how I'm looking forward to disconnecting myself from a lot of this stuff.

Gaye Francis (10:35):

He reckons he's going to go back to a wind watch when he retires.

Richard Robinson (10:38):

I am. Well, I think I might go for one for sooner than that, the way it's going.

Gaye Francis (10:43):

So it's just an interesting space that we're in. There's this push to automate and have AI do these things and start to make some of these decisions. But I think we've got to step back, especially in the health and safety sphere and just treat it with a little bit of caution.

Richard Robinson (11:03):

With any luck, I've managed to give you a substantial grounding for how you're going to address your children when it comes to the use of AI.

Gaye Francis (11:10):

Carefully, I think the answer is.

Richard Robinson (11:12):

Very carefully.

Gaye Francis (11:13):

So I think that's a good place to finish season seven. We will be back for season eight. Haven't quite got a topic in mind yet, but we will be back next season. So thank you again for joining us. Thanks, Richard.

Richard Robinson (11:24):

Thanks, Gaye.