Advancing the safety of AI-driven machinery requires closer collaboration with humans

As the adoption of highly automated off-road machinery increases, so does the need for robust safety measures. Conventional safety processes often fail to consider the health and safety risks posed by systems controlled by artificial intelligence (AI).

Marea de Koning, a doctoral researcher specialising in automation at Tampere University, conducts research with the aim of ensuring public safety without compromising technological advancements by developing a safety framework specifically tailored for autonomous mobile machines operating in collaboration with humans. This framework intents to enable original equipment manufacturers (OEM), safety & system engineers, and industry stakeholders to create safety systems that comply with evolving legislation.

Balance between humans and autonomous machines

Anticipating all the possible ways a hazard can emerge and ensuring that the AI can safely manage hazardous scenarios is practically impossible. We need to adjust our approach to safety to focus more on finding ways to successfully manage unforeseen events.

We need robust risk management systems, often incorporating a human-in-the-loop safety option. Here a human supervisor is expected to intervene when necessary. But in autonomous machinery, relying on human intervention is impractical. According to de Koning, there can be measurable degradations in human performance when automation is used due to, for example, boredom, confusion, cognitive capacities, loss of situational awareness, and automation bias. These factors significantly impact safety, and a machine must become capable of safely managing its own behaviour.

“My approach considers hazards with AI-driven decision-making, risk assessment, and adaptability to unforeseen scenarios. I think it is important to actively engage with industry partners to ensure real-world applicability. By collaborating with manufacturers, it is possible to bridge the gap between theoretical frameworks and practical implementation,” she says.

The framework intents to support OEMs in designing and developing compliant safety systems and ensure that their products adhere to evolving regulations.

Integrating framework to existing machinery

Marea de Koning started her research in November 2020 and will finish it by November 2024. The project is funded partly by the Doctoral School of Industry Innovations and partly by a Finnish system supplier.

De Koning’s next research project, starting in April, will focus on integrating a subset of her safety framework and rigorously testing its effectiveness. Regulation 2023/1230 replaces Directive 2006/42/ec as of January 2027, significantly challenging OEMs.

“I’m doing everything I can to ensure that safety remains at the forefront of technological advancements,” she concludes.

The research article presenting the findings titled A Comprehensive Approach to Safety for Highly Automated Off-Road Machinery under Regulation 2023/1230 was published in the prestigious Journal of Safety Science. The article provides valuable insights for policymakers, engineers and safety professionals.