Back to Articles

A Picture of Safety

Artificial intelligence to support safety programmes

by André Seichi Ribeiro Kuramoto, Petrobras

Share:

Considering all the knowledge and technologies of work safety engineering feasible to be implemented in the workplace, personal protective equipment is the last barrier to ensure safety for workers.

The programmes developed by companies for the proper use of PPE by workers have always been challenging. These programmes involve training, promoting positive culture, policies, and supervising the use of PPE by workers. Artificial intelligence for analysing images has developed significantly in recent years. As a result, software development companies have offered solutions to analyse worker images in real time and to support industries in the implementation of effective programmes to ensure adequate use of personal protective equipment for the workers.

PPE as the last resort

Ideally, for the safety of people, dangerous activities should not be performed by people, but by automated systems such as conveyors, actuators, robots, etc.

However, some risky activities in the heavy industry are still dependent on humans, such as welding, painting, equipment installation, ship repair inspection, automotive repair, civil construction, and others in assembly industries. The human presence in these activities is mainly due to the fact that automation technology is not yet developed enough, which ultimately means that the technology is not readily available, accessible or affordable for the companies.

When it is not feasible to remove people from risky activities, safety engineering makes use of knowledge and technology to design safe industrial facilities and equipment for workers. In addition, engineering controls and collective protection equipment also complement worker safety.

According to legislation and in order to reduce health risks, personal protective equipment (PPE) must always be regarded as a last resort to protect workers1. The design of safe facilities and equipment, as well as engineering controls, should always be considered first.

Some examples of PPE organised by categories commonly used in heavy industry are listed below:

  • Head protection – hard hats
  • Eye protection – goggles and visors
  • Respiratory protection – half face mask and full air-fed masks
  • Hearing protection – earmuffs and plugs
  • Hand protection – gloves
  • Foot protection – safety boots
  • Skin protection – pants and longsleeved clothes for working with chemicals, radiation hazards, welding, painting or adverse climatic conditions

Ensuring proper use of PPE

There is a wide range of types of PPE to provide safety for the workers. To choose a type of PPE the employer must consider the environment (temperature, humidity, ventilation, etc.) in which workers will be exposed to risk. Then, both safety and comfort when wearing the PPE are important issues. In addition, it is necessary to train and use policies to ensure that workers are not only wearing the PPE, but also wearing it in the proper manner.

The trainings address, for example, the risks present and why the PPE is needed. Moreover, training demonstrates how specifically to use, maintain, and store each type of PPE, and also the limitations of each variation of PPE, how to identify any defects in the equipment, and the procedure to obtain replacement PPE.

A strategic policy needs to include a PPE risk assessment, advice, and guidance for supervisors and employees. The policy should include1:

  • The legal duties
  • The promotion of a positive PPE culture
  • The roles and chores of supervisors and managers in ensuring the use of PPEs

Observing workers in their activities has been one of the roles of supervision in order to identify safe behaviour and also the proper use of PPE. This activity is carried out by sampling, given the time constraints and supervision resources.

Once difficulties in using PPE have been identified, it is the role of managers and supervisors to take actions such as training, promoting a positive culture of the use of PPE, and choosing types of effective and comfortable PPE for the workers.

Artificial intelligence

The development of technologies related to artificial intelligence (AI) and cognitive computing has enabled object in context detection and human pose estimation (identification of the position of people) at scale and in real time.

Machine learning and deep learning algorithms for image analysis are increasingly efficient. Since 2015, the accuracy of object detection in an image by machines exceeds the accuracy obtained by humans.

Good software frameworks are being developed in communities and open sources, and therefore are accessible for applications regardless of the size of the industry. Some examples of such frameworks are Tensorflow2, Pytorch3 and OpenCV4. These are efficient in creating datasets and implementing a type of artificial intelligence structure called artificial neural networks.

The cost of processing capacity to apply these frameworks was an important restriction in the past. However, currently, there is specific small size and low power hardware available at affordable prices (a few hundred dollars) to implement artificial neural networks in real time, which are capable of executing about four Tera logical/mathematical operations per second. This hardware can be placed close to the cameras that capture the images5,6,7. This type of processing is called Edge Computing. Thus, there is no need for high-speed networks for video streaming or large and expensive data centres to process the images.

This AI technology for object in context detection has become popular and has been applied to consumer products such as autonomous vehicles. The vehicles are equipped with multiple cameras that capture dozens of images per second. Then, an AI detects situations by analysing these images. The interpretation of images in addition to several other sensors (for example distance sensors) are inputs for the decision-making system of an autonomous vehicle.

“since 2015, the accuracy of object detection in an image by machines exceeds the accuracy obtained by humans”

Evolution of object detection techniques

Object detection uses machine learning techniques in which a model is commonly created from a previous step called training, or sometimes from manual design. The training is carried out with samples of the object to be detected. Using this model, AI is able to detect the trained objects in new images.

The first framework to present sufficient performance for object detection in real time was proposed by Viola and Jones in 20018. The main limitation of this framework is the need for the object to be fully visible in the images as it was presented before in the AI training step.

In 2012, Convolutional Neural Networks (CNN) became the standard for image classification after the performance achieved by the technique showed in9 during ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC 2012)10. The goal of this competition is to estimate the content of images by machines. The AI training step is made with a large hand-labelled dataset called ImageNet11. Some new images are presented with no information and the algorithms must specify what objects are present in the images and the location of objects.

After 2012, variants of CNN have been used to detect objects in context and the accuracy, as said before, in 2015 this detection exceeded human capacity. Recently other techniques have been developed that are also efficient for ‘object in context’ detection, such as YOLO (You Only Look Once)12.

AI as supervisor support

With the popularity of AI applications, scientists and data engineers, and software development companies have been developing systems to detect risk situations in the execution of human works.

Since 2017, object detection applications have been proposed by tech companies like Microsoft to identify incidents or risk scenarios in construction, industrial, and hospital environments13,14. A simple security camera of a CCTV (closed-circuit television) combined with AI algorithms can detect the hazard scenarios, for example, workers on hazardous proximity zones of active heavy equipment. Additionally, the detection can be used for automatic alerts or interactively setting the safety mode of equipment, as smart equipment is transforming into IoT (Internet of Things)-like devices in the Industry 4.0 revolution.

There is now some commercial software capable of detecting the use of PPE. The company Cortexica (now acquired by Zebra Technologies) showed its software combined with an actuator to block the entrance to the risk area when the worker was not wearing a specific PPE15. The companies NextCam and VEER Video Analytics offer analytics and PPE detection to improve safety and reduce the number of accidents at construction sites16,17. The company Bleeco highlights that their AI software toolset is proper to prevent accidents, improve safety culture, improve the productivity of occupational safety specialists18. Others companies Provectus, Vitech Lab and Cisco19,20,21 also offer software solutions to automate the 24/7 monitoring of the use of PPE in real time and improve safety.

For some years I have also been trying AI technologies for detecting objects in context with a focus on detecting risk situations for workers and specifically detecting the use of PPE by workers22,23.

All these applications aim to complement the supervisory activity to ensure safe behaviour and the proper use of PPE by workers. Thus, the limitation of supervision time for identifying deficiencies in the proper use of PPEs can be multiplied with the use of AI.

The responsibilities of supervision and management mentioned above are not replaced by AI, as the management actions to ensure the proper use of PPE is not carried out by the AI. The purpose of AI is to analyse the images and generate reports of the deficiencies in the proper use of PPE, statistical reports and storage of this information for later analysis.

The actions resulting from the analysis of the inappropriate use of PPE can be, for example, to adopt more comfortable PPE, such as lighter and more breathable equipment in order to avoid the eventual annoyance and overheating experienced when wearing PPE; to adapt the work environment to be suitable for the use of PPE. For instance, more ventilated and cooler environments so that the use of PPE does not cause too much heat to workers.

Conclusions

Today, there are still risky activities that are performed by humans and it is not feasible to replace the humans with automated systems because the technology is not yet developed enough. For these activities, safety engineering techniques are used since the design of the workplace and even the equipment used by them. Control engineering and collective protection equipment are also used; PPE is the last resort.

The programmes to ensure the proper use of PPE must involve training, company policies, and supervising. One of the most fundamental roles of a supervisor is observing workers regarding the proper use of PPEs. However, supervision resources are limited, and it is not viable to supervise all workers permanently.

Technologies related to object in context with AI have become popular in recent years. Since 2015, machines have been able to detect objects more accurately than humans.

Hence, it becomes interesting to use machines to detect the proper use of PPE. AI can be associated to images of CCTV systems already installed in factories to analyse images regarding the use of PPE. Software development companies have come up with AI solutions to oversee the proper use of PPE.

This technology is an input for the activity of supervising and managing worker safety. However, the responsibilities of supervision and management mentioned above are not replaced by artificial intelligence. The AI does not establish the management actions that will guarantee the proper use of PPE and moreover, safety for the workers; at least for now there are jobs that human still beats machine.

References

  1. Toby Hayward, “The Last Resort”, Heath and Safety Magazine International, Apr 2016. www. hsimagazine.com/article/the-lastresort-1232
  2. TensorFlow. www.tensorflow.org
  3. PyTorch. pytorch.org
  4. OpenCV. opencv.org
  5. Intel Movidus www.movidius.com/
  6. Coral AI coral.ai/
  7. nVidia Jetson developer.nvidia.com/ embedded-computing
  8. Paul Viola and Michael Jones, “Robust Real-time Object Detection”, International Journal of Computer Vision, 2001.
  9. Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, “ImageNet Classification with Deep ConvolutionalNeural Networks”, Advances in neural information processing systems 25(2) · January 2012.
  10. ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC 2012). image-net.org/ challenges/LSVRC/2012/ index#introduction
  11. ImageNet web site. www.image- net.org/
  12. Joseph Redmon and Ali Farhadi. “YOLOv3: An Incremental Improvement”. arxiv.org/ abs/1804.02767
  13. Microsoft Builds 2017 Day one May 10th 2017, “Workplace Safety Demonstration”. www.youtube.com/ watch?v=pL-c00M2CnI
  14. Cortexica. www.cortexica.com
  15. Real-time PPE Monitoring on the Edge. github.com/cortexica/intelrrk-safety
  16. NextCam. www.next-cam.com
  17. VEER – Video Analytics. veertec.com
  18. Bleenco. bleenco.com
  19. Provectus. provectus.com
  20. Vitech Lab. vitechlab.com
  21. Cisco. github.com/CiscoDevNet/ ppe-detection
  22. Artificial intelligence and computer vision improving safety. www. linkedin.com/feed/update/urn:li:acti vity:6480237103799156736/ and www.youtube.com/ watch?v=HQ-PJV1LP_Q
  23. Artificial Intelligence detecting the use of PPE. www.linkedin.com/ pulse/artificial-intelligence-detectinguse-ppe-andr%C3%A9-kuramoto/ and www.youtube.com/ watch?v=xwlvi5LIA2M

Author Details

André Seichi Ribeiro Kuramoto, Petrobras

André Seichi Ribeiro Kuramoto is an electronic engineer. In December 2012 he received a PhD in System Identification from the University of São Paulo, Brazil, Since 2016 he has developed his skills in innovation, agile, data science and artificial intelligence applications, as student at University of California, Los Angeles (UCLA), Stanford University and local universities in Brazil. His research areas of interest are system identification, machine learning, deep learning, time series forecasting, object detection and internet of things (IoT). Kuramoto’s current position is digital technologies consultant at Petrobras S. A., Brazil.