Questions about procedures, checklists, their adequacy and effectiveness are frustrating to many of us across industries. The popular belief – An accident happened because procedures were not followed. But could it also be that the procedures could not be followed? This is a theme that we explore in this article.
When safety tools serve another purpose
An accident happened because a procedure was not followed.’
This has become a popular belief when analysing accidents in the maritime industry. And the countermeasures to prevent accidents often invoke even more procedures. The underlying assumption is that procedures act as barriers between safe operations and accidents, and hence safety results from following the procedures. However, we rarely ask why people deviate from the procedures, or how exactly procedures ensure safety. And what is the purpose of procedures in the first place? This paper is an enquiry into procedures, checklists and similar instruments of formal knowledge incorporated in the safety management system (SMS) and their contribution towards managing safety.
Procedures and checklists
The ISM Code requires ship operators to establish procedures, plans, instructions and checklists for all key operations. For the purpose of this paper we will focus on procedures and checklists only. A procedure is typically a message from the managers to the employees about ‘How we carry out work in this organisation’. Examples may include procedures for maintenance of firefighting and lifesaving equipment, procedures for cargo operations, procedures for navigation, etc. A procedure may be anything from precise instructions for carrying out a task to a general policy statement; it can be specific or abstract; it can be of good or poor quality. A procedure may serve different purposes or even multiple purposes. It is critical to be aware of these differences when using procedures as a tool for managing safety, and to understand why procedures cannot always be followed. A checklist, on the other hand, serves as a reminder to ensure that in undertaking a task the essential actions (or checks) have been considered. A checklist could also be seen as a ‘precondition’ to be met before initiating a task. In the hierarchy of formal knowledge within the SMS, checklists may be used to break down high level procedures into a group of tasks and sub-tasks. For instance, navigation procedures may include a checklist for port arrival and departure, a checklist for navigation with pilot onboard, a checklist for anchoring etc. A checklist is therefore an integral part of a procedure.
Even the checklist for a routine operation such as departure from port appears highly problematic (see case study, below and right). It is apparent that the document serves multiple purposes – an aide-memoire, a work instruction, an instrument for allocating responsibility and monitoring the conduct of seafarers, as a record and even as a tool for risk assessment. On the face of it, imperfect checklist design simply reflects the competence of those responsible for implementing the safety management system. However, careful examination reveals that the problem arises due to the multiple and conflicting goals of the safety management system.
Bridge procedures and checklists – a case study
The checklist opposite was collected during an accident investigation involving a small offshore supply ship. It is representative of checklists used for navigation on ships by many companies. As such, it serves as an appropriate case for review and analysis.
The ‘Bridge Checklist’ shown here was a sub-document related to two procedures in the vessel’s safety management system: ‘Procedure for the start-up of bridge prior to departure’ and ‘Procedure for readying the bridge prior to arrival/port’. Both procedures were situated in the vessel safety management system section 7, ‘Shipboard Operation’, subsection ‘Arrival, departure and mooring procedures’, and referred to this checklist for arrival and departure from port.
Ideally, the checklist is to be used as an aide-memoire. However, in this case it appears that the entire description of starting up and readying the bridge has been transferred from the procedure to the checklist. Thus, the checklist has been extended from an aide-memoire to a work description.
The checklist consists of eight categories. Not all checks are phrased in the same manner. While some checks specify actions (eg ‘port notified’, ‘PAX boarding completed’, ‘AIS updated’) others merely identify equipment (eg ‘GPS’, ‘Log’, ‘NAVTEX’ and ‘Fire alarm and control panel’). There are only two options for completing the checklist – a tick or N/A (not applicable). If a task is not applicable, why should the action appear on the checklist in the first place? Furthermore, there is no room for recording negatively or commenting against an item in case of malfunction or Underperformance. The purpose of completing the checks is also unclear. For instance, does the tick confirm that the instrumentation is functional, or is it merely to ensure that it is turned on? And what if the equipment is functional but not up to the required standards of safe operation (for example, the gyro compass exhibiting high error or a radar with questionable performance)? Is the checklist intended to ensure that the tasks have been completed or is it merely for the purpose of record keeping?
Notice also the difference in the nature of tasks included in the checklist. At one extreme, the checklist is intended to ensure that even minor details such as window wipers are included.
At the other, the checklist includes a broad-brush statement to ensure ‘risk assessment carried out’. Once a detailed procedure has been designed for a routine operation, what is the purpose of incorporating risk assessment? At the bottom is a section to ensure that a responsible person has completed and signed the checklist. What is the purpose of including this section? More specifically, how does this improve the safety of operations?
The multiple goals of the SMS
Safety is often perceived as a standalone goal isolated from the wider activities and functions of the organisation. This is evident when designing a ‘safety management system’ as required under the ISM Code which is primarily focused on ensuring safety (and environmental protection). However, managing safety risks is not the end goal of any organisation. Rather, they are often looking to control the absence of safety for economic and reputational reasons. It is here we need to take a closer look at the ambiguous nature of checklists and procedures.
The paradox of self-regulation
A key purpose of the ISM Code was to move away from a prescriptive regulatory framework established by external authorities towards self regulation. Instead, the ISM Code would allow companies to design a safety management system that was best suited for their requirements with minimum interference from regulatory and commercial bodies. Ideally this would mean that procedures and checklists were designed to reflect the specific nature of each ship and shipboard operations.
In practice, however, this is far from achievable. For example, the charterers may require a certain task, checklist or an entire procedure to be added to the safety management system. In the example checklist on the previous page, the requirement to carry out a ‘risk assessment’ or ‘toolbox talk’ may not be there for operational reasons, but due to a specific legal or contractual requirement. An increased number of checks may even be required in order to look good to the customer. Similarly, if a charterer has requested (but not mandated) a certain checklist or procedure, it may be included in the spirit of ‘good customer relationships’. However, since the task was introduced for non-operational reasons, its presence may not always make sense to the seafarer.
This is the paradox of self-regulation, and explains why a safety management system designed to move away from prescriptive regulations and towards goal based self-regulation may not necessarily meet its original intentions.
Responsibility aversion
The ISM Code acknowledges the problem of allocating responsibility for the SMS to the Master of the vessel alone. The Code requires every company to appoint a Designated Person Ashore (DPA) with the overall responsibility of monitoring the safety of the entire fleet. But the allocation of responsibility becomes problematic when the end goal is to manage safety risks. In the event of an accident the decisions and competence of those responsible for managing safety ashore come under scrutiny.
One ‘solution’ to the problem lies in designing generic procedures and then leaving it up to the seafarer to decide between what is relevant and what could be left out as not applicable (N/A). There are practical reasons for this as well. Since procedures cannot always be designed for every conceivable situation, keeping them generic and open-ended allows companies to reduce the level of documentation within the safety management system. All this creates discretionary space for the end user of the procedure (in this case the seafarer) to manage the gap between what can be thought out and documented in procedures and what really happens in an uncertain, dynamic work environment.
In most cases the seafarer will succeed in bridging this gap effectively. However, in the event of an undesirable outcome such as an accident, the generic nature of procedures also creates ample buffer space for the shore based office. The investigation inevitably points to the seafarer, demanding that they should make better choices within the same discretionary space, while leaving aside the capabilities and competencies of those responsible for the design and implementation of the checklist. The ISM Code is intended to ensure that the responsibility for safety management is effectively shared between the ship and the company. However, an undesirable outcome may lead to risk and responsibility aversion from the shore side, especially when safety risks are meshed with commercial and reputational risks.
“When professional judgement is replaced with checklists, seafarers adopt a casual approach to these control measures”
The missing link
A careful examination of the sample checklist shows how responsibility is pushed downstream. While certain checks spell out clear expectations (for instance ‘AIS updated’), others are ambiguous and open-ended, such as ‘radars’, ‘GPS’, ‘auxiliary engines/generator’, ‘window wiper’ and ‘auto-pilots’. This missing text raises doubts over what is considered to be a ‘completed check’. For instance, if the auxiliary engines fail to perform prior to departure from port, does this mean that the vessel will be held up in port until the problem has been rectified? And what if one of the window wipers does not operate? How is the seafarer expected to react to this situation given the limited options on the checklist – tick off or decide it is not applicable?
By swinging between the two extremities of ensuring that even minor details such as window wipers are included, and at the same time that an overall risk assessment is carried out, the shore based management aims to encapsulate the risk management of everything in one document.
When a seafarer puts their signature at the bottom of the checklist, this is not only an affirmation that they have completed the checks, but also that they take overall responsibility for the imperfect and ambiguous nature of the checklist. In the event of an accident, it is issues such as this ambiguity that become the subject of detailed investigation. The risks arising from an imperfect procedure are transferred to seafarers who may not have had much involvement in designing those procedures in the first place.
‘Defensive proceduralism’
The problem is not limited to the ambiguous nature of procedures. It is also the volume of procedures, checklists and tasks that are introduced in the wake of an undesirable outcome. As one seafarer stated: ‘We had a communication breakdown with the engine room while departing from port – and the next thing was that two more checks were added to the checklist. They never remove anything, only keep adding.’
On the surface this may appear like an appropriate move to limit responsibility and safeguard the reputation of the company. The danger arises when everyone from the top management down to the seafarer becomes concerned with managing their own risks, at the expense of everyone else’s. All this creates a risk in itself.
When professional judgment is limited and replaced with procedures and checklists, seafarers respond by adopting a casual approach towards such control measures. Over time, checklists and procedures are met with a mere ‘autotick’ response. They move from facilitating the safety of operations towards being record-keeping for audit trails of risk management. Contrary to popular belief, the effectiveness of these documents in preventing accidents becomes questionable.
Barrier or catalyst?
We return to the original theme of discussion – an accident happened because a procedure was not followed. Starting from this point, we get into the mind-set that procedures are intended as ‘barriers’ (or deterrents) against accidents. We are therefore convinced that procedures should be followed. This paper was an attempt to initiate a discussion on this topic. By examining a specific case this paper has highlighted that procedures are not always introduced for safety reasons. Given the conflicting and competing goals of safety management system, procedures may be introduced for economic, legal, and reputational reasons. All this may turn procedures away from their original objective of managing safety risks. Delving deeper into the case, we find that where procedures are abstract and ambiguous, this may be a deliberate move to limit the risk and responsibility of the company.
When accidents happen, the imperfect design of procedures becomes a source of individualised risk for the end user.
When the blame game combines with the risk game, the result could be a defensive attitude to both designing and following procedures. This may turn procedures into imaginary documents which have little to do with the realities of work. The role of procedures in such instances becomes even more questionable.
Procedures may be intended to manage several risks; their role in enhancing safety may not always be as straightforward. An accident happened because procedures were not followed. Could it also be that procedures could not be followed?
About the authors
Nippin Anand works as a Principal Specialist, Safety Management System at DNV GL. The views expressed by the author in this paper may not be the views of the organisation that the author represents.
Øssur Jarleivson Hilduberg is Head of the Danish Maritime Accident Investigation Board.
The paper is an edited section of a recent report published by the Danish Maritime Accident Investigation Board (DMAIB) that undertakes a detailed investigation into the role of procedures in safety management system.
The full report can be accessed at http://bit.ly/2bKdtvA
