The Pipeline and Hazardous Materials Safety Administration (PHMSA) issued an Advisory Bulletin to operators of gas transmission and hazardous liquid pipeline facilities regarding their responsibilities under federal integrity management regulations to perform evaluations of their integrity management programs using meaningful performance metrics.
PHMSA’s integrity management regulations require operators to establish processes to evaluate the effectiveness of their integrity management programs. Program evaluation is one of the key required program elements as established in the integrity management rules. For hazardous liquid pipelines, Sec. Sec. 195.452(f)(7) and 195.452(k) require methods to measure program effectiveness.
Advisory Bulletin (ADB-20l2-10) states that a critical program element of an operator’s integrity management program is the systematic, rigorous evaluation of the program’s effectiveness using clear and meaningful metrics. When executed diligently, this self-evaluation process will lead to more robust and effective integrity management programs and improve overall safety performance. This process is critical to achieving a mature integrity management program and a culture of continuous improvement. Program evaluation is a required integrity management program element as established in Sec. Sec. 192.911(i) and 195.452(k) for gas transmission and hazardous liquid pipelines, respectively. In light of NTSB’s findings following the San Bruno gas transmission incident, PHMSA is reminding operators about the importance of these requirements.
PHMSA advises operators to critically review their processes and methods for evaluating integrity management program performance and take action to strengthen these processes where warranted. An effective operator performance evaluation process is expected to have the following characteristics:
- A well-defined description of the scope, objectives, and frequency of program evaluations.
- The use of periodic self-assessments, internal or external audits, management reviews, performance metrics analysis, benchmarking against other operators, or other self-critical evaluations to assess program effectiveness.
- Clear performance goals and objectives to measure the effectiveness of key integrity activities
- Clear assignment of responsibility for implementing required actions.
- Review and follow-up of program evaluation results, findings, and recommendations, etc., by appropriate company managers.
Operators are also advised that a clear and meaningful set of performance metrics is essential to program effectiveness. An effective program for measuring integrity management program effectiveness should have the following characteristics:
- A description of the type of performance measures to be used, along with the data sources, data validation and quality assurance activities, the frequency of data collection, and any normalization factors.
- A means to update the performance measures (if needed) to assure they are providing useful information about the effectiveness of integrity management program activities.
- The use of performance metrics data to check and calibrate the operator’s risk analysis tools to assure these best represent the performance of the operator’s specific assets.
The performance metrics that are required to be reported to PHMSA annually, such as the number of miles of pipeline assessed, number of anomalies found requiring repair or mitigation, etc., are a small subset of the overall suite of metrics used by an operator to evaluate its program. A much larger set of operator-specific metrics to be used internally is needed to effectively evaluate an integrity management program performance. Metrics should be developed for each of the following:
- Overall program effectiveness indicated by the number of releases, number of injuries or fatalities, volume released, etc.
- Specific threats that include both leading and lagging indicators for the important integrity threats on an operator’s systems. These include:
- Activity Measures that monitor the surveillance and preventive activities that are in place to control risk
- Deterioration Measures that monitor operational and maintenance trends to indicate if the program is successful or weakening despite the risk control activities in place. (Also identified as Operational Measures in ASME B31.8S.
- Failure Measures that reflect whether the program is effective in achieving the objective of improving integrity. (Also identified as Direct Integrity Measures in ASME B31.8S)
- Metrics that measure and provide insights into how well an operator’s processes associated with the various integrity management program elements are performing. Examples of such processes would include integrity assessment, risk analysis, the identification of preventive and mitigative measures, etc.
Finally, operators must keep records supporting the decisions, analyses, and processes developed and used in their evaluation of integrity management program effectiveness. These records should include those justifying the selection of performance metrics, the performance metric data and trends, and how these metrics are used to improve the integrity management program. Operators should also be diligently working to eliminate information and data gaps throughout their entire integrity management program.