Evaluate
Make sure your hearing conservation program is working with regular program evaluations. This will identify trends, magnify problem areas and drive improvement.
Is the hearing conservation program effective?
-
The ultimate goal of a hearing conservation program is to protect people from developing a hearing loss caused by working in hazardous noise. When all of the hearing conservation program (HCP) tasks have been completed, and all of the regulations have been checked and followed, it is important to evaluate the effectiveness of the overall hearing loss prevention program (HLPP). Is it actually preventing noise-induced hearing loss? Are there gaps? How can it be improved? Is the HLPP efficient and cost effective?
There are several ways to measure the effectiveness of the program. One is to specifically evaluate changes that occur as a result of the program, like the trend over time of the number of hearing loss cases. Other outcome measures can be tracked, like successful reduction of noise sources or exposures. Another approach is to look at the cost of delivering the HLPP, comparing it to the cost of implementing noise control to reduce the noise hazards. It can also be helpful to audit the HLPP for compliance and/or to review the company policies and practices to ensure alignment between what is on paper and what really happens in practice. A recent hearing conservation program checklist to assess effectiveness has been developed (Neitzel et al, 2017).
Conducting routine evaluations of program effectiveness is recommended. Companies choose to review aspects of the HCP at different times to spread the work load throughout the year or conduct an audit at the same time every year or every other year. Program evaluation can be done by using internal resources, contracting the service to outside subject matter experts, or through a combination of both. Finally, once the key findings are identified, the next step is to incorporate the recommendations into the HLPP.
Key Takeaways
-
- Effective hearing loss prevention programs prevent noise-exposed workers form having standard threshold shifts (STSs).
- Analysis of hearing conservation program records, policies and practices will help employers evaluate the quality of their efforts.
- Regular program evaluation can identify trends, detect gaps, and drive improvement.
- Evaluation can include cost analysis and return on investment (ROI)
Key Terms
-
- Audiometric database analysis
- Audit
- Checklist
- Hearing Loss Impairment formula
-
- Outcome measures
- Return on investment (ROI)
- Standard Operating Procedure
- Standard Threshold Shift
- Key Performance Indicator
Getting Started With Program Evaluation
-
Step 1. Identify the HLPP team members and responsibilities, including who should be included in program evaluation.
A feature of successful HLPPs is that there is a HLPP team with one person who is designated as the HLPP leader. A quick tool to help identify the team is to complete the “Who’s Responsible?” form. Doing this exercise may reveal duplications, gaps, or uncertainties regarding who is doing what task, and provide some guidance for next steps. The team leader can pull together the team for regular meetings to set goals, discuss issues, and evaluate the total program. Evaluating the program can be completed by the HLPP team, however in some cases, you could choose to contract with an independent service provider to perform this service.
-
Step 2. Determine the method(s) to be used for HLPP program evaluation
There are several approaches to program evaluation. Simple checklists can be used to determine if company policies and procedures are in compliance with regulations. A program evaluation compliance checklist for OSHA can be found by clicking here.
Another approach is to identify specific outcome measures to related to program success. If the HLPP team has determined goals, then outcome measures can be used to track progress towards completing the goal. Examples of outcome measures include:
- Noise reduction accomplished through engineering controls
- Number of employees enrolled in the HLPP
- Completion of hearing protection fit testing
- Trend of STS occurrences
- Hearing impairment rates
-
Step 3. Consider the schedule for program evaluation
Program evaluation can be an ongoing process in terms of ensuring that tasks are completed as planned and regulatory requirements are met. However special assessments can be scheduled periodically to dig deeper into the details. Some programs conduct annual internal audits as preparation for a potential external, unscheduled inspection.
-
Step 4. Review results and implement recommendations for improvement
Recommendations for improvement may be simple, such as identifying positions for hearing protection dispensers through the work space, or more complicated, such as developing a plan to increase the completion rate for audiometric retests for workers who experience a standard threshold shift. Program changes can be complex, such as strategically planning to implement engineering control projects to reduce noise exposure over 95 dBA TWA, for example.
-
Step 5. Conduct a cost benefit study
Operating an effective hearing conservation program costs money as well as time and energy. It is helpful to calculate the actual costs of delivering the program and compare that to the costs of reducing hazardous noise. A cost effectiveness analysis can reveal if resources are allocated appropriately or if changes could increase program effectiveness. Some tools exist to guide cost benefit studies, such as those discussed in this NIOSH blog.
What is required?
-
OSHA regulation 29 CFR 1910.95 requires that employers implement an “effective hearing conservation program”. In a letter of interpretation, OSHA explained that this term means “a hearing conservation program that prevents workplace noise from producing a standard threshold shift in the hearing ability of any employees.” It is expected that employers should make an effort to evaluate whether or not the HCP is effective, however there are no additional requirements specifying how to conduct a program evaluation.
Basics of Program Evaluation
-
Compliance Checklist
In some cases, a checklist for regulatory compliance is conducted to ensure that all regulatory requirements are being met with the existing company policies and procedures. In the true sense, a compliance checklist is different than an evaluation of program effectiveness, however it is a good place to start, in order to document that the hearing conservation program is in compliance. Various compliance checklists can be found on the internet, including this one provided by NIOSH.
Beyond the Basics
-
Program Audit
A hearing conservation program audit is more in depth than a compliance checklist. This typically involves interviewing members of the management, staff, and workforce. Records can be reviewed in detail and an effort is made to identify if everyday practices are aligned with the company policies and procedures. A “deep dive” program audit can be done by an internal team or may be contracted with an external, subject matter expert.
-
Tracking Outcome Measures
Outcome measures can be used to determine the effect of an intervention program. Examples are given below. Select one or more and track the results over time to identify trends and guide program decisions. These measures can be focused on the results of the audiometric database to track occurrences of hearing shift or hearing loss. Ideally, the occurrences of hearing shift, or STS, in the noise exposed group of workers should be the same as that of a non-noise exposure population of workers at the same facility. Using this approach would mean that hearing tests must be conducted on individuals who are not enrolled in the HCP, which may not be practical. In lieu of this, NIOSH has suggested that an STS rate of 3% or less is a good target.
-
STS Percentage
To calculate the incidence of STS, divide the number of STS cases by the number of annual tests and multiply the result by 100.
% STS = 100 x (# of STS/ # annual tests)
For example, a company that conducted 200 annual hearing tests found 9 cases of STS. The overall percent STS for the noise-exposed group is 100 x (9/200) or 4.5%. Tracking this number over time can help identify if the STS rate is acceptable and/or stable.
-
Audiometric Variability
Another approach is to measure the variability within the audiometric database for workers who have been tested over multiple years. An ANSI Technical Report details a statistical procedure to identify fluctuations in the audiometric data which is an indicator of quality. High variability in audiometric thresholds for a population can be a sign of hearing change, poor quality hearing tests, or other factors that compromise the integrity of the data. This ANSI report can be purchased as a stand-alone document however it is also reproduced in the CAOHC Hearing Conservation Manual Appendix M: ANSI S12.13 TR-2002 (R2010) American National Standard Technical Report Evaluating the Effectiveness of Hearing Conservation Programs through Audiometric Data Base Analysis and can be purchased at www.caohc.org.
-
Other outcome measures
Still other measures can be conducted on other aspects of the hearing conservation program, like tracking the number of people in critical exposure groups, noise control efforts that result in decreased exposures, or on achieving a target for completing hearing protection fit-testing.
Here are some examples to monitor change over time:
- Number of decibels reduced by noise control projects
- Personal Attenuation Rating after hearing protection fit testing
- Number of hearing protection types offered and used
- Change in compliance of hearing protection use after training
Have You Considered?
-
- Can you spread out the evaluation phase over time in order to study a single aspect of the program, such as the quality of the audiometric database?
- Would spending resources on noise control provide long term cost savings by reducing the number of people in the hearing conservation program?
- Conducting a survey of the noise-exposed workers to learn what their perceptions of the hearing conservation program are?
- Contracting with an external subject matter expert to conduct a “deep dive” analysis of the audiometric data variability, OSHA recordkeeping, and intervention practices?
- Creating a strengths, weaknesses, advantages, and threats (SWAT) review of the HLPP?
Resources to Learn About Program Evaluation
-
IMPORTANT NOTE: This information is based on selected current national requirements. Other country or local requirements may be different. Always consult User Instructions and follow local laws and regulations. This website contains an overview of general information and should not be relied upon to make specific decisions. Reading this information does not certify proficiency in safety and health. Information is current as of the date of publication, and requirements can change in the future. This information should not be relied upon in isolation, as the content is often accompanied by additional and/or clarifying information. All applicable laws and regulations must be followed.
