With increased government oversight and enforcement efforts, including mandatory compliance programs and more stringent enrollment screening requirements for Medicare and Medicaid providers mandated by the Patient Protection and Affordable Care Act (P.L. 111-148), having a compliance program in place is not enough. Providers must have methods in place to ensure that their compliance programs are effective. Accurately and appropriately measuring effectiveness is essential. The Office of Inspector General (OIG) identifies the development and implementation of regular, effective education and training program as one of the seven essential elements of compliance programs in its compliance program guidances. Therefore, a good place to start evaluating the effectiveness of the organization’s compliance program is to assess the effectiveness of the organization’s compliance education and training.
Effectiveness is a factor of outcome, according to Jillian Bower, MPH, a manager of the Compliance Resource Center, in her article “How Effective Is Your Compliance Training?” in the November/December 2011 issue of the Journal of Health Care Compliance. “[U]sing process data such as the number of people trained is not a measure of outcome but of output,” Bowers said, adding that “the key question is what the employees learned from the training.” The means to evidence compliance education and training effectiveness requires addressing two issues: (1) delivering comprehensive compliance training programs; and (2) evidencing that they were effective in gaining understanding and acceptance by participants, Bowers explained.
The means to effective training is choosing the appropriate training approach. Training must fit the culture, composition of the workforce, and the needs of the organization as well as include a method to measure the participants’ understanding of the training message. Training can be live, on video, computerized, or written self-study. Training can be provided by internally or by outside experts. Lesson quizzes and tests should be included to evidence how well the lessons were learned.
Factors to consider when selecting the training method are level of expertise and understanding of compliance issues by the trainers, level of professional experience in delivering programs of this type, fee charges, travel expenses, flexibility in scheduling, the make-up of the target audience.
Live training. According to Bowers, experts a two-tier live presentation by a facilitator that uses case studies believe the most effective approach especially if a highly trained and skilled facilitator delivers it. Background information on the organization’s compliance program and the standards of conduct are discussed followed by participants applying the standards of conducts to resolve issues presented in scenarios or case studies.
The straight lecture with slides approach, however, has been shown to be among the least effective methods and should be avoided, Bowers noted saying that “People involved in health care do not react well to someone lecturing them on proper behavior and conduct in the workplace. She advised against presentations and slides that detail the elements of laws and regulations that go beyond what most employees would be reasonably expected to understand.
Videos. In Bowers opinion, talking head video presentations may be less effective than lectures and should be limited to eight to 12 minutes. Their value and purpose is primarily as a basic orientation or introduction to the compliance training, especially when it includes a personal message from the chief executive officer (CEO). Other than that, it can be counter-productive, she said.
On the other hand, Bowers explained that using role playing videos to present case studies for training has a number of advantages including that the message can be communicated quickly and effectively and participants can access the program at any time and proceed at their own pace. In addition, role play can guide participants on how to analyze and handle a potentially difficult compliance issue.
Computer-based or online training. Computer-based training is likely to have high perceived credibility from the trainees and has many of the benefits of role playing videos but at a much lower cost, according to Bowers. It is available at any time the participants wish to access the program and proceed at their own pace. Additionally, many of the companies that provide computer-based training also provide periodic updates to the content. Quizzes and tests are often included that can be used to evidence the training effectiveness to outside parties, Bowers said.
Written self-study. Bowers believes that written self-study training is a difficult method of delivery and is normally limited to selected persons noting that the general workforce is not likely to do well with this approach. The major advantages of this method are scheduling flexibility and ease of assignment to the training; however, evidencing that the training was effective would require a separate testing protocol, she added.
Bowers identified two ways to evidence training:
- One way is to have a test at the end of the training that measures participant understanding of the lessons. If the training uses paper documents which are used over again, Bowers recommends organizations keep control over them to avoid others having the test questions in advance of the program. Participants may be quizzed after each lesson in the program, followed by a test at the end of training. The results of the tests should be kept as part of the participant’s permanent record and maintained by the compliance office or human resource management, Bowers advised.
- A second way is through questionnaires and surveys. This can be disseminated separately from training or in conjunction with training as further confirming evidence, Bowers said. A knowledge survey asks employees about their understanding of the compliance program and related concepts, as well as other key issues that are included in compliance training programs.