Kusserow on Compliance: Tips for measuring compliance program effectiveness

When possible, it is important to have a variety of metrics to evidence the effectiveness of the compliance program at the executive and board level. These metrics also can contribute to budget justification for the compliance office. Yet, focusing on process metrics, such as the number of hotline complaints handled or employees trained, does not speak to how effective these processes are and, alone, provide organizations with little insight into how successful the compliance programs really is.

The great majority of hotline calls tend to be human resource related or not highly significant to the compliance program, and any decline in the number of hotline reports cannot be explained by numbers alone. Declines may be a result of a more compliant work environment or perhaps loss of confidence in reporting problems. Increases in call volume have similar problems in deducing the reason for the change. Raw training numbers really do not answer the question as to how well the employees understood the lessons learned in compliance training.

The following tips are better avenues to explore to truly measure a compliance program’s effectiveness:

  • Conduct an overview of the whole compliance program. Do not limit compliance oversight and evaluation to any specific areas, but, instead examine all seven elements of an effective compliance program, as described by the HHS Office of Inspector General (OIG). This includes determining whether the structure of the program beginning at the board and executive levels have been properly chartered and is operating in accordance to that charter with respect to the organization and management of compliance office. The same should hold true for compliance guidance, which can come in the form of a code and policies, education, communication, investigation, sanction screening, investigations undertaken, document management, monitoring/auditing of high risk areas, etc. The best method to ensure this compliance is to seek outside experts to conduct a compliance program effectiveness review. This is something that the OIG has advocated for from the earliest days in providing compliance guidance.
  • Compliance document management. A well-managed records management program provides evidence to confirm the effectiveness of the program. Controlling and managing compliance-related policies and procedures are among the most challenging areas for compliance officers. Among the challenges to meet are standardizing the processes for developing the policies; setting up controls for policy review; keeping the policies current with changing standards and regulations; properly citing them in authority; ensuring they are consistent with other policies; developing the policies so that they are understandable by all covered person; training all covered persons on complying with the provisions; monitoring the implementation to verify the policies are being followed; and measuring effectiveness of the policy in achieving the desired outcome. Keeping track of this process can add valuable metrics supporting an active and alert compliance program.
  • Ongoing monitoring and ongoing auditing of high-risk areas. This idea refers to the notion that managers are responsible for overseeing their own operations including: (1) keeping current with changes in rules, regulations, and applicable laws; (2) developing internal controls, policies, and procedures to comply with them; (3) training staff on these rules; and (4) taking steps in monitoring or verifying compliance with these new guidelines. The process of ongoing monitoring and auditing of high risk areas can be best described as continuous control monitoring with the purpose of detecting compliance and risk issues that will enable providers to address issues quickly. This should be designed to test for inconsistencies, duplication, errors, policy violations, missing approvals, incomplete data, dollar or volume limit errors, or other possible breakdowns in internal controls. There should also be ongoing auditing of manager monitoring obligations and responsibilities. This should verify managers are meeting their monitoring obligations and validating effectiveness in the results. The auditing team may involve the compliance office, internal or external auditors, other program managers, or any combination thereof. The combination of ongoing monitoring and auditing will provide more clear and consistent data on compliance.
  • Effectiveness of compliance training. Developing a comprehensive training program and delivering it to all employees is only part of the equation. The real question is whether the employees understand, accept, and retain the lesson points. There are two major methods to measure the training effectiveness and both approaches should be considered. The first is to have a quiz at the end of the training to ensure that participants remembered key points. Most vendors using web-based training programs offer that sort of testing as part of their services. That is good for short term memory, but the follow-up question is whether they retain their understanding in the long run. Employee compliance surveys can cover key factors about compliance and assist in determining levels of retention.
  • Employee surveys. Employee surveys are among the best and most inexpensive means for evaluating, evidencing, and benchmarking compliance program effectiveness. These surveys can provide important metrics to supply to the compliance officer, executive leadership, and the board to show how well compliance is understood and accepted among employees. Unfortunately, it is also one of the least utilized tools for doing so. Surveying is one of the two methods suggested by the OIG in their Compliance Program Guidance for Hospitals and Supplemental Guidance for Hospitals. The agency noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” It further stated it “recommend[s] that organizations should evaluate all elements of a compliance program through employee surveys.” Results from a professionally and independently administered survey can provide a very powerful and credible report to the compliance oversight committees, as well as independent evidence for any outside authority questioning the program. They can also identify relative strengths in the compliance programs, as well as those areas requiring special attention. There are two general types of surveys that can be employed to evidence compliance program effectiveness, a compliance culture survey and a compliance knowledge survey. Periodic surveys can provide valuable insights into employee knowledge of the Code of Conduct and policies as well as applicable laws and regulations. For many the right survey tool can also provide benchmarks to the organization by comparing results between each survey; and if the survey is anchored in industry results, they can be benchmarked against the universe.
  • Keep up to date on compliance with outside authorities. The regulatory and enforcement environment is changing at a fast rate. It is not easy to stay current with issues or always know how to address them once identified. Subscribing to or monitoring newsletters and blogs are always helpful, but having experts on call is probably worth considering as well. Experts not only stay abreast of changes but know how to adapt to and address them. Consider co-sourcing with a compliance expert firm that can be on call when needed and avoid scrambling for assistance with a problem occurs.


Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2016 Strategic Management Services, LLC. Published with permission.