Kusserow’s Corner: Evidencing and Benchmarking Compliance Program Effectiveness

Increasingly Compliance Officers are being asked by their executive leadership and boards to provide evidence of compliance program effectiveness. More specifically, they are being asked to provide metrics as part of this evidencing. In most cases, the answer is may be easier to produce than one might think—think surveys. Surveys are among the best and least expensive means for evaluating, evidencing, and benchmarking compliance program effectiveness. Unfortunately, they are also one of the least-utilized tools for doing so. Surveying is one of the two methods suggested by the HHS Office of Inspector General (OIG) in its Compliance Program Guidance for Hospitals and Supplemental Guidance for Hospitals. The OIG noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” It further reinforced this by stating it “recommends that organizations should evaluate all elements of a compliance program through “employee surveys.” Results from a professionally and independently administered survey can provide a very powerful report to the compliance oversight committees, as well as credible independent evidence for any outside authority questioning the program. Survey results can also identify relative strengths in the compliance programs, as well as those areas requiring special attention. There are two general types of surveys that can be employed to evidence compliance program effectiveness, “compliance culture survey” and “compliance knowledge survey”.

Either approach can provide great insights into how effective the compliance program has been, and can signal both strengths as well as areas of potential weakness warranting attention. There is the added benefit of signaling to employees that (a) their opinions are valued; (b) the organization is committed to them as individuals; and (c) their input is being used to make positive changes. These messages can have a powerful influence on increased compliance, reduced violations, and heightened integrity. In either case, surveys must be kept to a reasonable length, generally no more than 20 to 30 minutes to complete. If it takes longer, employees may become apathetic and begin responding in a careless fashion.

Compliance Culture Survey

Culture surveys are useful in measuring change in the compliance environment over a period of time. Culture surveys focus on the beliefs, attitudes, and values that guide the thinking and behavior of employees within an organization. They can measure the outcome of the compliance program; and examine the extent to which individuals, coworkers, supervisors, and leaders demonstrate commitment to compliance. These can be extremely useful tools for assessing the current state of the compliance climate or culture of an organization. They are usually presented in a Likert Scale format that offer a series of gradation of answers wherein respondents are asked whether they “Strongly Disagree,” “Disagree,” are “Neutral,” “Agree,” or “Strongly Agree,” with the statement presented in each item. A sixth option, “Don’t Know,” may be offered to respondents who feel they lack the knowledge needed to answer the particular question.

Compliance Knowledge Surveys

Compliance knowledge surveys are used most often with maturing compliance programs to learn about the progress of the compliance program in reaching the employee population. They test knowledge of the compliance program’s structure and operations, including the understanding of the role of the Compliance Officer, how the hotline functions, etc. This is useful in providing empirical evidence of the advancement of program knowledge, understanding, and effectiveness. Compliance knowledge surveys approach generally use only dichotomous questions that are simple questions with “Yes,” “No,” and “I Don’t Know” answer choices. This approach creates the simplest of all the closed-ended questions, and as such is extremely easy for respondents to answer. As such, all the questions on the survey are referred to as nominal, and no binomial statistics are applied.

Benchmarking Compliance Program Progress

If an organization decides to conduct an annual compliance survey, it is advisable to alternate between the two types, rather than use the same one each time. Using the same one every year diminishes its value and does not take into consideration that changes in the employee population do not occur overnight and results may not materially differ from one period to another. The better approach to annual surveys is alternating between the two different types, or conducting surveys every other year. Using different survey tools has the advantage of providing different dimensions and perspectives on a compliance program.

Tips on Surveying

  1. Anyone can draft a questionnaire in a matter of hours, but that does not mean it will produce reliable, valid and credible results, especially to an outside party.
  2. Internally developed and administered surveys may be questioned as to potential bias or reliability, both in preparation and administration.
  3. Use a valid and independently administered survey tested over many organizations.
  4. Professionally developed/tested surveys are less costly than those developed and delivered in house.
  5. Surveys must ensure confidentiality of participants.
  6. Using a firm specializing in health care compliance is surprisingly inexpensive.
  7. Ideal surveys should provide added value if they can provide comparative data to others who have taken the identical survey.
  8. For optimum credibility, all survey reports should certify a valid and reliable survey tool was used that was tested over many clients and that the administration and report development met independent and objective standards.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow’s Corner Newsletter

Copyright © 2014 Strategic Management Services, LLC. Published with permission.