Kusserow’s Corner: Evidencing and Benchmarking Compliance Program Effectiveness

Surveys are among the best, but least utilized, means for evaluating, evidencing, and benchmarking compliance program effectiveness, and is one of the two methods suggested by the HHS Office of Inspector General (OIG) in its Compliance Program Guidance for Hospitals and Supplemental Guidance for Hospitals. The OIG noted “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff” and reinforced this by stating it “recommends that organizations should evaluate all elements of a compliance program through ‘employee surveys.’” Results from a professionally administered survey can provide a very powerful report to compliance oversight committees, as well as to any outside authority questioning the program. Surveys can also identify relative strengths in the compliance programs, as well as areas requiring special attention.

There are two general types of surveys that can be employed to evidence compliance program effectiveness: “compliance culture survey” and “compliance knowledge survey.” Either approach can provide great insights into how effective the compliance program has been in changing and improving compliance within an organization. Survey responses can signal not only strengths in the compliance program, but areas of potential weakness warranting attention. They can also communicate a strong positive message to employees. Surveys also have the added benefit of signaling to employees that (a) their opinions are valued; (b) the organization is commited to them as individuals; and (c) their input is being used to make positive changes. These messages can have a powerful influence on increased compliance, reduced violations, and heightened integrity. In either case, surveys must be kept to a reasonable length, generally no more than 20 to 30 minutes to complete. If it takes longer, the employee may become apathetic and begin responding in a careless fashion.

Compliance Culture Survey

Culture surveys are useful in measuring change in the compliance environment over a period of time. Culture and knowledge surveys  are derived from very different ontological traditions and perspectives. They focus on the beliefs and values that guide the thinking and behavior of employees within an organization. They can measure the outcome of the compliance program; and examine the extent to which individuals, coworkers, supervisors, and leaders demonstrate commitment to compliance. This can be extremely useful for assessing the current state of the compliance climate or culture of an organization. Surveys are usually presented in a Likert Scale format that offer a series of gradation of answers wherein respondents are asked whether they “Strongly Disagree,” “Disagree,” are “Neutral,” “Agree,” or “Strongly Agree,” with the statement presented in each item. A sixth option, “Don’t Know,” may be offered to respondents who feel they lack the knowledge needed to answer the particular question.

Compliance Knowledge Surveys

Knowledge surveys are used most often with mature compliance programs to learn about the progress of the compliance program in reaching the employee population. They test knowledge of the compliance program structure and operations, including the understanding of the role of the Compliance Officer, how the hotline functions, etc. This is useful in providing empirical evidence of the advancement of program knowledge, understanding and effectiveness. Compliance knowledge surveys approach generally use only dichotomous questions that are simple questions with “Yes,” “No,” and “I don’t know” answer choices. This approach creates the simplest of all the closed-ended questions, and as such is extremely easy for respondents to answer. As such, all the questions on the survey are referred to as nominal, and no binomial statistics is applied for the single “Yes,” “No,” and “I don’t know” answers and serves as the dependent variable or one of the dependent variables in an analysis.

It is worth noting that although anyone can draft a questionnaire and conduct a survey, but it does not mean all surveys will produce reliable, valid and credible results, especially to an outside party; surveys may be questioned as to potential bias or reliability. It is advisable to use a valid and independently web-based administered survey that has been tested over many organizations and ensures participant confidentiality. Using a professional survey service specializing in health care compliance is surprisingly inexpensive and less costly than developing and delivering a survey in house.

Benchmarking Compliance Program Progress

The OIG noted that “the existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.” One of the great benefits of using surveys is that they can be used to meet that standard. Using a standardized survey instrument can do this two ways. First, if the survey being used is anchored in a database of users, the organization can benchmark them against that universe. Second, an initial survey can establish a baseline from which future surveys can be used to benchmark progress of the compliance program. Either type of survey can be used for benchmarking purposes. If an organization decides to conduct an annual compliance survey, it is advisable to alternate between the two types, rather than use the same one each time. Normally significant changes take a little time to show results. Also, using the same survey every year diminishes its value and having information from the two different types of surveys will provide increased insights about the compliance program.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow’s Corner Newsletter

Copyright © 2014 Strategic Management Services, LLC. Published with permission.