Kusserow on Compliance: Measuring compliance program effectiveness using validated and reliable knowledge surveys

The OIG from its earliest compliance guidance documents has recommended the use of “[q]uestionnaires developed to solicit impressions of a broad cross section” of the workforce. Evaluating effectiveness through the use of questionnaires or surveys can measure the compliance culture and/or knowledge of the organization. Such surveying of employees are one of the two methods suggested for evidencing compliance program effectiveness by the HHS OIG in its Compliance Guidance for Hospitals and Supplemental Guidance for Hospitals. The agency noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” It further reinforced this by stating it “recommends that organizations should evaluate all elements of a compliance program through “employee surveys.”   The OIG also stated that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”

Steve Forman, CPA, has 35 years experience as a compliance officer and health care compliance consultant. He has used compliance surveys for over 20 years to measure program effectiveness and has found them to be an extremely inexpensive method to provide great insight into the compliance program’s effectiveness. However, he notes that it is critical that the survey being used has been professional developed, as well as validated and tested over many organizations. In addition, it is necessary for employees to have confidence in the fact that their scoring will not be attached to them. This means that the survey needs to be independently administered that ensures the confidentiality and anonymity of participants. It is very useful for organizations gaining feedback from employees by querying them on their knowledge of the compliance program elements drawn from their general observations and personal experiences. Results from a survey can evidence employees’ knowledge; awareness and understanding of the compliance program are used to identify positives and weaknesses of the compliance program.  It can provide empirical evidence of the advancement of program knowledge, understanding, and effectiveness.

Jillian Bower has been overseeing administration of knowledge surveys with health care organizations for more than 6 years at the Compliance Resource Center (CRC). The CRC has been employing compliance surveys since 1993.  The most popular survey for Compliance Officers is the Compliance Knowledge Survey© that tests the knowledge of the compliance program’s structure and operations, including the understanding of the role of the Compliance Officer, how the hotline functions, etc. It specifically focuses on the OIG’s seven elements of an effective compliance program and uses simple closed-ended questions with “Yes and “No” answers choices that requires no more than 20 to 30 minutes to complete. Reports from this survey runs 30 pages or more that includes tips for addressing weaknesses and benchmarks results against the universe of those who have used the same survey three ways; (a) overall results, (b) by topic, and (c) individual questions. The biggest benefit of the Compliance Knowledge Survey© is being able to benchmark the results of an organization with the universe of those that have used the same survey by overall results, topical areas, and by question.

Carrie Kusserow with 15 years experience as a compliance officer and consultant has found that reports of survey results can evidence both strengths in the compliance program, as well as areas opportunities for improvements in the Compliance Program. It is one way that compliance program effectiveness can be objectively measured with credible metric evidence. Using the same survey over time, permits measurements that can benchmark progress in Compliance Program development and in tracking improvements.

Al Bassett, JD, has assisted in building and evaluating compliance program effectiveness more than just about anyone in the country over the last 20 years. He has routinely employed employee surveys as a tool to obtain the most out of a compliance effectiveness review. He has found that a compliance knowledge survey parallels and reinforces his findings from document reviews, observation of program operations, and interviews of key staff. In addition, he has surveys administered to provide the foundation for focus group meetings. Findings from a survey can identify potential weakness, but does explain the “why” for the issue. He cautions that for reliable and credible result, the survey should be professionally developed and administers.  From experience he notes that internally developed questionnaires naturally raise employee suspicion that the questions are being designed to bias the results in favor of the organization.  There is also the concern that if administered internally, anonymity in responding to questions would be lost. Another issue is that the credibility of the results is not likely to provide convincing evidence to any outside authorities. A properly developed survey will also address a response-set bias, where respondents may always answering the questions as “yes” or “no”. It is therefore important to have a few reverse scored questions included.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on
Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.