Kusserow on Compliance: Measuring compliance program effectiveness using validated and reliable knowledge surveys

The OIG from its earliest compliance guidance documents has recommended the use of “[q]uestionnaires developed to solicit impressions of a broad cross section” of the workforce. Evaluating effectiveness through the use of questionnaires or surveys can measure the compliance culture and/or knowledge of the organization. Such surveying of employees are one of the two methods suggested for evidencing compliance program effectiveness by the HHS OIG in its Compliance Guidance for Hospitals and Supplemental Guidance for Hospitals. The agency noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” It further reinforced this by stating it “recommends that organizations should evaluate all elements of a compliance program through “employee surveys.”   The OIG also stated that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”

Steve Forman, CPA, has 35 years experience as a compliance officer and health care compliance consultant. He has used compliance surveys for over 20 years to measure program effectiveness and has found them to be an extremely inexpensive method to provide great insight into the compliance program’s effectiveness. However, he notes that it is critical that the survey being used has been professional developed, as well as validated and tested over many organizations. In addition, it is necessary for employees to have confidence in the fact that their scoring will not be attached to them. This means that the survey needs to be independently administered that ensures the confidentiality and anonymity of participants. It is very useful for organizations gaining feedback from employees by querying them on their knowledge of the compliance program elements drawn from their general observations and personal experiences. Results from a survey can evidence employees’ knowledge; awareness and understanding of the compliance program are used to identify positives and weaknesses of the compliance program.  It can provide empirical evidence of the advancement of program knowledge, understanding, and effectiveness.

Jillian Bower has been overseeing administration of knowledge surveys with health care organizations for more than 6 years at the Compliance Resource Center (CRC). The CRC has been employing compliance surveys since 1993.  The most popular survey for Compliance Officers is the Compliance Knowledge Survey© that tests the knowledge of the compliance program’s structure and operations, including the understanding of the role of the Compliance Officer, how the hotline functions, etc. It specifically focuses on the OIG’s seven elements of an effective compliance program and uses simple closed-ended questions with “Yes and “No” answers choices that requires no more than 20 to 30 minutes to complete. Reports from this survey runs 30 pages or more that includes tips for addressing weaknesses and benchmarks results against the universe of those who have used the same survey three ways; (a) overall results, (b) by topic, and (c) individual questions. The biggest benefit of the Compliance Knowledge Survey© is being able to benchmark the results of an organization with the universe of those that have used the same survey by overall results, topical areas, and by question.

Carrie Kusserow with 15 years experience as a compliance officer and consultant has found that reports of survey results can evidence both strengths in the compliance program, as well as areas opportunities for improvements in the Compliance Program. It is one way that compliance program effectiveness can be objectively measured with credible metric evidence. Using the same survey over time, permits measurements that can benchmark progress in Compliance Program development and in tracking improvements.

Al Bassett, JD, has assisted in building and evaluating compliance program effectiveness more than just about anyone in the country over the last 20 years. He has routinely employed employee surveys as a tool to obtain the most out of a compliance effectiveness review. He has found that a compliance knowledge survey parallels and reinforces his findings from document reviews, observation of program operations, and interviews of key staff. In addition, he has surveys administered to provide the foundation for focus group meetings. Findings from a survey can identify potential weakness, but does explain the “why” for the issue. He cautions that for reliable and credible result, the survey should be professionally developed and administers.  From experience he notes that internally developed questionnaires naturally raise employee suspicion that the questions are being designed to bias the results in favor of the organization.  There is also the concern that if administered internally, anonymity in responding to questions would be lost. Another issue is that the credibility of the results is not likely to provide convincing evidence to any outside authorities. A properly developed survey will also address a response-set bias, where respondents may always answering the questions as “yes” or “no”. It is therefore important to have a few reverse scored questions included.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on
Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.

Kusserow on Compliance: Focus group meetings provide valuable information about compliance program effectiveness

In the HHS Office of Inspector General (OIG) Compliance Program Guidance for Hospitals and Supplemental Compliance Program Guidance for Hospitals, the agency urges using techniques to solicit impressions of a broad cross-section of the hospital’s employees and staff regarding all elements of a compliance program. One method for doing this is conducting focus group meetings.  An employee compliance focus group involves employees who are assembled to participate in a guided discussion about compliance issues. The focus group facilitator is a critical role, and those taking it on should be well-trained on how to run the group and handle difficult situations.

Carrie Kusserow has 15 years’ experience as a compliance officer and consultant. She has employed employee focus groups frequently and found them an easy and efficient way to uncover ideas for improving the employee compliance environment.   She warns, however, that it may be easy to gather a group of employees in a room to ask a few questions and have a “discussion,” but it is quite something else to ensure that the discussion yields reliable information that can serve as a basis for decision-making. A productive focus group is much more than a chat session.  It is important that a clear purpose statement is explained to the focus group to provide a concise and clear stated purpose for the meeting and how the information collected will be used.  There are other benefits that can be derived from individuals being included in these focus group conversations. They feel honored to be able to contribute to addressing organizational problems and that management values employee input.

It is doubtful there is anyone else in the country that has more experience in facilitating compliance focus groups than Al Bassett, J.D. He is a nationally recognized compliance expert, employing this technique in evaluating compliance programs for over 15 years. It has been his standard practice to include in compliance reviews one or more focus group meetings.  He has found using this technique is best after issues have been identified and documented during the review and evaluation.   The findings may identify the “what” but not the “why” or how best to address the issues.  From his experience, Bassett says that this approach ensures that employees will be many times more engaged, because they are not being asked to identify problems but to address those already identified. As a rule, employees trust an outside facilitator more than someone from management or the compliance office, as employees feel like they can speak more freely about their workplace. He even discourages participation by anyone from management or compliance in the group for the same reason.  It is also important that the facilitator make it clear that comments made will not be identified to any individual.  Also, the meeting should be scheduled during work hours in a private room, where employees will not be overheard, and last no more than 60 to 90 minutes.  Bassett has found that the optimum size of the focus group is eight.

Steve Forman, CPA, has been facilitating focus group meetings on compliance-related issues for over 20 years. He has found that it is critical that the facilitator remain neutral and have the skills to keep the discussion on track, steer the conversation away from a general gripe session, and avoid an uninspired, silent meeting.  He has found that one of the best ways to encourage an open and useful focus group meeting is to use results of a compliance survey as the basis of discussion. The facilitator can then open the meeting by asking participants if they could shed additional light on the reason why there may be a problem, as suggested in survey response to certain questions. This can assist if certain issues require deeper probing and more nuanced evaluation. This approach relieves participants from having to identify problems. In this way, the identification is already done, and their feedback as to why opens the discussion nicely.  Potential weaknesses may have been identified, but they don’t always point to action steps for improvements.  Therefore, the group discussion can begin with the issues having been established from general employee feedback. The focus group can then be focused on providing insights as to why employees answered questions the way they did and what can be done to address those concerns.

9 tips for compliance officers

Compliance officers should heed the following advice:

  1. Consider including focus group meetings in the scope of work of any engagement to evaluate the effectiveness of the compliance program to assist in finding solutions to identified issues.
  2. Whenever there is a compliance survey administered, consider using focus groups to help explain why employees responded the way they did.
  3. Use an experienced outside facilitator to conduct compliance focus group meetings, as he or she will be considered more trustworthy and neutral.
  4. Members of management or compliance staff should not present in the group, as it has a chilling effect.
  5. Limit participants in a focus group to eight and duration of the session to 60-90 minutes
  6. Do not attach any comments made in the focus group meeting to the individual who made them.
  7. The facilitator should ask probing questions (e.g., what, why) and paraphrase or repeat back certain phrases so that ideas can be clarified.
  8. Questions presented need to be designed to solicit specific views of participants on issues.
  9. Key focus group success comes from acting on the ideas generated during the discussion.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.

Kusserow on Compliance: Compliance culture a key measure of program effectiveness

The compliance culture is the set of shared attitudes, values, goals, and practices that characterizes an institution or organization when it comes to compliance with laws, regulations, rules, standards, code of conduct, and policies.   Oversight agencies believe the compliance program should be a change agent in promoting a culture of compliance that creates an environment less likely to have regulatory or enforcement problems.  This means establishing a culture in which everyone in the work environment embraces and adheres to rules, regulations, laws, code of conduct, and policies.  The Department of Justice (DOJ) and the HHS Office of Inspector General (OIG) frequently encounter organizations with compliance programs that exist on paper, but that culturally failed to be effective in operation. Compliance officers should find means to evidence that the culture of the organization matches the compliance goals.

Positive compliance culture promotes good business

Carrie Kusserow, with over 15 years’ experience as a compliance officer and consultant, makes the case that a good compliance culture is also good for business and does not just serve as a “cost center.” She notes there are many positive benefits to be derived from the effort. She offered the following points in her argument.

  • Organizations are less likely to have liabilities, arising from wrongful behavior.
  • Evidence suggests compliance-committed organizations are more efficient.
  • Lower employee turnover occurs when the organization culture is to abide by rules and standards.
  • There exists greater employee commitment to compliance with laws, rule, code of conduct and policies.
  • Employees feel less pressure to compromise company standards to achieve company goals.
  • Employees are empowered to report wrongful behavior and misconduct internally, not externally.

Compliance culture surveys evidence compliance program effectiveness

Steve Forman, CPA has been using compliance culture surveys for the last 20 years, both as a compliance officer and as a compliance consultant. He believes that one of the best and most inexpensive methods for evaluating, evidencing, and benchmarking compliance program effectiveness is through a compliance culture survey that measures employee perceptions of ethical culture and/or the compliance program. He likes using this type of survey, alternately with a compliance knowledge survey that tests employee knowledge of the program. He points to the fact that the OIG recommends this in its Compliance Program Guidance, wherein it noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as . . . using questionnaires (employee surveys) . . . developed to solicit impressions of a broad cross-section of . . . employees and staff.” Results from a professionally administered survey provide a very powerful and credible report to the compliance oversight committee, as well as to any outside authority questioning the program. They can also identify relative strengths in the compliance programs, as well as those areas requiring special attention that are invaluable for compliance officers.

Compliance survey benefits

Conducting a compliance survey provides numerous benefits to an organization.  For example, it can:

  • provide outcome measurements for the compliance program;
  • serve as critical evidence in determining the degree of effectiveness of the compliance program;
  • identify program strengths and potential weakness warranting attention;
  • evidence the extent of individual and leader commitment to compliance;
  • assess the current state of the compliance climate or culture of an organization;
  • communicate a positive message that employee opinions and perceptions are valued;
  • underscore organization commitment to employees;
  • increase management attention on what is being measured;
  • provide metrics as to progress in developing an effective compliance program;
  • benchmark compliance program effectiveness improvement;
  • signal the organization as to employee attitudes and perceptions;
  • tell employees that what they believe and understand is important; and
  • provide guidance as to where improvements are needed.

Benchmarking compliance program progress

Jillian Bower, with many years of experience in administering compliance surveys, as well as serving as interim compliance officer, notes the OIG compliance guidance says that “the existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”  Surveys can be used to meet that standard. If the survey being used is anchored in a large database of users, the organization can benchmark them against that universe, viewed as very important by most organizations. Furthermore, an initial survey can establish a baseline from which future surveys can be used to benchmark progress of the compliance program. The surveys can benchmark and measure change in the compliance environment over a period of time. However, Bower warns it is inadvisable to use the same survey annually, as significant changes among the work force takes time to show results.

Alena Treen, of the Compliance Resource Center (CRC), has many years’ experience in administering compliance surveys. She explained that culture surveys focus on the beliefs and values which guide the thinking and behavior of employees within an organization. They are usually presented in a Likert Scale format that offer a series of gradation in which respondents are asked whether they “Strongly Disagree,” “Disagree,” are “Neutral,” “Agree,” or “Strongly Agree,” with the statement presented in each item. This is in contrast with a compliance knowledge survey designed to learn how much employees know about the program with questions answerable as yes or no. She notes it is highly advisable to use a valid and independently web-based administered survey that has been tested over many organizations and ensures participant confidentiality. Using a professional survey service specializing in health care compliance is surprisingly inexpensive and less costly than developing and delivering a survey in house that doesn’t carry the same level of credibility. The CRC has been using the Compliance Benchmark Survey© since 1993 and has been employed by hundreds of health care organizations and over a half million surveyed population. Treen normally deals with reports that are about 50 pages in length that provide advice on each topical area and question as to how improvements may be made.   Clients find that comparing their results with the universe to be the most beneficial information.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.