Kusserow on Compliance: Measuring compliance program effectiveness using validated and reliable knowledge surveys

The OIG from its earliest compliance guidance documents has recommended the use of “[q]uestionnaires developed to solicit impressions of a broad cross section” of the workforce. Evaluating effectiveness through the use of questionnaires or surveys can measure the compliance culture and/or knowledge of the organization. Such surveying of employees are one of the two methods suggested for evidencing compliance program effectiveness by the HHS OIG in its Compliance Guidance for Hospitals and Supplemental Guidance for Hospitals. The agency noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” It further reinforced this by stating it “recommends that organizations should evaluate all elements of a compliance program through “employee surveys.”   The OIG also stated that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”

Steve Forman, CPA, has 35 years experience as a compliance officer and health care compliance consultant. He has used compliance surveys for over 20 years to measure program effectiveness and has found them to be an extremely inexpensive method to provide great insight into the compliance program’s effectiveness. However, he notes that it is critical that the survey being used has been professional developed, as well as validated and tested over many organizations. In addition, it is necessary for employees to have confidence in the fact that their scoring will not be attached to them. This means that the survey needs to be independently administered that ensures the confidentiality and anonymity of participants. It is very useful for organizations gaining feedback from employees by querying them on their knowledge of the compliance program elements drawn from their general observations and personal experiences. Results from a survey can evidence employees’ knowledge; awareness and understanding of the compliance program are used to identify positives and weaknesses of the compliance program.  It can provide empirical evidence of the advancement of program knowledge, understanding, and effectiveness.

Jillian Bower has been overseeing administration of knowledge surveys with health care organizations for more than 6 years at the Compliance Resource Center (CRC). The CRC has been employing compliance surveys since 1993.  The most popular survey for Compliance Officers is the Compliance Knowledge Survey© that tests the knowledge of the compliance program’s structure and operations, including the understanding of the role of the Compliance Officer, how the hotline functions, etc. It specifically focuses on the OIG’s seven elements of an effective compliance program and uses simple closed-ended questions with “Yes and “No” answers choices that requires no more than 20 to 30 minutes to complete. Reports from this survey runs 30 pages or more that includes tips for addressing weaknesses and benchmarks results against the universe of those who have used the same survey three ways; (a) overall results, (b) by topic, and (c) individual questions. The biggest benefit of the Compliance Knowledge Survey© is being able to benchmark the results of an organization with the universe of those that have used the same survey by overall results, topical areas, and by question.

Carrie Kusserow with 15 years experience as a compliance officer and consultant has found that reports of survey results can evidence both strengths in the compliance program, as well as areas opportunities for improvements in the Compliance Program. It is one way that compliance program effectiveness can be objectively measured with credible metric evidence. Using the same survey over time, permits measurements that can benchmark progress in Compliance Program development and in tracking improvements.

Al Bassett, JD, has assisted in building and evaluating compliance program effectiveness more than just about anyone in the country over the last 20 years. He has routinely employed employee surveys as a tool to obtain the most out of a compliance effectiveness review. He has found that a compliance knowledge survey parallels and reinforces his findings from document reviews, observation of program operations, and interviews of key staff. In addition, he has surveys administered to provide the foundation for focus group meetings. Findings from a survey can identify potential weakness, but does explain the “why” for the issue. He cautions that for reliable and credible result, the survey should be professionally developed and administers.  From experience he notes that internally developed questionnaires naturally raise employee suspicion that the questions are being designed to bias the results in favor of the organization.  There is also the concern that if administered internally, anonymity in responding to questions would be lost. Another issue is that the credibility of the results is not likely to provide convincing evidence to any outside authorities. A properly developed survey will also address a response-set bias, where respondents may always answering the questions as “yes” or “no”. It is therefore important to have a few reverse scored questions included.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on
Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.

Kusserow on Compliance: Using culture surveys to measure compliance program effectiveness

Organizations are increasingly making use of employee perception and attitudes in measuring the compliance culture through using surveys. Increasingly, board directors, given their oversight duties and personal exposure, see the value of an independently administered survey to better understand the status of the compliance program. Using surveys to measure compliance culture has long been advocated by regulatory bodies.  The U.S. Sentencing Commission explicitly recognizes the significance of culture in its 2004 Amendments to the Federal Sentencing Guidelines, stating that businesses must “promote an organizational culture that encourages ethical conduct and a commitment to compliance with the law.”  The HHS Office of Inspector General (OIG) from its earliest compliance guidance documents has recommended the use of “[q]uestionnaires developed to solicit impressions of a broad cross section” of the workforce to evaluate program effectiveness that can measure the compliance culture of the organization. In the Compliance Program Guidance for Hospitals and Supplemental Compliance Program Guidance for Hospitals, it states that “fundamentally, compliance efforts are designed to establish a culture within a hospital that promotes prevention, detection and resolution of instances of conduct that do not conform to Federal and State law, and Federal, State and private payor health care program requirements, as well as the hospital’s ethical and business policies . . . As part of the review process, the compliance officer or reviewers should consider techniques such as using questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff . . . Organizations should evaluate all elements of a compliance program through employee surveys.” In its Compliance Guidance for Nursing Facilities, it recommended evaluations of the compliance program through “employee surveys, management assessments, and periodic review of benchmarks established for audits, investigations, disciplinary action, overpayments, and employee feedback.”

Jillian Bower, a health care compliance consultant, has been overseeing, on behalf of the Compliance Resource Center, the administration of the Compliance Benchmark Survey© for six years to assess the compliance culture.  It has been administered to hundreds of health care organizations and more than a half million surveyed population since 1993. It measures perceptions and attitudes of employees on a number of issues, as they relate to them personally, their immediate work group, their supervisor, and the management of the organization. The results provide important insights as to organization’s strengths on which to advance the compliance culture and areas warranting special attention.  Results from the survey can measure the outcome of the compliance program and examine the extent to which individuals, coworkers, supervisors, and leaders demonstrate commitment to compliance. These can be extremely useful tools for assessing the current state of the compliance climate or culture of an organization.  Reports from the survey run 30-50 pages and include tips for addressing weaknesses. They also benchmark results against the huge universe of those who have used the same survey in three ways: (1) overall results, (2) by category, and (3) individual questions. Use of the same survey has an advantage to benchmark or measure survey results from the current year against past years. Most importantly for those using the survey, results can be benchmarked against peer organizations that utilized the same survey. This provides invaluable metrics of program effectiveness.

Steve Forman, CPA, has 35 years’ experience as a full-time compliance officer at major health care organizations, as well as in assisting organizations in developing and evaluating their compliance programs.   In all his evaluations of compliance programs, he urges clients to consider including a survey of employees because the combination of the consultant’s findings and recommendations are reinforced in the survey report. He also cautions that to have a valid and tested survey instrument that produces reliable results is a serious business.  The survey needs to be developed by experts who validate and test the instruments over many organizations.  Any survey developed in-house is not likely to meet these standards.  They also can be viewed by employees as suspect and designed to bias the results in favor of management and will lack credibility to any outside authorities. As such, they will have little value in providing credible evidence of an effective compliance program.  Furthermore, the administration of the survey process is critical to useful results.  It must be independently administered that ensures the confidentiality and anonymity of participants.

Al Bassett, J.D., is another nationally recognized compliance expert with over 30 years’ experience in the OIG and as a consultant. He has found the use of surveys to evidence compliance program effectiveness is quite inexpensive, costing a small fraction of a full consultant-led compliance program effectiveness evaluation. As such, many of his clients opt for conducting the survey as a standalone engagement.  Another feature of using standardized surveys is that they can be supplemented with organization-specific questions and/or open-ended questions designed to provide more dimension to the information gathered.  He frequently links survey results in conducting focus group meetings and interviews when evaluating program effectiveness to shed additional light on the reason why there may be a problem, as might be suggested in survey response to certain questions. This can assist if certain issues require deeper probing and more nuanced evaluation.

Carrie Kusserow is another expert with 15 years’ experience as a compliance officer and consultant who has used compliance culture surveys extensively. She believes surveys can provide great insight into the compliance program’s effectiveness and very importantly can benchmark the progress of the program.  Reports can identify both strengths in the compliance program and potential gaps needing attention. Results can also be used as a benchmark for measuring progress and track improvements in the operation of the compliance program over time.  This is very important. She cites the OIG Compliance Program Guidance for Hospitals that states that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.” Another benefit of using an employee survey is that it can communicate a strong message to employees that their opinions are valued, the organization is committed to them as individuals, and their input is being used to make positive changes.  These messages can have a powerful influence on increased compliance, reduced violations, and heightened integrity.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.

Kusserow on Compliance: Compliance culture a key measure of program effectiveness

The compliance culture is the set of shared attitudes, values, goals, and practices that characterizes an institution or organization when it comes to compliance with laws, regulations, rules, standards, code of conduct, and policies.   Oversight agencies believe the compliance program should be a change agent in promoting a culture of compliance that creates an environment less likely to have regulatory or enforcement problems.  This means establishing a culture in which everyone in the work environment embraces and adheres to rules, regulations, laws, code of conduct, and policies.  The Department of Justice (DOJ) and the HHS Office of Inspector General (OIG) frequently encounter organizations with compliance programs that exist on paper, but that culturally failed to be effective in operation. Compliance officers should find means to evidence that the culture of the organization matches the compliance goals.

Positive compliance culture promotes good business

Carrie Kusserow, with over 15 years’ experience as a compliance officer and consultant, makes the case that a good compliance culture is also good for business and does not just serve as a “cost center.” She notes there are many positive benefits to be derived from the effort. She offered the following points in her argument.

  • Organizations are less likely to have liabilities, arising from wrongful behavior.
  • Evidence suggests compliance-committed organizations are more efficient.
  • Lower employee turnover occurs when the organization culture is to abide by rules and standards.
  • There exists greater employee commitment to compliance with laws, rule, code of conduct and policies.
  • Employees feel less pressure to compromise company standards to achieve company goals.
  • Employees are empowered to report wrongful behavior and misconduct internally, not externally.

Compliance culture surveys evidence compliance program effectiveness

Steve Forman, CPA has been using compliance culture surveys for the last 20 years, both as a compliance officer and as a compliance consultant. He believes that one of the best and most inexpensive methods for evaluating, evidencing, and benchmarking compliance program effectiveness is through a compliance culture survey that measures employee perceptions of ethical culture and/or the compliance program. He likes using this type of survey, alternately with a compliance knowledge survey that tests employee knowledge of the program. He points to the fact that the OIG recommends this in its Compliance Program Guidance, wherein it noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as . . . using questionnaires (employee surveys) . . . developed to solicit impressions of a broad cross-section of . . . employees and staff.” Results from a professionally administered survey provide a very powerful and credible report to the compliance oversight committee, as well as to any outside authority questioning the program. They can also identify relative strengths in the compliance programs, as well as those areas requiring special attention that are invaluable for compliance officers.

Compliance survey benefits

Conducting a compliance survey provides numerous benefits to an organization.  For example, it can:

  • provide outcome measurements for the compliance program;
  • serve as critical evidence in determining the degree of effectiveness of the compliance program;
  • identify program strengths and potential weakness warranting attention;
  • evidence the extent of individual and leader commitment to compliance;
  • assess the current state of the compliance climate or culture of an organization;
  • communicate a positive message that employee opinions and perceptions are valued;
  • underscore organization commitment to employees;
  • increase management attention on what is being measured;
  • provide metrics as to progress in developing an effective compliance program;
  • benchmark compliance program effectiveness improvement;
  • signal the organization as to employee attitudes and perceptions;
  • tell employees that what they believe and understand is important; and
  • provide guidance as to where improvements are needed.

Benchmarking compliance program progress

Jillian Bower, with many years of experience in administering compliance surveys, as well as serving as interim compliance officer, notes the OIG compliance guidance says that “the existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”  Surveys can be used to meet that standard. If the survey being used is anchored in a large database of users, the organization can benchmark them against that universe, viewed as very important by most organizations. Furthermore, an initial survey can establish a baseline from which future surveys can be used to benchmark progress of the compliance program. The surveys can benchmark and measure change in the compliance environment over a period of time. However, Bower warns it is inadvisable to use the same survey annually, as significant changes among the work force takes time to show results.

Alena Treen, of the Compliance Resource Center (CRC), has many years’ experience in administering compliance surveys. She explained that culture surveys focus on the beliefs and values which guide the thinking and behavior of employees within an organization. They are usually presented in a Likert Scale format that offer a series of gradation in which respondents are asked whether they “Strongly Disagree,” “Disagree,” are “Neutral,” “Agree,” or “Strongly Agree,” with the statement presented in each item. This is in contrast with a compliance knowledge survey designed to learn how much employees know about the program with questions answerable as yes or no. She notes it is highly advisable to use a valid and independently web-based administered survey that has been tested over many organizations and ensures participant confidentiality. Using a professional survey service specializing in health care compliance is surprisingly inexpensive and less costly than developing and delivering a survey in house that doesn’t carry the same level of credibility. The CRC has been using the Compliance Benchmark Survey© since 1993 and has been employed by hundreds of health care organizations and over a half million surveyed population. Treen normally deals with reports that are about 50 pages in length that provide advice on each topical area and question as to how improvements may be made.   Clients find that comparing their results with the universe to be the most beneficial information.

Richard P. Kusserow served as DHHS Inspector General for 11 years. He currently is CEO of Strategic Management Services, LLC (SM), a firm that has assisted more than 3,000 organizations and entities with compliance related matters. The SM sister company, CRC, provides a wide range of compliance tools including sanction-screening.

Connect with Richard Kusserow on Google+ or LinkedIn.

Subscribe to the Kusserow on Compliance Newsletter

Copyright © 2017 Strategic Management Services, LLC. Published with permission.