Is statistical sampling in audits, FCA cases, and recoupment valid?

The government has used random sampling for a very long time as a way to provide sufficient evidence of valid audits and intent or “reckless disregard” False Claims Act (FCA) cases. While the government considers a random sample a valid sample, “’random’” is not necessarily ‘valid’, according to Tracy M. Field, partner, Parker, Hudson, Reiner & Dobbs, LLP and Sandra Miller, partner, Womble Carlyle, Sendridge & Rice LLP. Health care providers must manage and defend against statistical evidence derived from a government audit or presented to a court. Field and Miller presented their viewpoints and practical tips in a session on March 26, 2017, at the Health Care Compliance Association Annual Compliance Institute.

Statistics in audits

Inferential statistics include a probe audit to ensure that there is a good understanding of the population and study design, according to the presenters. By definition, inferential statistics samples items to determine what a population might look like by selecting a random sample. Providing the example of 20 quarters pulled at random from a box of coins, the presenters asked “What do you know about the population based on the sample selected?” Do the quarters represent the actual coin content of the box? Are the sampled items, in this example, coins, providing a normal distribution or a skewed distribution that could be biased? Can the 90 percent confidence interval be “correct” for very imprecise data? Their answer was that we don’t always know how many quarters versus nickels are in the box and what how that concept relates to statistical samplings in claims audits.

Multiple strata. For audits to be more precise, claims are audited by identifying multiple strata. The presenters noted that a sampling unit for an Office of Inspector General (OIG) is a claim, but they stressed that “a beneficiary’s claim is really a cluster of claims which is less precise.” In another example of government audits, the multiple strata involved Current Procedural Terminology® (CPT®) codes but the audits did not take into account the payment variables or the number of claim lines sampled.

Error rates. According to the presenters, the government threshold for error rates is 5 percent in settlements. In addition, for Discovery Samples, OIG uses a 5 percent error rate to determine the full sample size, however, error rates can vary, specifically in Provider Reimbursement Review Board cases. Presenters recommended that providers speak to their legislators regarding audit issues and error rates.

Statistical sampling in False Claims Act cases

In FCA cases, statistics are used to prove the intent of the provider and establish damages. The presenters referred to court cases as they identified questions for providers to ask including whether the case involve medical necessity of the services, whether a realtor can use statistical sampling to prove both liability and damages, whether the sampling reflects patients that may need more rehabilitation, and whether patients are individually considered?

The presenters specifically pointed out the arguments in the brief of the U.S. ex rel Michael and Whitesides v. Agape case before the fourth circuit court, where the defendants argued that “statistical evidence is poorly adapted to providing the falsity and knowledge of elements of FCA liability generally, […] particularly in this case, which involves clinical judgments, such as whether a patient is terminal ill, which is “a highly individualized, context- specific, and uncertain.” In addition, the brief noted that “courts have consistently rejected attempts to use statistical sampling to prove liability in fraud cases.”


The brief in the Agape case explained the recoupment process as an administrative proceeding initiated by a claims processor to recover overpayments through the reduction of future Medicare payments, is a contractual set-off and is far different from an FCA case, according to the presenters. The recovery is limited to the actual amount of the overpayment plus interest while the FCA exposes defendants to treble damages and a fine of at least $5000 per claim. The burden of proof is on the payee to prove that it is entitled to the amount paid.  Further, sampling and extrapolation in recoupment action are authorized by statute if there is evidence of sustained or high level payment error (42 U.S.C. §1395ddd(f)(3)).

OIG challenges industry to come up with an upgraded statistical sampling tool

CMS handles more than a trillion dollars in Medicare and Medicaid claims every year. Because not every claim can be scrutinized, statistical sampling is essential for effective oversight of these claims. The current sampling tool, RAT-STATS, was originally designed by the HHS Office of Inspector General (OIG) to give nonexperts a robust method for selecting statistically valid samples. It is the primary statistical tool for OIG’s Office of Audit Services. Although OIG does not require the use of RAT-STATS, many providers download the software and use it in their efforts to fulfill the claims review requirements for corporate integrity agreements or provider self-disclosure protocol.

The OIG has recently announced the launch of the Simple Extensible Sampling Tool Challenge (Challenge) to develop the foundation for an upgraded version of RAT-STATS. According to the OIG, while the current version of RAT-STATS is well validated, its user interface can be difficult to navigate and the underlying code makes the software costly to update. Therefore, the OIG needs a new, modern version of the software that is easy to use and can be extended in a cost-effective manner.


The RAT-STATS software was originally created in 1978 and has gone through several upgrades since then. Unlike a full statistical package that attempts to answer all types of questions for a wide range of users, RAT-STATS serves as a streamlined solution to handle the specific task of developing valid statistical samples and estimates within the health care oversight setting.

For example, an OIG investigator may pull a simple random sample in order to estimate damages for a provider suspected of fraud. RAT-STATS then generates valid pseudo-random numbers and outputs all of the information needed to replicate the sample. Once the investigator finishes reviewing the sample, he or she can then enter the results into RAT-STATS to get the final statistical estimate. While the investigator may need some basic training in statistics, they do not need the same level of expertise as would be required to navigate the many options available in a full-service statistical or data analysis package.

The Challenge

In order to complete the Challenge participants must create a software package that replicates the operation of four of the functions of the original RAT-STATS software: (1) single stage random numbers;
(2) unrestricted attribute appraisal; (3) unrestricted variable appraisal; and (4) stratified variable appraisal.

Teams of one or more members can participate in this Challenge. Each team must have a captain. Individual team members and team captains must register in accordance with the registration process set forth in the Federal Register notice.  The team captain is to serve as the corresponding participant
with OIG about the Challenge and to submit the team’s Challenge entry. While the OIG will notify all registered Challenge participants by email of any amendments to the Challenge, the team captain is expected to keep the team members informed about matters germane to the Challenge.

Submissions must meet all of the 20 rules and requirements outlined in the Federal Register notice. The technical specifications behind the four RAT-STATS functions along with 10 test datasets are available on the OIG website.

The Challenge began on September 28, 2016. The submission period runs from September 28, 2016, to May 15, 2017. The judging period runs from September 28, 2016, to June 15, 2017. A winner will be announced no later than July 1, 2017. The grand prize is $25,000.