OIG challenges industry to come up with an upgraded statistical sampling tool

CMS handles more than a trillion dollars in Medicare and Medicaid claims every year. Because not every claim can be scrutinized, statistical sampling is essential for effective oversight of these claims. The current sampling tool, RAT-STATS, was originally designed by the HHS Office of Inspector General (OIG) to give nonexperts a robust method for selecting statistically valid samples. It is the primary statistical tool for OIG’s Office of Audit Services. Although OIG does not require the use of RAT-STATS, many providers download the software and use it in their efforts to fulfill the claims review requirements for corporate integrity agreements or provider self-disclosure protocol.

The OIG has recently announced the launch of the Simple Extensible Sampling Tool Challenge (Challenge) to develop the foundation for an upgraded version of RAT-STATS. According to the OIG, while the current version of RAT-STATS is well validated, its user interface can be difficult to navigate and the underlying code makes the software costly to update. Therefore, the OIG needs a new, modern version of the software that is easy to use and can be extended in a cost-effective manner.

Current RAT-STATS

The RAT-STATS software was originally created in 1978 and has gone through several upgrades since then. Unlike a full statistical package that attempts to answer all types of questions for a wide range of users, RAT-STATS serves as a streamlined solution to handle the specific task of developing valid statistical samples and estimates within the health care oversight setting.

For example, an OIG investigator may pull a simple random sample in order to estimate damages for a provider suspected of fraud. RAT-STATS then generates valid pseudo-random numbers and outputs all of the information needed to replicate the sample. Once the investigator finishes reviewing the sample, he or she can then enter the results into RAT-STATS to get the final statistical estimate. While the investigator may need some basic training in statistics, they do not need the same level of expertise as would be required to navigate the many options available in a full-service statistical or data analysis package.

The Challenge

In order to complete the Challenge participants must create a software package that replicates the operation of four of the functions of the original RAT-STATS software: (1) single stage random numbers;
(2) unrestricted attribute appraisal; (3) unrestricted variable appraisal; and (4) stratified variable appraisal.

Teams of one or more members can participate in this Challenge. Each team must have a captain. Individual team members and team captains must register in accordance with the registration process set forth in the Federal Register notice.  The team captain is to serve as the corresponding participant
with OIG about the Challenge and to submit the team’s Challenge entry. While the OIG will notify all registered Challenge participants by email of any amendments to the Challenge, the team captain is expected to keep the team members informed about matters germane to the Challenge.

Submissions must meet all of the 20 rules and requirements outlined in the Federal Register notice. The technical specifications behind the four RAT-STATS functions along with 10 test datasets are available on the OIG website.

The Challenge began on September 28, 2016. The submission period runs from September 28, 2016, to May 15, 2017. The judging period runs from September 28, 2016, to June 15, 2017. A winner will be announced no later than July 1, 2017. The grand prize is $25,000.