National Quality Forum Falls Short on Health Care Quality Measurement Projects

HHS needs to use all of its monitoring tools to help address the performance of the National Quality Forum (NQF), complete testing of retooled quality measures, and comprehensively plan for its quality measurement needs, according to a recent report released by the GAO. The report was completed as required by the Medicare Improvements for Patients and Providers Act of 2008 (MIPPA), and is the second of two such reports MIPPA required GAO to submit on NQF’s contract with HHS.

The Department of Health and Human Services (HHS) was initially required under MIPPA to enter into a 4-year contract with an entity that would perform various activities related to health care quality measurement. HHS awarded this job in January, 2009 to the National Quality Forum (NQF), which is a nonprofit organization that endorses health care quality measures and recognizes certain ones as national standards. When the Patient Protection and Affordable Care Act (P.L. 111-148) (PPACA) was signed into law in 2010, it established additional duties for NQF.

As of August 2011, NQF had failed to meet time lines on 18 of 26 projects in nine contract categories during the first two years of the four-year HHS contract, the report said. For more than half of the projects, NQF did not meet or did not expect to meet the initial time frames approved by HHS.

One project completed by NQF was to convert previously-endorsed quality measures to an electronic format compatible with electronic health records, or “retool” them. Although the project was expected to be completed by September 2010, completion was delayed by 3 months, which NQF claims was caused by HHS’ expansion of the project’s scope and complexity. According to HHS, the first set of 44 retooled measures had errors requiring correction, including errors in electronic coding. HHS and NQF agreed that the estimated time frames were “overly ambitious, given the scope and complexity of the work.” HHS officials said “the technical complexity and labor required to complete the project was greater than anticipated.” Additionally, because the project was not completed on time, HHS did not have all the retooled measures that it expected to include in its Electronic Health Records (EHR) Incentive Program.

 NQF was further required to convene an expert review panel to examine the retooled measures to ensure proper formatting and correct logic, which was also completed five months later than required. GAO noted that the delay of this project was also a contributing factor to NQF exceeding its estimated cost for its entire contract activity related to electronic health records by about $560,000 in the second contract year.

Although HHS monitored NQF’s progress using monthly progress reports and they approved changes to time frames and costs, HHS failed to use all of the monitoring tools that were required under their contract. GAO noted that, HHS did not conduct an annual performance evaluation to assess timeliness and cost issues, which could have helped to inform NQF’s future scope of work. It was not until August 2011 that HHS began enforcing the provision for NQF to submit a financial graph that compares monthly costs for each contract activity with cost estimates. This information is not included in monthly progress reports, GAO stated, reasoning that these tools could have provided further detailed information “to help identify instances in which NQF might have been at risk of not meeting time frames or exceeding cost estimates, which could have provided HHS an opportunity to make any appropriate changes to NQF’s activities.”

A comprehensive plan that identifies HHS’ measurement needs and time frames for obtaining endorsed measures under PPACA requirements need to be established by HHS, according to the GAO. Without a plan in place, GAO believes that HHS may “be limited in its efforts to prioritize which specific measures it needs to develop and to have endorsed by NQF during the remainder of the NQF contract.” This could potentially result in HHS being unable to ensure that the agency receives the quality measures needed to meet PPACA requirements, including time frames for implementing quality measurement programs.

 HHS neither agreed nor disagreed with these recommendations, while NQF concurred with many of the findings in the report.