Are hospital graders making the grade?
November 6, 2013 in Medical Technology
By Jordan Rau
Judge not, that ye be not judged. Irked by the growing number of report cards assessing the quality of hospitals, a New York state hospital association has taken this biblical admonition to heart by putting out a report card grading the quality of hospital graders. Five of the 10 report cards that were evaluated were given low marks.
“HANYS’ Report On Report Cards,” published Monday by the Healthcare Association of New York State, used nine criteria to assess report cards issued by government or private organizations. These principles included whether the report card used measures grounded in scientific evidence, whether the data were recent and whether hospitals had a chance to review the findings before publication to pinpoint errors.
“We had been hearing more and more from members their general frustration of all the different report cards,” said Kathleen Ciccone, director of HANYS’ Quality Institute. “It’s so time consuming for them to be able to respond to the reports, to be able to see what’s useful about them. They’re really looking for some guidance on how to use the information.”
The scoring ranged from three stars, for the report cards that met all or nearly all of the criteria, to half a star for the report cards that met few or none. U.S. News World Report did the worst, earning half a star for its hospital rankings, which incorporate surveys of the opinions of doctors around the country. “A subjective perception of hospital reputation is not a scientifically proven measure to evaluate hospitals’ processes of care,” HANYS wrote.
Four other report cards received only one star. HANYS dinged the Leapfrog Hospital Safety Score, which gives out grades of A to F, for relying “heavily on unvalidated survey data” that it collects directly from hospitals.
Consumer Reports’ Hospital Safety Ratings also received a sole star. HANYS faulted the nonprofit for rolling up data that came from different sources and spanned different time frames. Some of the Consumer Reports’ data use insurance billing records, which HANYS considers inferior to data taken from medical charts. Seven of the 10 report cards, including Medicare’s own Hospital Compare website, were faulted for using billing records, even though there are no other publicly available measurements in many areas of hospital quality such as mortality.
Healthgrades’ America’s Best Hospitals and Truven Health Analytics Top 100 Hospitals each received one star. HANYS said Healthgrades does not detail its rating method with enough specificity for others to replicate the calculations. Healthgrades and Truven violated another HANYS’ criteria because they include measures that are either not used by the federal government or not endorsed by the National Quality Forum, a nonprofit that seeks industry consensus on which quality measures are reliable.
[See also: HealthGrades defends its best hospital list.]