THURSDAY, Dec. 23 (HealthDay News) -- Hospital-wide patient death rates may not be a reliable way to assess a facility's quality of care, a new study indicates.

Researchers in Boston compared four measures of hospital patient death rates. They found that the measures yielded both higher- and lower-than-expected rates for the same Massachusetts hospitals for the same year.

The findings add to an ongoing debate about the value of using death rates as a measure of health-care quality, said lead author Dr. David M. Shahian, a Harvard Medical School professor of surgery at Massachusetts General Hospital (MGH), and colleagues.

"It's troubling that four different methods for calculating hospital mortality rates as a measure of quality should yield such different results. Measurement theory -- not to mention plain common sense -- suggests that there is a problem," Shahian said in an MGH news release.

The study was commissioned by the Massachusetts government, which wanted to find a way to measure hospital quality for the public report cards required under the state's 2006 health care reform law.

The researchers compared four measures of hospital-wide death rates that were created by different vendors. Each vendor received identical data about 2.5 million patient discharges from all 83 general acute care hospitals in Massachusetts over a three-year period.

"The results of the horse race are that it's really not clear who won," study co-author Dr. Lisa I. Iezzoni, Harvard Medical School professor of medicine at MGH, said in the news release. "The problem is that we were measuring the success of each of these measures against a gold standard we simply do not have: an objective measure of hospital quality."

The researchers told the state they could not recommend any of the four vendors' tools.

"But the results should not be interpreted as an indictment of any particular technique," study co-author Sharon-Lise T. Normand, Harvard Medical School professor of health care policy (biostatistics) and a professor of biostatistics at Harvard School of Public Health, said in the news release.

Instead, Normand said, the findings "call into question the concept of measurement of hospital-wide mortality, as four different methods yielded four different results. Thus, this may not be a good way to assess hospital quality."

The study results are published in the Dec. 23 issue of the New England Journal of Medicine.

More information

The U.S. Agency for Healthcare Research and Quality urges patients to do their homework before they choose a hospital.