If you’re an older person having a major operation these days, it is very likely that your hospital is receiving a “report card” on their performance. These reports are designed to prompt hospitals to improve in areas where they perform poorly. That’s the good news.
The not-so-good news: Those “report cards” do not seem to be making things better for patients.
A new study published Feb. 3 in JAMA by a team from the University of Michigan Medical School shows no difference in surgical safety among 263 hospitals taking part in a major national quality effort, and 526 similar hospitals that weren’t involved. The study analyzed data from 1,226,000 seniors enrolled in Medicare who had one of 11 major operations at those hospitals over a decade.
The initiative, called the American College of Surgeons National Surgical Quality Improvement Program or ACS-NSQIP, has been run since the early 2000s. Trained nurses at participating hospitals record data about every operation carefully, and send it to a secure central database. The ACS crunches the data from all ACS-NSQIP hospitals and shares quality reports, allowing hospitals and doctors see how their overall performance stacks up against others.
But this quality reporting, the authors find, is not enough alone to accelerate the pace of improvement in surgical safety — nor cost savings.
That doesn’t mean such efforts should stop, or aren’t worthwhile, they hasten to add.
But it does point to the need for better efforts to ensure hospitals put the ACS-NSQIP data to good use, and work with other hospitals in their region to develop and share “best practices” that lead to improvements across hospitals. The ACS, and other large health care quality organizations such as the Michigan Surgical Quality Collaborative, have recently begun to emphasize the importance of collaboration.
“Although ACS-NSQIP hospitals are improving over time, so are other non-participating hospitals,” says Dr. Nicholas Osborne, the study’s lead author and a vascular surgeon at the U-M Health System’s Frankel Cardiovascular Center. “Our study suggests that the ACS-NSQIP is a good start, but that reporting data back to hospitals is not enough. The ‘drilling down’ that is needed to improve quality using these reports is better suited for regional collaboratives.”
Osborne notes that the analysis is the first to use a control group of hospitals to study the impact of ACS-NSQIP participation — and that the team matched each ACS-NSQIP hospital with not one, but two control hospitals. Patients treated at the two types of hospitals were generally similar, though ACS-NSQIP hospitals were larger and did more operations and were more likely to be nonprofits or teaching hospitals.
The 11 types of operations analyzed were esophagectomy, pancreatic resection, colon resection, gastrectomy, liver resection, ventral hernia repair, cholecystectomy, appendectomy, abdominal aortic aneurysm repair, lower extremity bypass, and carotid endarterectomy.
The lack of an effect from ACS-NSQIP participation could be due to many things, the authors say, including hospitals may not have used the reports to improve care, or quality improvement efforts by hospitals using their data may have fallen short of affecting the four items the study evaluated. Many hospitals may not have the infrastructure needed to develop effective strategies to improve care. In addition, outside factors, such as reimbursement-driven efforts to improve safety, improvements in care across all hospitals or selective referral of patients to high-volume hospitals, could also have played a role in improving safety at all hospitals.
“Knowing where you perform poorly is the important first step,” says Osborne. “But the next leap from measuring outcomes to improving outcomes is much more difficult.”
Adds senior author Dr. Justin Dimick, “Better approaches for engaging surgeons, better systems for supporting them in change efforts, and better tools for helping them re-engineer care are clearly needed. Future national and regional quality improvement initiatives must be aimed at not only providing feedback to participants, but also providing an infrastructure for implementing change.”
Dimick, Osborne and co-authors Andrew Ryan and Jyothi Thumma are members of the U-M Center for Healthcare Outcomes & Policy. Dimick and Osborne are members of the U-M Institute for Healthcare Policy and Innovation. Osborne is a member of the VA Center for Clinical Management Research, and Dimick and Ryan have appointments in the U-M School of Public Health. Co-author Lauren Nicholas is at the Johns Hopkins University Bloomberg School of Public Health. Dimick is a consultant and has an equity interest in ArborMetrix Inc., which provides software and analytics for measuring hospital quality and efficiency.
