Correlation or Causation – Developments in Medical Science


Like operations, medications and fad diets, health care policy is usually appraised with a simple question: Does it work, or doesn’t it? The assumption is that new regulations and practices either cut costs and improve care, or they don’t; and so, by simple extension, should be either supported or struck down.


Now a fascinating study asks all of us to reconsider [how we judge the success or failure of certain health care initiatives]. Focusing on outcomes in obesity surgery, the study illustrates the breathtaking speed at which clinical medicine can progress, [and how wrong we can be about whether our medical initiatives are actually bringing about improvements in patients’ lives.]


Doctors have attempted to use surgery to treat obesity since the 1950s, but it wasn’t until the early 1990s that doctors and patients began to view bariatric surgery [(stomach stapling, etc)] as an acceptable alternative to diets, exercise and medications. Fuelled by dramatic and well-televised outcomes, like patients clocking in losses of 100 pounds or more, the number of operations rose exponentially despite the fact that they were still considered experimental and that complications could be significant. Pulmonary embolism, leaks from the new intestinal connections and even death were all known to result from bariatric surgery; and the rates of such mishaps were not always systematically documented.


[As a result, in order to improve patient safety,] in 2006, the Centres for Medicare and Medicaid Services (C.M.S) [(the publicly funded health insurance programmes in the United States.)] began restricting payment for bariatric surgery to so-called centres of excellence. To become such a centre, hospitals needed first to fulfil criteria set by one of two organizations, the American College of Surgeons or the American Society for Metabolic and Bariatric Surgery. The criteria included having two bariatric surgeons on staff and performing at least 125 obesity operations a year. [The idea being that by allowing only excellent surgeons to perform the surgery, patients would be less likely to suffer complications after their operation … and initially it seemed to work as the success rate of surgery went up and the number of post-operative complications went down.]


[However,] in the years since 2006, surgeons developed new operations, incorporated safer techniques and better equipment, and began conducting large-scale research on complication rates and safety measures [all of which would have improved the chances of successful surgery for anyone, anywhere. This raises the question of whether we can be sure that it was actually the decision to use only centres of excellence that led to the improvements in survival rates, rather than these other improvements that would have happened at any other hospital in the country?] Yes, Medicare patients who had bariatric surgery after 2006 tended to do better, confirming a common belief that the C.M.S. coverage decision contributed to the improvement in outcomes and general upsurge in quality of care. But it wasn’t clear if the improvements were due to the policy, or were simply a result of the advances that had occurred over time.


To answer this question, the authors of the new study reviewed the discharge records of more than 20,000 Medicare patients who underwent bariatric surgery before and after the national coverage decision. As with previous studies, they found that Medicare patients who had their operations after 2006 did better than those who had them earlier.


The researchers then compared these outcomes to those of patients who were not covered by Medicare and therefore not restricted to having their operations done at centres of excellence. Even after adjusting for individual patient risk factors and the specific type of bariatric procedure performed, they found no differences in complication rates or outcomes between Medicare and non-Medicare patients. Moreover, they discovered that many of the improvements had been under way prior to 2006.


In other words, the much-heralded policy of funnelling patients to centres of excellence has had little effect on how patients do. [Thus this is a perfect example of how something that looks like a case of causation (restricting surgery to centres of excellence caused better results as patients suffered fewer complications) was actually a case of correlation (restricting surgery to centres of excellence just happened to coincide with a reduction in post-operative complications) that was actually caused by something else. In this case, the general improvement in surgical techniques in this field since 2006.]


The results of this study have prompted the C.M.S. to reconsider its policy. As a number of patients who may benefit from bariatric surgery do not currently choose to have it done because of the difficulty of travelling to a centre of excellence for the surgery and the subsequent regular medical check-ups


Not all bariatric surgeons agree with lifting the 2006 coverage restrictions; but Dr. Dimick and his co-authors believe that reviewing all the research and reassessing policy choices is important, particularly in a field that has advanced as quickly as theirs. “Policymaking has to be a dynamic process that reconsiders the evidence as it comes out,” Dr. Dimick said. “If you’re going to keep designating centres of excellence, you need to be sure they are actually excellent and actually offering better care than other institutions.”


Dr. Pauline Chen, April 4th , 2013