Sabermetrics and Surgical Outcomes
Justin Dimick
|
Justin Dimick, a former
American Greco-Roman
varsity wrestling champion
at Cornell, is currently
practicing general
and minimally invasive
surgery the University
of Michigan. He trained
in epidemiology at
Dartmouth and completed
his medical and
surgical training at Johns
Hopkins University. His
research is conducted at the University of Michigan’s
Centre for Healthcare Outcomes and Policy.
At December’s University Rounds on Surgical
Epidemiology and Statistics, Justin used a picture of the
University of Michigan football stadium filled with over
a 100,000 people to drive home the point that there over
100,000 deaths each year following surgery in the United
States. He asked our audience “if surgical performance follows
a bell-shaped curve, which of you is in the lower half
of the curve?” He asks this question whenever he speaks;
it universally elicits a negative response. Like the 95%
of teenage drivers who consider themselves to be in the
top 5% in driving performance, we tend to overestimate
ourselves. At the current time, there is great interest and
attention focused on the outcomes of surgery.
The National Surgical Quality Improvement Program
(NSQIP) - the American College of Surgeons Program
(Surgical Spotlight, Winter 2010, pp. 14-15 or http:// www.surgicalspotlight.ca/Article.aspx?ver=Winter_201 0&f=KoCaterpillarGraphs) on quality improvement,
teaches that “the road to quality improvement is paved
with DATA.” Justin used the popular movie “Money
Ball”to illustrate how important is to use the right data
to improve outcome. The manager of the Oakland
Athletics, Billy Beane (played by Brad Pitt) was able to
bring his team into the playoffs regularly, and establish
an amazing record with the lowest budget in the major
leagues ~ 40 million dollars per year. He outmanaged
the New York Yankees despite their annual budget of
226 million dollars. Beane relied on sabermetrics - the
mathematical and statistical analysis of baseball records
originated by Bill James. James recognized that the ability
to get on base was what scored runs and won games,
not the usual metrics used to choose and recruit players,
such as running speed, fielding dexterity, and batting
average. Similarly, the usual metrics for measuring or
predicting surgical outcomes may not be using the right
data. The commonly accepted formula is: Severity of illness
+ quality of care + random error = outcome.
|
But severity of illness as a risk factor is overemphasized
in Justin’s view. Though predictive of complications, it
is the ability to rescue from complications, not their
incidence, that determines hospital mortality. That is
why teams, rather than individual surgeons, account
for lower mortality at large volume hospitals. The
individual surgeon proved to be an important factor in
only a few operations, like carotid endarterectomy. The
team proved more influential in most operations like
pancreatectomy. These findings were recently published
in a landmark article in the New England Journal of
Medicine. They clarify and emphasize the critical importance
of the quality of care.
A second helpful contribution to improving outcomes
was a simplification of the burdensome documentation
process of care. In a recent analysis of the NISQUIP
data, Dimick and his colleagues found that they could
simplify their data collection, as five variables gave as
robust a correlation with outcome as the 20 variables
currently collected in the NISQUIP program.
Alice Wei asked about analyzing the rescue process.
Justin responded that this is a complex component
which is currently being analyzed using qualitative
and quantitative methods. For example, what is the
impact on outcome of adding of a rapid response team?
David Latter asked whether mortality was overused. It
can be accurate, but impoverished in that it does not
address some of the significant goals of surgery. Dimick
responded that mortality had proven to be very unhelpful
in analysis in bariatric surgery, as there were only two
deaths in 6000 analyzed cases. The complication rate is
far more significant. Gail Darling asked about the analysis
of length of stay, which proved to be much greater
following esophagectomy in high volume hospitals. This
was a reflection of the rescue process at low volume
hospitals, which probably led to earlier deaths with less
prolonged rescue procedures.
Both Dimick in Michigan and David Urbach in Ontario
are analyzing ‘what goes on in the operating room’ as a
factor in outcome. Perioperative antibiotics and other
nonsurgical factors have been well analyzed. Technical
proficiency and innate skill are probably high leverage
components, but these are inadequately studied. The
Michigan group is using the skills lab to assess innate
surgical skill and videotaping operating room cases to
assess technical proficiency. They use an OSATS scale
(developed by Richard Reznick and his colleagues) to
assess skill of the operating room team. They find that
the details of the operation that are the usual metrics,
such as stapled versus hand -sewn anastomoses have not
proven to be potent factors.
M.M.
(1) Ghaferi, A.,John D. Birkmeyer, J., Dimick, J.Variation in
Hospital Mortality Associated with Inpatient Surgery; N Engl
J Med 2009; 361:1368-1375
|