Truth and Truthiness in Surgery
“The current practice of surgery is an accumulated wisdom, mixing fact, opinion and magical thinking in unknown proportions.” (Jeff Matthews).
This provocative opening statement and a definition
of truthiness from Steven Colbert opened the
Crossgrove lecture at UHN by Jeff Matthews. Jeff is a
Surgeon Scientist whose background includes medical
school and surgical training at Harvard and hepatobiliary
surgical training in Bern, Switzerland. He has
been Chair of Surgery at the University of Chicago
since 2006.
Jeff Matthews
|
Jeff opened with an interesting Venn diagram of Plato’s notion of knowledge,
located at the intersection of truth and belief.
He cited the evidence we usually rely upon - the textbook
that states that 90% of febrile episodes in the first
48 hours after surgery are caused by atelectasis, and the
confirmatory ritual of a chest X-ray showing atelectasis
followed by a response to chest physiotherapy and incentive
spirometry. “The relationship between atelectasis
and fever is unsupported by clinical or experimental
evidence. Atelectasis does not induce fever in animals.
Spontaneous or therapeutic pneumothorax in humans is
not associated with fever, and the incidence of postoperative
atelectasis is similar with and without postoperative
fever. Nevertheless, we are led by thought leaders and
tradition to believe in this persistent myth.” (1) This and
other very convincing anecdotes set the stage for a rational
deconstruction of evidence-based medicine.
Q: Are antibiotics unequivocally indicated in pancreatitis?
A: Yes, according to some meta-analysis and Cochrane
reports. No, according to the same sources at a different
time.
Q: Is mechanical bowel preparation indicated for
colorectal surgery?
A: “In three meta-analyses of nine randomized trials in
over 1,500 patients, no mechanical prep was better than
mechanical prep to prevent anastomotic leak, wound
infection and septic and non-septic complications (2).
Despite the evidence, nearly all colorectal surgeons use
mechanical bowel prep.”
“John Ioannadis shows convincingly that most published
research findings are false (3). The prevalence of false
positives is related to emphasis on P values, flexible trial
designs, data manipulation, and large scale data mining.
Bias in the medical literature derives from publication
bias - favouring publication of positive or hot results;
time lag bias - as negative results generally follow considerably
after positive ones; citation bias - because the visibility
of positive studies is increased by subsequent citation;
reporting bias - as authors highlight positive over
negative results; and prevailing field bias - supporting
entrenched opinion. Highly cited studies showing strong
treatment effects are often followed later by reports of
smaller effects, or contradictory findings. Only 44% are
ever confirmed, 24% are never challenged.
|
Truth is somewhat subjective as different frames of reference
are used by different observers of the same events,
and different relative weight assigned to the details of
the observation results in selective assembly of information
to reconstruct past events. This is the so-called
Rashomon effect, illustrated in Kurosawa’s classic motion
picture in which four individual witnesses describe a
crime in four mutually contradicting ways.
“Knowledge is filtered through the imperfect lens of
human perception, resulting in cognitive distortion. Our
biases influence our perceptions. Cognitive distortion
can be broken into group-think, professional deformation
(how members of our own profession would look
at the situation without the broader view, focusing too
much on one aspect of an event), framing - leading
to conclusions that are highly dependent on how the
data are presented, and confirmation bias- searching
for information in a way that confirms our preconceptions
and vested interests. Surgeons are susceptible to
cognitive distortion and bias because of our hierarchical
organizational structure, the influence of our mentors,
the narrative style of our rationalizations, (particularly in
the culture of the morbidity and mortality conference), a
predilection for linear causality, i.e. what chain of events
is decided upon as the cause of an action or decision,
and belief in ourselves and our results for economic and
reputational reasons.
“Mechanisms to repair or compensate for cognitive
distortion include: rules of thumb (heuristics), multidisciplinary
teams to broaden the view of decision
makers, and decision support tools such as standardized
nomenclature, classifications of clinical presentations,
risk stratification, scoring systems, algorithms and decision
trees. Many protocols and algorithms, however, can
create a false aura of certainty and an illusion of mathematical
precision. Though we can quantify the predicted
mortality for a particular operation for a particular
disease, when it comes to the application in a particular
patient, there are multiple qualitative and subjective factors
which heavily influence the actual outcome. What
are this patient’s comorbidities, anatomic circumstances,
family circumstances, value system, local institutional
resources, comorbidities? Personalized medicine is a current
favourite belief system, promising a path to precise
biasfree decision-making. Multidimensional large scale
data sets promise early detection, improved prognostic
prediction, and the development of precisely tailored
therapy. The reality is that scientific reductionism and
the acquisition of data is outpacing methodology, and
the exaggerated potential for individualized care is leading
to a paradoxical depersonalization of medicine.”
Professor Matthews closed with a statement that evidence-
based medicine is an important concept, but
it is insufficient to encompass all forms of knowledge
needed for good care. Tacit knowledge (described by the
Hungarian polymath Michael Polanyi, who famously
said: “We know more than we can tell”) is the knowledge
gained in a local context with traditions and experiences
that are not universally shared, and cannot be dissociated
from unconscious and subjective elements. “The meaning
between words, intuition and judgment are part of
the concept of tacit knowledge.”
“In surgical education that puts so much emphasis on
facts, there should be a focus on uncertainty. Learning to
make decisions with incomplete information, ability to
recognize false precision, over simplification, overconfidence
and drift toward trying to be the decisive surgeon.
“In summary, truth and truthiness inevitably coexist.
Evidence is allusive and fluid. Knowledge is inseparable
from bias and experience, and evidence-based medicine
is an incomplete epistemology that inadequately
accounts for the complexity and nuances of clinical
decision-making.” This stimulating lecture closed with a
delightful quote from Nobel laureate and University of
Chicago Professor of Physics Enrico Fermi. After hearing
a lecture by Niels Bohr, Fermi said: “Before I came here I
was confused about the subject. Having listened to your
lecture, I am still confused, but on a higher level.”
Tom Waddell asked if this de-emphasis on the factual
and rational basis of surgical practice is a call for a return
to the apprentice model. Matthews answered that longer
exposure to surgical decision-making, for example during
a general surgery residency prior to entering a specialty
like plastic surgery has led to better performance.
The exact content of what was learned in that preparation
is hard to specify.
M.M.
1. I have reported the lecture here from my notes. Quotation
marks are used to convey a sense of participation to the
reader. They should not be taken as evidence of an accurate
record of what was said [Ed.]
2. Slin, K. British Journal of Surgery, 2004, vol. 91, pp 1125-
30
3. PLOS Medicine 2005, vol. 2, pp 696-701
|