the metablog

Are there any truly universal anaesthesia rules?

Sun, 21 Jul 2019 11:35:00 +0000

It’s not uncommon to hear anaesthesiologists drop verbal markers of universal truths into their clinical utterances: always, every, never. Often it's the most banal practices ("always attach the cannula tegaderm so") that generate our greatest passions.

However there are few, if any, universalisms in anaesthesia. Almost every rule has caveats and exceptions, reflecting the shades-of-grey reality of patient needs and human physiology.

When clinicians decree something is always or never so, they are often confusing truth with convention. Sometimes confusing an absolute with what is instead dogma. A minority reveal the rigidity of their thinking, impeding rather than enhancing the performance of their practice.

Rules are still useful. They are important guardrails, a mental model to keep us on the road of safety unless there is very good reason to cautiously edge onto the gravely shoulder, or even turn down a completely new path. But always with the understanding of the compromise we have intentionally made, reminding us of the cost that may need to be paid.

Universal anaesthesia rules become a problem when we dogmatically extol them without understanding the foundations of why – or use them as a blunt tool to browbeat our colleagues and assistants.

Rules are shortcuts to express the tension between the benefits and risks of different anaesthetic decisions. All difficult airways in specialty exams might require mandatory awake fiberoptic intubation, but in a real world of patient refusals and modern airway toys & techniques, it is a more nuanced decision. In fact, patient refusal might be the closest we get to an absolute rule – but even this is a negotiable area of greys.

As a mental model for normalising risk, rules are an efficient way to communicate our prioritisation of the risk-benefit tension. A tool for education and standard setting, without requiring the immediate overhead of qualifying and rationalising.

First learn the rules. Then master the rules. And only then can you break the rules.

The danger comes when we confuse the mental model – the guardrails – for the reality. Misidentifying a tool to guide perception and practice, as something we mistakenly believe reality can be bent to. When we make this error, reality has a nasty habit of reasserting itself with unsentimental brutality.

I made my mistakes so you don’t have to. Don’t repeat my mistakes. Make your own mistakes. Make better mistakes.

Browse articles by Journal, Year or Issue

Thu, 18 Oct 2018 02:33:00 +0000

As well as suggesting interesting articles online or in your weekly emailed metajournal, you can now also browse articles by journal, year or even just scroll through a single journal issue. Simply click on the journal name or issue date above an abstract to focus your abstract browsing.

So whether you want to see the latest articles from the British Journal of Anaesthesia, the top articles of 2017 from the American Journal of Emergency Medicine, or simply browse through the October issue of Current Opinion in Anesthesiology, metajournal can keep you up to date.

Of course, signed-in users can also search across any of these scopes – say, if you want to see the top remifentanil abstracts from IJOA, or find that elusive C-MAC c-spine article you vaguely remember seeing in a 2017 Am J Emerg Med issue – just hit the Search... button.

The 4th Horseman: Research Fraud & Mountains of Fujii

Thu, 10 Nov 2016 17:12:00 +0000

The fourth and final horseman of the medical research apocalypse is the scourge of medical research fraud. Although certainly not a new problem, the scale and potential impact of research deceit is unlike anything faced previously.

Academic fabrication, falsification, and plagiarism (FFP) make up the breadth of academic fraud – and sadly, anesthesia is the number one specialty by volume. In 2012 our eyes were opened to the sheer scale of the problem as the largest medical research fraud in history was exposed.

Trust, responsibility and the Fujii fraud

Medical research involves a lot of trust. The trust of patients and the public, the trust of publishers and the trust of the research-consuming clinician. Unfortunately we often overlook our responsibility to ensure that our trust is not misused.

"...with increasing amazement, we notice that the results reported by Fuji et al. are incredibly nice ..." wrote Kranke, Apfel and Roewer in their April 2000 letter, politely challenging Yoshitaka Fujii's PONV research.¹

And so began the very slow unravelling of the biggest academic fraud in the history of medicine. Despite a meaningless response from Fujii to that first challenging letter, there was no investigation or further questioning from the various anesthesia journals. In fact Anesthesia & Analgesia went on to publish another 11 articles by Fujii over the next decade.²^,³

Other major anesthesia journals did not escape unscathed, merrily publishing Fujii's growing stack of articles: Anaesthesia, The British Journal of Anaesthesia, The European Journal of Anaesthesia, The Canadian Journal of Anesthesia, Acta Anaesthesiologica Scandinavica, Pediatric Anesthesia, Anaesthesia and Intensive Care... the list goes on.

The story is interesting both because of the lengths that one man went to falsifying post-operative nausea and vomiting research (hardly the sexiest of anesthesia topics) and that we then overlooked the truth for almost two decades. The passive culpability of major anesthesia journals and the impressive statistical analysis by John Carlisle⁴ that ultimately brought down the Fujii fraud add further color to this story.

As the second decade of the 21st century draws to a close, the same statistical techniques, paired with a little data mining, provide a new defence against research fraud.

19 years of fraud – 172 papers

Other than a brief two years in Canada, Yoshitaka Fujii has spent most of his career in Japan split between the University of Tsukuba and the prestigious Toho University School of Medicine. His primary research focus was the management of post-operative nausea and vomiting (PONV), specifically 5HT3-antagonists – and in particular granisetron.

Fujii appears to have began his fraud in 1993 with the unexciting paper "Contractility of fatigued diaphragm is improved by dobutamine" published in the CJA.⁵ Over the next 19 years he published an incredible 249 papers, 172 of which have now been withdrawn from publication having been found to be either totally fabricated or including fictional data.

"It is as if someone sat at a desk and wrote a novel about a research idea..." – Japanese Society of Anaesthesiologists investigating committee.⁶

It is unclear what drove Fujii to deceive on such a grand scale. While the sheer number of papers give great weight to his deceit and its potential to shift medical management – his research areas were neither exciting or celebrated areas of our specialty. Nausea and vomiting; diaphragm contractility; neuromuscular block reversal; propofol injection pain; and attenuation of intubation responses – no single article could be argued to be practice changing. Few articles were widely cited.

Fujii received no awards or prizes for any of his studies, although he did unsuccessfully submit some for consideration. Neither does it appear he received any honoraria or other financial reward. Was reputational advantage alone enough to motivate him; winning the career security that comes with being a well-published associate professor at a prestigious hospital and research institute?

Joachim Boldt and a taste for starch

Anesthesia has the dubious honor now of holding not just the first place on the research-fraud leaderboard, but also second place with German anesthesiologist and researcher Joachim Boldt's impressive "up to 90" fraudalent articles.

Boldt was once considered a leader in colloid research, publishing extensively on hydroxyethyl starch use. An investigation into 91 of his publications showed study files missing or incomplete for the majority; and a lack of ethics approval, registration or even formal recording of subject consent in many.⁷ In ten of the studies false data was also identified.

Reubin and Pfizer

In 2010, Dr Scott Reubin, former Professor of Anesthesiology at Tufts in Boston, was released after 6 months in prison for medical fraud. Dubbed by Scientific American a 'Medical Bernie Madoff' after publishing 21 fraudulent articles, what Reubin lost to Fujii in volume he made up for in clinical impact. His pain medicine research focused on COX-2 inhibitors Vioxx and Celebrex, and the analgesia benefits of Lyrica and Effexor, particularly after orthopedic surgery.

"We are talking about millions of patients worldwide, where postoperative pain management has been affected by the research findings of Dr. Reuben," says Steven Shafer, editor in chief of ... Anesthesia & Analgesia, which published 10 of Reuben's fraudulent papers. Paul White, another editor at the journal, estimates that Reuben's studies led to the sale of billions of dollars worth of ... Pfizer's Celebrex (celecoxib) and Merck's Vioxx (rofecoxib).⁸

Why anesthesia and research fraud?

"In fact, of the 2,200 papers that journals have retracted since 1970, Reuben, Boldt and Fujii ... account for roughly 285, or nearly 13%." – Adam Marcus, Retraction Watch.⁹

Anesthesia now appears over-represented in research fraud – why? The controlled and measured environment of an operating room creates a lot of opportunity for clinical research. Much of this focuses on specific interventions within a context of standard patient care. The majority happen in the normal hospital environment. This ready access to large volumes of data usable by small research teams, combined with a lack of oversight that appears in a busy surgical setting, provides the means to commit fraud even on a grand scale.

Anesthetists and anesthesiologists tend to be more mobile than other specialists. In many countries it is common for anesthetists to work at multiple institutions, further muddying the waters of research accountability.

This provides means, but what about motivation? Institutional environments rewarding clinicians for publication for the sake of publication provide tempting incentives. When the motivator of medical research becomes decoupled from actually improving patient care, the purpose of investigating and answering clinical questions is quickly lost.

Bringing us back to responsibility

It is easy to blame anesthesia journals for failing to catch these research frauds earlier. But you and I are as much to blame as any overwhelmed journal editor for failing to keep the system accountable. Whether by passively supporting the endless publication of me-too research, or supporting institutional cultures that encourage a publish-or-perish mentality – we are equally culpable. We have a responsibility to question and challenge what is researched and where it is published.

Many readers of the articles of Fujii, Boldt and Reubin noted the lucky neatness of certain parts of their reported data. A tiny few penned a letter back to the journals. An even smaller fraction did anything further.

Hopefully the example that John Carlisle provides in his epic analysis revealing the Fujii fraud is sufficient inspiration for us all. In the same way that anesthesia has become a leader in risk management, patient safety and progressive training, it would go a long way to restore public trust if we now rose to become the leader in research integrity.

So, do you read critically?

Kranke P, Apfel CC, Roewer N, Fujii Y. Reported data on granisetron and postoperative nausea and vomiting by Fujii et al. Are incredibly nice! Anesth Analg. 2000 Apr;90(4):1004-7. ↩
To their credit A&A were instrumental in ultimately uncovering the fraud and have lead the charge in undoing the damage inflicted by Fujii. Shafer SL. Fujii Statement Of Concern. Anesth Analg. 2012 Mar 7. ↩
Yentis SM. Lies, damn lies, and statistics. Anaesthesia. 2012 May;67(5):455-6. ↩
Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia. 2012 May;67(5):521-37. (full text) Carlisle investigated the distribution of independent variables between study groups in Fujii's research: "The published distributions of 28/33 variables (85%) were inconsistent with the expected distributions, such that the likelihood of their occurring ranged from 1 in 25 to less than 1 in 1 000 000 000 000 000 000 000 000 000 000 000 (1 in 10³³), equivalent to p values of 0.04 to < 1 × 10^-33 , respectively." ↩
Fujii Y, Toyooka H, Ebata T, Amaha K. Contractility of fatigued diaphragm is improved by dobutamine Can J Anaesth. 1993 May;40(5 Pt 1):453-8. ↩
Akst J. Anesthesiologist Fabricates 172 Papers The Scientist, Jul 3 2012. ↩
Ludwigshafen Hospital. Hospital Presents Results of Final Report: Committee Completes Investigation in the Case of Dr Boldt. 09 August 2012 ↩
Borrell, B. A Medical Madoff: Anesthesiologist Faked Data in 21 Studies. Scientific American, March 10, 2009. (pdf) ↩
Marcus A. Does anesthesiology have a problem? Retraction Watch. 2 July 2012. ↩

The 3rd Horseman: Significance & Relevance

Sun, 16 Oct 2016 14:42:00 +0000

Arguably the most important piece of the evidence-based-medicine puzzle is when we ask ourselves:
"Is this evidence significant? – Is this relevant to my patients and my practice?"

When we talk about the 'quality' of a published research work we largely mean what the epidemiologists refer to as 'internal validity' – the extent to which the study's conclusions are actually warranted given the methodology and results. Internal validity looks only at the study design, conduct and interpretation, and takes into account bias and confounders. While important, internal validity is not alone sufficient.

The significance of a piece of evidence to medicine in general, along with it's relevance to our own practice, is referred to as the external validity. I think that for your and my practice this is often what matters most.

Really, external validity just describes how well the results and conclusions can be generalized to situations and people beyond those in the study.

I think of significance as the cumulative generalizability of a piece of evidence for the specialty and for wider medicine, integrated with how well the evidence agrees with what is already known. Relevance describes how applicable the evidence is to my hospital, my practice – and my patients.

It has significance for you, and relevance for me.

There are publications that have such great significance that even though their relevance to your own practice is low, it is still very important that you know of them. For example a critical care specialist with limited obstetric practice should still understand the importance of the Magpie Trial for halving the risk of eclampsia in pre-eclamptic pregnant women. Most 'landmark' trials fall into this group of greatly-significant medical articles.

In contrast, an article might be of low general significance but have great relevance to my or your own practice, such as Tandoc's 2011 trial showing that adjuvant dexamethasone prolongs the duration of interscalene blocks. Not particularly relevant unless you have some regional anesthesia interest!

The challenge is that significance *and *relevance are imperfectly linked, in fact there is at times almost an inverse relationship – the most personally relevant articles are unlikely to be the most significant. While most clinicians could make a reasonable judgement of the significance of any individual piece of medical research, determining relevance is much, much harder.

Only you can determine the relevance of the evidence for your practice and your patients – and there lies the challenge of the third horseman. As medicine increasingly specializes with ever greater levels of distinction ("I only operate on mitral valves"), and the volume of publication accelerates, the probability that any single, random article you pickup will be personally relevant is falling exponentially.

The only way to combat this is by having a non-random way of finding personally relevant, practice-changing evidence, and simultaneously not missing those general articles of great significance.

The 2nd Horseman: Quality Evidence

Mon, 03 Oct 2016 00:00:00 +0000

Our understanding of what makes for quality medical research has improved dramatically over the past three decades. We understand that research must be ethical; should be reproducible; free of bias, so that we may make accurate conclusions; and that confounders be minimised and controlled for. We understand that prospective is best, and large blinded randomized trials are king.

We can articulate that a study must be appropriately powered to answer the question we are asking – but also not over-powered so that we waste resources and goodwill, or continue a study after an answer is known.

To meet the needs of 'quality medical research' ambitious study designs have been developed: massive, multicenter randomized controlled trials; longterm cohort studies; nested case-control and case cohort. Advanced biostatistic techniques, like multiple logistic regression analysis are now commonly used, along with combining tens or even hundreds of smaller studies into single large meta-analyses.

Surely (medical) truth is now knowable?

Nonetheless, despite the obvious improvement in the quality of many large, landmark trials, our appreciation of what makes a quality trial has highlighted how even today many do not live up to the standard required.

"It usually comes as a surprise ... to learn that some (perhaps most) published articles belong in the bin, and should certainly not be used to inform practice."
– Trisha Greenhalgh, How to read a paper.

Medical meta-researcher, Dr. John Ioannidis, depressingly concludes that 90% of medical research is fundamentally flawed. In his landmark 2005 paper in PLoS Medicine, Ioannidis demonstrated that 80% of non-randomized studies were wrong, and among randomized controlled studies 25% were incorrect. Even large, multicenter, randomized clinical trials were predictably wrong in 10% of studies.

"Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true."
– Dr. John Ioannidis

To further explore the extent of the problem Ioannidis investigated just under 50 of the most significant and highly regarded medical research findings from 1990 to 2003. Of 45 that concluded their interventions were effective, 34 had had their hypothesis retested. Of these 34, over 40% (14) were subsequently shown to be incorrect or exaggerated. Forty percent of some of the most highly regarded, practice-changing medical evidence from the 20th century subsequently disproven!¹

Ioannidis then focused on how clinicians adjust their views when highly cited evidence is later refuted by better quality research. Do we correct our misconceptions when quality, contrary evidence is available? Ioannidis' team investigated this by examining the persistence in the belief of several big-ticket 1990s errors: vitamin E's supposed cardiovascular benefits, beta carotene's anti-cancer effects and oestrogen's Alzheimer-protection.² Although early observational epidemiological studies had supported these theories, all had been subsequently refuted by large, randomised controlled trials a decade later. Surely researchers had adjusted their understanding of the evidence?

Surprisingly the earlier disproven observational studies were all still positively cited in 50% or more of peer reviewed publications - despite the well-established contrary evidence! It seems that even when we have conflicting better-quality evidence, the established belief persists independently of its merit.

What are we to do?

The quality of medical evidence is far poorer than we belief, even among the those studies that we perceive as most reliable. But most worryingly, even when bad evidence is subsequently disproven and corrected, the incorrect conclusions persist in the minds of doctors for decades.

We need be both more critical and questioning of research conclusions even as we incorporate new knowledge into our practise – but simultaneously we must avoid becoming wedded to the dogmatic; quick to change our decisions and care when the weight of evidence guides us.

Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005 Jul 13;294(2):218-28. ↩
Tatsioni A1, Bonitsis NG, Ioannidis JP. Persistence of contradicted claims in the literature. JAMA. 2007 Dec 5;298(21):2517-26. ↩