Established, experienced...and wrong

I've heard versions of this "increasing confidence" aphorism for years, but recently wondered it came from. This seems to be its source - A Skeptic's Medical Dictionary, by Michael O'Donnell:

     Clinical experience. Making the same mistakes with increasing confidence over an impressive number of years.

     Evidence-based medicine. Perpetuating other people's mistakes instead of your own.

(Cited in The Lancet - and see the book review in The BMJ.)

Shrikant Kalegaonkar pointed out that Oscar Wilde said something similar - and it is as exquisite as you would expect. It's here, in his 1890 The Picture of Dorian Gray:

He began to wonder whether we could ever make psychology so absolute a science that each little spring of life would be revealed to us. As it was, we always misunderstood ourselves and rarely understood others. Experience was of no ethical value. It was merely the name men gave to their mistakes. Moralists had, as a rule, regarded it as a mode of warning, had claimed for it a certain ethical efficacy in the formation of character, had praised it as something that taught us what to follow and showed us what to avoid. But there was no motive power in experience. It was as little of an active cause as conscience itself. All that it really demonstrated was that our future would be the same as our past, and that the sin we had done once, and with loathing, we would do many times, and with joy.

It was clear to him that the experimental method was the only method by which one could arrive at any scientific analysis of the passions; and certainly Dorian Gray was a subject made to his hand, and seemed to promise rich and fruitful results. 

From the sublime to the ridiculous. I've another to add to this picture. It's based on an aphorism I coined myself in a piece I wrote in The BMJ in 2004 - cartoon version and post here on Statistically Funny:

     Promising treatment. The larval stage of a disappointing one.

Calling treatments "promising" is a problem that seems to afflict all sides - including evidence-based medicine (EBM). As I'm lampooning the worst side of clinical experience with this cartoon though, it seems only fair for balance to have a shot at EBM at the same time. Here's where I've written about problems in EBM this year at MedPage Today and PLOS Blogs.

The most important article recently on this subject, though, is from Trisha Greenhalgh and colleagues: Six 'biases' against patients and carers in evidence-based medicine. 

(Cartoon spruced up and text added on 12 September 2015.)

Effectiveness delusions - don't become a statistic!

To inoculate yourself against "significant" effects that might not improve health, have a look at papers by Ioannidis and Gotzsche. Want to know more about the risks of relying only on biomarkers? Here's an explanation of their pitfalls at PubMed Health.

As if data could speak for itself...

Screening for disease - hoping for a miracle

Disappointment in early intervention, and the cycle begins anew: even earlier intervention in even more people who will mostly not get sick anyway. Read more about why starting the disease clock ticking earlier sometimes helps, but often doesn't:  http://1.usa.gov/xywTjg

Torturing the data - a cry from the heart

A take on a classic saying - can't face the data mining boom without it! You can read about the dangers of data analyses that weren't pre-planned and multiple testing (data dredging) here. Another important related read? Epidemiologist John Ioannidis' "Why most published research findings are false."

Medical research - too big to fail?

Read about what it would take to get better research for better health care in Testing Treatments (access in full)

Metaanalyses (325 BCE - 278 BCE)

For the January 2012 issue of the newsletter for the International Society for Evidence-Based Health Care

Evidence-based ethics committee

Could we evidence our way to a better research ethics system? A bit more formal evaluation has always seemed to me like a very good idea. When I posted the version of this cartoon in 2012, though, I didn't know of any controlled evaluation - despite the critical importance of research ethics and the potential for ethics regulation processes to do harm themselves.

I updated the cartoon when I saw a controlled study related to research ethics for the first time. Mary Dixon-Woods and colleagues studied adding an ethics officer. They wanted to know if that could make the process more efficient and improve the quality of outcomes.

It didn't go exactly according to plan - 31% of the time there was no contact between the ethics officer and the committee before the meeting. There wasn't an appreciable impact on outcomes - and it didn't speed up the process either. Hats off to all concerned: we're a little less ignorant about research ethics committees than we were before.

Dixon-Woods cited a scoping review that showed how thin on the ground solid knowledge about what could make research ethics review more reliably effective. Here's hoping this new study spurs copycats!

Disclosure: I spent years on national research ethics committees in Australia, but don't on any now. I am a member of the human ethics advisory group for PLOS One, and was a member of the BMJ's ethics committee for several years.

Update: 3 September 2016 

Losing the (forest) plot

Systematic reviewing: what's not to love?

Heaven's Department of Epidemiology

Watch out for risk's magnifying glass - and cut your risk of being tripped up by 82%!

Whenever you see something tripling - or halving - a risk, take a moment before you let the fear or optimism sink in.

Relative risks are critically important statistics. They help us work out how much we might benefit (or be harmed) by something. But it all depends on knowing your baseline risk - your risks to start with.

If my risk is tiny, then even tripling or halving it is only going to make a minuscule difference: a half of 0.01% isn't usually a shift I'd even notice. Whereas if my risk is 20%, tripling or halving could be a very big deal. Unless you know a great deal about the risks in question - or your own baseline risk, you need more than a relative risk to make any sense out of data.

There's a good introduction to absolute and relative risks at Smart Health Choices. 

This is one of the 5 shortcuts to keep data on risks in perspective, at Absolutely Maybe.

Cartoon and content updated on 3 June 2017: This post was originally the cartoon only, from my blog post for the British Journal of Sports Medicine.