Where is the Evidence for “Evidence-Based” Therapy?*

1 Where is the Evidence for Evidence - based Therapy?* Jonathan Shedler Evidence - based therapy has become quite the term Evidence - based comes from medicine. It gainedattention in the 1990s and was, at the time, a call for criticalthinking. It reflected the recognition that we ve always done itthis way is not a good enough reason to keep doing decisions should reflect clinical judgment, patients values and preferences, and relevant scientific Evidence - based has come to mean something very dif-ferent in the psychotherapy world. The term has been appropri-ated to promote a particular ideology and agenda. It has becomea code word for manualisedtreatment most often, brief, highly-structured cognitive behavioural therapy (CBT). Manualised means the therapy is literally conducted by following an instruc-tion manual. The treatment may be pre-scripted in a way thatleaves little room for understanding patients as the Evidence - based therapy movement lies what Iwill call the master narrative , a narrative that increasinglydominates the mental health landscape.

2 The master narrativegoes something like this: In the dark ages, therapists practiceduntested, unscientific therapy. Science shows that Evidence - based therapies are superior. This narrative has become a justi-fication for all-out attacks on traditional ( , psychodynamic)therapy that is, psychotherapy that fosters self-understandingand insight in the context of a meaningful, ongoing therapy Journal of Psychological Therapies in Primary Care, Vol. 4, May 2015: pp. 47 59.* 2015 by Jonathan Shedler, 10/07/2015 14:26 Page 47 Here is a small taste of what proponents of Evidence - based therapy have been saying in public: The empirically supportivepsychotherapies are still not widely practiced. As a result, manypatients do not have access to adequate treatment (Hollon et al., 2002;emphasis added). Notice the linguistic sleight-of-hand: If it is notan Evidence - based ( , manualised) treatment, it is Mischel of Columbia University wrote, The disconnectbetween what clinicians do and what science has discovered is anunconscionable embarrassment (Mischel, 2008; emphasis added).

3 When this master narrative gets into the media, things getworse. The venerable Washington Postran an article titled, Is yourtherapist a little behind the times? (Baker et al., 2009). It likenedtraditional (read, psychodynamic) therapy to pre-scientific medi-cine when healers commonly used ineffective and often injuri-ous practices such as blistering, purging and bleeding . Newsweeksounded a similar note in an article titled, Ignoring the Evidence :why do psychologists reject science? (Begley, 2009).Notice how a form of McCarthyism enters the proponents of brief, manualised therapies have appro-priated the term Evidence - based for their own use, it becomesdifficult to have an intelligent conversation about what consti-tutes good therapy toquestion claims for Evidence - based therapy is to risk the accusation of being anti-science .You may be thinking that in light of the strong claims for Evidence - based therapies and the denigration of psychody-namic or insight-oriented therapies there must be amazinglystrong scientific Evidence for their benefits.

4 There is not. Thereis a yawning chasm between what we are told research showsand what research actually empirical research really shows is that Evidence - based therapies are ineffectivefor most people most of the Part I, I discuss what empirical research really shows. In Part2, I take a closer look at some troubling practices in Evidence - based therapy I: what research really showsResearch shows that Evidence - based therapies are weaktreat-ments. Their benefits are trivial. Most patients do not get the trivial benefits do not Shedler48 THE JOURNAL OF PSYCHOLOGICAL THERAPIES IN PRIMARY 10/07/2015 14:26 Page 48 This may be very different from what you have heard else- Where . You may be thinking, who is this guy? And why shouldI believe him? I will revisit this question at the conclusion. I amnot asking you to believe me. That is why I will be referencingprimary gold standard of Evidence in Evidence - based therapyresearch is the randomised controlled trial.

5 Patients with acertain diagnosis are randomly assigned to either a treatment orcontrol group, and the study compares the two mother of all randomised controlled trials for psycho -therapy is the National Institute of Mental Health (NIMH)Treatment of Depression Collaborative Research Program. Itwas the first really large multi-site research project investigatingwhat are now called Evidence - based therapies. The studyincluded three active treatments: manualised CBT, manualisedinterpersonal therapy (IPT), and antidepressant medication. Thecontrol group got a placebo pill and clinical management, butnot psychotherapy. The study was initiated in the mid-1970sand the first major publications started coming out around the past twenty-five years, we have been told that thisstudy showed that CBT, IPT, and antidepressant medication are empirically validated treatments for depression. We havebeen told that these treatments were scientifically proven to bepowerful and effective.

6 I will focus on CBT because that is whatgets the most attention and, of course, it is the theme of claims for the benefits of CBT were based on the findingthat CBT was statistically significantly more effective than theplacebo control group. Statistically significant does not meanwhat most people think. Set aside preconceptions about theword significantand consider, instead, the actual difference inthe NIMH study between the CBT group and the control groupthat got the sugar primary outcome measure in the NIMH study was the 54-point Hamilton Depression Rating Scale. The difference betweenthe CBT treatment group and the control group was point difference is trivial and clinically does not pass the So what? test. It does not pass the Doesit matter? test. It does not pass the Why should anyone care? JOURNAL OF PSYCHOLOGICAL THERAPIES IN PRIMARY CARE49 Where is the Evidence for Evidence - based Therapy? 10/07/2015 14:26 Page 49 How can there be such a mismatch between what we havebeen told versus what the study actually found?

7 You may bewondering whether the original researchers did not present thedata clearly. That is not the case. The first major research reportfrom the NIMH study was published in 1989 in Archives ofGeneral Psychiatry(Elkin et al., 1989). The authors wrote: Therewas limited Evidence of the specific effectiveness of interper-sonal psychotherapy and none for cognitive behaviour therapy (emphasis added). That is what the original research report 1994, the principle investigator wrote a comprehensivereview of what we had really learned from that study, titled The NIMH Treatment of Depression Collaborative ResearchProgram. Where we began and Where we are (Elkin, 1994).Writing in careful academic language, the principle investiga-tor stated: What is most striking in the follow-up findings is therelatively small percentage of patients who remain in treatment,fully recover, and remain completely well throughout the 18-month follow-up period. The percentage was so small that it raises questions about whether the potency of the short-termtreatments for depression has been oversold (Elkin, 1994, p.)

8 131).What was that percentage, actually? It turns out that only24% of the patients in the study got well and stayed way of saying that is that about 75% the overwhelm-ing majority did notget well. How can this be? We have beentold the opposite for the last twenty-five years. We have beentold that manualised CBT is powerful and can now revisit the term significant. In the Englishlanguage, the word significantis a synonym for important ormeaningful. But that is not what the term means in statistics. Instatistics, it is a technical term of art. It means that a researchfinding probably did not occur by chance. In the NIMH study,there was a point difference between the CBT group and thecontrol group. That is clinically meaningless nobody woulddispute that. But the difference was statistically significant ,meaning it probably did not occur by are few other fields Where people talk about signifi-cance instead of talking about actual benefits.

9 When aresearcher emphasises statistical significance , something isbeing hidden. If there is a meaningful treatment benefit, onetalks about that, not significance . If we have a drug that is50 THE JOURNAL OF PSYCHOLOGICAL THERAPIES IN PRIMARY CAREJ onathan 10/07/2015 14:26 Page 50effective in lowering blood pressure, we say the drug decreasedblood pressure by so much. If we had an effective weight lossprogramme, we would say the average person in the prog -ramme lost twenty-five pounds, or thirty pounds, or we had a drug that lowered cholesterol, we would talk abouthow much it lowered would not talk about significant differences . Whenresearchers emphasise statistical significance , something isbeing NIMH findings were published more than twenty-fiveyears ago. Surely, research findings for CBT must haveimproved over time. So let us jump ahead to the most recentstate-of-the-art randomised controlled trial for depression(Driessen et al.)

10 , 2013). This recent study included 341 depressedpatients who were randomly assigned to sixteen sessions ofmanualised CBT or sixteen sessions of manualised psychody-namic therapy. This study was published in 2013 in the AmericanJournal of authors wrote, One notable finding was that only the patients achieved remission (Dreissen et al., 2013, ). They continued, Our findings indicate that a substantialproportion of patients .. require more than time-limited ther-apy to achieve remission. In other words, about 75% of patientsdid not get well. It is essentially the same finding reported in theNIMH study a quarter of a century appropriate conclusion to be drawn from both of thesemajor studies is that brief, manualised treatments are ineffectivefor most depressed patients most of the I have reviewed the earliest major study and the mostrecent. What about all the research in between? The results arelargely the same. The research is summarised in a review paperby lead author Drew Westen (Westen et al.