Evidence-Based Practice in ASD Gina Green, Ph. D, BCBA-D Association of Professional Behavior Analysts info@apbahome. net Gina Green 2011 DO NOT REPRODUCE OR DISTRIBUTE Ó 1
“Those who fall in love with practice without science are like a sailor who enters a ship without a helm or a compass, and who never can be certain whither he is going. ” -- Leonardo da Vinci 2
Topics § § § Evidence-based practice Tools for evaluating evidence about interventions EBP guidelines Current evidence about interventions for ASD Implementing evidencebased practice guidelines 3
Evidence-based practice (EBP) l l Using best available scientific evidence, combined with clinical expertise and consumer preferences, to guide decisions about practice Gaining support in several disciplines -- e. g. , medicine, behavior analysis, psychology, speech-language pathology, education 4
EBP l Many practices are touted as “evidence-based. ” But are they really? l “Evidence” comes in many flavors. l Some forms of evidence are better (more reliable and credible) than others. l Good EBP protocols look to scientifically sound studies for evidence of the effects of practices. 5
Why base ASD intervention practices on scientific evidence? l There are scores of interventions for ASD. Many are very popular, and are promoted enthusiastically. All are claimed to be effective by someone, many remarkably so. l l l Popularity should not be mistaken for proof. Enthusiasm is no substitute for evidence. It just does not stand to reason that all interventions are comparably effective. 6
Why scientific evidence? l Many interventions are supported mainly by opinions, speculations, personal accounts, and uncontrolled or poorly controlled studies. Basing practices on those forms of “evidence” has many risks: l l l Wasted money, time, energy Exploitation of vulnerable people Physical and emotional injuries Lost opportunities to make real advances Reinforcement and perpetuation of practices that impede progress The most tried-and-true way to reduce those risks is to rely on scientific methods to separate opinions and speculations from demonstrated facts. 7
Why scientific evidence (cont’d) l l Science provides the best tools for determining which practices work, how well, with what side effects, for which consumers. Practice based on scientific evidence l l l Allows consumers to make fully informed choices Protects consumers, practitioners, and funding agencies Prevents needless waste of resources Scientific research has enhanced the health, skills, and quality of life for people with many conditions. No less than people with other conditions, people with ASD deserve interventions that have proved effective in rigorous studies. With large numbers of people with ASD diagnoses seeking services, it is essential to invest time, effort, and money in interventions that are most likely to produce meaningful improvements. 8
Evaluating treatments: Sources of strong scientific evidence l Systematic comparisons of a treatment with no treatment or an alternative treatment -- i. e. , controlled studies l Between-groups experimental designs -- one group receives treatment, similar (control) group does not; all participants measured at beginning of study and again after some period of time; group mean (average) scores after treatment are compared statistically. l l l If treatment group mean differs from control group mean after treatment, must be due to treatment Single-case experimental designs -- each individual experiences control (baseline) and treatment conditions with replications; behavior or biology measured repeatedly over time; data are examined for clinically important differences between treatment and control conditions Necessary to rule out other explanations for apparent Tx effects (e. g. , practice, maturation, other interventions) 9
Evaluating treatments: Sources of strong scientific evidence l l Careful selection and assessment of participants Objective, accurate, and reliable measurement of the intervention as well as its effects Replications by multiple investigators Peer review and publication in scientific journals 10
Not-so-scientific evidence l l Indirect, subjective evaluations of Tx effects l Surveys, questionnaires, interviews, personal accounts Uncontrolled, descriptive studies -- no comparison of control and treatment conditions l Assessment reports l Epidemiological studies l Longitudinal studies l Qualitative studies l Single-group studies 11
Not-so-scientific evidence l Theories and speculations based on descriptive studies or personal observations. Examples: l “The [brains/blood/urine/hair] of some people with autism contain certain substances or structures. I believe those things cause autism. Therefore, I recommend treatment Y. ” l “The [communication/cognitive/social] skills of some people with autism have X characteristics. I believe those skills develop in a certain way. Therefore, I recommend treatment Y. ” 12
Not-so-scientific evidence l l l Retrospective “chart reviews” Conference presentations (few are peer reviewed) Workshop handouts (unless supporting scientific studies are cited) Attractively packaged treatment manuals, videos, materials, but little or no scientific research on outcomes Reports in nonscientific journals, self-published books and journals, and on many websites Self-reports, anecdotes, and testimonials 13
The quality of evidence Most credible: l Directly tested l l Publicly verified Measured l Direct Objective l Accurate l Reliable (consistent) Experimental (controlled) studies Peer reviewed l l l l Least credible: Untested speculations, interpretations Unobservable, unverified Subjective or unquantified l Indirect (surveys, interviews, third-party reports) l Personal observations, opinions l Tests of unknown accuracy, reliability Descriptive, uncontrolled studies; case reports No or inadequate review 14
How good EBP guidelines are developed l l l Team of expert reviewers and methodologists assembled Peer-reviewed empirical research literature searched l Criteria developed for including and excluding articles Uniform standards are developed for evaluating critical components of every study. Highest ratings: l l l Research design -- Controlled studies with adequate comparisons of experimental (treatment) and control/comparison conditions, between groups or within subjects Measurement of dependent variable(s) -- direct, objective, accurate, reliable Measurement of independent variable(s) (treatment integrity) -- direct, objective, accurate, reliable Participant ascertainment -- Diagnosis and evaluation by independent evaluators using valid and reliable methods Effect(s) -- Clinically and/or statistically significant differences between comparison/control and treatment conditions; evidence of generality 15
EBP guidelines development l l Reviewers trained to use evaluation criteria. Each study evaluated by reviewers for quality of evidence produced; limitations, harms, costs also considered l l Preferably, each study is evaluated by more than one reviewer, and interrater agreement is checked periodically. Evaluations of all studies on a particular intervention are aggregated. 16
EBP guidelines development l l Interventions with strong supporting evidence are recommended. Those proved ineffective, harmful, or interventions lacking adequate scientific evaluation are not recommended. Promising interventions warranting further research may be identified. General recommendations may be translated into specific suggestions for practitioners, consumers, and funding agencies. 17
EBP guidelines development l Guidelines are Reviewed by other professionals, consumers l Updated and revised periodically Process includes input from many sources, standardized methods, multiple checks and balances so as to minimize influence of personal biases, maximize objectivity l l 18
EBP guidelines for autism: An example l NY Dept. of Health Early Intervention Program Clinical Practice Guidelines for Young Children with Autism (1999) l Multidisciplinary panel and independent methodologist Used protocol developed by Agency for Health Care Policy and Research, U. S. Public Health Service to evaluate peer-reviewed scientific research (group and single-case design studies), make recommendations based on the strength of the evidence Extensive peer review Recommended ABA Limited to ages 0 -6 yrs l Being updated this year l l 19
EBP guidelines: A promising effort l l National Autism Center’s National Standards Project Multidisciplinary panel reviewed hundreds of scientific studies on all behavioral and educational interventions and special diets (but no biomedical interventions) for ages 0 -21 l l Focused interventions “Package” interventions Comprehensive intervention programs Protocol integrated features of several established protocols for evaluating evidence about 20 treatments
National Standards Project l l Searched peer-reviewed published research using specific inclusion/exclusion criteria Reviewers evaluated scientific merit of each study by rating each component on 0 -5 scale: l l l Design -- group and single-case Measurement of dependent variable(s) - tests, scales, checklists, etc. and direct behavioral observation Measurement of independent variable (treatment integrity) Participant ascertainment Generalization of effects Also rated treatment effects as beneficial, ineffective, adverse, or unknown 21
National Standards Project l All reviewer ratings aggregated to determine strength of evidence regarding every intervention l Identified interventions that are l l l Established -- sufficient evidence from several strong scientific studies to conclude intervention produces favorable outcomes Emerging -- one or more studies suggest favorable outcomes, but not enough evidence to draw any conclusions Unestablished -- little or no scientific evidence; claims based on very poorly controlled studies, testimonials, unverified clinical observations, opinions, speculations Ineffective/harmful -- sufficient scientific evidence to conclude intervention is ineffective or harmful Most interventions lumped together in large categories 22
NSP: Main findings l Large majority of established interventions come from applied behavior analysis, behavioral psychology: l l l Antecedent package Behavioral package Comprehensive behavioral treatment for young children Joint attention intervention Modeling Naturalistic teaching strategies Peer training package Pivotal response treatment Schedules Self-management Story-based intervention package 23
NSP: Main findings l Most emerging interventions also come from ABA and behavioral psychology, though some are developmental, cognitive. More research needed to draw conclusions: l l l l l Augmentative and alternative communication device Cognitive behavioral intervention Developmental relationship-based treatment Exercise Exposure package Imitation-based interaction Initiation training Language training (production) Language training (production & understanding) Massage/touch therapy 24
NSP: Main findings l More emerging interventions: l l l Multi-component package Music therapy Peer-mediated instruction Picture Exchange Communication System Reductive package Scripting Sign instruction Social communication intervention Social skills package Structured teaching Technology-based treatment Theory of mind training 25
NSP: Main findings l l Unestablished treatments: l Academic interventions l Auditory integration training l Facilitated Communication l Sensory integration therapy l Gluten free/casein free diet Authors noted that l Other reviewers have found some of these interventions harmful. l There are many other interventions for ASD that have not been evaluated in scientific studies. 26
National Standards Project l l Established interventions described in terms of skills increased, problem behaviors decreased, age groups and diagnoses for which they have proved effective Recommendations for treatment teams: l l Give serious consideration to established treatments. Don’t start with emerging treatments, but consider if team determines that established treatments are inappropriate. Consider unestablished treatments only after additional research shows them to be beneficial. Reports can be purchased at http: //www. nationalautismcenter. org/learning/bookstore. p hp 27
Current evidence about interventions for autism l Summaries that follow are drawn from reviews of research by l l l l l Matson et al. (1996) Smith (1996, 1999) NY State Department of Health (1999) Ellis, Singh, & Ruane (1999) Dawson & Watling (2000) Romanczyk, Arnstein, Soorya, & Gillis (2003) Carr & Firth (2005) Levy & Hyman (2005) Jacobson, Foxx, & Mulick (2005) Malone, Gratz, Delaney, & Hyman (2006) Myers, Johnson, & Council on Children with Disabilities (2007) Rogers & Vismara (2008) Dawson (2008) Reichow & Wolery (2009) Eikeseth (2009) National Standards Project (2009) Green (2010) Association for Science in Autism Treatment (http: //www. asatonline. org/intervention/autismtreatments. htm as of January 2, 2011) 28
l Lots of anecdotes and speculations, little or no scientific data: Vitamin A, cod liver oil, chelation, detoxification, gold salts, marijuana, Options/Son. Rise, Whole Life Therapy (Higashi), touch therapy, deep pressure therapy, music therapy, art therapy, animal therapy, holding therapy, water therapy, visual integration therapy, metronome therapy, oral-motor therapy, attachment (bonding) therapy, Irlen lenses, Rhythmic Entrainment Intervention, craniosacral manipulation, “Floor Time” (DIR), Giant Steps, Gentle Teaching, Fast For. Word, “rapid prompting, ” Relationship Development Intervention, SCERTS, social skills groups, many OT techniques, antifungal medications, hyperbaric oxygen, hormones, many psychotropic medications, neurosurgery, fatty acids, etc. 29
l l l Found ineffective in scientific studies: Vitamin B 6, DMG, patterning (Doman-Delacato), psychoanalytic play therapy, sensory integration and other sensorimotor therapies, secretin, special diets, “eclectic” (mixedmethod) comprehensive treatment, typical early intervention/special education Found harmful in scientific studies: Facilitated Communication, auditory integration training, intravenous immune globulin, withholding vaccinations Limited scientific evidence: TEACCH, developmental approaches, social stories, PECS, PROMPT, fluency training, amino acids, some psychoactive drugs (but most have negative side effects) 30
l Substantial scientific evidence of effectiveness: l l Some drugs (risperidone, Ritalin), but they have adverse side effects, and have not proved more effective than behavioral interventions for problem behaviors Many (but not all) applied behavior analysis (ABA) techniques and “packages” 31
l Substantial scientific evidence of effectiveness: l Some models of early, comprehensive, intensive ABA (25 -40 hrs/wk, year around, 1 - 4 years; designed and overseen by qualified behavior analysts) l l 7 controlled between-groups studies of Lovaas/UCLA model 4 controlled between-groups studies of other ABA models Main findings: ~45% of children had large improvements in cognitive, language, and adaptive skills, school placement, autism characteristics; ~45% had moderate improvements, ~10% small improvements No scientific studies of other comprehensive ABA models (e. g. , Applied Verbal Behavior, CABAS, Pyramid, Competent Learner Model, all”naturalistic” models, LEAP) 32
Incorporating effective interventions into everyday practice l Use reviews of scientific evidence and evidencebased practice guidelines as starting point for selecting interventions for people with ASD l l Check ASAT website periodically for updates: http: //www. asatonline. org/resources/autismtreatments. htm Consider individual and family characteristics, strengths, needs, goals, and preferences Also consider practitioner competencies, other available resources Evaluate effects of every intervention on individual and significant others carefully and objectively 33
Summary l Science isn’t perfect or failsafe, but using scientific evidence to guide intervention decisions l Empowers consumers and practitioners to make truly informed choices l Increases the likelihood that the person with ASD will benefit l Assures that consumers and practitioners are alerted about possible side effects l Makes accountability possible l Focuses precious resources where they are most likely to produce measurable benefits and long-term cost savings 34
Some sources of scientifically sound info about interventions for ASD l Association for Science in Autism Treatment -- www. asatonline. org/intervention/autismtreatments. ht m NY Dept. of Health Early Intervention Program -www. health. state. ny. us/nysdoh/eip/index. htm l Autism Watch -- www. autism-watch. org (also see www. Quackwatch. org) l Autism Speaks ABA section -www. autismspeaks. org/whattodo/what_is_aba. php l Cambridge Center for Behavioral Studies -www. behavior. org/autism l 35
For qualifications to practice applied behavior analysis, see l l l Behavior Analyst Certification Board -- www. BACB. com Association of Professional Behavior Analysts -www. APBAhome. net Association for Behavior Analysis Autism Special Interest Group Consumer Guidelines -- http: //www. apbahome. net/guidelines_consumer. php l American Psychological Association Specialty in Behavioral Psychology -http: //www. apa. org/crsppp/archivbehav. html 36
