Overview of Evidence Based Practice Charles Wilson, MSSW, Executive Director of Chadwick Center The Sam and Rose Stein Chair on Child Protection Rady Children’s Hospital-San Diego www. cachildwelfareclearinghouse. org
How Things Change A Problem is Recognized Action-Any Action
Action- Creation of Orphan Trains • Between 1854 and 1929 100, 000 -200, 000 children were placed in new families via the Orphan Trains. http: //www. orphantraindepot. com • Children were taken in small groups of 10 to 40, under the supervision of at least one adult, and traveled on trains to selected stops along the way, where they were taken by families in that area. http: //www. pbs. org/wgbh/amex/orphan/teachers. html
How Things Change A Problem is Recognized Action-Any Action Informed Action Series of Trail and Errors Adjustments. Some Better. Some Worse
Trial Family Foster Care and Error Orphanages and Boarding schools Tennessee Preparatory School for Dependent Children
How Things Change A Problem is Recognized Informed Action-Based on Science
So how do we know what works vs. mere marketing hyperbole? Let the Buyer Beware
Thought Field Therapy “Thought field therapy with Callahan techniques® is a powerful therapy exerted through nature's healing system to balance the body's energy system. This therapy promotes stress management and stress relief as well as the reduction or elimination of anxiety and anxiety related problems. This includes help for weight control and weight loss, trauma Roger J. Callahan, Ph. D or sleep difficulties, depression, addictions and the disorders associated with past trauma including nightmares and post traumatic stress disorder. ” (underlines added) Retrieved from http: //www. tftrx. com/, November 17, 2006
More Claims for TFT Q. How Can TFT Benefit You? – What Kind of Problems Can Be Helped? • • • Anxiety and Stress Personal fears or your children’s fears Anger and Frustration Eating or smoking or drinking problems Loss of loved ones Social or public speaking fears Sexual or intimacy problems Travel anxiety including fear of flying or driving on the freeways Nail biting Cravings Low moods and mood swings Retrieved from http: //www. tftrx. com/profaq. php? PHPSESSID= f 4 cf 66 c 40 b 9678 b 742 b 82989 fee 7 b 377# on November 17, 2006
NPR All Things Considered, March 29, 2006 “According to psychologist Roger Callahan, the creator of thought field therapy, major problems like depression can be cured quickly with this method. He says post-traumatic stress disorder is easily dispatched in 15 minutes, and even the most serious cases of anxiety, addiction and phobias are likewise subject to quarter-hour cures. ”
Research on TFT? “Has any research been carried out on TFT? There have been no control (sic) studies on the success of TFT” From the Thought Field Therapy Training Center of La Jolla Retrieved from http: //thoughtfield. com/faqs. htm on November 17, 2006
Distinguishing groundless marketing claims from reality The Problem: All sorts of “interventions” are available out there.
Waiting Room Sign Ben Saunders MUSC
Evidence Based Social Work “Professional judgments and behaviors should be guided by two interdependent principals: 1. When ever possible, practice should be grounded on prior findings that demonstrate empirically…that they are likely to produce predictable, beneficial, and effective results. 2. Every clients system, over time should be evaluated” Evidence Based Practice Manual Oxford University Press 2004 Albert Roberts, Ph. D Kenneth Yeager, Ph. D, LISW
Global Definition of EBP The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Including Both Individual clinical expertise The best available clinical evidence from systematic research -David Sackett
Huge Policy Implications • Should policy makers support adoption of EBP? – If so, which ones –When are they “Ready for Prime time” • What is the standard of evidence? – If so, how best can they support adoption? • What are the pitfalls of a state or national policy level adoption of EBP? – Impact on Innovation – Misapplication of good models? -One size does not fit all – Watering down of empirically based practice-danger of implementing in name only – Ideology vs. Science- who is the judge of the science? • Should we limit what we do to EBP?
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials (Gordon C Smith, Jill P Pell, 2005) The perception that parachutes are a successful intervention is based. largely on anecdotal evidence • Observational data have shown that their use is associated with, morbidity and mortality due to both failure of the intervention and mechanical complications. In addition, “natural history" studies of free fall indicate that failure to take or deploy a parachute does not inevitably result in an adverse outcome. . . The effectiveness of an intervention has to be judged relative to nonintervention. • Understanding the natural history of free fall is therefore imperative. • If failure to use a parachute were associated with 100% mortality then any survival associated with its use might be considered evidence of effectiveness. Therefore, studies are required to calculate the balance of risks and benefits of parachute use.
Why Evidence-Based Practice Now? • A growing body of scientific knowledge • Increased interest in consistent application of quality services • Increased interest in outcomes and accountability by funders • Past missteps in spreading untested “best practices” that turned out not to be as effective as advertised • Because they work !!
Problems in the Child Abuse Field in the U. S. • Empirical evidence of efficacy has not been a common criteria for treatment selection in the child maltreatment field. • Lack of outcome research for many commonly used interventions. • Ready willingness among some to use, embrace, promote, and staunchly defend practices that have no evidence for their efficacy and questionable theoretical bases. • Poor dissemination of the significant clinical outcome research that has been done. • Ineffective approaches to continuing education. • Poor adoption of empirically supported treatments in real world clinical settings. • Disconnection between current scientific knowledge and practice in the field.
Scared Straight
TF-CBT
Reactive Attachment Disorder and Attachment Therapy …pioneered by psychoanalyst Aaron Lederer, the RAD Consultancy’s creator and director. His methods yield remarkable results within weeks. Retrieved from http: //www. radconsultancy. com/, November 17, 2006
Why should we worry about using Evidence Supported Treatments?
Institute of Medicine: Apply the Principles and Methods of Evidence Based Practice Integration of: • Best Research Evidence • Best Clinical Experience • Consistent with Client Values • http: //www. shef. ac. uk/scharr/ir/netting/ • http: //ebmh. bmj. com/ • http: //cebmh. com/ • http: //www. cebm. utoronto. ca/
Understand Adoption of Innovation MTFC 1991 Innovators Early Late Majority Early Adopters Traditionalists
Common Errors When Deciding about Intervention Effectiveness • Reliance solely on individual anecdotes and remembered cases. – “That child made such amazing changes during treatment. ” • Confusing client satisfaction with clinical improvement. – “The family just loved coming to therapy. Never missed a session during their 3 years of therapy. Amazing. Too bad they had to move away. ” • Misattribution of the cause of change. • Failure to appreciate resilience and natural recovery. – “The family got multiple services and wrap around care. ” – “With treatment her PTSD resolved in about 3 months after the rape. ” • Guru effect in training and treatment adoption. – “I heard Dr. Mc. Dreamy is doing a level II training. And, it’s in San Diego in January!” – “Those videos were just so amazing! I have got to try that. ” Ben Saunders MUSC
What to look for in a Practice? • Treatment or intervention protocol that has at least some scientific, empirical research evidence for its efficacy with its intended target problems and populations. • Evidence may be based on a variety of research designs. – Randomized Clinical Trial (RCT) – Controlled studies without randomization – Open trials, pre- post-, or uncontrolled studies – Multiple baseline, single case designs • The degree to which we are persuaded that the treatment is effective will vary by the quality of empirical support. – Number of RCT’s – Replication by researchers other than the treatment developers – Sampling, sample size used, comparison treatment, effect size • Various methods have been developed for classifying the level of empirical support enjoyed by treatment approaches. – Should be useful for front-line practitioners
CEBC Website: www. cachildwelfareclearinghouse. org
Current Data on Visitors to the Website Total Number of Visits to the Website 46, 635 Percentage of Total Visitors from over 131 International Countries Percentage of Total Visitors from U. S. 14% 86% Percentage of Total Visitors from California 33% Data based on numbers as of September 1, 2007
CEBC’s Definition of Evidence-Based Practice for Child Welfare § Best Research Evidence § Best Clinical Experience § Consistent with Family/ Client Values (m odified from The Institute of Medicine) http: //www. iom. edu/
The California Evidence-Based Clearinghouse for Child Welfare (CEBC) In 2004, the California Department of Social Services, Office of Child Abuse Prevention contracted with the Chadwick Center for Children and Families, Rady Children’s Hospital-San Diego in cooperation with the Child and Adolescent Services Research Center to create the CEBC. The CEBC was launched on 6/15/06.
Advisory Committee The Advisory Committee is composed of 15 members drawn from a broad cross-representation of communities and organizations. There are representatives from: California Department of Social Services Child Welfare Departments from California Counties Child Welfare Director’s Association (CWDA) California Child Welfare Training Leaders Public and Private Community Partners Within the State The role of the Advisory Committee is to: Determine the topical areas for the CEBC Ensure the CEBC remains up-to-date with emerging evidence. Assist in disseminating the products of the CEBC. Provide feedback on the utility of the CEBC products.
National Scientific Panel The National Scientific Panel is composed of five core members and up to 10 selected Topical Experts. The Panel is nationally recognized as leaders in child welfare research and practice, and who are knowledgeable about what constitutes best practice/evidence-based practice. The Panel assists in identifying relevant practices and research and provide guidance on the scientific integrity of the CEBC products.
Scientific Rating Scale and Relevance to Child Welfare Scale
Rating Scale Development • Goals: – Multiple categories – High standard for top ratings – Randomized Controlled Trials – Clearly defined criteria – Focus on peer-reviewed research and ability to replicate program
Gold Standard for Evidence • Randomized controlled trial (RCT) – Participants are randomly assigned to either an intervention or control group. This allows the effect of the intervention to be studied in groups of people who are the same, except for the intervention being studied. – Any differences seen in the groups at the end can be attributed to the difference in treatment alone, and not to bias or chance.
Peer-Reviewed Research • Peer review – A process used to check the quality and importance of research studies. It aims to provide a wider check on the quality and interpretation of a study by having other experts in the field review the research and conclusions.
Efficacy vs. Effectiveness • Efficacy focuses on whether an intervention works under ideal circumstances and looks at whether the intervention has any impact at all. • Effectiveness focuses on whether a treatment works when used in the real world. – An effectiveness trial is done after the intervention has been shown to have a positive effect in an efficacy trial.
Scientific Rating Scale
6. Concerning Practice • If multiple outcome studies have been conducted, the overall weight of evidence suggests the intervention has a negative effect upon clients served. and/or • There is a reasonable theoretical, clinical, empirical, or legal basis suggesting that, compared to its likely benefits, the practice constitutes a risk of harm to those receiving it.
5. Evidence Fails to Demonstrate Effect • Two or more randomized, controlled outcome studies (RCT's) have found that the practice has not resulted in improved outcomes, when compared to usual care. • If multiple outcome studies have been conducted, the overall weight of evidence does not support the efficacy of the practice.
4. Acceptable/Emerging Practice. Effectiveness is Unknown • There is no clinical or empirical evidence or theoretical basis indicating that the practice constitutes a substantial risk of harm to those receiving it, compared to its likely benefits. • The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it. • The practice is generally accepted in clinical practice as appropriate for use with children receiving services from child welfare or related systems and their parents/caregivers. • The practice lacks adequate research to empirically determine efficacy.
3. Promising Practice Same basic requirements as Level 4 plus: • . At least one study utilizing some form of control (e. g. , untreated group, placebo group, matched wait list) has established the practice’s efficacy over the placebo, or found it to be comparable to or better than an appropriate comparison practice. The study has been reported in published, peerreviewed literature. • Outcome measures must be reliable and valid, and administered consistently and accurately across all subjects. • If multiple outcome studies have been conducted, the overall weight of evidence supports the efficacy of the practice.
2. Well Supported-Efficacious Practice Same basic requirements as Level 3 plus: • Randomized controlled trials (RCTs): At least 2 rigorous RCTs in highly controlled settings (e. g. University laboratory) have found the practice to be superior to an appropriate comparison practice. -The RCTs have been reported in published, peer-reviewed literature. • The practice has been shown to have a sustained effect at least one year beyond the end of treatment, with no evidence that the effect is lost after this time.
1. Well supported - Effective Practice Same basic requirements as a Level 2 plus: • Multiple Site Replication: At least 2 rigorous randomized controlled trials (RCTs) in different usual care or practice settings have found the practice to be superior to an appropriate comparison practice. - The RCTs have been reported in published, peerreviewed literature. – The practice has been shown to have a sustained effect at least one year beyond the end of treatment, with no evidence that the effect is lost after this time.
Child Welfare Ratings • Not every program that is evidence-based will work in a Child Welfare setting… • We also examined each program’s experience and fit with Child Welfare systems and families
Relevance to Child Welfare Scale 1. High: The program was designed or is commonly used to meet the needs of children, youth, young adults, and/or families receiving child welfare services. 2. Medium: The program was designed or is commonly used to serve children, youth, young adults, and/or families who are similar to child welfare populations (i. e. in history, demographics, or presenting problems) and likely included current and former child welfare services recipients. 3. Low: The program was designed to serve children, youth, young adults, and/or families with little apparent similarity to the child welfare services population.
Child Welfare Outcomes • We also examined whether programs had included outcomes from the Child and Family Services Reviews in their peerreviewed evaluations: Safety Permanency Well-being
Common Continuing Education Dissemination Model One day workshop Book Therapist Use Tx with appropriate clients
X Laying the Groundwork for Implementing Evidence Based Practice
Levels of Implementation Fixen et al • Paper Implementation • Process Implementation • Performance Implementation Fixsen, D. , Naoosm, S. , Blasé, K. , Friedman, R. , Wallace, F. (2005)
Institute for Healthcare Improvement Model Environmental Context Organizational Context Microsystem Community, Government, Funders Organizations Departments and Programs Within Organizations Patient and Community Social Workers, Therapists, Medical Professionals and Families
Transtheoretical Model of Change 5 Stages of Change • Precontemplation • Compliant Status Quo • Contemplation • Changes in orientation • Preparation • Planning for change • Organizational and environmental readiness • Action • Training • Maintenance • Monitoring/Institutionalization Driven at each stage by: Self Efficacy & Decisional Balance
Components of Implementation • Select a Solution that Fits a Problem • Prepare the internal and external environment Supervision and Leadership Buy-in • Acquire knowledge and skills • Use practice with support, supervision and consultation • Adapt practice to environment • Monitor fidelity • Teach others • Institutionalize Practice
Practice Selection Attributes that can facilitate adoption • Relative Advantage- clear, unambiguous advantage in either effectiveness or cost effectiveness • Costs- training/materials/on-going consultation-loss productivity during start up- costs of delivery • Compatibility-How compatible is the practice with the organizational and workforce’s values, norms, and clinical traditions and orientation • Complexity –perceived as more simple to use and to implement • Trialability- able to experiment with in a limited basis • Observability of Benefits –outcomes or interim results/measures • Reinvention- if can adapt, refine or otherwise modify it to meet own needs • Risk- if there is higher certainty of outcomes • Task Issues- If relevant to performance of intended users work and improved task performance • Knowledge- if knowledge can be codified and transferred from one context to another • Augmentation/Support- if provided with training/consultation From Greenhalgh et al
Organizational Readiness • • Organizational Culture/Traditions/History Leadership Supervision Capacity to evaluate change-Know if it is working Support of Opinion Leaders Connections with other supportive organizations/individuals Does organization have the technology to support the change Staff readiness
Staff Readiness Staff Directly and Indirectly involved • Understand What Benefits Will the Adoption of the EBP Bring • Meaning-What does the change mean to the staff? • What concerns will staff have about adoption • How congruent are the trainers in orientation and values with the staff • Presence of Champions
Readiness of External Environment • Congruence with Community/Cultural/Family Values • Referral Source Understanding and Support • Funding Source Support • Political Support • Role of Social Influence/Demand for Services – Role Social Movement Theory
Supportive Implementation Model Administrative Leadership and Support for EBT Obtain client feedback Technical Assistance Supervision Expert Consultation Therapist Training Use EST with appropriate clients Materials Community/Consumer Support for EBT
Finding Evidence Supported Treatments on the Web • www. nctsn. org • www. cachildwelfareclearinghouse. org/ • http: //modelprograms. samhsa. gov/template. cfm? CFID=11 9292&CFTOKEN=55491051 • www. strengtheningfamilies. org/ • www. ncptsd. va. gov/topics/treatment. html • www. childtrends. org • www. wsipp. wa. gov • http: //ebmh. bmjjournals. com/ • www. cochrane. org • www. campbellcollaboration. org • www. colorado. edu/cspv/blueprints/model/overview. html
Contact Information Download reports from: www. chadwickcenter. org E-mail: cwilson@rchsd. org www. cachildwelfareclearinghouse. org
Скачать презентацию Overview of Evidence Based Practice Charles Wilson MSSW
ee991994222661d00bb9f9d260074ee0.ppt