Ecodevelopmental and Systemic Modeling and Implementing High Fidelity

Ecodevelopmental and Systemic Modeling and Implementing High Fidelity Interventions in Real World Settings Guillermo Prado 1, 2, Hilda Pantin 1, Seth Scwartz 1, Jose Szapocznik 1 & Daniel J. Feaster 1, 2 1 Center for Family Studies, University of Miami 2 Stempel School of Public Health, Florida International University 1

The Ecodevelopmental Model and Methodological Questions 2

The Center for Family Studies is interested in the role of ecodevelopmental context in the prevention and treatment of adolescent behavior problems, drug abuse, and HIV/AIDS. 3

Ecodevelopmental Theory Incorporates three primary, integrated components: (a) Social Ecological Theory (b)(b) Developmental Theory (c) Emphasis on Social Interactions 4

Social Ecological Theory Bronfenbrenner (1979, 1986) Highlights the Multiple Influences on Adolescent Development, including: (a) Macrosystems (e. g. , cultural & societal values) (b) Exosystems (e. g. , parents’ exosystemic stressors and social support for parents) (c) Mesosystems (e. g. , parental monitoring of peers and collaboration with youth’s school) (d) Microsystems (e. g. family functioning) 5

Developmental Component Emphasizes the changing nature of youth, contexts, & their interdependence over time e. g. , family functioning is influenced not only by parents’ current social support & work stress, but also by previous levels of social support & work stress e. g. , current family functioning in turn influences both present and future levels of adolescent behavior problems 6

Social-Interactional Component Risk and protection are expressed in the patterns of direct transactions between individuals within and across the different contextual levels e. g. , when parents engage in supportive interactions with individuals outside the family, they are more likely to parent their children in supportive rather than harsh ways. 7

An Ecodevelopmental Perspective on Prevention 8

Context of Adolescent Behavior Problems, Drug Use, and Risky Sexual Behavior Social-Cultural Context Parental Resources/Stressors Parents’ Social Support Parents’ Work Stress Cultural Mismatch Family Microsystem Parent-adolescent communication about sex Positive Parenting Parent-Adolescent Communication Marital Conflict Parental Involvement Family support Family-School Relations Parental involvement in school Monitoring homework School Bonding Academic Achievement Immigration Policy Language Problems Family-Peer Relations Parental monitoring of peers Supervision of situations of sexual possibility Peers Substance use w/ friends Sexually active friends Poverty 9

Ecodevelopmental Model of Problem Behaviors Social Support for Parents Adolescent Acculturation Family Exosystemic Stressors Parent Acculturation Parental Monitoring of Peers Parental-Adolescent Communication about Sex Peer Sexual Behavior Family Relations Early Adolescent Sexual Initiation Social Cognitive Mediators re. Sex Early Adolescent Substance Use Peer Substance Use Early Adolescent Problem Behaviors Social Cognitive Mediators re. Drug Use 10

Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves multiple levels of nesting u Repeated Observations Individuals Families Ecosystemic levels across developmental stages u Statistically, data are non-independent— Substantively people and their social contexts are interdependent 11

Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves longitudinal processes u Ecodevelopmental processes are interrelated and influence each other over time F How do we model these interrelationships over time? u Ecodevelopmental processes develop over time, and a snapshot of such a process (e. g. , family functioning at baseline) is not accurate F How do we account for this in our model? 12

Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves influences from multiple systems u Ecodevelopmental models involve two and three level interactions F How do we model these moderation effects when the variables are observed? F How do we model these moderation effects when the variables are latent? 13

Multiple levels of nesting in effectiveness trials Time is nested within adolescents who are nested within families who are nested with therapist who are nested both within treatment & site Treatment crosses site 14

Implementing a High Fidelity Systemic Treatment in Drug Abuse Treatment Centers 15

Plan of Section n Clinical Trials Network (CTN) n BSFT Design Issues—Level of Control n BSFT Experience in High Fidelity Implementation 16

Clinical Trials Network n n NIDA funded network to test the effectiveness of efficacious treatments in real world settings As of 9/05: u u u n Nodes = 17 States = 34+Puerto Rico CTPs = 152 Protocols = 27 (11 closed to enrollment, 5 in development) Currently 88 CTPs involved in 11 open studies Mission: To implement science-based efficacious treatments in community settings AND to show that these implementations are an improvement over current practice 17

Brief Strategic Family Therapy —BSFT, CTN 0014 n Brief Strategic Family Therapy is a systemic, process-focused family therapy u 4 months with weekly sessions u Up to 8 booster sessions over 8 months u Focus on changing repetitive (maladaptive) interactions within the family Note that the focus is on underlying processes, not crises F May use a crisis as a content focus, but therapy addresses underlying “everyday” processes F n BSFT has been the focus of over 30 years of research at the Center for Family Studies 18

BSFT-Therapy Components Three major techniques F Joining (Engaging Participants into Treatment) • Balanced across all family members • Must join with most powerful F Diagnosis (Family Relationships and Roles) F Restructuring (Implementing the Treatment Plan) • • • Work in Present—Enactments Reframing Shifting boundaries 19

BSFT CTN 0014 n n n 8 Sites 60 participants per site on average 480 adolescent participants Drug Use assessed monthly for 12 months Full assessments at Baseline, 4 months, 8 months and 12 months u Delinquent Behaviors, Conduct Problems, Sexual risk behaviors, Adolescent Pro-social Activities, & Family Functioning 20

BSFT—Randomization Participants will be randomized to BSFT or the clinic’s standard outpatient treatment n Note, randomization is at the individual level, not at the clinic level n 21

Design Considerations for Effectiveness Trials n Level of Control u Homogeneity of Study Population (Szapocznik) u Standardization & Monitoring of Treatment (Szapocznik) u n Standardization & Monitoring of Control (Feaster) Handling of Site Variance (Feaster) u Efficacy study—Fixed Effect u Effectiveness—Random Effect 22

BSFT: Heterogeneous Treatment Population n Inclusion Criteria: u 12 -17 years of age u Any illicit drug use in last 30 days u Lives or is expected to with “family” u Reside in the same geographic area as CTP u Signed consent & assent n Exclusion Criteria: u Not living with family (halfway house, institution, etc. ) u Suicidal or homicidal risk must be stabilized, first u Current or pending severe criminal charges if likely to lead to incarceration u If already receiving drug treatment services 23

Choice for BSFT: Full Control of Experimental Condition n Extensive Training and Supervision u 5 months of training u Weekly supervision n Training and Supervision are considered integral to the BSFT model 24

Multiple Levels of Clinical Supervision & Adherence Monitoring n Clinical Supervision Weekly conference calls with a BSFT supervisor. Supervision includes: u u n National Clinical Supervisor u u n weekly face-to-face sessions with each clinical supervisor Regularly sits in on selected supervision calls with sites Adherence Ratings of Videotapes u u n videotape review case discussion and planning. Randomly selected sessions trained independent raters in Miami Failure to adhere to the BSFT model u u Definition-<70% adherence for 3 consecutive sessions Consequences & Corrective Action F F F no new cases until 80% in two consecutive sessions Increased supervision & retraining until meets criteria If criteria not met before conclusion of current cases withdrawn 25 from study (at discretion of clinical supervisor)

Therapist Consent and Selection Process n Identification of volunteers u Consent u Demographics u Views of Adolescent Drug Abuse – Q-sort n Selection u Interviews (Site PI, National Study Director/ Coordinator, and BSFT Head Training Supervisor) u Family Session n Randomization u Academic training u Years of clinical experience 26

Training Phase for BSFT Therapists n n Five-month clinical training program Workshops u Four 3 -day workshops Week 1 (Workshop 1) *Miami F Week 3 (Workshop 2) F Week 5 or 6 (Workshop 3) F Week 13 (Workshop 4) F n Supervision u Weekly group supervision F n n Each therapist will have ½ hour for videotape review and ½ for case discussion Pilot Cases: 2 -4 cases for each therapist Certification 27

Implementation Phase n Active Cases u Caseload builds over time u Study caseload will range from 2 -8 cases F (minimum n Supervision = 0; maximum = 10) u Weekly group supervision F Review of videotapes F Review of clinical forms F Treatment planning F Each therapist (2 active) will have 30 -45 minutes for videotape review and 30 -45 minutes for case discussion 28

Adherence Ratings By Site Good Adherence is 3 29

Adherence Ratings n Show adequate adherence n Some variability across sites n Supervisors’ ratings uniformly higher than independent raters’ 30

31

Some Statistical Approaches and Areas for Further Research in Systemic Modeling and Design of Effectiveness Trials 32

Plan of Section: Specification of Site Effects & Bounds of Inference n Power analysis and Trial Planning n Time Structure of Models & Reciprocal Effects n Need for Simulation Research n Questions about Mplus & Simulation n 33

Design Considerations for Effectiveness Trials n Level of Control u u Homogeneity of Study Population (Szapocznik) Standardization & Monitoring of Treatment (Szapocznik) u Standardization & Monitoring of Control (Feaster) n Handling of Site Variance (Feaster) u Efficacy study—Fixed Effect u Effectiveness—Random Effect 34

Considerations for Choice of Comparison Condition n Standardized Control Group u Smaller sample size (sites & participants) u High internal validity u Lacks ecological validity for n CTPs Treatment As Usual u Larger sample size (sites & participants) u High external validity u Highly variable across sites u Minimum site size— within site comparisons 35

Choice for BSFT: Treatment As Usual Compare BSFT to the population of treatments in the community Currently drug abuse treatment has considerable variability in treatment approach and implementation 36

Handling of Site Variance n Fixed Effect (control for site & remove variance from error): Cannot generalize statistically n Random Effect: Can generalize effect beyond the clinics included u Both site & site X treatment are random u Implications on power of study 37

Choice for BSFT: TAU n Differences in TAU at each site implies larger variance of the random Site X Treatment effect n Sites need not be randomly assigned, but need to describe the generality of clinics (if not randomly selected) 38

Fidelity of Implementation Variability in fidelity across sites will increase the site and site by treatment effects 39

Specifics of BSFT Statistical Plan Hypothesis 1: BSFT will be significantly more effective than TAU in reducing adolescent drug abuse, defined as the percentage of days with positive drugs use. 40

Example of Expected Trajectories 41

Secondary Hypotheses BSFT will be significantly more effective than TAU in : – Reducing: u Delinquent behaviors & conduct problems u Sexually risky behaviors – Increasing: u Prosocial activities (school, employment) u Family functioning (parenting, parent- adolescent communication) 42

Analysis Strategy n n Randomization at the Individual Client Level Multi-level Growth Curve with 3 levels u Level 1 is within person—Time u Level 2 is between persons (and incorporates treatment assignment) u Level 3 is between site n n Controlling for baseline drug use (ANCOVAtype specification) Note: not accounting for all levels of nesting in primary hypotheses (therapist effects examined in post-hoc analyses) 43

Proposed Model Level 1—Time: Time, a, is centered at Month 4 Level 2—Between Adolescents: Level 3—Between Sites: 44

Multisite Power Analyses (Raudenbush & Liu, 2001) Procedure assumes single post test (not growth curve) We have 12 monthly drug use assessments However, drug use is not normally distributed 45

Methodological Issues Most research on design implications of site effects has been done by statisticians with drug trial experience n Aim is to prevent site effects or justifying ignoring them if they exist n Therefore, little prior evidence published on site variability & particularly on the site by treatment interaction n 46

Methodological Issues n Power Analysis u Lack of prior info makes difficult u Lack of software (Raudenbush u Simulation in M-Plus n & Liu, 2001) Have been doing simulations u In SAS, have difficulty with 8 sites identifying the site & site X treatment variance in growth curve framework u Specifically, it may not be possible to identify the covariance between the site and site X treatment random components u Simulations do not exactly match Raudenbush & Liu 47

Resource Allocation Issue Random site X treatment effect with variability in Treatment as Usual requires a large sample n Necessity of effectiveness research if to generalize to new clinics n Some disagree (due in part to costs) n u Simulation to show Type 1 error u Look at potential mistakes of policy based on inappropriate overly precise estimates 48

Reciprocal Effects Person-specific cross-lagged panel model n Could be formulated in a latent difference score framework n Model is important for systemic phenomenon like family interactions n Illustrated within a person using coping and distress data n 49

“SETA-Fam” Hypotheses b SET a Family Functioning Target Woman’s Outcomes d e c Dotted Line Refers to Hypotheses in SETA Family Members’ Outcomes 50

Within A Person Distress 1 a Distress 2 c e e f Coping 2 e f d d b Distress 4 c f d a Distress 3 c z Coping 1 a b Coping 3 b Coping 4 Test a through d as random effects 51

Within a Family Distressj 1 a Distressj 2 c e d f e Distressk 2 Distressj 4 c f e f d d b a Distressj 3 c z Distressk 1 a b Distressk 3 b Distressk 4 Test a through d as random effects 52

Distress M=. 61* AR V=. 02* Cross Lagged w/AR Random Distress T 1 Distress T 2 Distress T 3 . 19* . 03 Coping T 1 Coping T 2 Coping T 3 Adj. BIC=2206. 9 Coping M=. 55* AR V=. 03 Distress T 4 Coping T 4 53

How does Model Fit Compare to Growth Curve Specification? Distress Baseline. 77* Coping Baseline -. 46* Distress Linear Distress Quadratic V=0 . 9999* Coping Linear Adj. BIC=2219. 5, *Difficulty with Psi Coping V=0 Quadratic 54

Selected Issues to Address n Questions to be addressed: u Parameter interpretation u Comparative Model Fit/Discrimination u Initial Conditions u Different time frames both within and between individuals (data is collected on only one time frame, though with variability in exact times of collection) n Simulations u Question, is it possible to vary parameters in a simulation within Mplus and save the parameter settings to the generated data file? u [AR@. 3 to. 8 by. 1]. u AR @ 0 to 5 by. 5. 55

56

57