SCANLab Meta-analysis of neuroimaging data What, Why, and How Tor D. Wager Columbia University http: //www. columbia. edu/cu/psychology/tor/
Uses of meta-analysis in neuroimaging • Meta-analysis is an essential tool for summarizing the vast and growing neuroimaging literature Wager, Lindquist, & Hernandez, in press SCANLab http: //www. columbia. edu/cu/psychology/tor/
Uses of meta-analysis in neuroimaging • Assess consistency of activation across laboratories and task variants • Compare across many types of tasks and evaluate the specificity of activated regions for particular psychological conditions • Identify and define boundaries of functional regions SCANLab • Co-activation: Develop models of functional systems and pathways Wager, Lindquist, & Kapan, 2007 http: //www. columbia. edu/cu/psychology/tor/
Functional networks in meta-analysis • Use regions or distributed networks in a priori tests in future studies SCANLab http: //www. columbia. edu/cu/psychology/tor/
Meta-analyses of cognitive control Authors Chein et al. Year Method 2002 Density (Gaussian) Clustering of peaks, Wager et al. 2003 chi-square Wager et al. 2004 KDA, Spatial MANOVA Buchsbaum 2005 ALE et al. Chein & 2005 Density (Gaussian) Schneider Laird et al. 2005 ALE Psychological focus Verbal working memory SCANLab Working memory Attention/task switching Wisconsin Card Sorting Practice effects in cognitive control Stroop interference Owen et al. 2005 ALE N-back working memory Neumann et ALE, Co-activation 2005 Stroop interference al. "replicator dynamics" Costafreda 2006 Spatial location Verbal fluency in left IFG et al. Gilbert et al. 2006 Spatial location/Chisquare/classifier Episodic memory, Multitasking, Mentalizing in BA 10 Nee et al. KDA, logistic regression Cognitive control/interference 2007 Van 2008 Snellenberg MKDA/KDA * & Wager Cognitive control and memory http: //www. columbia. edu/cu/psychology/tor/
Meta-analyses of emotion & motivation SCANLab Authors Year Method Chi-square within Phan et al. 2002 regions Spatial location (K-S Murphy et al. 2003 Test) Wager et al. 2003 KDA, Chi-square Kringelbach 2004 Spatial location et al. Psychological focus Phan et al. Emotion 2004 Qualitative Emotion Reinforcers in OFC Baas et al. 2004 Chi-square Northoff et 2005 Clustering of peaks al. Amygdala lateralization Krain et al. Decision-making 2006 ALE Wager et al. 2008 MKDA, Chi-square Kober et al. 2008 MKDA, Co-activation * Self-referential processes Emotion http: //www. columbia. edu/cu/psychology/tor/
Meta-analyses of disorders Authors Year Zakzanis et 2000 al. Zakzanis et 2003 al. Whiteside et 2004 al. Glahn et al. Method Psychological focus SCANLab Effect sizes Schizophrenia Effect sizes Alzheimer's disease Effect sizes Obsessive-compulsive disorder 2005 ALE Fitzgerald et 2006 ALE al. Dickstein et 2006 ALE al. Van Snellenberg 2006 Effect sizes et al. Spatial location Steele et al. 2007 ("unwarped") Working memory in schizophrenia Depression, DLPFC ADHD Schizophrenia and working memory Depression, frontal cortex Valera et al. 2007 Effect sizes Brain structure in ADHD Etkin & Wager Anxiety disorders 2007 MKDA, Co-activation Hoekert et al. 2007 Effect sizes Emotional prosody in schizophrenia http: //www. columbia. edu/cu/psychology/tor/
Meta-analyses of language SCANLab Authors Turkeltaub et al. Year Method 2002 ALE Psychological focus Single-word reading Jobard et al. 2003 Clustering of peaks Word reading Brown et al. 2005 ALE Speech production Language, left cortical hemisphere Vigneau et al. 2006 Clustering of peaks Ferstl et al. 2008 ALE, Co-activation Text comprehension "replicator dynamics" Turkeltaub 2002 ALE et al. Jobard et al. 2003 Clustering of peaks Single-word reading Word reading Brown et al. 2005 ALE Speech production Vigneau et al. 2006 Clustering of peaks Language, left cortical hemisphere Ferstl et al. 2008 ALE, Co-activation Text comprehension "replicator dynamics" http: //www. columbia. edu/cu/psychology/tor/
Meta-analyses of other stuff SCANLab Authors Joseph Grezes & Decety Kosslyn & Thompson Year Method 2001 Spatial location 2001 Qualitative Action 2003 Logistic regression Visual imagery Nielsen et al. 2004 Kernel density/multivariate Gottfried & 2005 Spatial location Zald Nickel & 2005 Clustering of peaks Seitz Petacchi et 2005 ALE al. Average maps in Lewis 2006 CARET Postuma & 2006 Co-activation Dagher Zacks Psychological focus Object recognition: category specificity 2008 Cognitive function Olfaction in OFC Parietal cortex Auditory function, cerebellum Tool use Basal ganglia Mental rotation http: //www. columbia. edu/cu/psychology/tor/
SCANLab Using meta-analysis to evaluate consistency: Why? http: //www. columbia. edu/cu/psychology/tor/
Locating emotion-responsive regions 164 PET/f. MRI studies, 437 activation maps, 2478 coordinates SCANLab http: //www. columbia. edu/cu/psychology/tor/
Why identify consistent areas? • Making statistic maps in neuroimaging studies involves many tests (~100, 000 per brain map) • Many studies use uncorrected or improperly corrected p-values SCANLab Long-term Memory # of Maps P-value thresholds used Uncorrected Corr. How many false positives? A rough estimate: 663 peaks, 17% of reported activations Wager, Lindquist, & Kaplan, 2007 http: //www. columbia. edu/cu/psychology/tor/
Consistency SCANLab Emotion: 163 studies Consistently Activated Reported peaks regions 163 studies http: //www. columbia. edu/cu/psychology/tor/
Ventral surface Lateral surface (R) Medial surface (L) SCANLab vm. PFC Gyrus rectus Central sulcus dm. PFC p. OFC BF rd. ACC Pre SMA pg. ACC dm. PFC PCC OCC CM, MD sg. ACC m. TC s. TC a. INS lat. OFC l. FG Kober et al. , in press, NI Deep nuclei TC vm. PFC http: //www. columbia. edu/cu/psychology/tor/
SCANLab Using meta-analysis to evaluate specificity: Why? http: //www. columbia. edu/cu/psychology/tor/
Disgust responses: Specificity in insula? Insula SCANLab http: //www. columbia. edu/cu/psychology/tor/
Disgust responses: Specificity in insula? SCANLab Search Area: Insula Feldman-Barrett & Wager, 2005; Phan, Wager, Taylor, & Liberzon, 2002; Phan, Wager, Liberzon & Taylor, 2004 http: //www. columbia. edu/cu/psychology/tor/
Meta-analysis plays a unique role in answering… SCANLab The Neural Correlates of Task X • Is it reliable? – Would each activated region replicate in future studies? – Would activation be insensitive to minor variations in task design? • Is it task-specific? – Predictive of a particular psychological state or task type? – Diagnostic value? http: //www. columbia. edu/cu/psychology/tor/
SCANLab Using meta-analysis to evaluate consistency: How? http: //www. columbia. edu/cu/psychology/tor/
Meta-analysis: Multilevel kernel density estimate (MKDE) Monte Carlo: Expected maximum proportion Under the null hypothesis Peak coordinate locations (437 maps) Permute blobs within study maps … Damasio, 2000 Liberzon, 2000 Kernel convolution SCANLab Wicker, 2003 Apply threshold E Significant regions Weighted average … Comparison indicator maps Wager, Lindquist, & Kaplan, 2007; Etkin & Wager, in press Proportion of activated Comparisons map (from 437 comparisons) http: //www. columbia. edu/cu/psychology/tor/
MKDA: Key points SCANLab • Statistic reflects consistency across studies. Study comparison map is treated as a random effect. Peaks from one study cannot dominate. • Studies are weighted by quality (see additional info on handouts for rationale) • Spatial covariance is preserved in Monte Carlo. Less sensitive to arbitrary standards for how many peaks to report. http: //www. columbia. edu/cu/psychology/tor/
Whether and how to weight studies/peaks MKDA analysis weights by sqrt(sample size) and study quality (including fixed/random effects) Study quality weight Weighted proportion of Sample size for activating map c studies SCANLab Weighted average Activation indicator (1 or 0) for map c Fixed effects Random effects http: //www. columbia. edu/cu/psychology/tor/
Monte Carlo Simulation • Simulation vs. theory (e. g. Poisson process) • Simulation allows: SCANLab – Non-stationary spatial distribution of peaks (clumps) under null hypothesis; randomize blob locations – Family-wise error rate control with irregular (brain -shaped) search volume – Cluster size inference, given primary threshold Monte Carlo: E(max(P|H 0)) http: //www. columbia. edu/cu/psychology/tor/
Compare with Activation Likelihood Estimate (ALE), Kernel Density Analysis (KDA) Peak coordinates Combined across studies Kernel convolution Density kernel SCANLab Apply significance threshold Peak density or Significant results ALE map OR ALE kernel Ignores the fact that some studies report more peaks than others! Density kernel: Chein, 1998; Phan et al. , 2002; Wager et al. , 2003, 2004, 2007, in press Gaussian density kernel + ALE: Turkeltaub et al. , 2002; Laird et al. , 2005; others http: //www. columbia. edu/cu/psychology/tor/
Comparison with other methods MKDA KDA/ALE SCANLab • Statistic reflects consistency • Peaks are lumped together, study across studies. Study comparison is fixed effect. Peaks from one map is treated as a random effect. study can dominate, studies that Peaks from one study cannot report more peaks dominate. • Studies are weighted by quality • No weighting, or z-score weighting (problematic) • Spatial covariance is preserved in • Spatial covariance is not Monte Carlo. Less sensitive to preserved in Monte Carlo. arbitrary standards for how many Effects of reporting standards peaks to report. large. See handouts for more comparison points http: //www. columbia. edu/cu/psychology/tor/
ALE approach • Treats points as if they were Gaussian probability distributions. • Summarize the union of probabilities at each voxel: probability of any peak “truly” lying in that voxel SCANLab is the probability that peak Xi lies in a given voxel The bar indicates the complement operator Null hypothesis: No peaks lie in voxel Alt hypothesis: voxel At least one peak lies in http: //www. columbia. edu/cu/psychology/tor/
ALE meta-analysis SCANLab • Analyst chooses smoothing kernel • ALE analysis with zero smoothing: – Every voxel reported in any study is significant in the meta-analysis • Test case: 3 -peak meta analysis, one peak activates in voxel: ALE statistic: Highest possible value! • In practice: 10 – 15 mm FWHM kernel http: //www. columbia. edu/cu/psychology/tor/
Comparison across methods: Inference Property KDA ALE Multilevel KDA Kernel Spherical Gaussian Spherical Interpretation of statistic Null hypothesis Num nearby peaks Interpretation of significant result Assumptions Generalize to SCANLab Prob. that at least one peak nearby Peaks are not No peaks truly spatially consistent activate More peaks lie near One or more peaks voxel than expected lies at this voxel by chance Num. study maps activating nearby Study maps are not spatially consistent A higher proportion of studies activate near voxel than expected by chance 1. Study is fixed Activation ‘blobs’ effect (homogenous are spatially sample of studies) independent under 2. Peaks are the null hypothesis spatially independent under the null hypothesis New peaks from New study maps same studies http: //www. columbia. edu/cu/psychology/tor/
Comparison: Correction and Weighting SCANLab Property KDA ALE Multilevel KDA Multiple comparisons FWER FDR Weighting None, or weight peaks by z-score None FWER (recommended) or FDR Weight studies by sample size, fixed/random effects, quality http: //www. columbia. edu/cu/psychology/tor/
Density analysis: Summary SCANLab Working memory Executive WM Long-term memory Memory Inhibition Task switching Response selection Wager et al. , 2004; Nee, Wager, & Jonides, 2007; Wager et al. , in press; Van Snellenberg & Wager, in press http: //www. columbia. edu/cu/psychology/tor/
SCANLab Using meta-analysis to evaluate specificity: How? http: //www. columbia. edu/cu/psychology/tor/
Specificity • Task-related differences in relative activation frequency across the brain: SCANLab – MKDA difference maps (e. g. , Wager et al. , 2008) • Task-related differences in absolute activation frequency – Nonparametric chi-square maps (Wager, Lindquist, & Kaplan, 2007) • Classifier systems to predict task type from distributed patterns of peaks (e. g. , Gilbert) http: //www. columbia. edu/cu/psychology/tor/
MKDA Difference maps: Emotion example SCANLab Experienced Perceived • Approach: – Calculate density maps for two conditions, subtract to get difference maps – Monte Carlo: Randomize blob locations within each study, recalculate density difference maps and save max – Repeat for many (e. g. , 10, 000) iterations to get max distribution – Threshold based on Monte Carlo simulation http: //www. columbia. edu/cu/psychology/tor/
Emotion example: Selective regions Experience > Perception > Experience OFC TP SCANLab Amy IFG Hy Midb OFC a. Ins TP OFC va. Ins Hy dm. PFC Amy m. OFC Hy Midb PAG va. Ins IFG pg. ACC PAG TP TP CB Amy OFC CB a. Ins Amy Wager et al. , in press, Handbook of Emotion http: //www. columbia. edu/cu/psychology/tor/
Task-brain activity associations in meta-analysis SCANLab Study contrast map Region/V Task oxel 1 condition Study 1 1 Disgust Study 2 0 Fear Study 3 1 Disgust Study 4 1 Happiness Study 5 0 Anger … … … Study N 0 Sadness Measures of association: Chi-square • But requires high expected counts (> 5) in each cell. Not appropriate for map-wise testing over many voxels Fisher’s exact test (2 categories only) Multinomial exact test • Computationally impractical! Nonparametric chi-square • Approximation to exact test • OK for low expected counts http: //www. columbia. edu/cu/psychology/tor/
Nonparametric chi-square: Details SCANLab Study contrast map Region/V Task oxel 1 condition Study 1 1 Disgust Study 2 0 Fear Study 3 1 Disgust Study 4 1 Happiness Study 5 0 Anger … … … Study N 0 Sadness Idea of exact test: • Conditionalize on marginal counts for activation and task conditions. • Null hypothesis: no systematic association between activation and task • P-value is proportion of nullhypothesis possible arrangements that can produce distribution across task conditions as large as observed or larger. http: //www. columbia. edu/cu/psychology/tor/
Nonparametric chi-square: Details SCANLab Study contrast map Region/V Task oxel 1 condition Study 1 0 1 Study 2 Study 3 Study 4 Study 5 … Study N 0 1 0 … 1 Disgust Fear Disgust Happiness Anger … Sadness Permutation test: • Permute activation indicator vector, creating null-hypothesis data (no systematic association) • Marginal counts are preserved. • Test 5, 000 or more samples and calculate P-value based on observed null-hypothesis distribution http: //www. columbia. edu/cu/psychology/tor/
Density difference vs. Chi-square • Relative vs. absolute differences SCANLab Experience Perception Voxels (one-dimensional brain) Chi-square Density http: //www. columbia. edu/cu/psychology/tor/
Can we predict the emotion from the pattern of brain activity? Experienced SCANLab Perceived • Approach: predict studies based on their pattern of reported peaks (e. g. , Gilbert, 2006) • Use naïve Bayesian classifier (see work by Laconte; Tong; Norman; Haxby). Cross-validate: predict emotion type for new studies that are not part of training set. http: //www. columbia. edu/cu/psychology/tor/
Classifying experienced emotion vs. perceived emotion: 80% accurate SCANLab Experience EXP vs. PER DMPFC vs. Pre-SMA PAG vs. Ant. thalamus Perception Deep cerebellar nuc. vs. Lat. cerebellum http: //www. columbia. edu/cu/psychology/tor/
Outline: Why and How… • Consistency: Replicability across studies SCANLab – Consistency in single-region results: MKDA – Consistency in functional networks: MKDA + Co-activation • Specificity and “reverse inference” – Brain-activity – psychological category mappings for individual brain regions: MKDA difference maps; Nonparametric Chi-square – Brain-activity – psychological category mappings for distributed networks Applying classifier systems to meta-analytic data http: //www. columbia. edu/cu/psychology/tor/
Extending meta-analysis to connectivity SCANLab Study contrast map Region/V oxel 1 Region/V oxel 2 Study 1 1 0 Study 2 0 0 Study 3 1 1 Study 4 1 1 Study 5 0 0 … … … Study N 0 1 N = 45 Region 1 No Region 1 Yes Region 2 Yes 6 23 Region 2 No 12 4 Co-activation: If a study (contrast map) activates within k mm of voxel 1, is it more likely to also activate within k mm of voxel 2? Measures of association: Kendall’s Tau-b Fisher’s exact test Nonparametric chi-square Others… http: //www. columbia. edu/cu/psychology/tor/
Kendall’s Tau: Details • Ordinal “nonparametric” association between two variables, x and y • Uses ranks; no assumption of linearity or normal distribution (Kendall, 1938, Biometrika) • Values between [-1 to 1], like Pearson’s correlation SCANLab Tau is proportion of concordant pairs of observations sign(x diff. between pairs)= sign(y diff. between pairs) Tau = (# concordant pairs - # discordant pairs) / total # pairs http: //www. columbia. edu/cu/psychology/tor/
Meta-analysis functional networks: Examples • Emotion: Kober et al. (in press), 437 maps SCANLab http: //www. columbia. edu/cu/psychology/tor/
Meta-analysis of emotion Acknowledgements Lisa Feldman Barrett Luan Phan Steve Taylor Israel Liberzon Students Hedy Kober Lauren Kaplan Jason Buhle Jared Van Snellenberg Statistics Martin Lindquist Meta-analysis of cognitive function SCANLab Ed Smith Tom Nichols Derek Nee John Jonides Ed Smith Funding agencies: National Science Foundation National Institute of Mental Health http: //www. columbia. edu/cu/psychology/tor/
SCANLab Weighting http: //www. columbia. edu/cu/psychology/tor/
Whether and how to weight studies/peaks • Studies (and peaks) differ in sample size, methodology, analysis type, smoothness, etc. • Advantageous to give more weight to more reliable studies/peaks SCANLab • Z-score weighting – Advantages: Weights nominally more reliable peaks more heavily – Disadvantages: Small studies can produce variable results. Reporting bias: High z-score peaks are high partially due to error; “capitalizing on chance” • Must convert to common Z-score metric across different analysis types in different studies http: //www. columbia. edu/cu/psychology/tor/
Whether and how to weight studies/peaks • Alternative: Sample-size weighting SCANLab – Advantages: • Weights studies by the quality of information their peaks are likely to reflect • Avoids overweighting peaks reported due to “capitalizing on chance” – Disadvantages: Ignores relative reliability of various peaks within studies http: //www. columbia. edu/cu/psychology/tor/
SCANLab MKDA vs. ALE: Comparison chart http: //www. columbia. edu/cu/psychology/tor/
SCANLab More details on reverse inference http: //www. columbia. edu/cu/psychology/tor/
Is brain activity diagnostic of a particular psychological state? SCANLab Pleasure? Punishing wrongdoers Forward inference Reverse inference Brain activity P(Brain | Psy) Given a psychological We observe state brain activity P(Psy | Brain) Can we infer Given brain psychological pleasure? activity ‘Forward’ and ‘reverse’ inference are not the same! Reverse inference requires comparing across many psychological states! http: //www. columbia. edu/cu/psychology/tor/
The predictive value problem: Worked example For a brain region to be used as a marker of pleasure SCANLab – The brain region must respond consistently to pleasure – The brain region must respond specifically to pleasure (not activated by other things) Non-pleasure P(Brain|no pleasure) =. 4 1 -Specificity Ventral caudate P(pleasure) =. 1 Prior P(Brain|Pleasure) =. 9 Forward inference; Sensitivity Caculate reverse inference: P(Pleasure|Brain) =. 2 http: //www. columbia. edu/cu/psychology/tor/
SCANLab More details on connectivity http: //www. columbia. edu/cu/psychology/tor/
SCANLab More details on MKDA difference maps and nonparametric chi-square maps http: //www. columbia. edu/cu/psychology/tor/
Скачать презентацию SCANLab Meta-analysis of neuroimaging data What Why and
67f3764e280803e0024b4be8eacff3bd.ppt