Experiment Basics Variables Psych 231 Research Methods in

Experiment Basics: Variables Psych 231: Research Methods in Psychology

n Journal Summary 1 due in labs this week Reminders

n n n Independent variables (explanatory) Dependent variables (response) Extraneous variables n n n Control variables Random variables Confound variables Many kinds of Variables

n n Scales of measurement Errors in measurement n n Reliability & Validity Sampling error Measuring your dependent variables

μ = 71 Population Everybody that the research is targeted to be about Sampling error X = 68 Sample Sampling The subset of the population that actually participates in the research

Population Sampling to make data collection manageable Inferential statistics used to generalize back n Sample Sampling Allow us to quantify the Sampling error

n Goals of “good” sampling: – Maximize Representativeness: – To what extent do the characteristics of those in the sample reflect those in the population – Reduce Bias: – A systematic difference between those in the sample and those in the population n Key tool: Random selection Sampling

n Probability sampling n n Have some element of random selection Non-probability sampling n n n Simple random sampling Cluster sampling Stratified sampling Quota sampling Convenience sampling Random element is removed. Susceptible to biased selection There advantages and disadvantages to each of these methods n n I recommend that you check out table 6. 1 in the textbook pp 127 -128 Here is a nice video (~5 mins. ) reviewing some of the sampling techniques (Statistics Learning Centre) Sampling Methods

n Every individual has a equal and independent chance of being selected from the population Simple random sampling

n n n Step 1: Identify clusters Step 2: randomly select some clusters Step 3: randomly select from each selected cluster Cluster sampling

n Step 1: Identify distribution of subgroups (strata) in population 8/40 = 20% n 20/40 = 50% 12/40 = 30% Step 2: randomly select from each group so that your sample distribution matches the population distribution Stratified sampling

n n Step 1: identify the specific subgroups (strata) Step 2: take from each group until desired number of individuals (not using random selection) Quota sampling

n Use the participants who are easy to get (e. g. , volunteer sign-up sheets, using a group that you already have access to, etc. ) Convenience sampling

n Use the participants who are easy to get (e. g. , volunteer sign-up sheets, using a group that you already have access to, etc. ) n College student bias (World of Psychology Blog) “Who are the people studied in behavioral science research? A recent analysis of the top journals in six sub-disciplines of psychology from 2003 to 2007 revealed that 68% of subjects came from the United States, and a full 96% of subjects were from Western industrialized countries, specifically those in North America and Europe, as well as Australia and Israel (Arnett 2008). The make-up of these samples appears to largely reflect the country of residence of the authors, as 73% of first authors were at American universities, and 99% were at universities in Western countries. This means that 96% of psychological samples come from countries with only 12% of the world's population. ” Henrich, J. Heine, S. J. , & Norenzayan, A. (2010). The weirdest people in the world? (free access). Behavioral and Brain Sciences, 33(2 -3), 61 -83. Convenience sampling

n n Independent variables Dependent variables n Measurement • Scales of measurement • Errors in measurement n Extraneous variables n n n Control variables Random variables Confound variables Variables

n Control variables n n Holding things constant - Controls for excessive random variability Random variables – may freely vary, to spread variability equally across all experimental conditions n Randomization • A procedure that assures that each level of an extraneous variable has an equal chance of occurring in all conditions of observation. n Confound variables n n Variables that haven’t been accounted for (manipulated, measured, randomized, controlled) that can impact changes in the dependent variable(s) Co-varys with both the dependent AND an independent variable Extraneous Variables

n Divide into two groups: n n men women n Instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. n Women first. Men please close your eyes. Okay ready? n Colors and words

Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 1

n n n Okay, now it is the men’s turn. Remember the instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. Okay ready?

Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 2

n So why the difference between the results for men versus women? n Is this support for a theory that proposes: n n “Women are good color identifiers, men are not” Why or why not? Let’s look at the two lists. Our results

List 1 Women Matched List 2 Men Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green Mis-Matched

n What resulted in the performance difference? n n Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green n n Our manipulated independent variable (men vs. women) The other variable match/mis-match? Because the two variables are perfectly correlated we can’t tell This is the problem with confounds IV Co-vary together Confound DV Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green

n n Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green What DIDN’T result in the performance difference? Extraneous variables n Control • # of words on the list • The actual words that were printed n Random • Age of the men and women in the groups • Majors, class level, seating in classroom, … n These are not confounds, because they don’t co-vary with the IV Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green

n Our goal: n To test the possibility of a systematic relationship between the variability in our IV and how that affects the variability of our DV. n Control is used to: • Minimize excessive variability • To reduce the potential of confounds (systematic variability not part of the research design) Experimental Control

n Our goal: n To test the possibility of a systematic relationship between the variability in our IV and how that affects the variability of our DV. T = NRexp + NRother + R Nonrandom (NR) Variability NRexp: Manipulated independent variables (IV) • Our hypothesis: the IV will result in changes in the DV NRother: extraneous variables (EV) which covary with IV • Condfounds Random (R) Variability • Imprecision in measurement (DV) • Randomly varying extraneous variables (EV) Experimental Control

n Variability in a simple experiment: T = NRexp + NRother + R Treatment group NR other NR exp R Absence of the treatment Control group (NRexp = 0) NR other R “perfect experiment” - no confounds (NRother = 0) Experimental Control: Weight analogy

n Variability in a simple experiment: T = NRexp + NRother + R Control group Treatment group NR exp R R Difference Detector Our experiment is a “difference detector” Experimental Control: Weight analogy

n If there is an effect of the treatment then NRexp will ≠ 0 Control group Treatment group R NR exp R Difference Detector Our experiment can detect the effect of the treatment Experimental Control: Weight analogy

n Potential Problems n n Confounding Excessive random variability Difference Detector Things making detection difficult

n Confound n If an EV co-varies with IV, then NRother component of data will be present, and may lead to misattribution of effect to IV IV DV Co-vary together EV Potential Problems

n Confound n Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but could be due to the NRother R NR other NR exp R Difference Detector Experiment can detect an effect, but can’t tell where it is from Confounding

n Confound n Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but could be due to the NRother These two situations look the same R NR other R NR NR exp other R Difference Detector There is an effect of the IV Confounding R Difference Detector There is not an effect of the IV

n Excessive random variability n If experimental control procedures are not applied • Then R component of data will be excessively large, and may make NRexp undetectable Potential Problems

n If R is large relative to NRexp then detecting a difference may be difficult R R NR exp Difference Detector Experiment can’t detect the effect of the treatment Excessive random variability

n But if we reduce the size of NRother and R relative to NRexp then detecting gets easier n So try to minimize this by using good measures of DV, good manipulations of IV, etc. R NR exp R Difference Detector Our experiment can detect the effect of the treatment Reduced random variability

n How do we introduce control? n Methods of Experimental Control • Constancy/Randomization • Comparison • Production Controlling Variability

n Constancy/Randomization n If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate • Control variable: hold it constant • Random variable: let it vary randomly across all of the experimental conditions Methods of Controlling Variability

n Comparison n An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is often the absence of the treatment Training group • • • No training (Control) group Without control groups if is harder to see what is really happening in the experiment It is easier to be swayed by plausibility or inappropriate comparisons Useful for eliminating potential confounds Methods of Controlling Variability

n Comparison n An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is often the absence of the treatment • Sometimes there a range of values of the IV 1 week of Training group 2 weeks of Training group 3 weeks of Training group Methods of Controlling Variability

n Production n The experimenter selects the specific values of the Independent Variables 1 week of Training group 2 weeks of Training group 3 weeks of Training group • Need to do this carefully • Suppose that you don’t find a difference in the DV across your different groups • Is this because the IV and DV aren’t related? • Or is it because your levels of IV weren’t different enough Methods of Controlling Variability

n n So far we’ve covered a lot of the about details experiments generally Now let’s consider some specific experimental designs. n n Some bad (but common) designs Some good designs • • 1 Factor, two levels 1 Factor, multi-levels Between & within factors Factorial (more than 1 factor) Experimental designs

n Bad design example 1: Does standing close to somebody cause them to move? n n n “hmm… that’s an empirical question. Let’s see what happens if …” So you stand closely to people and see how long before they move Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”) Poorly designed experiments

n Bad design example 2: n n Testing the effectiveness of a stop smoking relaxation program The participants choose which group (relaxation or no program) to be in Poorly designed experiments

n Bad design example 2: Non-equivalent control groups Independent Variable Dependent Variable Training group Measure No training (Control) group Self Assignment Measure participants Random Assignment Problem: selection bias for the two groups, need to do random assignment to groups Poorly designed experiments

n Bad design example 3: Does a relaxation program decrease the urge to smoke? n Pretest desire level – give relaxation program – posttest desire to smoke Poorly designed experiments

n Bad design example 3: One group pretest-posttest design Dependent Variable participants Add another factor Independent Variable Dependent Variable Pre-test Training group Post-test Measure Pre-test No Training group Post-test Measure Problems include: history, maturation, testing, and more Poorly designed experiments

n Good design example n How does anxiety level affect test performance? • Two groups take the same test • Grp 1 (moderate anxiety group): 5 min lecture on the importance of good grades for success • Grp 2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough n 1 Factor (Independent variable), two levels • Basically you want to compare two treatments (conditions) • The statistics are pretty easy, a t-test 1 factor - 2 levels

n Good design example n How does anxiety level affect test performance? Random Assignment Anxiety Dependent Variable Low Test Moderate Test participants 1 factor - 2 levels

Good design example n How does anxiety level affect test performance? One factor Use a t-test to see if anxiety low moderate 60 80 test performance n these points are statistically different Observed difference between conditions T-test = Difference expected by chance low Two levels 1 factor - 2 levels moderate anxiety

n Advantages: n n Simple, relatively easy to interpret the results Is the independent variable worth studying? • If no effect, then usually don’t bother with a more complex design n Sometimes two levels is all you need • One theory predicts one pattern and another predicts a different pattern 1 factor - 2 levels

n Disadvantages: n “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea Interpolation test performance What happens within of the ranges that you test? low 1 factor - 2 levels moderate anxiety

n Disadvantages: n “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea Extrapolation test performance What happens outside of the ranges that you test? low moderate anxiety 1 factor - 2 levels high

n n For more complex theories you will typically need more complex designs (more than two levels of one IV) 1 factor - more than two levels n n Basically you want to compare more than two conditions The statistics are a little more difficult, an ANOVA (Analysis of Variance) 1 Factor - multilevel experiments

n Good design example (similar to earlier ex. ) n How does anxiety level affect test performance? • Two groups take the same test • Grp 1 (moderate anxiety group): 5 min lecture on the importance of good grades for success • Grp 2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough • Grp 3 (high anxiety group): 5 min lecture on how the students must pass this test to pass the course 1 Factor - multilevel experiments

Random Assignment Dependent Variable Low Test Moderate Test High participants Anxiety Test 1 factor - 3 levels

low mod high 60 80 60 test performance anxiety low mod high anxiety 1 Factor - multilevel experiments

n Advantages n n Gives a better picture of the relationship (function) Generally, the more levels you have, the less you have to worry about your range of the independent variable 1 Factor - multilevel experiments

2 levels test performance 3 levels low moderate anxiety low mod high anxiety Relationship between Anxiety and Performance

n Disadvantages n n Needs more resources (participants and/or stimuli) Requires more complex statistical analysis (analysis of variance and pair-wise comparisons) 1 Factor - multilevel experiments

n The ANOVA just tells you that not all of the groups are equal. n If this is your conclusion (you get a “significant ANOVA”) then you should do further tests to see where the differences are • High vs. Low • High vs. Moderate • Low vs. Moderate Pair-wise comparisons