The Scientific Method Common Mistakes in Applying

The Scientific Method

Common Mistakes in Applying the Scientific Method • The scientist prefers one outcome over another (bias) • Ignoring or ruling out data that do not support the hypothesis; tendency to find something “wrong” with evidence that does not support the hypothesis (not treating all data in the same way) • Failure to estimate quantitatively systematic errors • Faulty interpretation of statistical data

Bad conclusions can be drawn when there is… • Selection bias: a distortion of evidence or data that arises from the way the data are collected • Reversed burden of proof: burden of proof should rest on those making a claim, not on the critic • Assertion that claims which have not been proven false must be true (and vice versa)

Correlation vs Causation • Correlation implies causation is a logical fallacy by which two events that occur together are claimed to be cause and effect. • It is dangerous to: – ignoring the possibility that the correlation is coincidence or that both correlated events have another common cause – draw conclusions about causation from statistical correlations. If you only have A and B, a correlation between them does not let you infer A causes B, or vice versa.

The Simpsons (Season 7, “Much Apu about Nothing”) • Homer: Not a bear in sight. The “Bear Patrol” must be working like a charm! • Lisa: That’s specious reasoning, Dad. • Homer: Thank you, dear. • Lisa: By your logic I could claim that this rock keeps tigers away. • Homer: Oh, how does it work? • Lisa: It doesn’t work. • Homer: Uh-huh. • Lisa: It’s just a stupid rock. But I don’t see any tigers around, do you? • Homer: Lisa, I want to buy your rock.

Case Study: Willie Soon • Associated with the George C. Marshall and Fraser Institutes • Has received funding from the American Petroleum Institute (this is acknowledged in his main paper) • While it doesn’t prevent him from doing good science, this calls into question his ability to be unbiased

Claim: “[T]he 20 th century is not unusually warm or extreme. ” • This was not based on a quantitative analysis. • The authors considered anomalous conditions to be warm, cold, wet or dry relative the 20 th century. • The relationship between wet/cold and warm/dry is not 1: 1. • Authors do not properly distinguish between local and global temperature changes • No attempt was made to estimate the errors in using proxy data • No way to resolve short time scales

Claim: CO 2 has nothing to do with global climate change • The authors can only responsibly claim one point in their paper – They have a model with two free parameters – They can fit these parameters to global temperature change data – In their model, their best fit includes a higher contribution from the sun than green house gases – Just because the data fits a model does not imply the model is true. (One would have to test all possible models in order to prove that any one model was true. ) • A convincing argument would entail performing a simulation that produced output that matched the data without fitting *Acknowledged funding sources in paper: Electric Power Research Institute, Mobil Foundation, Texaco Foundation and American Petroleum Institute

Some useful references about Correlation and Causation… • • • http: //www. onpedia. com/encyclopedia/Correlation-implies-causation-(logical-fallacy) http: //www. sciencebuddies. org/science-fair-projects/project_scientific_method. shtml http: //teacher. pas. rochester. edu/phy_labs/appendixe. html http: //en. wikipedia. org/wiki/Pseudoscience http: //www. venganza. org/ Willie Soon Papers and Rebuttal • • • http: //adsabs. harvard. edu/full/1996 Ap. J. . . 472. . 891 S http: //www. cfa. harvard. edu/~wsoon/1000 yrclimatehistory-d/Jan 30 Climate. Researchpaper. pdf http: //www. atmos. washington. edu/~davidc/ATMS 211/articles_optional/Mann_on_Soon 2003. pdf