Department of Statistics What Researchers Need to Know

Department of Statistics What Researchers Need to Know about the Statistical Consulting Process or Working with the Statistician as a Research Colleague Walt Stroup Professor & Chair, UNL Department of Statistics 1 December 2006 SSP Core Facility

Department of Statistics Outline for Talk I. Role of a Statistician II. Statistical View of Research Process Ø From planning to finished product III. Essential features of an effective partnership Ø What is expected of you; what you should expect IV. What can go wrong & what to do about it 1 December 2006 SSP Core Facility

Department of Statistics Cool Quote – Source Unknown § Writers tell stories with words § Painters tell stories with pictures § Musicians tell stories with music § Statisticians tell stories with numbers However There must be a story to tell or the words, pictures, music, numbers --- and sophisticated methodology --are “. . . full of sound and fury, signifying nothing. . . ” 1 December 2006 SSP Core Facility

Department of Statistics So. . . Statisticians: The object of consulting is The object of consulting collaboration is The Science not the math nor the statistical techniques 1 December 2006 SSP Core Facility

Department of Statistics Cool Quote – Source Unknown § There are three subjects everybody thinks they can be an expert in with no formal training: − Law − Medicine − Statistics NOT !!! 1 December 2006 SSP Core Facility

Department of Statistics Role of Statistics in Research § Another cool quote, source also unknown: § “Think of ’Statistics’ as ‘the rules of evidence for scientific inquiry’” § Webster: “Statistics is a branch of mathematics. . . ” § Better: Oxford English Dictionary – “Statistics is the science of data collection, analysis, and reporting for the purposes of characterizing a population, testing a theory, or making a decision 1 December 2006 SSP Core Facility

Department of Statistics Some Influences My Major Professor: R. L. Anderson “This is the worst data set I ever saw! Why didn’t they talk to me first? Unbelievable!” Gerald 90% of the value of statistics is in what happens before any data are collected Hahn When people see me after the fact I often feel that the only thing I can do is write the death certificate “Don’t argue – take data” “You need to learn to think of yourself not as a statistical consultant but as a research consultant” 1 December 2006 SSP Core Facility

Department of Statistics E. O. Wilson not the exact quote, but true to spirit: What sets good research apart is usually not one’s cleverness in finding the answer but one’s skill in • asking the right question, and • phrasing it carefully and well to which I would add: “. . . and collecting the right data. . . ” 1 December 2006 SSP Core Facility Theme? Planning!

Department of Statistics 1 December 2006 SSP Core Facility

Department of Statistics Steps in Research Planning Protocol Description What is the Research Question? Answer Data analysis ? Conclusions New questions Relevant Evidence to answer question (response variables Collect data How much evidence? (sample size? ) 1 December 2006 How will evidence be collected – design? (clinical trial model? sample survey? ) SSP Core Facility How will evidence be used to answer (hypothesis test? )

Department of Statistics When is this Planning Applicable? § Grant writing § Preparing a dissertation proposal § Journal Publication § Informing Policy Decisions 1 December 2006 SSP Core Facility

Department of Statistics late Senator Daniel Patrick Moynihan we in this nation face a choice we can inform public policy with sound research or we can continue to make policy on the basis of data-free ideology 1 December 2006 SSP Core Facility

Department of Statistics Another of “Walt’s mentors”: Dal Kratzer § defined statistics as “experimentation science” § Protocol description form § Intended for pharmaceutical research & development § With word-smithing and tweaking, readily applicable as a template for research planning 1 December 2006 SSP Core Facility

Department of Statistics Protocol Description Form What are the objectives – these must be translated into specific questions that can be answered with data Key: operating definitions What is the relevant evidence? Distinguish between essential variables and auxilliary variables What are the measurement units? This implied defining the target population & how you intends to sample / observe it What is the treatment design? Experiment design? Power analysis – how many observations do I need Murphy’s Law – what can go wrong? what will I do? How will I analyze the data? How does this relate to objectives? Plan for reporting? Target audience? Intended purpose 1 December 2006 SSP Core Facility

Department of Statistics Working with Statistician § Haste makes waste § Build a relationship § Plan to educate statistician about your research − Two way street − You’re not a statistical expert; the statistician is not an expert in your discipline § “De-jargonization” 1 December 2006 SSP Core Facility

Department of Statistics What you don’t want 1 December 2006 SSP Core Facility

Department of Statistics Types of statistical investigations § “Data mining” − characterization, “fishing expedition” − looking for patterns that “might” be there − latent class, factor analysis, other multivariate methods § Associations − “cross-tabs”, log-linear models, path analysis − often based on survey or retrospective data § Treatment or factor effect − based on experiment design or quasi-design − “clinical trial” model; closest thing to cause & effect 1 December 2006 SSP Core Facility

Department of Statistics Essential Parts of the Process - I § What is the research question? − It is easy to be vague when you think you aren’t − Operating definitions are essential § Is “Math in the Middle” effective? − what do you mean by “effective”? − compared to what? − once “effective” has an operating definition, can you measure it? how? − if you can’t, you don’t have an operating definition yet 1 December 2006 SSP Core Facility

Department of Statistics Essential Parts of the Process - II § Power § First − what is the population? − what elements represent the population? − how do you (ethically) observe the elements? § Then − what are the relevant sources of variation among the elements? − what is the magnitude of variation assoc with these sources? − how big an effect is considered important? 1 December 2006 SSP Core Facility

Department of Statistics What power analysis is NOT 1 December 2006 SSP Core Facility

Department of Statistics Essential Parts of the Process - III § Power § Should be a dialog § At its best − mutual learning − dress rehearsal for data collection and analysis § INCLUDE: what can go wrong & what is my plan? − missing data happen − missing data are inevitable 1 December 2006 SSP Core Facility

Department of Statistics Essential Parts of the Process - IV § Stay away from jargon and “discipline-speak” − applies to you − applies to statistician § Ask questions − don’t let “stat-speak” go by − it’s not your ignorance, it’s the statistician’s language skills § Answer questions − if a good statistician asks you a question, it’s because she is curious & trying to get herself in the best position to be helpful − questions are not a hidden agenda to find fault or embarrass you 1 December 2006 SSP Core Facility

Department of Statistics Essential Parts of the Process - V § If possible, face-to-face meetings are better § Haste makes waste − last minute, rush-job consulting invites trouble § Another “mentor quote” − “we don’t have time to do it right, but we have plenty of time to do it over” EGO 1 December 2006 SSP Core Facility

Department of Statistics Analysis § Vastly easier in well-designed study § Vastly better if statistician has been involved all along § Parsimony − if two analyses show essentially same results and plausibly satisfy assumptions, the simpler analysis is better − 90% rule § However − just because it is traditional does not make it right − computer has had huge impact on statistical methods 1 December 2006 SSP Core Facility

Department of Statistics Analysis & Reporting § Part of statistician’s job is to help write report − material & methods: § statistical design § analysis − result and discussion § help explain and interpret § Statistical science itself is a work in progress − your project may motivate statistical research − methods may be unfamiliar to reviewers − statistician may need to educate referees / editors 1 December 2006 SSP Core Facility

Department of Statistics Etiquette & Ethics § Authorship − would research / grant have been possible without statistician’s efforts? − NO means a co-authorship should be offered § Funding − external funding is lifeblood of research university − statistician’s dilemma in academia: − “do I close my door to colleagues in allied disciplines to write my own grants, or do I participate in interdisciplinary teams? ” − for latter, co-PI grant arrangements with real $$$ essential 1 December 2006 SSP Core Facility

Department of Statistics More Etiquette & Ethics § Special Note on Graduate (Dissertation) Research § Common Unfortunate Situation: Graduate Student as Ping-Pong Ball § When statistician advises graduate student on design and/or analysis (ideally, “and” – ed. ) student, student’s major professor, and statistician should be on same page § Ideal – statistician on Ph. D committee § But at least – meet face to face periodically 1 December 2006 SSP Core Facility

Department of Statistics Final Thought § Most Statistics departments are seriously understaffed – UNL no exception § Not a complaint – just a reality § Conflicting demands − teaching − research − consulting § Applied statistics need interdisciplinary work – just as quality research needs engaged statistics § BUT – you may have to be a nag 1 December 2006 SSP Core Facility

that’s all thanks Any Questions?