LIS 397 1 Introduction to Research in Library

LIS 397. 1 Introduction to Research in Library and Information Science Introduction to Survey Techniques R. E. Wyllys Copyright 2003 by R. E. Wyllys Last revised 2003 Feb 3 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Experiments vs. Surveys • Experiments – Important variables can be controlled – Experiments can often be repeated with different values of important variables • Surveys – Important variables cannot be controlled – Surveys must try to gather enough data to include different values of important variables School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Steps in the Research Process • Decide on objectives • Formulate hypotheses • Decide how to test – What data needed? – How to analyze? – How to collect? • Review feasibility of testing hypotheses • Design data collection & analysis • Review feasibility of testing hypotheses • Pilot-test data-collection instruments • Collect data • Process data • Perform statistical and other analyses • Assemble results and decide whether hypotheses are true • Write up results; draw conclusions; make recommendations School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Data Collection via Surveys • Surveys – Gathering of data (observations) by direct contact with people • In all forms of surveys, people know they are being observed • Hence, survey data are subject to interference by the Hawthorne Effect School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

The Hawthorne Effect People who know that they are being observed modify their behavior not only consciously but also unconsciously School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

The Hawthorne Effect • Hawthorne Effect interferes with data collection by surveys • Interference is unavoidable: people know they are responding • Alternative: Unobtrusive Observation – At best, difficult – At worst, useless (without mind-reading) • Solution: collect survey data but be alert to Hawthorne Effect interference School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

The Hawthorne Effect • Example of problems with unobtrusive observation – Library patron’s borrowed item can be unobtrusively identified from charge-out transaction record – But item may be borrower’s 4 th choice not 1 st choice, may have been borrowed for self or for someone else, etc. School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

The Hawthorne Effect • Example of problems with known (i. e. , obtrusive) observation – Library patron’s borrowed item can be identified by asking him or her at departure from circulation desk – Borrower may be willing to say whether item was borrowed for someone else – Borrower may not be willing to say what his or her 1 st choice really would have been School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Person-to-Person Interaction • Exists in addition to Hawthorne Effect • Involves – Voice inflections – Facial expressions – Body language • Is inevitable, even with best efforts by surveyor to minimize interaction • Can be detrimental but also useful School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Interaction can Involve Cultural Differences that are Hard to Recognize • “Comfortable” face-to-face distance when people are free to move • Direct vs. Averted gaze – Averted gaze: a sign of respect in some cultures, of dishonesty in others • When are you late for an appointment? – Arab vs. German vs. American cultures – Differences within U. S. School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Surveys & Level of Interaction • A spectrum of types of surveys – Questionnaire: mail, email, Web-based (minimal person-to-person interaction) • Web-based questionnaires are a relatively recent but increasingly important form of questionnaire. An excellent overview of their use is: Gunn, H. Web-based Surveys: Changing the Survey Process. First Monday. 2002 December; 7(12) – Group-administered questionnaire – Self-administered questionnaire – Interview: face-to-face, telephone (maximal person -to-person interaction) School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Data-Collection Alternatives Compared Factor Face-to-Face Cost Time Sample size/Dollar Data quantity/Respndnt Widely dispersed sample Reaches special locations Interaction with respndnt? Degree of interviewer bias Non-response bias Visual stimuli presented? Fieldworker training reqrd? High Med. Small High No Yes High Low Yes Phone Mail Med. Low Med. Maybe Yes Med. Low No Yes Low High Large Low Yes No No None High Maybe No From: Alreck, P. ; Settle, R. The Survey Research Handbook. 2 nd ed. 1995 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Surveys Categorized by Purpose • Enumerative (exploratory, descriptive) Surveys – Aim to find out general characteristics of target group • Analytic Surveys – Aim to gather data that will serve in test(s) of hypothesis(ses) School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Surveys Categorized by Target • Exhaustive surveys – Aim for response from every member of group (population) being studied – Are likely to fail in that aim, unless surveyor has power over target group • Sample surveys – Aim for response from every member of a sample selected from target group – May succeed in that aim if well planned and conducted School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Survey Techniques • Sample surveys are – Usually preferable over exhaustive surveys – Require answers to two questions • What population (group) do you want to make inferences about? • How can you obtain a reasonable (i. e. , sufficiently large and sufficiently random) sample of your target population? School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Types of Sampling • General Random Sampling – Choose your sample elements randomly from entire population • Stratified Sampling – Define strata (e. g. , ethnic, income, occupational) within your population; then sample randomly from each stratum • Cluster Sampling – Subsets of population defined geographically • Quota Sampling – Interviewers seek specified numbers of respondents from each stratum School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Types of Sampling • General random sampling is the easiest to handle mathematically • Stratified, cluster, and quota sampling all require special mathematical handling; you should seek expert advice if you decide these methods are needed – In many institutional settings such advice is available from within the institution: e. g. , • Planning staffs within school systems and cities • Marketing staffs within companies • Mathematics faculty within universities and community colleges School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Dealing with Non-Response • Responding to a questionnaire is usually voluntary – Respondents tend to be those with strong feelings, pro or con, about the subject; hence, initial responses may be biased • Follow-up needed to check for bias – Select sample of initial non-respondents – Intensively seek responses from them – If returns from initial non-respondents look like those of initial respondents, you’re OK; if not, you’re in trouble School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Identifying Non-Respondents for Follow-Up • How to identify non-respondents in anonymous surveys? – Method 1: Mark questionnaires with individual IDs; cover letter must assure respondents that IDs will be ignored (e. g. , removed and discarded) if they respond – Method 2: With questionnaire enclose postcard with ID; ask respondents to return postcard at same time as they return questionnaires in separate envelopes School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Survey Question Types • Closed: fixed set of possible responses (e. g. , multiple-choice, true or false) • Open (free-answer): response in respondent’s own words School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Closed Questions • Advantage – Easy to process responses • Disadvantages – Hard to set alternative responses initially – Respondents may misunderstand alternatives – Respondent may find no available response that fits his or her situation – Alternatives set tone of responses (cf. economic slant vs. ethnic slant) even if “other” is available – No probing possible outside of initially defined responses School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Open Questions • Advantages – Easy to set up – Respondents can provide unexpected or unusual responses truly suited to their situations • Disadvantages – Hard to process responses – Response affected by space on questionnaire or pace of interview – Subject to “off the top of one’s head” spurious or unreliable responses School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Off the Top of One’s Head Pollsters have for years conceded that the people they question are quick to offer an opinion, but sometimes are equally quick to change it. “They flip-flop like a fish, ” says Daniel Yankelovich, chairman of Yankelovitch, Skelly & White Inc. , a public attitude research concern. [YSW] has devised a new polling. . . index, which comprises four questions [to] determine how well informed, personally affected, deeply considered and generally stable is an interviewee’s opinion on a particular issue. . . It’s called the “mushiness index. ” From: Birnbaum, J. H. “’Mushiness Index’ Devised. ” Wall Street Journal. 1981 March 23 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Neutrality of Questions • Phrase questions as neutrally as possible. – Don’t ask: “Do you believe that hard candies are harmful to the teeth? ”* – Do ask: “Some people say that eating hard candies is bad for your teeth, but others say that it makes no difference. How do you feel about this? ”* • Alternate the order of alternatives, presenting each order to half the respondents *From: Oppenheim, A. N. Questionnaire Design and Attitude Measurement. 1966 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Questions on Attitudes • Attitudes can be assessed by rankings or ratings – Ranking: Whole list of items is labeled in order of respondent’s preference (usually 1 = top); each item gets unique rank – Ratings: Each item is provided a rating on some scale (e. g. , 1 to 5, 1 to 10; usually higher numbers denote greater favor); item ratings need not be unique School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Likert Scales: A Widely Used Way to Assess Attitudes • 1 Strongly agree (or favor) • 2 Agree (or favor) • 3 Neutral • 4 Disagree (or disfavor) • 5 Strongly disagree (or disfavor) • • 1 Very strongly agree 2 Strongly agree 3 Agree 4 Neutral 5 Disagree 6 Strongly disagree 7 Very strongly disagree Note: The extra shades of distinction provided by more than 5 levels are usually useless. School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Verbal Frequency Scales • • • 1 Always 2 Often 3 Sometimes 4 Rarely 5 Never • • 1 Always 2 Almost always 3 Often 4 Sometimes 5 Rarely 6 Almost never 7 Never Note: The extra shades of distinction provided by more than 5 levels are usually useless. School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Nature of Question Sequence • Resist temptation to ask irrelevant questions (even though it might be “nice to know the answer to this question too, as long as we’re going to all the trouble and expense of sending out this questionnaire”) • Irrelevant questions may arouse suspicions of respondents and lead them to toss questionnaire into wastebasket • The longer the questionnaire, the more likely it will wind up in a wastebasket School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Nature of Question Sequence • Don’t begin with personal, specific questions; leave these till later if truly needed • Do begin with easy, impersonal questions relevant to “stated purpose” of survey • Place questions in an order that makes sense to respondent rather than suits convenience of response processors (e. g. , all multiplechoice questions together) • Use “funnel” approach with “filter” questions to enable respondents to skip irrelevant subsets of questions School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Funnel-Approach Questions For instance, suppose we want to know whether some people avoid hard candies because they are said to be harmful to the teeth. It would not do to ask them a question such as, “Do you believe that hard candies are harmful to the teeth? ” or “Do you avoid eating hard candies because you feel that they are harmful to the teeth? ” These would be grossly leading questions. . . Obviously, it would be valuable if we could get the respondent to say spontaneously that he avoids hard candies because they are damaging to the teeth, before we suggest it to him. Therefore, we may start off with some very broad questions, such as: “What is your opinion of hard candies? ” “What do you think of people who eat hard candies? ” Each question provides the respondent with an opportunity to mention the issue of dental decay spontaneously. Next, we might ask more restricted questions, such as: “Do you eat hard candies at all? ” “Do you allow your children to eat hard candies? ” From: Oppenheim, A. N. Questionnaire Design and Attitude Measurement. 1966 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Filter Questions A filter question is used to exclude a respondent from a particular question sequence if those questions are irrelevant to him. [Suppose that] we might wish to ask for some factual information about candybuying behavior and hard-candy purchases. Obviously, if the respondent never buys these sweets then there is no point in asking about frequency, weight, type of shop, type of container, color preferences and so forth. Therefore, our illustrative question sequence will be preceded by a filter question, such as, “Do you buy hard candies from time to time? ” or “Have you bought any hard candies in the last two weeks? ” If the answer is negative, the interviewer will be instructed to skip the next few questions and proceed to the beginning of the next question sequence. From: Oppenheim, A. N. Questionnaire Design and Attitude Measurement. 1966 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Pilot-Test Your Survey • Pilot-testing is essential – No matter how hard you try to frame your questions clearly, somebody will misinterpret at least one of them • Pilot tests must use people like the intended target group (not just your easily available, obliging colleagues) – Don’t pilot-test on college students if your target group is users of a ghetto library School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Special Aspects of Interviews • Interview is an interpersonal situation • Interaction between respondent and interviewer occurs at both conscious and subconscious levels (bear the Hawthorne effect in mind) School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Special Aspects of Interviews • Respondent-interviewer interaction may be both good and bad – Interviewer can clarify questions, detect misunderstandings, follow up on unusual responses, encourage respondent to continue till completion – Interviewer may fail to understand respondent’s responses, may unintentionally mislead respondent – Respondent may misinterpret explicit and/or hidden cues from interviewer School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Interviewer-Respondent Differences in Perception The woman interviewer, in describing her feelings about the male respondent, remarks: “I just didn’t trust the guy. . . He made me creep. . . [He probably prefers] somewhat sadistic” movies. The respondent’s image of the interviewer is that [she was] pleasant, polite, attractive, and. . . “suitable to my idea of an interviewer. . . I’d like to know her better. . . [I think she] liked [me]. I had such a feeling, I don’t know why. ” In addition to [her] intense hostility. . . the interviewer. . . reports that she “was particularly worried and depressed” that day and “in a special hurry to finish the interview” [which] was conducted in the street. From: Hyman, Herbert H. Interviewing in Social Research. 1975 School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Guidelines for Interviews • Interviewer must establish sufficient rapport to put respondent at ease • Interviewer must exercise care against giving respondent clues as to interviewer’s opinions or “expected, ” “right, ” “acceptable” responses • Interviewer must thoroughly understand purposes of survey • Interviewer must practice interviews in pilot tests School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

General Survey Considerations • Respondent needs introduction to survey, including – Sponsorship of survey • Should be respected, neutral institution or organization • Sponsor should differ from focus of survey – Ostensible purpose of survey • Should be respectable • Should be related to real purpose, but real purpose may need to be disguised slightly so that survey can work • Surveyor needs to be alert to ethical complications of sponsorship and purpose School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

Anonymity and Confidentiality • Anonymity – Possible with questionnaires but not interviews – Workarounds with interviews • Provide respondent with questionnaire to be placed in locked box or mailed in postage-paid envelope School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science

It Takes Hard Work to Do a Good Survey! School of Information - The University of Texas at Austin LIS 397. 1, Introduction to Research in Library and Information Science