Measurement 17 871 Spring 2003 Topics in

Measurement 17. 871 Spring 2003

Topics in Measurement • • From abstraction to measure Sources of error What to do about error Practical ways to improve measurement

The Mapping Theory Observation X Y x y ex ey

Mapping from the Abstract to the Measurement • Some abstract things we try to measure – Alienation – Moral decay – Democracy – Party identification – Fear of defeat – Terrorism – Ideology

Sources of Error in Measurement (Mosteller) • • • Conceptual or design error Bad breaks in random sampling Survey question wording Non-random out-selection Transcription errors Calculation & mechanization errors

What to Do About Error • Practice safe data – Know where your data come from – Watch for anomalies – Use multiple measurement techniques – Collect as much data as possible and disaggregate

Practical things to do about measurement • Distinction between a measure and an indicator – Measure: straightforward quantification of a variable of interest – Indicator: quantification of a variable that is believed (or known) to be highly correlated with the “real” variable of interest

Examples of Measures • • • Income in $$ Age in years Votes Number of wars (hmmmmmm…. . ) Campaign contributions • Measurement issues tend to focus on the quality of the data-gathering method, especially sampling

Examples of Indicators • Public opinion – Party identification – Trust in government – Ideology • Economics – Gross domestic (national) product • Characterizations of political systems – Freedom – Transparency – Democracy

How do we generate indicators? • Trust others and hope for the best – GDP, etc. – Presidential approval • Use a single measure and hope for the best (or convince ourselves it’s OK) – – 7 -point ideology scale 7 -point party identification scale Codings of “wars” or “rally events” Bulletin of the Atomic Scientist “nuclear clock” • Use multiple measures creatively

Multiple-measure indicators • “Trust in government” battery – How much of the time do you think you can trust the government in Washington to do what is right? – Would you say the government is pretty much run by a few big interests looking out for themselves or that it is run for the benefit of all the people? – Do you think that people in the government waste a lot of money we pay in taxes, waste some of it, or don’t waste very much of it? – Do you think that quite a few of the people running the government crooked, not very many are, or do you think hardly any of them are crooked? • External political efficacy battery – Sometimes politics and government seem so complicated that a person like me can’t really understand what’s going on – People like me don’t have any say about what the government does – Public officials don’t care much what people like me think.

Trust and Efficacy over Time

Other Multiple-Variable Indicators • Americans for Democratic Action Support Scores • Freedom House freedom assessment • MIT Teaching Quality • Transparency International’s “Corruption Perceptions Index” • DNominate Scores

Scaling Or, How to Create Multiple Indicators More Generally

Imagine an unobserved factor that gives rise to observable factors Democracy Healthiness Racism Conservatism Religiosity Measure 1 b 1 Unobserved factor b 2 bn b 1 Polyarchy b 2 b 3 b 4 e 1 Measure 2 . . . • • • Measure n Fairness of elections e 3 e 1 Meaningfulness of elections e 2 Freedom of expression e 3 Unbiased reporting of official pronouncements e 4

Fairness of elections Freedom of expression Two-factor (pure) polyarchy example A B Polyarchy C D E F G H Polyarchy I J

Fairness of elections What We Observe in the Pure Case Freedom of expression

Fairness of elections Freedom of expression Two-factor (impure) polyarchy example A B Polyarchy C D E F G H Polyarchy I J

Fairness of elections What We Observe in the Impure Case Polyarchy

Bhutan Congo, Lebanon Hungary USSR, Libya, Iran, China Jamaica, Greece Egypt, Sri Lanka, Mexico Argentina, U. S. , Finland Peru, India, Israel Source: http: //www. nd. edu/~mcoppedg/crd/polydat. htm

Simple way to create multipleindicator scale with just 1 factor