Experimental Methods in Nonmarket Valuation Kyrre Rickertsen Based

Experimental Methods in Nonmarket Valuation Kyrre Rickertsen Based on Book Chapter with Frode Alfnes Workshop in National Chung Cheng University in Chia-Yi, Taiwan March 8, 2011 2111 2005

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Outline Presentation § Introduction § Incentive compatible (IC) valuation mechanisms § Validity of bid and choices – Laboratory versus field experiments – Internal validity (design and other issues) – External validity – validity of results outside the laboratory § Some empirical results § Recommendations 2 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Introduction 1 § When (high-quality) market data are available, they are usually preferred to experimental data § Market data are frequently unavailable – New product or product with new characteristics • In peak year 1995, 16, 900 food and beverage products introduced in the US (Nestle 2002) • Controversial technologies used in production – Growth hormones, GMO, irradiation, and cloning – New labeling or information • Nutritional, organic, and sustainable labeling 3 – Maybe little variability in market data (e. g. , prices) – Effects of socioeconomic variables? www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Introduction 2 § Alternatives are: – Stated preference data • Can ask about nonexistent products BUT – Lack economic incentives – Hypothetical bias (e. g. , List and Gallet 2001) – Experimental data (hypothetical market data) § Experiments also used to develop economic theory, for example: – WTP – WTA disparity (e. g. , Shogren et al. 1994 - AER) – Preference reversals (e. g. , List 2002 - AER) 4 – Coherent arbitrariness (Ariely et al. 2003 – QJE) www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Introduction 3 § In experimental valuation studies, participants make bids or choices with real products and real money § Experiments used for a long time in marketing: – As a method of research, controlled experimentation is universally used among scientists. They conduct experiments on a small scale to discover facts from which overall conclusions may be reached. In the gradual evolution of marketing research from an art to a science, some practitioners have endeavored to employ experimental methods where possible. William Applebaum and Richard F. Spears (1950: 505) 5 – The first scientific article describing a test marketing experiment is Ginzberg (1936) in AER. – Test marketing was the preferred method • New products or marketing strategies tested in a few stores www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Introduction 4 § Applied economists started to use experimental methods around 1990 to study the value of quality attributes of foods (e. g. , Menkhaus et al. 1992 – JARE; Buhr et al. 1993 – JARE) – Little reference to previous marketing literature – Focus on WTP while previous marketing literature focused on market shares – This line of research is in the focus of this presentation 6 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 1 § A mechanism is incentive compatible (IC) given that the utility of the participant cannot be increased by submitting a different bid or making a different choice § IC mechanisms include (complete listing see table 1): – Sealed bid auctions • 2 nd price, nth price, random nth price – Becker-De. Groot-Marschak (BDM) mechanism – Price list experiments – Real choice (RC) experiments § Non-IC mechanisms include: 7 – Dutch and first-price auction – The price paid is not independent of the winner’s bid www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 2 Sealed-Bid Auctions with Endogenous Market Prices § 2 nd price (Vickrey 1961) – Participants submit bids and one unit of the good is sold to the highest bidder for a price equal to second-highest bid – Two problems • May not engage low-value bidders (especially in a multiple trial setting with posted prices) • Some may see auction as competition with one winner § A random nth price auction where n-1 units of the good is sold for the nth highest bid – Engage high- and low-value bidders 8 – n -1 winners reduce any auction winning utilities – Lottery effect? www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 3 Sealed-Bid Auctions with Endogenous Market Prices § In the endowment approach, the participants are endowed with a product and bid for an upgrade (Hayes et al. 1995) – The bid reflects the difference in value between the two alternatives – The bid (and thereby the value difference) may be affected by all sorts of anchors § In full bidding approach, the participants bid on two (or more) products and one is randomly chosen for sale (Hoffmann et al. 1993) – The difference in bids reflects the difference in value between the two alternatives – Bid differences less affected by anchors 9 § See Lusk and Shogren (2007: 95 -112) and Alfnes (2009) for detailed discussions www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 4 Becker-De. Groot-Marschak (BDM) Mechanism § No auction but strategically equivalent to IC auctions § Each participant submits a sealed bid § Sales price is randomly drawn from a distribution from zero to a price that is higher than the anticipated max bid § If bid higher than price, then the participant purchases one unit for the drawn price § Can be conducted with only one participant – Easy to do at a grocery store § Lottery element? 10 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 5 Price List Experiments § a) Real dichotomous choice – Offer the participant to buy a commodity for a price. If yes, sale is completed § b) Multiple price list (payment card method) – A list of questions on the form • “At a price of $8. 75 I will buy _____ I will not buy ____” • One row is implemented as the final sale § c) Open-ended choice experiment – How many units do you want to purchase at different prices? – Can calculate demand curves 11 – But people tend to stock up on storable products at low prices and the interpretation of the demand curve becomes difficult www. umb. no

NORWEGIAN UNIVERSITY OF LIFE SCIENCES Department of Economics and Resource Management IC Valuation Mechanisms 6 Real Choice Experiments 12 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 7 Real Choice Experiments § Extension of stated choice experiment – But your choices have real economic consequences – Need to have the product § Participants make choices in a series of scenarios § Product attributes (including prices) vary between scenarios § One scenario randomly drawn as binding § Advantage: The choice task is similar to choices made in a food store § Weakness: Individual WTP is not observable – WTP has to be estimated based on choices of all participants – Estimated WTP for each participants affected by the choices of other participants 13 – Estimated WTP for each participants affected by model specification www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 8 Comparisons of Mechanisms § In theory, sealed bid auctions and BDM should result in identical bids and bid differences – In practice, not always the case § Induced value experiments – Sealed-bid auctions produce more accurate bids than BDM (Noussair et al. 2004; Lusk and Rousu 2006) § Homegrown value experiments – Impossible to test whether bids equal homegrown values (private and unknown to experimenter) – Many comparisons with mixed results (see paper) 14 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES IC Valuation Mechanisms 9 Choice of Mechanism § Choice of mechanism a trade-off – Auctions perform better than BDM in induced value experiments – 2 nd price auction perform well for high-value bidders while random nth price performs better for low-value bidders • Recently most studies use n > 2 – BDM easy to implement on individual basis and in the field – RC experiments resembles choices in grocery stores but do not give individual WTP values 15 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Laboratory versus Field Experiments § In marketing literature – Laboratory experiments are conducted in a laboratory setting with a high degree of control of external factors – Field experiments are conducted in an actual market place with less control but a familiar context for the specific choice § Harrison and List (2004) have a refined categorization depending on – Students versus general population – Field goods versus induced values – Field context versus laboratory 16 – Subjects know or do not know that they participate in an experiment www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Internal Validity 1 § Internal validity: – The ability to demonstrate that observed correlations are causal (Roe and Just 2009) – Does the experiment control for alternative explanations? § Some factors affecting the internal validity – Training is important (Plott and Zeiler 2005) • Most IC mechanisms are unfamiliar and small differences may affect the bids as discussed above (2 nd vs. nth price) – Presence of field substitutes (Harrison et al. 2004) • Bids are truncated by prices of field substitutes • But is cod the only field substitute to cod? 17 – Or is fish in general or maybe meat the field substitute? • Bid differences may be more robust than bid levels www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Internal Validity 2 § Bid affiliation and posted prices in multiple-round auctions § In induced value experiments, it has been shown that it takes several rounds to reach theoretical WTP § What about homegrown value experiments? Increasing prices typically observed – We have bid affiliation when bids in later rounds are affected by posted prices in earlier rounds • Some participants may try to buy at low prices and discover that it does not work, i. e. , “market discovery” • Some participants may take high posted prices as a signal of quality and revise their bids, i. e. , “preference learning” 18 • Some participants may consider the auction as one competition, i. e. , “auction winning utilities” www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Internal Validity 3 § No easy solution to bid affiliation § Problem is the interpretation of the bids (and the internal validity) § One-shot auctions with training with a different product is recommended by Harrison et al. (2004) – Do we find the “correct” WTP for unfamiliar food products in the first shot? – Does it solve the problem to use a different training product? • Anchors • Incidental prices • Coherent arbitrariness 19 § Recent papers seem to use few but more than one round www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Internal Validity 4 § Participants use numbers that are presented to them as anchors § Examples on anchors – Reference price of field substitute – Distribution of prices in price list experiments – The limits of the price distribution in BDM – Bids in training sessions – Posted prices 20 www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices Internal Validity 5 § Arbitrary numbers can also serve as anchors – Nunes and Boatwright (2004) found that exposing the participants to prices that were completely unrelated to the goods in an auction (incidental prices) had substantial effects on the bids (e. g. , sweatshirt price affected the bids for CDs) – Ariely et al. (2003) found that absolute bids on ordinary products (like computer keyboards) were affected by random numbers such as the participants’ social security numbers • Participants were asked to write down their social security number before bidding. High numbers resulted in high bids in the following auction (arbitrariness). • The relative valuations were not affected (coherent) 21 § Given random anchors, we recommended to use the full bidding approach and relative WTP (or alternatively bid differences when many zero bids) www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices External Validity 1 § External validity – The ability to generalize the relationships found in a study to other contexts (Roe and Just 2009). For example, from lab to market – Mixed results; see paper § Several factors may reduce the external validity § Unfamiliar environment: – People may behave different in a laboratory than a retail setting (Levitt and List 2007) – People are not in laboratory because they need to buy food – The time and day of the experiment are decided by researcher 22 • Corrigan and Rousu (2008) found that people that intended to buy bananas on the day of the experiment bid more, i. e. , regular buyers may behave as non-buyers in an experiment – Experiments in a store setting may be preferred but lack of control and limited possibilities to trainwww. umb. no participants

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices External Validity 2 § Nature of decision task different in store and experiment – Only a small number of products – A lot of focus on the (few) products – Can typically only buy one unit of one product – May have to bid for products instead of choosing among them – Random draws may resemble lotteries § Representative samples – Self-selected students may behave differently than representative consumers • Low age, low income, more education, strong beliefs – Nonstudent samples may be unrepresentative outside sample area 23 – Users may have different preferences than the general population • For many food products, 10% of users buy almost everything www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices External Validity 3 § Information – In experiments there is usually a focus on a few products and, consequently, few prices. Information is typically provided (e. g. , Power. Point etc). • Participants encouraged to make “rational” decisions using the information • Small samples encourage the use of within- rather than betweensample tests – Within-sample tests draw attention to the treatment and gives larger effects than between-sample tests (Johansson-Stenman and Svedsäter 2008) 24 – In food store many products, less emphasis on information about the product and many prices • Many purchases based on habits www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Validity of Bid and Choices External Validity 4 § Presence of researchers – People may change behavior when they are scrutinized • Behave like they believe the experimenter want • Give a socially desirable impression – Especially for product attributes with dimensions such as animal friendly, “fair trade”, GMO, etc § Repeated purchase – People may bid high because they want to try the product (preference learning) 25 – A high premium in the laboratory does not necessarily imply repeated purchase of the good www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Examples on Empirical Results § Food marketing issues: Packing technologies, use of insecticides, use of different types of feed, grading for tenderness, labels etc – Segments of consumers with different preferences – Typically some increased mean WTP for new product • Do not investigate the profitability of the new product § Controversial technologies: GMO, hormone treatment, irradiation – Segments of consumers with different preferences – US participants more positive than European § Effects of information – Moderate effects of scientifically balanced information – Stronger effects of information from environmental groups, etc 26 – Effects of negative information dominate effects of positive information www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Our Recommendations 1 No Universal Agreement § Do not deceive participants or lie to them – You do not need to tell participants everything but what you tell them should be true § Use representative consumers – Students differ from other consumers • Likely to have other preferences concerning sensory quality of foods, attitudes towards animal welfare or controversial technologies • Different sensitivity for prices – Student sample are OK for testing designs and theoretical studies – No purpose in including vegetarians in a study of beef attributes 27 • Consumption constraint www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Our Recommendations 2 § Make sure the participants understand the mechanism – Many procedures are unfamiliar and have lottery elements • Inform and train participants by using the same mechanism and number of goods as in the experiment § Calculate relative WTP values – According to micro theory relative prices are important – Systematic anchoring effects will cancel out – WTP differences can be used with many zero bids § Use a context as free of scrutiny as possible 28 – Let participants feel relaxed and not scrutinized. Fill in any forms in a room allowing for some privacy and let the participant identify themselves only through numbers that are unknown for the other participants www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Our Recommendations 3 § Use mechanisms with strong real economic incentives – If you have 15 participants there are stronger incentives in a median price than a second price auction § Collect background information – Socioeconomic, attitudinal, and knowledge variables can be used to define segments – Segments of interest for policy makers and industry § Let participants taste unfamiliar products – Tasting reduces sensory uncertainty, which potentially either can reduce bids (due to risk aversion) or increase bids (participant wants to taste a new product) 29 – Tasting results in increased external validity www. umb. no

School of Economics and Business NORWEGIAN UNIVERSITY OF LIFE SCIENCES Our Recommendations 4 § Treat all products equally – Rotate order of presentation between sessions – Taste all or none of the products – Use full bidding format in experimental auctions § Delete participants with a nonresponse to all alternatives, i. e. , always bid zero or choose “none of the alternatives” – May reflect that participant does not like any of products – May reflect inconvenience, lack of trust in the experiment, or pure laziness § Let participants evaluate multiple items of heterogeneous products 30 – To avoid deception by letting the participants bid on as many items as there are participants and, next, let each of the participants draw one item as their binding product www. umb. no

Thank You for Your Attention! 2111 2005