Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Belief Elicitation and Behavioral Incentive Compatibility

Danz, David and Vesterlund, Lise and Wilson, Alistair J (2022) Belief Elicitation and Behavioral Incentive Compatibility. American Economic Review. (In Press)

Accepted Version

Download (1MB) | Preview
PDF (Online Appendices)
Supplemental Material

Download (2MB) | Preview
[img] Other (Plain Text Bibliography)

Download (12kB)


Subjective beliefs are crucial for economic inference, yet behavior can challenge the elicitation. We propose that belief elicitation should be incentive compatible not only theoretically but also in a de facto behavioral sense. To demonstrate, we show that the binarized scoring rule, a state-of-the-art elicitation, violates two weak conditions for behavioral incentive compatibility: (i) within the elicitation, information on the incentives increases deviations from truthful reporting; and (ii) in a pure choice over the set of incentives, most deviate from the theorized maximizer. Moreover, we document that deviations are systematic and center-biased, and that the elicited beliefs substantially distort inference.


Social Networking:
Share |


Item Type: Article
Status: In Press
CreatorsEmailPitt UsernameORCID
Danz, Daviddanz@pitt.edudanz0000-0001-5474-445X
Vesterlund, Lisevester@pitt.eduvester0000-0002-3476-3183
Wilson, Alistair Jalistair@pitt.edualistair0000-0001-8118-2376
Date: February 2022
Date Type: Acceptance
Journal or Publication Title: American Economic Review
Schools and Programs: Dietrich School of Arts and Sciences > Economics
Dietrich School of Arts and Sciences > Economics > Economics Working Papers
Refereed: Yes
References: Abdellaoui, Mohammed, Driouchi, Ahmed, and L'Haridon, Olivier, “Risk aversion elicitation: reconciling tractability and bias minimization,” Theory and Decision 71, 1 (2011), pp. 63-80. Abeler, Johannes, Nosenzo, Danielle, Raymond, Collin, “Preferences for Truth-Telling,” Econometrica, 87, 4, (2019), pp. 1-118. Ahrens, Steffen, and Ciril Bosch-Rosa. “The Motivated Beliefs of Investors Under Limited Liability.” Working paper, 2019. Aksoy, Billur, Ian Chadd, and Boon Han Koh. “Hidden Identity and Social Preferences: Evidence from Sexual Minorities.” Working paper, 2021. Aoyagi, Masaki, Guillaume R Fréchette, and Sevgi Yuksel. 2021. “Beliefs in Repeated Games.” Working paper, 2021. Avoyan, Ala and Schotter, Andrew, “Attention in Games: An Experimental Study” (2020), European Economic Review, 124, 103410 (2020), pp. 1-28. Babcock, Linda, Recalde, Maria P, Vesterlund, Lise, and Weingart, Laurie, “Gender differences in accepting and receiving requests for tasks with low promotability,” American Economic Review 107, 3 (2017), pp. 714-47. Benjamin, Daniel, “Errors in Probabilistic Reasoning and Judgment Biases,” in Bernheim Douglas, DellaVigna, Stefano, Laibson, David, eds., Handbook of Behavioral Economics: Applications and Foundations 1, Volume 2, North-Holland, (2019), pp. 69-186. Bhatt, M, Camerer, CF. “Self-referential thinking and equilibrium as states of mind in games: fMRI evidence,” Games and Economic Behavior 52, 2, (2005), pp.424-459. Blanco, M, Engelmann, D, Koch, AK, Normann H-T, “Belief elicitation in experiments: is there a hedging problem?” Experimental Economics, 13, 4, (2010), pp.412-438. Bordalo, Pedro, Coffman, Katherine, Gennaioli, Nicola, and Shleifer Andrei, “Beliefs about Gender,” American Economic Review 109, 3 (2019), pp.739-773. Bruner, David M, “Changing the probability versus changing the reward,” Experimental Economics 12, 4 (2009), pp. 367-385. Burfurd, Ingrid and Wilkening, Tom, “Experimental guidance for eliciting beliefs with the Stochastic Becker-DeGroot-Marschak mechanism.” Journal of the Economic Science Association 4, 1, (2018), pp. 15-28. Cantoni, D., Yang, D.Y., Yuchtman, N, Zhang, YJ., “Protests as Strategic Games: Experimental Evidence from Hong Kong's Antiauthoritarian Movement,” Quarterly Journal of Economics, 134, 2, (2019), pp. 1021-1077. Cason, Timothy N. and Plott, Charles R. “Misconceptions and Game Form Recognition: Challenges to Theories of Revealed Preference and Framing,” Journal of Political Economy, 122 (2014), pp. 1235-1270. Cason, Timothy N., Sharma, Tridib, and Vadovič, Radovan. “Correlated beliefs: Predicting outcomes in 2× 2 games.” Games and Economic Behavior, 122 (2020), pp. 256-276. Castillo, Marco E, Philip J Cross, and Mikhail Freer. 2019. “Nonparametric Utility Theory in Strategic Settings: Revealing Preferences and Beliefs from Proposal-Response Games.” Games and Economic Behavior 115 (2019), pp. 60-82. Charness, Gary, Dufwenberg, Martin “Promises and Partnership,” Econometrica. 74, 6, (2006), pp. 1579-1601. Charness, Gary, Oprea, Ryan, and Yuksel, Sevgi. “How do people choose between biased information sources? Evidence from a laboratory experiment.” Journal of the European Economic Association, 19:3, (2021), pp. 1656-91. Chen, Yan, and YingHua He. n.d. “Information Acquisition and Provision in School Choice: A Theoretical Investigation.” forthcoming Economic Theory, (2021), 10.1007/s00199-021-01376-3 Choi, Syngjoo, Byung-Yeon Kim, Jungmin Lee, and Sokbae Lee. “Institutions, Competitiveness and Cognitive Ability.” Working paper, 2020. Colzani, Paola, and Luıs Santos-Pinto. “Does Overconfidence Lead to Bargaining Failures?” Workding paper, 2020 Corazzini, Luca, Stefano Galavotti, and Paola Valbonesi. “An Experimental Study on Sequential Auctions with Privately Known Capacities.” Games and Economic Behavior 117 (2019), pp. 289-315. Danz, David, Madarász, Kristóf, Wang, Stephanie, “The Biases of Others: Projection Equilibrium in an Agency Setting,” CEPR Discussion Paper, DP12867, (2018). Danz, David, Gupta, Neeraja, Lepper, Marissa, Vesterlund, Lise, and Winichakul, K. Pun, “Going virtual: A step-by-step guide to taking the in-person experimental lab online,” working paper, 2021. Danz, David, Vesterlund, Lise and Wilson, Alistair “Data and Code for : Belief Elicitation and Behavioral Incentive Compatibility.” American Economic Association [publisher], Inter-university Consortium for Political and Social Research [distributor]. Dargnies, Marie-Pierre, Hakimov, Rustamdjan, and Kübler, Dorothea, “Self-Confidence and Unraveling in Matching Markets,” Management Science 65, 12 (2019), pp. 5603-18. Dargnies, Marie-Pierre, “Men Too Sometimes Shy Away from Competition: The Case of Team Competition,” Management Science 58, 11, (2012), pp. 1982-2000. Dean, Mark, and Ortoleva, Pietro, “Allais, Ellsberg, and Preferences for Hedging,” Theoretical Economics 12 (2017):377-424. DeQuidt, Jonathan, Vesterlund, Lise, and Wilson, Alistair, “Experimenter Demand Effects,” Handbook of Research Methods and Applications in Experimental Economics (Edward Elgar Pub., edited by Arthur Schram and Aljaz Ule). Dianat, Ahrash, Echenique, Federico, and Yariv, Leeat, “Statistical Discrimination and Affirmative Action in the Lab,” CEPR Discussion Paper No. DP12915, (2019). Di Tella, Rafael, Perez-Truglia Ricardo, Babino Andres, Sigman Mariano, “Conveniently Upset: Avoiding Altruism by Distorting Beliefs about Others' Altruism,” American Economic Review 105, 11 (2015), pp. 3416-3442. Drerup, Tilman, Enke, Benjamin, and Von Gaudecker, Hans-Martin, “The precision of subjective data and the explanatory power of economic models,” Journal of Econometrics 200, 2 (2017), pp. 378-389. Dustan, Andrew, Kristine Koutout, and Greg Leo. “Second-Order Beliefs and Gender.” Working paper, (2020) Enke, Benjamin, and Graeber, Thomas, “Cognitive Uncertainty”. NBER Working Paper No. 26518, (2019). Enke, Benjamin, Schwerter, Frederik, and Zimmermann, Florian, “Associative memory and Belief Formation,” NBER Working Paper No. 26664, (2020). Erkal, Nisvan, Lata Gangadharan, and Boon Han Koh. 2020. “Replication: Belief Elicitation with Quadratic and Binarized Scoring Rules.” Journal of Economic Psychology 81: 102315. ———. “By Chance or by Choice? Biased Attribution of Others' Outcomes When Social Preferences Matter.” Working paper, (2021). ———. “Attribution biases in leadership: Is it effort or luck?” Working paper, (2019). Filippin, Antonio, and Marco Mantovani. “Risk Aversion and Information Aggregation in Asset Markets.” Working paper (2019). Fischbacher, Urs, “z-Tree: Zurich toolbox for ready-made economic experiments,” Experimental Economics 10, 2 (2007), pp. 171-178. — and Franziska Föllmi-Heusi, “Lies in Disguise—An Experimental Study on Cheating.” Journal of the European Economic Association, 11(3), (2013), pp 525- 547, Frederick, Shane, “Cognitive reflection and decision making,” Journal of Economic Perspectives 19, 4 (2005), pp. 25-42. Gächter, Simon and Renner, Elke, “The effects of (incentivized) belief elicitation in public goods experiments,” Experimental Economics 13, 3 (2010), pp. 364-377. Graeber, Thomas. “Inattentive Inference.” Working paper, (2021). Hao, Li and Houser, Daniel, “Belief Elicitation in the Presence of Naïve Respondents: An Experimental Study,” Journal of Risk and Uncertainty 44, 2 (2012), 161-80. Harrison, Glenn W and Phillips, Richard D, “Subjective beliefs and statistical forecasts of financial risks: The chief risk officer project,” in Contemporary Challenges in Risk Management (Springer, 2014), pp. 163-202. Healy, Paul J, “Epistemic Game Theory Experiments: Utility Elicitation and Irrational Play” mimeo, (2017). Healy, Paul J, “Explaining the BDM—Or any Random Binary Choice Elicitation Mechanism—To Subjects”, mimeo, (2018). Hillenbrand, Adrian and Schmelzer, André, “Beyond information: Disclosure, distracted attention, and investor behavior,” Journal of Behavioral and Experimental Finance 16 (2017), pp. 14-21. Holt, Charles A. and Smith, Angela M., “Belief Elicitation with a Synchronized Lottery Choice Menu That Is Invariant to Risk Attitudes,” American Economic Journal: Microeconomics 8, 1 (2016), pp. 110-39. Hossain, Tanjim and Okui, Ryo, “The binarized scoring rule,” Review of Economic Studies 80, 3 (2013), pp. 984-1001. ———, “Belief formation under signal correlation.” Working paper, (2019). Huffman, David, Raymond, Collin, Shvets, Julia, “Persistent Overconfidence and Biased Memory: Evidence from Managers,” mimeo, (2019). Hurley, TM, Shogren, JF. “An Experimental Comparison of Induced and Elicited Beliefs,” Journal of Risk and Uncertainty, 30, 2, (2005), pp. 169-188. Koutout, Kristine. 2020. “Gendered Beliefs and the Job Application Decision: Evidence from a Large-Scale Field and Lab Experiment.” LeCoq, C, Tremewan, J, Wagner, AK., “On the effects of group identity in strategic environments,” European Economic Review, 76, (2015), pp. 239-252. Li, Shengwu, “Obviously strategy-proof mechanisms.” American Economic Review 107, 11, (2017), pp. 3257-87. Manski, Charles F. "Measuring expectations." Econometrica 72.5 (2004): 1329-1376. Masiliūnas, Aidas, “Overcoming coordination failure in a critical mass game: strategic motives and action disclosure,” Journal of Economic Behavior & Organization 139 (2017), pp. 214-251. Meloso, Debrah, Nunnari, Salvatore, and Ottaviani, Marco, “Looking into crystal balls: a laboratory experiment on reputational cheap talk” Working paper, (2020). Nelson, Robert G and Bessler, David A, “Subjective probabilities and scoring rules: Experimental evidence,” American Journal of Agricultural Economics 71, 2 (1989), pp. 363-69. Niederle, Muriel and Vesterlund, Lise, “Do women shy away from competition? Do men compete too much?” Quarterly Journal of Economics 122, 3 (2007), pp.1067-1101. Offerman, Theo, Sonnemans, Joep, Van De Kuilen, Gijs, and Wakker, Peter P, “A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes,” The Review of Economic Studies, 76, 4 (2009), pp.1461-89. Oprea, Ryan, and Sevgi Yuksel. “Social Exchange of Motivated Beliefs.” Journal of the European Economic Association. Forthcoming, (2021). DOI: 10.1093/jeea/jvab035/6368096 Palfrey, Thomas R and Wang, Stephanie W, “On eliciting beliefs in strategic games,” Journal of Economic Behavior & Organization 71, 2 (2009), pp. 98-109. Rafkin, Charlie, Advik Shreekumar, and Pierre-Luc Vautrey. “When Guidance Changes: Government Stances and Public Beliefs.” Journal of Public Economics 196, (2021). Recalde, Maria, and Lise Vesterlund, “Gender Differences,” (2022), University of Pittsburgh working paper Renes, Sander, and Bauke Visser. “Markets Assessing Decision Makers and Decision Makers Impressing Markets: A Lab Experiment.” Working paper, (2019). Roth, Alvin E and Malouf, Michael W, “Game-theoretic models and the role of information in bargaining,” Psychological Review 86, 6 (1979), pp. 574. Schlag, Karl H and van der Weele, Joël J, “A Method to Elicit Beliefs as Most Likely Intervals,” Judgment and Decision Making 10, 5 (2015), 456-68. Schmidt, Tobias and Zankiewicz, Christian, “Binary Choice Belief Elicitation: An Adaptively Optimal Design” (2016). Schotter, Andrew and Trevino, Isabel, “Belief Elicitation in the Laboratory,” Annual Review of Economics 6,1 (2014) pp. 103-28, Sonsino, Doron, Yaron Lahav, and Amir Levkowitz. 2020. “The Conflicting Links Between Forecast-Confidence and Trading Propensity.” Journal of Behavioral Finance, (2020) pp. 1-18. Toussaert S. “Eliciting Temptation and Self-Control Through Menu Choices: A Lab Experiment,” Econometrica, 86, 3, (2018), pp. 859-889. Trautmann, Stefan T and van de Kuilen, Gijs, “Belief elicitation: A horse race among truth serums,” Economic Journal 125, 589 (2015), pp. 2116-35. Vanberg, C. “Why Do People Keep Their Promises? An Experimental Test of Two Explanations,” Econometrica, 76, 6, (2008), pp. 1467-1480. Wang, Stephanie W, “Incentive effects: The case of belief elicitation from individuals in groups,” Economics Letters 111, 1 (2011), pp. 30-33. Wilcox NT, Feltovich N. “Thinking Like a Game Theorist: Comment,” Mimeo, University of Houston, (2000). Wilson, Alistair J. and Vespa, Emanuel, “Paired-uniform scoring: Implementing a binarized scoring rule with non-mathematical language” (2018). Zimpelmann, Christian. 2021. “Stock Market Beliefs and Portfolio Choice in the General Population.” Working paper, (2021).
Article Type: Research Article
Date Deposited: 28 Feb 2022 19:14
Last Modified: 28 Feb 2022 19:14


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item