Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

The Experimenters' Dilemma: Inferential Preferences over Populations

Gupta, Neeraja and Rigotti, Luca and Wilson, Alistair (2021) The Experimenters' Dilemma: Inferential Preferences over Populations. (Unpublished)

Download (925kB) | Preview


We compare three populations commonly used in experiments by economists and other social scientists: undergraduate students at a physical location (lab), Amazon's Mechanical Turk (MTurk), and Prolific. The comparison is made along three dimensions: the noise in the data due to inattention, the cost per observation, and the elasticity of response. We draw samples from each population, examining decisions in four one-shot games with varying tensions between the individual and socially efficient choices. When there is no tension, where individual and pro-social incentives coincide, noisy behavior accounts for 60% of the observations on MTurk, 19% on Prolific, and 14% for the lab. Taking costs into account, if noisy data is the only concern Prolific dominates from an inferential power point of view, combining relatively low noise with a cost per observation one fifth of the lab's. However, because the lab population is more sensitive to treatment, across our main PD game comparison the lab still outperforms both Prolific and MTurk.


Social Networking:
Share |


Item Type: Article
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Gupta, Neerajaneg38@pitt.eduneg38
Rigotti, Lucaluca@pitt.eduluca0000-0002-1461-6055
Wilson, Alistairalistair.wilson@pitt.edualistair0000-0001-8118-2376
Date: 2021
DOI or Unique Handle: 10.48550/arxiv.2107.05064
Schools and Programs: Dietrich School of Arts and Sciences > Economics
Refereed: No
Uncontrolled Keywords: General Economics (econ.GN), FOS: Economics and business, FOS: Economics and business
Official URL:
Article Type: Research Article
Date Deposited: 16 May 2022 14:44
Last Modified: 16 May 2022 14:44


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item