Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Integrating Imperfect Machines and Unmindful Users: Assessing Human-Bot Hybrid Designs for Managing Discussions in Online Communities

Fu, Xinyu (2023) Integrating Imperfect Machines and Unmindful Users: Assessing Human-Bot Hybrid Designs for Managing Discussions in Online Communities. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Primary Text

Download (1MB) | Preview


Modern intelligent machines, developed through large-scale data instead of rules, are no longer passive tools waiting to be used, but take proactive actions and work as a co-worker with human agents. They are, admittedly, still imperfect. As a result, machines are often deployed in hybrid "assemblage" configurations that require active human interventions in the workflow. This dissertation examined such a collaborative workflow involving an artificial intelligence (AI) bot and a human to manage inappropriate discussions in the context of online communities. I simulated a bot-assisted news discussion forum where users can affirm or override bot-flagged inappropriate comments. Based on the Information Systems (IS) delegation framework, I examine the main attributes of human agents (cognition), agentic IS artifacts (imperfection), and the delegation mechanisms between these two agents: complacency potential. Cognition, or the "generation effect," occurs when people are asked to explain the bot's activities rather than being told of its flaws. The imperfection has two folds: bots' accuracy and bots' valence. Complacency potential refers to users' tendencies to over-rely on the bot and lack awareness of monitoring the bot's actions. With 1650 subjects over five lab experiments, I found that, on average, users aided by the bot achieved higher decision quality than the ones unaided by the bot. The users that were prompted to provide self-explanation were able to better detect the bot’s errors than others. By deploying the bot at lower accuracy levels, I found that digital platforms may compensate for the lower accuracy of bots by getting users’ active involvement, but there is a threshold level of a bot’s accuracy below which the bot will not improve the performance of users. Furthermore, the users who encountered a positive bot that recommended good content had higher levels of engagement with the online community relative to those who encountered a negative bot that flagged inappropriate content. Lastly, I found that users who provided a self-generated explanation about bots’ actions perceived a higher level of responsibility to monitor the bots’ performance, but remained willing to delegate the moderation work to the bot.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Fu, Xinyuxinyu.fu@pitt.eduxif340000-0001-7871-9953
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee CoChairGalletta, Dennis F.galletta@pitt.edu0000-0003-0442-5500
Committee CoChairRamasubbu, Narayannarayanr@pitt.edu0000-0002-6959-0729
Committee MemberGunarathne, Priyangapriyanga.gunarathne@katz.pitt.edu0000-0001-7003-7536
Committee MemberKirsch, Laurie
Committee MemberNah, Fiona
Date: 2 July 2023
Date Type: Publication
Defense Date: 14 April 2023
Approval Date: 2 July 2023
Submission Date: 1 July 2023
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 120
Institution: University of Pittsburgh
Schools and Programs: Joseph M. Katz Graduate School of Business > Business Administration
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Human-AI collaboration; algorithmic errors; error detection; generation effect; complacency potential
Date Deposited: 02 Jul 2023 22:47
Last Modified: 02 Jul 2023 22:47


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item