Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor

Forbes-Riley, K and Litman, D (2011) Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Communication, 53 (9-10). 1115 - 1136. ISSN 0167-6393

[img] Plain Text (licence)
Available under License : See the attached license file.

Download (1kB)


We evaluate the performance of a spoken dialogue system that provides substantive dynamic responses to automatically detected user affective states. We then present a detailed system error analysis that reveals challenges for real-time affect detection and adaptation. This research is situated in the tutoring domain, where the user is a student and the spoken dialogue system is a tutor. Our adaptive system detects uncertainty in each student turn via a model that combines a machine learning approach with hedging phrase heuristics; the learned model uses acoustic-prosodic and lexical features extracted from the speech signal, as well as dialogue features. The adaptive system varies its content based on the automatic uncertainty and correctness labels for each turn. Our controlled experimental evaluation shows that the adaptive system yields higher global performance than two non-adaptive control systems, but the difference is only significant for a subset of students. Our system error analysis indicates that noisy affect labeling is a major performance bottleneck, yielding fewer than expected adaptations thus lower than expected performance. However, the percentage of received adaptation correlates with higher performance over all students. Moreover, when uncertainty is accurately recognized and adapted to, local performance is significantly improved. © 2011 Elsevier B.V. All rights reserved.


Social Networking:
Share |


Item Type: Article
Status: Published
CreatorsEmailPitt UsernameORCID
Forbes-Riley, K
Litman, Ddlitman@pitt.eduDLITMAN
Centers: Other Centers, Institutes, Offices, or Units > Learning Research & Development Center
Date: 1 November 2011
Date Type: Publication
Journal or Publication Title: Speech Communication
Volume: 53
Number: 9-10
Page Range: 1115 - 1136
DOI or Unique Handle: 10.1016/j.specom.2011.02.006
Schools and Programs: Dietrich School of Arts and Sciences > Computer Science
Dietrich School of Arts and Sciences > Intelligent Systems
Refereed: Yes
ISSN: 0167-6393
Date Deposited: 30 Jul 2014 20:41
Last Modified: 02 Feb 2019 15:59


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item