Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

A Reinforcement Learning approach to evaluating state representations in spoken dialogue systems

Tetreault, JR and Litman, DJ (2008) A Reinforcement Learning approach to evaluating state representations in spoken dialogue systems. Speech Communication, 50 (8-9). 683 - 696. ISSN 0167-6393

[img] Plain Text (licence)
Available under License : See the attached license file.

Download (1kB)


Although dialogue systems have been an area of research for decades, finding accurate ways of evaluating different systems is still a very active subfield since many leading methods, such as task completion rate or user satisfaction, capture different aspects of the end-to-end human-computer dialogue interaction. In this work, we step back the focus from the complete evaluation of a dialogue system to presenting metrics for evaluating one internal component of a dialogue system: its dialogue manager. Specifically, we investigate how to create and evaluate the best state space representations for a Reinforcement Learning model to learn an optimal dialogue control strategy. We present three metrics for evaluating the impact of different state models and demonstrate their use on the domain of a spoken dialogue tutoring system by comparing the relative utility of adding three features to a model of user, or student, state. The motivation for this work is that if one knows which features are best to use, one can construct a better dialogue manager, and thus better performing dialogue systems. © 2008 Elsevier B.V. All rights reserved.


Social Networking:
Share |


Item Type: Article
Status: Published
CreatorsEmailPitt UsernameORCID
Tetreault, JR
Litman, DJdlitman@pitt.eduDLITMAN
Centers: Other Centers, Institutes, Offices, or Units > Learning Research & Development Center
Date: 1 August 2008
Date Type: Publication
Journal or Publication Title: Speech Communication
Volume: 50
Number: 8-9
Page Range: 683 - 696
DOI or Unique Handle: 10.1016/j.specom.2008.05.002
Schools and Programs: Dietrich School of Arts and Sciences > Computer Science
Dietrich School of Arts and Sciences > Intelligent Systems
Refereed: Yes
ISSN: 0167-6393
Date Deposited: 30 Jul 2014 20:43
Last Modified: 02 Feb 2019 15:59


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item