Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

An evaluation of decision-theoretic tutorial action selection

Murray, Robert Charles (2005) An evaluation of decision-theoretic tutorial action selection. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Primary Text

Download (1MB) | Preview


A novel decision-theoretic architecture for intelligent tutoring systems, DT Tutor (DT), was fleshed out into a complete ITS and evaluated. DT uses a dynamic decision network to probabilistically look ahead to anticipate how its tutorial actions will influence the student and other aspects of the tutorial state. It weighs its preferences regarding multiple competing objectives by the probabilities that they will occur and then selects the tutorial action with maximum expected utility. The evaluation was conducted in two phases. First, logs were recorded from interactions of students with a Random Tutor (RT) that was identical to DT except that it selected randomly from relevant tutorial actions. The logs were used to learn many of DT's key probabilities for its model of the tutorial state. Second, the logs were replayed to record the actions that DT and a Fixed-Policy Tutor (FT) would select for a large sample of scenarios. FT was identical to DT except that it selected tutorial actions by emulating the fixed policies of Cognitive Tutors, which are theoretically based, widely used, and highly effective. The possible action selections for each scenario were rated by a panel of judges who were skilled human tutors. The main hypotheses tested were that DT's action selections would be rated higher than FT's and higher than RT's. This was the first comparison of a decision-theoretic tutor with a non-trivial competitor. DT was rated higher than FT overall and for all subsets of scenarios except help requests, for which it was rated equally. DT was also rated much higher than RT. The judges preferred that the tutors provide proactive help and the study design permitted this information to be put to use right away to develop and evaluate enhanced versions of DT and FT. The enhanced versions of DT and FT were rated about equally and higher than non-enhanced DT except on help requests. The variability of the actions selected by both non-enhanced and enhanced versions of DT demonstrated more sensitivity to the tutorial state than the actions selected by non-enhanced and enhanced versions of FT.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Murray, Robert
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairVanLehn, Kurt
Committee MemberCooper, Gregory
Committee MemberAshley, Kevin Dashley@pitt.eduASHLEY
Committee MemberDruzdzel, Marek J.marek@sis.pitt.eduDRUZDZEL
Date: 5 October 2005
Date Type: Completion
Defense Date: 15 July 2005
Approval Date: 5 October 2005
Submission Date: 18 August 2005
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Institution: University of Pittsburgh
Schools and Programs: Dietrich School of Arts and Sciences > Intelligent Systems
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: calculus; decision theory; proactive help; student model; tutor model; utility; artificial intelligence; Bayesian network
Other ID:, etd-08182005-131235
Date Deposited: 10 Nov 2011 20:00
Last Modified: 15 Nov 2016 13:49


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item