Murray, Robert Charles
(2005)
An evaluation of decision-theoretic tutorial action selection.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
A novel decision-theoretic architecture for intelligent tutoring systems, DT Tutor (DT), was fleshed out into a complete ITS and evaluated. DT uses a dynamic decision network to probabilistically look ahead to anticipate how its tutorial actions will influence the student and other aspects of the tutorial state. It weighs its preferences regarding multiple competing objectives by the probabilities that they will occur and then selects the tutorial action with maximum expected utility. The evaluation was conducted in two phases. First, logs were recorded from interactions of students with a Random Tutor (RT) that was identical to DT except that it selected randomly from relevant tutorial actions. The logs were used to learn many of DT's key probabilities for its model of the tutorial state. Second, the logs were replayed to record the actions that DT and a Fixed-Policy Tutor (FT) would select for a large sample of scenarios. FT was identical to DT except that it selected tutorial actions by emulating the fixed policies of Cognitive Tutors, which are theoretically based, widely used, and highly effective. The possible action selections for each scenario were rated by a panel of judges who were skilled human tutors. The main hypotheses tested were that DT's action selections would be rated higher than FT's and higher than RT's. This was the first comparison of a decision-theoretic tutor with a non-trivial competitor. DT was rated higher than FT overall and for all subsets of scenarios except help requests, for which it was rated equally. DT was also rated much higher than RT. The judges preferred that the tutors provide proactive help and the study design permitted this information to be put to use right away to develop and evaluate enhanced versions of DT and FT. The enhanced versions of DT and FT were rated about equally and higher than non-enhanced DT except on help requests. The variability of the actions selected by both non-enhanced and enhanced versions of DT demonstrated more sensitivity to the tutorial state than the actions selected by non-enhanced and enhanced versions of FT.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
5 October 2005 |
| Date Type: |
Completion |
| Defense Date: |
15 July 2005 |
| Approval Date: |
5 October 2005 |
| Submission Date: |
18 August 2005 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
Dietrich School of Arts and Sciences > Intelligent Systems |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
calculus; decision theory; proactive help; student model; tutor model; utility; artificial intelligence; Bayesian network |
| Other ID: |
http://etd.library.pitt.edu/ETD/available/etd-08182005-131235/, etd-08182005-131235 |
| Date Deposited: |
10 Nov 2011 20:00 |
| Last Modified: |
15 Nov 2016 13:49 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/9173 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}