Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form
D-Scholarship @ Pitt Banner and Links to Homepage

A natural language intelligent tutoring system for training pathologists: Implementation and evaluation

El Saadawi, GM and Tseytlin, E and Legowski, E and Jukic, D and Castine, M and Fine, J and Gormley, R and Crowley, RS (2008) A natural language intelligent tutoring system for training pathologists: Implementation and evaluation. Advances in Health Sciences Education, 13 (5). 709 - 722. ISSN 1382-4996

[img] Plain Text (licence)
Available under License : See the attached license file.
Download (1kB)

Abstract

Introduction We developed and evaluated a Natural Language Interface (NLI) for an Intelligent Tutoring System (ITS) in Diagnostic Pathology. The system teaches residents to examine pathologic slides and write accurate pathology reports while providing immediate feedback on errors they make in their slide review and diagnostic reports. Residents can ask for help at any point in the case, and will receive context-specific feedback. Research questions We evaluated (1) the performance of our natural language system, (2) the effect of the system on learning (3) the effect of feedback timing on learning gains and (4) the effect of ReportTutor on performance to self-assessment correlations. Methods The study uses a crossover 2 × 2 factorial design. We recruited 20 subjects from 4 academic programs. Subjects were randomly assigned to one of the four conditions-two conditions for the immediate interface, and two for the delayed interface. An expert dermatopathologist created a reference standard and 2 board certified AP/CP pathology fellows manually coded the residents' assessment reports. Subjects were given the opportunity to self grade their performance and we used a survey to determine student response to both interfaces. Results Our results show a highly significant improvement in report writing after one tutoring session with 4-fold increase in the learning gains with both interfaces but no effect of feedback timing on performance gains. Residents who used the immediate feedback interface first experienced a feature learning gain that is correlated with the number of cases they viewed. There was no correlation between performance and self-assessment in either condition. © 2007 Springer Science+Business Media B.V.

Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: Article
Status: Published
Creators/Authors:
CreatorsEmailPitt UsernameORCID
El Saadawi, GM
Tseytlin, E
Legowski, E
Jukic, D
Castine, M
Fine, J
Gormley, R
Crowley, RS
Date: 1 December 2008
Date Type: Publication
Journal or Publication Title: Advances in Health Sciences Education
Volume: 13
Number: 5
Page Range: 709 - 722
DOI or Unique Handle: 10.1007/s10459-007-9081-3
Schools and Programs: School of Medicine > Biomedical Informatics
Refereed: Yes
ISSN: 1382-4996
MeSH Headings: Computer-Assisted Instruction--methods; Computer-Assisted Instruction--standards; Feedback, Psychological; Humans; Internship and Residency; Models, Educational; Natural Language Processing; Pathology--education; Problem-Based Learning--methods; Program Evaluation; Self-Assessment; User-Computer Interface; Writing--standards
Other ID: NLM NIHMS119432, NLM PMC2753375
PubMed Central ID: PMC2753375
PubMed ID: 17934789
Date Deposited: 05 Oct 2012 18:52
Last Modified: 02 Feb 2019 14:55
URI: http://d-scholarship.pitt.edu/id/eprint/15628

Metrics

Monthly Views for the past 3 years

Plum Analytics

Altmetric.com

Actions (login required)

View Item View Item