Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Effective Computer Access with Plan-Based Intelligent Screen Reader

Ma, Lin (2003) Effective Computer Access with Plan-Based Intelligent Screen Reader. Master's Thesis, University of Pittsburgh. (Unpublished)

Primary Text

Download (250kB) | Preview


The intelligent screen reader system observes a user¡¯s keystrokes when he or she is performing tasks, and infers an underlying plan structure. The system then generates an optimized script that allows the user to perform the same task more efficiently in the future. The intelligent screen reader system is particularly beneficial for visually impaired computer users.The JAWS for Windows screen reader was chosen for this prototype, since the JAWS scripting language provides flexibility to users for customizing their tasks within a specific application. The JAWS macro recorder, developed by Freedom Scientific Inc., records all the user¡¯s actions. Automated Synthesis of Plan Recognition Networks (ASPRN), developed by Intelligent Reasoning Systems Inc., takes a plan or a script representation as input and outputs a specially-constructed belief network that supports plan recognition. The Script Generation Interface, developed by the University of Pittsburgh, allows users to modify the actions recorded by the JAWS macro recorder and generate an optimized script. Development of the SGI involved integrating the SGI with the JAWS macro recorder, integrating the SGI with ASPRN, and implementing the SGI user interface.Preliminary usability testing involving the intelligent screen reader system was conducted with visually impaired users. Within limited actions and tasks, the results are satisfactory. Users felt the SGI was easy to learn and operate, and that it was efficient to create useful scripts with the intelligent screen reader system. Future work includes adding more plans to the plan library and creating a tutorial system.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Ma, Linlimst12@pitt.eduLIMST12
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairSimpson, Richardris20@pitt.eduRIS20
Committee MemberLitman, Dianelitman@cs.pitt.eduDLITMAN
Committee MemberBuning, Mary Ellenmbuning@pitt.eduMBUNING
Committee MemberMcCue, Michaelmmccue@pitt.eduMMCCUE
Date: 19 December 2003
Date Type: Completion
Defense Date: 9 December 2003
Approval Date: 19 December 2003
Submission Date: 13 December 2003
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Institution: University of Pittsburgh
Schools and Programs: School of Health and Rehabilitation Sciences > Health and Rehabilitation Sciences
Degree: MS - Master of Science
Thesis Type: Master's Thesis
Refereed: Yes
Uncontrolled Keywords: intelligent interface; computer access; plan recognition
Other ID:, etd-12132003-221824
Date Deposited: 10 Nov 2011 20:10
Last Modified: 15 Nov 2016 13:54


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item