Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form


Song, Tao (2009) DEVELOPMENT AND COMPARISON OF DIFFERENT METHODS OF EVALUATING FREE-RESPONSE ROC SYSTEMS. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Primary Text

Download (1MB) | Preview


Receiver Operating Characteristic (ROC) analysis has been widely used to evaluate diagnostic systems since the 1970s. In diagnostic imaging the decision task often needs the radiologist to locate the specific region on a subject that actually contains the abnormality. A Free-Response ROC experiment has been more and more accepted for evaluating this type of a diagnostic task. It entails detecting and marking the locations of all suspected abnormalities, as well as indicating a level of suspicion regarding the specific abnormality at each marked location. Several existing approaches of analyzing FROC data used the maximum rating to represent the multiple responses of a subject and then applied an analysis in an ROC concept to summarize the diagnostic system's discriminative ability in a randomly selected pair of actually negative and actually positive subjects. This dissertation proposes and evaluates new methods of subject-based discriminative ability by considering approaches based on the average of multiple ratings and approaches based on the stochastic order. Indices are also formulated by improving the JAFROC-type indices in literature, in order to summarize the diagnostic performance with correct location information. We also propose new indices that can penalize and reward for the number of correct and incorrect marks on the subjects. Asymptotic procedures are developed to compare the discriminative ability between two FROC systems. These asymptotic approaches are then extended to the multi-reader setting, taking into consideration the correlation and heterogeneity between readers. We also apply three different approaches to fit a smooth FROC curve, namely Box-Cox transformation approach, kernel smoothing approach and kernel regression approach. The public health significance of the work lies in our efforts to improve the statistical tools for evaluating medical diagnostic devices, which can help in the development of more specific and affordable diagnostic methods. Our contribution to early diagnosis could improve the timely recognition of reportable diseases.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairRockette, Howard Eherbst@pitt.eduHERBST
Committee MemberBandos, Andriyanb61@pitt.eduANB61
Committee MemberGur,
Committee MemberMazumdar, Satimaz1@pitt.eduMAZ1
Date: 29 January 2009
Date Type: Completion
Defense Date: 4 November 2008
Approval Date: 29 January 2009
Submission Date: 7 November 2008
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Institution: University of Pittsburgh
Schools and Programs: School of Public Health > Biostatistics
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Diagnostic performance; FROC; Nonparametric inference; Summary index
Other ID:, etd-11072008-093805
Date Deposited: 10 Nov 2011 20:04
Last Modified: 15 Nov 2016 13:51


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item