Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Evaluation of diagnostic performance using partial area under the roc curve

Ma, Hua (2014) Evaluation of diagnostic performance using partial area under the roc curve. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

[img]
Preview
PDF
Submitted Version

Download (1MB) | Preview

Abstract

Evaluation of diagnostic performance is critical in many fields including but not limited to diagnostic medicine. The Receiver Operating Characteristic (ROC) curve is the most widely used methodology for describing the intrinsic performance of diagnostic tests, with the area under the curve (AUC) being the most commonly used summary index of overall performance. The partial area under the ROC curve (pAUC), when focused on the range of practical/clinical relevance, is considered a more relevant summary index than the full AUC. However, several conceptual and analytical difficulties frequently prevent the pAUC from being used. First, in many diagnostic setting the relevant range is difficult to determine objectively. Second, in theory, due to potential use of less information, analysis based on the pAUC could lead to the loss of statistical precision and therefore would require larger sample sizes. Through mathematical derivation, extensive simulation studies and practical examples, this work investigates statistical properties when using the pAUC. First, this work demonstrates that in many practical scenarios inferences based on pAUC could be more powerful than inferences based on the full AUC. Thus, the use of the pAUC may lead to not only more clinically relevant but also more conclusive results in analyses of experimental data. Second, this investigation demonstrates that the advantages of pAUC-based inferences depend on the shape of ROC curves. The conventional binormal model does not always adequately describe scenarios where the pAUC is more statistically efficient. The bi-gamma family of concave ROC curves is shown to describe practically reasonable scenarios where either pAUC or full AUC could be advantageous. Programs for sample size estimation based on bi-gamma model are then developed. Finally, this work investigates the properties of pAUC-based inferences in scenarios where diagnostic results have substantial ties (or a "mass") at the lowest diagnostic results. For certain type of the ROC curves the existence of ties could lead to an increase in statistical efficiency. Forcing a diagnostic system to resolve ties could detrimentally affect reliability and conclusiveness of statistical inferences. In conclusion, this work provides investigators with insights into and tools for generating practically relevant conclusions using pAUC. The public health importance of this work stems from the relevance of the ROC analysis at different stages of development and regulatory approval of diagnostic systems in medicine. Enhanced methodology for evaluation of diagnostic accuracy helps in the development of improved diagnostic systems and could accelerate the delivery and clinical adoption of truly beneficial diagnostic technologies and/or clinical practices.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Ma, Huahum9@pitt.eduHUM9
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairBandos, Andriyanb61@pitt.eduANB61
Committee MemberGur, Davidgur@pitt.eduGUR
Committee MemberChang, Chung Chouchangj@pitt.eduCHANGJ
Committee MemberJeong, Jong Hyeonjeong@nsabp.pitt.eduJJEONG
Committee MemberRockette, Howard E.herbst@pitt.eduHERBST
Date: 29 September 2014
Date Type: Publication
Defense Date: 4 May 2014
Approval Date: 29 September 2014
Submission Date: 3 June 2014
Access Restriction: 1 year -- Restrict access to University of Pittsburgh for a period of 1 year.
Number of Pages: 127
Institution: University of Pittsburgh
Schools and Programs: School of Public Health > Biostatistics
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: diagnostic medicine; Receiver Operating Characteristic (ROC) curve; AUC; partial area under the ROC curve (pAUC); bi-gamma; binormal; straight-line; ROC curve with mass
Date Deposited: 29 Sep 2014 21:34
Last Modified: 15 Nov 2016 14:20
URI: http://d-scholarship.pitt.edu/id/eprint/21753

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item