Ma, Hua
(2014)
Evaluation of diagnostic performance using partial area under the roc curve.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
Evaluation of diagnostic performance is critical in many fields including but not limited to diagnostic medicine. The Receiver Operating Characteristic (ROC) curve is the most widely used methodology for describing the intrinsic performance of diagnostic tests, with the area under the curve (AUC) being the most commonly used summary index of overall performance. The partial area under the ROC curve (pAUC), when focused on the range of practical/clinical relevance, is considered a more relevant summary index than the full AUC. However, several conceptual and analytical difficulties frequently prevent the pAUC from being used. First, in many diagnostic setting the relevant range is difficult to determine objectively. Second, in theory, due to potential use of less information, analysis based on the pAUC could lead to the loss of statistical precision and therefore would require larger sample sizes. Through mathematical derivation, extensive simulation studies and practical examples, this work investigates statistical properties when using the pAUC. First, this work demonstrates that in many practical scenarios inferences based on pAUC could be more powerful than inferences based on the full AUC. Thus, the use of the pAUC may lead to not only more clinically relevant but also more conclusive results in analyses of experimental data. Second, this investigation demonstrates that the advantages of pAUC-based inferences depend on the shape of ROC curves. The conventional binormal model does not always adequately describe scenarios where the pAUC is more statistically efficient. The bi-gamma family of concave ROC curves is shown to describe practically reasonable scenarios where either pAUC or full AUC could be advantageous. Programs for sample size estimation based on bi-gamma model are then developed. Finally, this work investigates the properties of pAUC-based inferences in scenarios where diagnostic results have substantial ties (or a "mass") at the lowest diagnostic results. For certain type of the ROC curves the existence of ties could lead to an increase in statistical efficiency. Forcing a diagnostic system to resolve ties could detrimentally affect reliability and conclusiveness of statistical inferences. In conclusion, this work provides investigators with insights into and tools for generating practically relevant conclusions using pAUC. The public health importance of this work stems from the relevance of the ROC analysis at different stages of development and regulatory approval of diagnostic systems in medicine. Enhanced methodology for evaluation of diagnostic accuracy helps in the development of improved diagnostic systems and could accelerate the delivery and clinical adoption of truly beneficial diagnostic technologies and/or clinical practices.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
29 September 2014 |
| Date Type: |
Publication |
| Defense Date: |
4 May 2014 |
| Approval Date: |
29 September 2014 |
| Submission Date: |
3 June 2014 |
| Access Restriction: |
1 year -- Restrict access to University of Pittsburgh for a period of 1 year. |
| Number of Pages: |
127 |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
School of Public Health > Biostatistics |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
diagnostic medicine; Receiver Operating Characteristic (ROC) curve; AUC; partial area under the ROC curve (pAUC); bi-gamma; binormal; straight-line; ROC curve with mass |
| Date Deposited: |
29 Sep 2014 21:34 |
| Last Modified: |
15 Nov 2016 14:20 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/21753 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}