Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Use of Area Under the Curve (AUC) from Propensity Model to Estimate Accuracy of the Estimated Effect of Exposure

Zhang, Zhijiang (2007) Use of Area Under the Curve (AUC) from Propensity Model to Estimate Accuracy of the Estimated Effect of Exposure. Master's Thesis, University of Pittsburgh. (Unpublished)

[img]
Preview
PDF
Primary Text

Download (673kB) | Preview

Abstract

Objective: To investigate the relationship between the area under the Receiver Operating Characteristic curve (AUC) of the propensity model for exposure and the accuracy of the estimated effect of the exposure on the outcome of interest.Methods: A Monte Carlo simulation study was performed where multiple realizations of three binary variables: outcome, exposure of interest and a covariate were repeatedly generated from the distribution determined by the parameters of the "propensity" and "main" models and the prevalence of the exposure. "Propensity" model was a logistic regression with the exposure of interest as a dependent variable and a single covariate as an "independent" variable. "Main" model was a logistic regression with outcome as a dependent variable, exposure of interest and covariate as "independent" variables. A total of 500 simulations were performed for each considered combination of the model parameters and the prevalence of the exposure. AUC was estimated from the probabilities predicted by the propensity score model. The accuracy of the estimated effect of exposure was primarily assessed with the square root of Mean Square Error (RMSE); the fifth and ninety-fifth percentile of the empirical distribution of the estimator were used to illustrate a range of not unlikely deviations from the true value.Results: The square root of Mean Square Error of the estimated effect of exposure increases as AUC increases from 0.6 to 0.9. Varying values for parameters of the propensity score model or the main effect model does not change the direction of this trend. As the proportion of exposed subjects changes away from 0.5 the RMSE increases, but the effect of AUC on RMSE remains approximately the same. Similarly, as sample size changes from 50 to 100 or 200, the RMSE of effect estimate decreases on average, but the effect of AUC on RMSE remains approximately the same. Also, the rate of change in RMSE increases with increasing AUC; the rate is the lowest when AUC changes from 0.6 to 0.7 and is highest when AUC changes from 0.8 to 0.9.Conclusions: The AUC of the propensity score model for exposure provides a single, relatively easy to compute, and suitable for various kind of data statistic, which can be used as an important indicator of the accuracy of the estimated effect of exposure on the outcome of interest. The public health importance is that it can be considered as an alternative to the previously suggested (Rubin, 2001) simultaneous consideration of the conditions of closeness of means and variances of the propensity scores in the different exposure groups. Our simulations indicate that the estimated effect of exposure is highly unreliable if AUC of the propensity model is larger than 0.8; at the same time AUCs of less than 0.7 are not associated with any substantial increase of inaccuracy of the estimated effect of exposure.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Zhang, Zhijiangzhz23@pitt.edu; zhijiang2004@yahoo.comZHZ23
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairBandos, Andriyanb61@pitt.eduANB61
Committee MemberTang, Gonggot1@pitt.eduGOT1
Committee MemberKip, Kevin Ekipk@edc.pitt.edu
Date: 21 August 2007
Date Type: Completion
Defense Date: 11 July 2007
Approval Date: 21 August 2007
Submission Date: 30 July 2007
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Institution: University of Pittsburgh
Schools and Programs: Graduate School of Public Health > Biostatistics
Degree: MS - Master of Science
Thesis Type: Master's Thesis
Refereed: Yes
Uncontrolled Keywords: Accuracy; AUC; Propensity; Logistic regression; MSE
Other ID: http://etd.library.pitt.edu/ETD/available/etd-07302007-164554/, etd-07302007-164554
Date Deposited: 10 Nov 2011 19:55
Last Modified: 15 Nov 2016 13:47
URI: http://d-scholarship.pitt.edu/id/eprint/8735

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item