Towards a Learning Health System: Using Reinforcement Learning to Optimize Treatment Decisions in Sepsis PatientsKennedy, Jason Neal (2021) Towards a Learning Health System: Using Reinforcement Learning to Optimize Treatment Decisions in Sepsis Patients. Master's Thesis, University of Pittsburgh. (Unpublished) This is the latest version of this item.
AbstractSepsis, a syndrome defined by dysregulated host immune response to infection and acute organ dysfunction, affects 1.7 million Americans annually and accounts for more than 1 in 5 deaths worldwide. International clinical practice guidelines recommend early sepsis identification and a one-size-fits-all treatment bundle of broad spectrum anti-microbials, intravenous (IV) fluids, and vasopressors. Emerging evidence suggests, however, that an individualized, precision treatment approach may improve early sepsis care. We developed a precision treatment policy for IV fluids and vasopressors in early sepsis using model-free Q-learning in clinical Electronic Health Record (EHR) data. We analyzed 30,687 patients presenting with Sepsis-3 within 6 hours of hospital arrival using features in the EHR from 14 UPMC hospitals between 2013-2017. We extracted 38 model features (e.g., demographics, vital signs, laboratory variables) in 4-hour timesteps from hospital arrival until 48-hours after estimated sepsis onset. We defined patient states using K-means clustering and defined an action space that was a 5 5 matrix of IV fluid and vasopressor doses, including no drug administered and doses divided into observed dose quartiles. Awards and penalties were applied maximizing 90-day patient survival. We assessed model performance using weighted importance sampling and demonstrated that the expected value of the Q-learning model treatment policy was significantly higher than that of human clinicians. We demonstrated that model performance in patient- and hospital- level subgroups mostly greatly exceeded clinician performance among subgroups of older patients, those with higher illness severity, and history of recent hospitalization. In conclusion, we demonstrate that patients with early sepsis treated per a precision treatment policy of IV fluids and vasopressors developed using model-free Q-learning may have improved 90-day survival compared to those treated by standard protocol. Precision sepsis treatment strategies should be explored further, including among key clinical subgroups. Public Health Significance: Sepsis is an important public health problem; even small care improvements may make a significant global impact. We demonstrate that a precision treatment strategy using IV fluids and vasopressors may improve sepsis patient survival. These results serve as the foundation for future study, including the development of clinical decision support tools for use at the bedside. Share
Details
Available Versions of this Item
MetricsMonthly Views for the past 3 yearsPlum AnalyticsActions (login required)
|
![[feed]](/images/feed-icon-32x32.png){kind=icon}