Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Towards a Learning Health System: Using Reinforcement Learning to Optimize Treatment Decisions in Sepsis Patients

Kennedy, Jason Neal (2021) Towards a Learning Health System: Using Reinforcement Learning to Optimize Treatment Decisions in Sepsis Patients. Master's Thesis, University of Pittsburgh. (Unpublished)

This is the latest version of this item.

Download (1MB) | Preview


Sepsis, a syndrome defined by dysregulated host immune response to infection and acute organ dysfunction, affects 1.7 million Americans annually and accounts for more than 1 in 5 deaths worldwide. International clinical practice guidelines recommend early sepsis identification and a one-size-fits-all treatment bundle of broad spectrum anti-microbials, intravenous (IV) fluids, and vasopressors. Emerging evidence suggests, however, that an individualized, precision treatment approach may improve early sepsis care.

We developed a precision treatment policy for IV fluids and vasopressors in early sepsis using model-free Q-learning in clinical Electronic Health Record (EHR) data. We analyzed 30,687 patients presenting with Sepsis-3 within 6 hours of hospital arrival using features in the EHR from 14 UPMC hospitals between 2013-2017. We extracted 38 model features (e.g., demographics, vital signs, laboratory variables) in 4-hour timesteps from hospital arrival until 48-hours after estimated sepsis onset. We defined patient states using K-means clustering and defined an action space that was a 5  5 matrix of IV fluid and vasopressor doses, including no drug administered and doses divided into observed dose quartiles. Awards and penalties were applied maximizing 90-day patient survival. We assessed model performance using weighted importance sampling and demonstrated that the expected value of the Q-learning model treatment policy was significantly higher than that of human clinicians. We demonstrated that model performance in patient- and hospital- level subgroups mostly greatly exceeded clinician performance among subgroups of older patients, those with higher illness severity, and history of recent hospitalization.

In conclusion, we demonstrate that patients with early sepsis treated per a precision treatment policy of IV fluids and vasopressors developed using model-free Q-learning may have improved 90-day survival compared to those treated by standard protocol. Precision sepsis treatment strategies should be explored further, including among key clinical subgroups.

Public Health Significance: Sepsis is an important public health problem; even small care improvements may make a significant global impact. We demonstrate that a precision treatment strategy using IV fluids and vasopressors may improve sepsis patient survival. These results serve as the foundation for future study, including the development of clinical decision support tools for use at the bedside.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Kennedy, Jason Nealjnk28@pitt.edujnk280000-0001-7604-899X
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairBuchanich,
Committee CoChairTang,
Committee MemberCarlson, Jenna
Committee MemberChang, Chung-Chou
Committee MemberSeymour, Christopher
Committee MemberYouk,
Date: 12 May 2021
Date Type: Publication
Defense Date: 26 April 2021
Approval Date: 12 May 2021
Submission Date: 28 April 2021
Access Restriction: 1 year -- Restrict access to University of Pittsburgh for a period of 1 year.
Number of Pages: 89
Institution: University of Pittsburgh
Schools and Programs: School of Public Health > Biostatistics
Degree: MS - Master of Science
Thesis Type: Master's Thesis
Refereed: Yes
Uncontrolled Keywords: Sepsis, Critical Care, Reinforcement Learning, Q-learning, Precision Medicine, Personalized Care, Clinical Decision Support
Date Deposited: 13 May 2021 02:30
Last Modified: 12 May 2022 05:15

Available Versions of this Item

  • Towards a Learning Health System: Using Reinforcement Learning to Optimize Treatment Decisions in Sepsis Patients. (deposited 13 May 2021 02:30) [Currently Displayed]


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item