ETD

Structured Strategies for Learning and Exploration in Sequential Decision Making

Public Deposited
Analytics

202 pages

Title
  • Structured Strategies for Learning and Exploration in Sequential Decision Making
Date modified
  • 08/25/2025
Creator
Abstract
  • Solving Markov decision processes (MDPs) efficiently is challenging in many cases, for example, when the state space or action space is large, when the reward function is sparse and delayed, and when there is a distribution of MDPs. Structures in the policy, value function, reward function, or state space can be useful in accelerating the learning process. In this thesis, we exploit structures in MDPs to solve them effectively and efficiently. First, we study problems with concave value function and basestock policy and leverage these two structures to propose an approximate dynamic programming (ADP) algorithm. Next, we study the exploration problem in unknown MDPs, introduce structured intrinsic reward to the problem, and propose a Bayes-optimal algorithm for learning the intrinsic reward. Finally, we move to problems with structured state space (slow and fast state), build a hierarchical model which exploits the structure, and propose ADP algorithms for the hierarchical model.
Administrative Note
Date created
Identifier
Keyword
Language
Resource type
Degree discipline
  • Industrial Engineering
Degree grantor
  • University of Pittsburgh
Degree level
  • Doctoral
Degree name
  • Doctor of Philosophy
Rights statement

MLA citation style (9th ed.)

Wang, Yijia. Structured Strategies for Learning and Exploration In Sequential Decision Making. . 2022. d-scholarship.pitt.edu/concern/etds/e797c2f0-2a80-40dd-9a26-c2fd98ee08d5.

APA citation style (7th ed.)

W. Yijia. (2022). Structured Strategies for Learning and Exploration in Sequential Decision Making. https://d-scholarship.pitt.edu/concern/etds/e797c2f0-2a80-40dd-9a26-c2fd98ee08d5

Chicago citation style (CMOS 17, author-date)

Wang, Yijia. Structured Strategies for Learning and Exploration In Sequential Decision Making. 2022. https://d-scholarship.pitt.edu/concern/etds/e797c2f0-2a80-40dd-9a26-c2fd98ee08d5.

Note: These citations are programmatically generated and may be incomplete.

In Collection:
Thumbnail Title Date Uploaded Visibility Actions
42524_wangyj_etdPitt2022.pdf 42524_wangyj_etdPitt2022.pdf 2025-08-22 Public Download