Optimisation in the Presence of Model Ambiguity in Markov Decision Process

event calendar icon

14 Nov 2023

11:30 -12:30

Times are shown in local time.

Open to: Members of University of Cambridge

event map pin icon

Room W2.02 (Cambridge Judge Business School)

Trumpington St



United Kingdom

Join our Operations and Technology Management seminar

Speaker: Professor Brian Denton, University of Michigan

About the seminar topic

Optimising sequential decision-making under uncertainty is essential in many contexts, including inventory control, finance, healthcare, and many others. One of the most common model formulations is the Markov decision process (MDP). However, ambiguity in the MDP model parameters can introduce challenges because recommendations from MDPs depend on the underlying model, and there are often multiple plausible models. To address this problem, we present a framework in which a decision-maker considers multiple models of the MDP’s ambiguous parameters and seeks to find a policy that maximises an aggregate measure of performance with respect to each of these models of the MDP, such as weighted rewards, regret, or worst-case performance.

I will discuss connections to other models in the stochastic optimisation literature, complexity results, and solution methods for solving these problems. I’ll illustrate the approach with 2 examples, one in the context of preventative treatment for cardiovascular disease and the other in the context of machine maintenance. Finally, I’ll conclude with a summary of the most important takeaway messages from the study.

Speaker bio

Brian Denton is the Stephen M Pollock Professor of Industrial and Operations Engineering. His research interests are in sequential decision-making and optimisation under uncertainty with applications to healthcare delivery, supply chain management, and other topics related to allocating scarce resources. Before joining the University of Michigan, he worked at IBM, Mayo Clinic, and North Carolina State University. His honours and awards include the National Science Foundation Career Award, the INFORMS Daniel H Wagner Prize, the Institute of Industrial Engineers Outstanding Publication Award, and the Canadian Operations Research Society Best Student Paper Award.

He has served on several editorial boards including Manufacturing and Service Operations Management, Medical Decision Making, Operations Research, and Production and Operations Management. He has co-authored over 100 journal articles, conference proceedings, book chapters, and patents. He is an elected Fellow of INFORMS and IISE and a past President of INFORMS.


If you would like to register, or know more about this event, please email Luke Slater.