HONG KONG BAPTIST UNIVERSITY
FACULTY OF SCIENCE

Department of Computer Science Seminar
2019 Series

Markov Decision Processes: Robustness and Modern Applications

Dr. Clint Chin Pang Ho
Research Fellow
Imperial College Business School
UK

Date: March 6, 2019 (Wednesday)
Time: 4:30 - 5:30 pm
Venue: SCT909, Cha Chi Ming Science Tower, Ho Sin Hang Campus

Abstract
Since the 1950s, Markov decision processes (MDPs) have been a mainstream approach to model dynamic decision problems under uncertainty. Despite their long history and many successful practical implementations, the application of MDPs is challenged by the requirement to specify the underlying stochastic process exactly.

In the first part of my talk, I propose robust MDPs to alleviate the burden of precisely specifying this stochastic process. Instead of requiring the transition probabilities to be known precisely, robust MDPs assume that these probabilities are merely known to be contained in an uncertainty set (that can be estimated from domain knowledge or past data). By optimizing in view of the worst transition probabilities within this uncertainty set, robust MDPs yield decisions that hedge against estimation errors. Unfortunately, robust MDPs pose significant computational challenges if solved with traditional techniques. To this end, I will propose several efficient algorithms that solve robust MDPs in nearly at the same time complexity as standard (non-robust) MDPs.

In the second part of this talk, I will discuss a real-world application in which a personalized marketing strategy has to be derived from customer data. I model this problem via customer-specific MDPs, and I will discuss how efficient solutions can be determined even for instances with large numbers of customers.

Biography
Clint Chin Pang Ho is a Junior Research Fellow (now known as Imperial College Research Fellow) in the Imperial College Business School. Before that, he received a BS in Applied Mathematics (Summa Cum Laude) from the University of California, Los Angeles (UCLA) in 2011, an MSc in Mathematical Modeling and Scientific Computing from the University of Oxford in 2012, and a PhD in computational optimization from Imperial College London in 2016. Clint was a visiting researcher at the University of Texas at Austin in 2018, and he is currently visiting the Chinese University of Hong Kong. Clint studies optimization algorithms and computational methods for structured problems, as well as their applications in machine learning, operations research, and business analytics.

********* ALL INTERESTED ARE WELCOME ***********
(For enquiry, please contact Computer Science Department at 3411 2385)

http://www.comp.hkbu.edu.hk/v1/?page=seminars&id=512
Photos  Slides