标题：Computing monotone policies for Markov decision processes: a nearly-isotonic penalty approach * * This work was partially supported by the Swedish Research Council under contract 2016-06079 and the Linnaeus Center ACCESS at KTH.
摘要：AbstractThis paper discusses algorithms for solvingMarkov decision processes(MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by exploiting the monotone property The first stage is a linear program formulated in terms of the joint state-action probabilities. The second stage is a regularized problem formulated in terms of the conditional probabilities of actions given states. The regularization uses techniques fromnearly-isotonic regression. While a variety of iterative method can be used in the first formulation of the problem, we show in numerical simulations that, in particular, thealternating method of multipliers(ADMM) can be significantly accelerated using the regularization step.
关键词：Keywordsstochastic controlMarkov decision process (MDP)l1-regularizationsparsitymonotone policyalternating direction method of multipliers (ADMM)isotonic regression
