TU Darmstadt / ULB / TUbiblio

Correlation Priors for Reinforcement Learning

Alt, B. and Šošić, A. and Koeppl, H. (2019):
Correlation Priors for Reinforcement Learning.
In: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Kanada, 09.12.-13.12.2019, [Online-Edition: https://papers.nips.cc/paper/9564-correlation-priors-for-rei...],
[Conference or Workshop Item]

Abstract

Many decision-making problems naturally exhibit pronounced structures inherited from the characteristics of the underlying environment. In a Markov decision pro- cess model, for example, two distinct states can have inherently related semantics or encode resembling physical state configurations. This often implies locally cor- related transition dynamics among the states. In order to complete a certain task in such environments, the operating agent usually needs to execute a series of tempo- rally and spatially correlated actions. Though there exists a variety of approaches to capture these correlations in continuous state-action domains, a principled solution for discrete environments is missing. In this work, we present a Bayesian learning framework based on Pólya-Gamma augmentation that enables an analogous reasoning in such cases. We demonstrate the framework on a number of common decision-making related problems, such as imitation learning, subgoal extraction, system identification and Bayesian reinforcement learning. By explicitly modeling the underlying correlation structures of these problems, the proposed approach yields superior predictive performance compared to correlation-agnostic models, even when trained on data sets that are an order of magnitude smaller in size.

Item Type: Conference or Workshop Item
Erschienen: 2019
Creators: Alt, B. and Šošić, A. and Koeppl, H.
Title: Correlation Priors for Reinforcement Learning
Language: English
Abstract:

Many decision-making problems naturally exhibit pronounced structures inherited from the characteristics of the underlying environment. In a Markov decision pro- cess model, for example, two distinct states can have inherently related semantics or encode resembling physical state configurations. This often implies locally cor- related transition dynamics among the states. In order to complete a certain task in such environments, the operating agent usually needs to execute a series of tempo- rally and spatially correlated actions. Though there exists a variety of approaches to capture these correlations in continuous state-action domains, a principled solution for discrete environments is missing. In this work, we present a Bayesian learning framework based on Pólya-Gamma augmentation that enables an analogous reasoning in such cases. We demonstrate the framework on a number of common decision-making related problems, such as imitation learning, subgoal extraction, system identification and Bayesian reinforcement learning. By explicitly modeling the underlying correlation structures of these problems, the proposed approach yields superior predictive performance compared to correlation-agnostic models, even when trained on data sets that are an order of magnitude smaller in size.

Divisions: 18 Department of Electrical Engineering and Information Technology
18 Department of Electrical Engineering and Information Technology > Institute for Telecommunications > Bioinspired Communication Systems
18 Department of Electrical Engineering and Information Technology > Institute for Telecommunications
DFG-Collaborative Research Centres (incl. Transregio)
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet > B: Adaptation Mechanisms
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet > B: Adaptation Mechanisms > Subproject B4: Planning
Event Title: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Event Location: Vancouver, Kanada
Event Dates: 09.12.-13.12.2019
Date Deposited: 15 Oct 2019 05:48
Official URL: https://papers.nips.cc/paper/9564-correlation-priors-for-rei...
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)

View Item View Item