TU Darmstadt / ULB / TUbiblio

Entropic Risk Measure in Policy Search

Nass, David ; Belousov, Boris ; Peters, Jan (2022)
Entropic Risk Measure in Policy Search.
International Conference on Intelligent Robots and Systems (IROS). Macau, China (03.-08.11.2019)
doi: 10.26083/tuprints-00020551
Conference or Workshop Item, Secondary publication, Postprint

Abstract

With the increasing pace of automation, modern robotic systems need to act in stochastic, non-stationary, partially observable environments. A range of algorithms for finding parameterized policies that optimize for long-term average performance have been proposed in the past. However, the majority of the proposed approaches does not explicitly take into account the variability of the performance metric, which may lead to finding policies that although performing well on average, can perform spectacularly bad in a particular run or over a period of time. To address this shortcoming, we study an approach to policy optimization that explicitly takes into account higher order statistics of the reward function. In this paper, we extend policy gradient methods to include the entropic risk measure in the objective function and evaluate their performance in simulation experiments and on a real-robot task of learning a hitting motion in robot badminton.

Item Type: Conference or Workshop Item
Erschienen: 2022
Creators: Nass, David ; Belousov, Boris ; Peters, Jan
Type of entry: Secondary publication
Title: Entropic Risk Measure in Policy Search
Language: English
Date: 2022
Place of Publication: Darmstadt
Year of primary publication: 2022
Publisher: IEEE
Book Title: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Collation: 6 Seiten
Event Title: International Conference on Intelligent Robots and Systems (IROS)
Event Location: Macau, China
Event Dates: 03.-08.11.2019
DOI: 10.26083/tuprints-00020551
URL / URN: https://tuprints.ulb.tu-darmstadt.de/20551
Corresponding Links:
Origin: Secondary publication service
Abstract:

With the increasing pace of automation, modern robotic systems need to act in stochastic, non-stationary, partially observable environments. A range of algorithms for finding parameterized policies that optimize for long-term average performance have been proposed in the past. However, the majority of the proposed approaches does not explicitly take into account the variability of the performance metric, which may lead to finding policies that although performing well on average, can perform spectacularly bad in a particular run or over a period of time. To address this shortcoming, we study an approach to policy optimization that explicitly takes into account higher order statistics of the reward function. In this paper, we extend policy gradient methods to include the entropic risk measure in the objective function and evaluate their performance in simulation experiments and on a real-robot task of learning a hitting motion in robot badminton.

Status: Postprint
URN: urn:nbn:de:tuda-tuprints-205513
Classification DDC: 000 Generalities, computers, information > 004 Computer science
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Intelligent Autonomous Systems
TU-Projects: EC/H2020|640554|SKILLS4ROBOTS
Date Deposited: 22 Nov 2022 09:52
Last Modified: 23 Nov 2022 08:07
PPN:
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details