Koert, Dorothea ; Kircher, Maximilian ; Salikutluk, Vildan ; D'Eramo, Carlo ; Peters, Jan (2021)
Multi-Channel Interactive Reinforcement Learning for Sequential Tasks.
In: Frontiers in Robotics and AI, 2020, 7
doi: 10.26083/tuprints-00019239
Artikel, Zweitveröffentlichung, Verlagsversion
 | Es ist eine neuere Version dieses Eintrags verfügbar. |
Kurzbeschreibung (Abstract)
The ability to learn new tasks by sequencing already known skills is an important requirement for future robots. Reinforcement learning is a powerful tool for this as it allows for a robot to learn and improve on how to combine skills for sequential tasks. However, in real robotic applications, the cost of sample collection and exploration prevent the application of reinforcement learning for a variety of tasks. To overcome these limitations, human input during reinforcement can be beneficial to speed up learning, guide the exploration and prevent the choice of disastrous actions. Nevertheless, there is a lack of experimental evaluations of multi-channel interactive reinforcement learning systems solving robotic tasks with input from inexperienced human users, in particular for cases where human input might be partially wrong. Therefore, in this paper, we present an approach that incorporates multiple human input channels for interactive reinforcement learning in a unified framework and evaluate it on two robotic tasks with 20 inexperienced human subjects. To enable the robot to also handle potentially incorrect human input we incorporate a novel concept for self-confidence, which allows the robot to question human input after an initial learning phase. The second robotic task is specifically designed to investigate if this self-confidence can enable the robot to achieve learning progress even if the human input is partially incorrect. Further, we evaluate how humans react to suggestions of the robot, once the robot notices human input might be wrong. Our experimental evaluations show that our approach can successfully incorporate human input to accelerate the learning process in both robotic tasks even if it is partially wrong. However, not all humans were willing to accept the robot's suggestions or its questioning of their input, particularly if they do not understand the learning process and the reasons behind the robot's suggestions. We believe that the findings from this experimental evaluation can be beneficial for the future design of algorithms and interfaces of interactive reinforcement learning systems used by inexperienced users.
| Typ des Eintrags: | Artikel |
|---|---|
| Erschienen: | 2021 |
| Autor(en): | Koert, Dorothea ; Kircher, Maximilian ; Salikutluk, Vildan ; D'Eramo, Carlo ; Peters, Jan |
| Art des Eintrags: | Zweitveröffentlichung |
| Titel: | Multi-Channel Interactive Reinforcement Learning for Sequential Tasks |
| Sprache: | Englisch |
| Publikationsjahr: | 2021 |
| Publikationsdatum der Erstveröffentlichung: | 2020 |
| Verlag: | Frontiers |
| Titel der Zeitschrift, Zeitung oder Schriftenreihe: | Frontiers in Robotics and AI |
| Jahrgang/Volume einer Zeitschrift: | 7 |
| Kollation: | 19 Seiten |
| DOI: | 10.26083/tuprints-00019239 |
| URL / URN: | https://tuprints.ulb.tu-darmstadt.de/19239 |
| Zugehörige Links: | |
| Herkunft: | Zweitveröffentlichung aus gefördertem Golden Open Access |
| Kurzbeschreibung (Abstract): | The ability to learn new tasks by sequencing already known skills is an important requirement for future robots. Reinforcement learning is a powerful tool for this as it allows for a robot to learn and improve on how to combine skills for sequential tasks. However, in real robotic applications, the cost of sample collection and exploration prevent the application of reinforcement learning for a variety of tasks. To overcome these limitations, human input during reinforcement can be beneficial to speed up learning, guide the exploration and prevent the choice of disastrous actions. Nevertheless, there is a lack of experimental evaluations of multi-channel interactive reinforcement learning systems solving robotic tasks with input from inexperienced human users, in particular for cases where human input might be partially wrong. Therefore, in this paper, we present an approach that incorporates multiple human input channels for interactive reinforcement learning in a unified framework and evaluate it on two robotic tasks with 20 inexperienced human subjects. To enable the robot to also handle potentially incorrect human input we incorporate a novel concept for self-confidence, which allows the robot to question human input after an initial learning phase. The second robotic task is specifically designed to investigate if this self-confidence can enable the robot to achieve learning progress even if the human input is partially incorrect. Further, we evaluate how humans react to suggestions of the robot, once the robot notices human input might be wrong. Our experimental evaluations show that our approach can successfully incorporate human input to accelerate the learning process in both robotic tasks even if it is partially wrong. However, not all humans were willing to accept the robot's suggestions or its questioning of their input, particularly if they do not understand the learning process and the reasons behind the robot's suggestions. We believe that the findings from this experimental evaluation can be beneficial for the future design of algorithms and interfaces of interactive reinforcement learning systems used by inexperienced users. |
| Status: | Verlagsversion |
| URN: | urn:nbn:de:tuda-tuprints-192395 |
| Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Intelligente Autonome Systeme |
| Hinterlegungsdatum: | 04 Aug 2021 08:03 |
| Letzte Änderung: | 09 Aug 2021 07:28 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
Verfügbare Versionen dieses Eintrags
- Multi-Channel Interactive Reinforcement Learning for Sequential Tasks. (deposited 04 Aug 2021 08:03) [Gegenwärtig angezeigt]
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum