Flentge, Felix (2005)
Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning.
Johannes Gutenberg-Universität Mainz
Ph.D. Thesis, Bibliographie

Abstract

This doctoral thesis deals with the development of a function approximator and its application to methods for learning discrete and continuous actions:

1. A general function approximator ? Locally Weighted Interpolating Growing Neural Gas (LWIGNG) ? is developed from Growing Neural Gas (GNG). The topological neighbourhood structure is used for calculating interpolations between neighbouring neurons and for applying a local weighting scheme. The capabilities of this method are shown in several experiments, with special considerations given to changing target functions and changing input distributions.

2. To learn discrete actions LWIGNG is combined with Q-Learning forming the Q-LWIGNG method. The underlying GNG-algorithm has to be changed to take care of the special order of the input data in action learning. Q-LWIGNG achieves very good results in experiments with the pole balancing and the mountain car problems, and good results with the acrobot problem.

3. To learn continuous actions a REINFORCE algorithm is combined with LWIGNG forming the ReinforceGNG method. An actor-critic architecture is used for learning from delayed rewards. LWIGNG approximates both the state-value function and the policy. The policy is given by the situation dependent parameters of a normal distribution. ReinforceGNG is applied successfully to learn continuous actions of a simulated 2-wheeled robot which has to intercept a rolling ball under certain conditions.

Send an inquiry

Show editorial Details