List of Videos

Open loop control of the Barrel
Barrel performing a “lolloping” behavior
Emerging cooperativity in a chain of TwoWheeleds
The chain of robots with decentralized control in a regular maze
Sensitivity and adaptation to environmental changes
Destabilization and Emergence.
The role of embodiment and situatedness with wheeled robots.
Chain of wheeled robots exploring a maze.
Locomotion of Slider Armband with decentralized control.
Sweeping mode
The Semni robot with homeokinetic controller
Semni getting excited at its resonance frequency
Oscillatory behavior of Semni
Different rolling modes of the Spherical
Spherical in a circular basin
Spherical adapts to a circular corridor
Whole body movement with Slinging Snake
Sweeping mode of Slinging Snake with high frequency catastrophe
Sensitivity and tolerance to sensor failure
Walk-like behavior of the Rocking Stamper
Creativity in unexpected situations
Emergence of nontrivial modes — Precession of the Barrel
Decay of nontrivial modes
Creativity in unexpected situations
Deprivation of internal forward model and recovery shown with the TwoWheeled
The challenge for self-organization
Swinging legs.
Suspended Humanoid
Dog “playing” with barrier
Dog climbing over a barrier
The HippoDog — The effect of the anatomy
Initial development of the Humanoid (I)
Initial development of the Humanoid (II)
Later developments
Humanoid exercise at high bar
Humanoid in Rhönrad
Fight, Fight, Fight
More Fighting
The Snake adapting to its environment
How the Snake may manage to get out of the pit
Emergence and decay of collective modes
Self-rescue scenario with the Humanoid
The world of playful machines
Armband learns to locomote — weakly guided
Armband quickly learns to locomote
Armband changing the direction of motion
Illustration of the interaction of different materials.

Chapter 2  Self-Organization in Nature and Machines

Abstract: Self-organization in the sense used in natural sciences means the spontaneous creation of patterns in space and/or time in dissipative systems consisting of many individual components. Central in this context is the notion of emergence meaning the spontaneous creation of structures or functions that are not directly explainable from the interactions between the constituents of the system. This chapter presents at first several examples of prominent self-organizing systems in nature with the aim to identify the underlying mechanisms. While self-organization in natural systems shares a common scheme, self-organization in machines is more diversified. An exception is swarm robotics because of the similarity to a system of many constituents interacting via local laws as encountered in physics (particles), biology (insects), and technology (robots). This chapter aims at providing a common basis for a translation of self-organization effects to single robots considered as complex physical systems consisting of many constituents that are constraining each other in an intensive manner.

Chapter 3  The Sensorimotor Loop

Abstract: This chapter aims at providing a basic understanding of the sensorimotor loop as a feedback system. First we will give some insights into the richness of behavior resulting from simple closed loop control structures in a robotic system called the Barrel. This richness is a lesson we can learn from dynamical systems theory: even very simple systems can produce highly complicated behavior. Nearly everything is possible in such a feedback system that is provided with enough energy from outside. Surprisingly, this is accomplished even with extremely simple, fixed controllers, to which we will restrict ourselves here. In later chapters we will see how the homeokinetic principle makes theses systems adaptive and drives them towards specific working regimes of moderate complexity, loosely speaking somewhere between order and chaos.

3.2  Dominated by Embodiment: The Barrel

3.2.2  Open Loop Control

Video 3.1: Open loop control of the Barrel The robot is controlled by a periodic control signal driving the internal weights with a phase shift of π/2. Starting with a very low frequency of the controller signal, the frequency is doubled at time 2:15, 2:40, 3:05, and 3:30. At 3:45 the Barrel is accelerated by a force (red dot) but is seen to return rapidly to the original mode of behavior. The higher frequencies very clearly reveal the difficulties in realizing a fast motion under the open loop control paradigm.
back to List of Videos

3.2.3  Closed Loop Control

Video 3.2: Barrel performing a “lolloping” behavior. A fixed closed loop controller (C11=C12=2, C21=C22=−1, h1,2 = 0) was used. The robot jumps by rapidly moving the internal weight while rolling from left ro right.
back to List of Videos

3.3  Analyzing the Loop

3.3.5  Effective Bifurcation Point and Explorative Behavior

Video 3.3: Emerging cooperativity in a chain of TwoWheeleds. The arena has no obstacles. Control is completely decentralized, but the individual wheels spontaneously cooperate in the working regime close to the effective bifurcation point.
back to List of Videos

Video 3.4: The chain of robots with decentralized control in a regular maze. Spontaneous cooperativity and sensitive reactions to collisions helps the chain to navigate in the maze without any proximity sensors.
back to List of Videos

Chapter 4  Principles of Self-Regulation — Homeostasis

Abstract: We have seen in Chap. 3 that there are specific working regimes in closed sensorimotor loops where the agents exhibit interesting behaviors. The challenge is now to develop general principles so that the agent finds these regions by itself. One essential point at this level of autonomy is the ability to survive in hostile situations, which, as a first prerequisite, requires a certain stability against external perturbations. An example of this is homeostasis, one of the prominent self-regulation scenarios in living beings. This chapter introduces Ashby´s homeostat as a concrete example from cybernetics and develops a general principle of self-regulation as a first step towards a general basis for the self-organization of behavior.

Chapter 5  A General Approach to Self-Organization — Homeokinesis

Abstract: In this chapter we will introduce the concept of homeokinesis, formulate it in mathematical terms, and develop a first understanding of its functionality. The preceding chapter on homeostasis made clear that the objective of “keeping things under control” cannot lead to a system which has a drive of its own to explore its behavioral options in a self-determined manner. This is not surprising since the homeostatic objective drives the controller to minimize the future effects of unpredictable perturbations. This chapter uses a different objective, the so called time-loop error, derives learning rules by gradient descending that error and discusses first consequences of the new approach. Minimizing the time-loop error is shown to generate a dynamical entanglement between state and parameter dynamics that has been termed homeokinesis since it realizes a dynamical regime jointly involving the physical, the neural, and the synaptic dynamics of the brain-body system.

5.2  Homeokinetic Learning

5.2.3  Self-Actualization, Adaptivity, and Sensitivity — Example

Video 5.1: Sensitivity and adaptation to environmental changes. The robot starts with a unit initialization, such that it moves only slowly. After a short time the feedback strength has risen sufficiently such that the robot starts moving steadily forward and backward, bouncing at the walls. Then a heavy trailer is connected to the robot. Initially the robot hardly moves anymore, but the feedback strength is adapted so that stronger and stronger actions are performed. Settings: єc=1, єA=0.25.
back to List of Videos

5.2.4  The Homeokinetic Barrel

Video 5.2: Destabilization and Emergence. Behavior of the Barrel when starting in a situation where the center of gravity of the Barrel is very low so that the situation is physically stable. The homeokinetic learning is seen to destabilize the system quite rapidly (a few seconds real time), getting the Barrel into moving. The parameter dynamics of both the forward model and the controller can be followed in the panels at the left and right upper corners, respectively. The panels depict the course of the parameters in a time window of 250 steps corresponding to about 10 sec. After some time stable rolling patterns are emerging. Later on (at time 01:05) the Barrel was stopped by applying a physical force to it (note the red dot that pulls the robot). After releasing the Barrel, the system recovers again in a very short time and resumes the rolling patterns in a slightly modified form.
back to List of Videos

Chapter 6  From Fixed-Point Flows to Hysteresis Oscillators

Abstract: Homeokinesis realizes the self-organization of artificial brain-body systems by gradient descending the time-loop error, a quantity that is truly internal to the robot since it is defined exclusively in terms of its sensorimotor dynamics. Homeokinesis can therefore be considered as a self-supervised learning procedure with the special effect of making the brain-body system self-referential. We will study this phenomenon here in an idealized one-dimensional world in order to identify key features of our self-referential dynamical systems independently of any specific embodiment effects. In particular, we will gain some insight into the entanglement of state and parameter dynamics and investigate the way how the latter induces behavioral variability.

6.4  Embodiment and Situatedness — Robotic Experiments

6.4.1  Wheeled Robots

Video 6.1: The role of embodiment and situatedness with wheeled robots. (a) Three different wheeled robot with each wheel controlled by its own homeokinetic controller. The TwoWheeled mostly rotates, the FourWheeled drives mostly straight, and the LongVehicle does both equally likely; (b) LongVehicles of different length in a maze. The robots may get stuck, but manage to find out again. Upon being blocked, actions are very gentle because the sensor values are small and thus the activity in the sensorimotor loop goes down rapidly. Due to the sensitization effect in the learning dynamics, the feedback strength is increasing rapidly so that stronger and stronger actions are performed. Settings: єc=0.02, єA=0.01.
back to List of Videos

6.4.2  Spontaneous Cooperation in a Chain of Wheeled Robots

Video 6.2: Chain of wheeled robots exploring a maze. Five TwoWheeled robots connected to a chain by passive joints. Control is decentralized with each wheel being controlled individually by a homeokinetic controller. The position of the red robot is used for the analysis. Settings: єcA=0.01.
back to List of Videos

6.4.3  Emergent Locomotion of the Slider Armband

Video 6.3: Locomotion of Slider Armband with decentralized control. Each of the 18 joints is controlled individually by a one-dimensional controller with the corresponding joint position as the only input information. Nevertheless, after some time a spontaneous cooperation of the segments occurs that causes the robot to locomote. It can even surpass obstacles. At the walls the robot manages to invert its velocity.
back to List of Videos

Chapter 7  Symmetries, Resonances, and Second Order Hysteresis

Abstract: This chapter is a continuation of the preceding chapter to two-dimensional systems. We will identify new features of our self-referential dynamical system again independently of any specific embodiment effects. The additional dimension opens the possibility for state oscillations, where the entanglement of state and parameter dynamics will lead to interesting phenomena and induces behavioral variability. The most prominent effects are driving oscillatory systems into a second order hysteresis (by a self-organized frequency sweeping effect) and getting into resonance with latent oscillatory modes of the controlled system.

7.3  Second Order Hysteresis

7.3.3  Frequency Sweeping in Real Systems

Video 7.1: Sweeping mode.
Behavior of the Barrel with homeokinetic control.
back to List of Videos

Chapter 8  Low Dimensional Robotic Systems

Abstract: In this chapter we will demonstrate the performance of the homeokinetic control system when applied to physical robots. We will recognize many of the effects observed in idealized world conditions, shown in the previous chapters, but most dominantly witness new features originating from the interaction of the learning dynamics with the respective embodiment. Among them are non-trivial sensorimotor coordination, excitation of resonance modes, the adaptation to different environments – all emerging from the unspecific homeokinetic learning rules. The entanglement effect is seen to make emerging motion patterns transient so that the behavioral options are explored and a playful behavior is observed. In order to keep things simple enough for analysis we consider here only low-dimensional systems and leave the high-dimensional ones for Chap. 10.

8.1  Semni

Video 8.1: The Semni robot with homeokinetic controller. Until the first cut in the video only joint angle sensors and position control were used. At the beginning a frequency wandering is observed . Eventually the legs hits the ground and the movement is hindered. The system enters a bias dominated dynamics where a slow oscillation occurs. Both joints get synchronized through the mechanical limits and the robot performs a series of flips. After the first cut all sensors and the voltage control were used. Note the change in the behavior, which becomes more compliant with the movement of the robot. Different frequencies of oscillations are developing, depending on the current mode of behavior and the pose of the robot. Later in the clip the sensitivity of the controller can be seen. The robot is touched very gently and the behavior changes rather strongly. Also note the reaction when the robot is turned over. Importantly the controller adapts very quickly and engages the body into a different oscillatory mode.
back to List of Videos

Video 8.2: Semni getting excited at its resonance frequency. Setting: Robot with loaded head, acceleration sensors, and voltage control. At the beginning the leg moves in synchrony with the body at a rather high frequency. The robot slowly slides into one direction. Then the robot is manually turned over and starts to swing at a much lower frequency. The robot’s resonance frequency is exited and the amplitude rises such the robot falls over to the previous position.
back to List of Videos

Video 8.3: Oscillatory behavior of Semni. Two videos of the Semni robot in the same setting as in Video 8.1. Top: Here the oscillations are interrupted by hand and the robot remains on the other side. Bottom: A longer clip showing the rocking behavior.
back to List of Videos

8.2  Spherical

Video 8.4: Different rolling modes of the Spherical.
back to List of Videos

Video 8.5: Spherical in a circular basin. The robot starts to roll around randomly. Then it enters a circular trajectory as suggested by the environment.
back to List of Videos

Video 8.6: Spherical adapts to a circular corridor. Later in the clip the robot repeatedly balances along the wall for some time.
back to List of Videos

8.3  Slinging Snake

Video 8.7: Whole body movement with Slinging Snake. Three Slinging Snakes are placed in a square arena. (a) Initially they only move little but slowly more and more strong movements are observed. Nevertheless, due to the underactuated nature, the robot cannot be pulled straight along the ground. Eventually the learning rate is increased and the robots start to pick up the swaying movement of their body and enter a fast rotational mode. (b) The robots are slowed down or accelerated by collisions.
back to List of Videos

Video 8.8: Sweeping mode of Slinging Snake with high frequency catastrophe. The initial phase illustrates very nicely how the robot enters the spinning mode. The frequency rises quickly and then is lowered again until the robot comes to a rest. It will not come out of this situation.
back to List of Videos

8.4  Rocking Stamper

Video 8.9: Sensitivity and tolerance to sensor failure. The two infrared sensors at the back are mounted to the trunk with hinges so that their pitch can be changed during the experiment. (a) The robot is perturbed by moving the hand into the sensor range. The robot reacts immediately by moving the pole to the opposite side. (b) One of the sensors gets disabled and enabled by a paper hull. The robot adapts to the new situation quickly and reenters a rocking mode.
back to List of Videos

Video 8.10: Walk-like behavior of the Rocking Stamper. The robot rocks in a way that slow forward locomotion occurs. The first three frames (from left to right) show one full swing. The remaining ones show how the robot travels. Frames times relative to the first frame: 0.5, 0.9, 3.2, and 18 sec.
back to List of Videos

8.2  Barrel

Video 8.11: Creativity in unexpected situations. The Barrel was put into an upright position by an external force about 10 seconds ago. Internal axes are horizontal now so that the sensor values, the inclination of the internal axes, are zero apart from some small sensor noise. The forward model does not get any reliable information in this situation so that rapid forgetting sets in. This is counteracted by the controller, which increases weights so that small perturbations are amplified. Note that the parameters are quite shaken at the beginning of the clip due to the moving of the barrel. After some time the motion of the internal weights become so strong that the Barrel is tossed over. The dynamics of the parameters of the model and the controller can be followed in the panels at the left and right upper corners, respectively. The panels depict the course of the parameters in a time window of 250 steps corresponding to about 10 sec. Note that the scales of the panels change, in particular the model parameters change by two orders of magnitude. The rapidly oscillating parameters are the bias terms of both model and controller.
back to List of Videos

Video 8.12: Emergence of nontrivial modes — Precession of the Barrel. Out of the upright position there are different modes that can emerge. In the video you see the emergence of a precession mode which lasts for more than five minutes. Parameter and model dynamics are depicted in the panels in the right and left upper corners (swapped w. r. t. Video 8.2). Note that the scales of the panels change.
back to List of Videos

Video 8.13: Decay of nontrivial modes. One special feature of our approach is that modes do not last forever, instead the simultaneous learning of model and controller most often leads to a slow change of the parameters so that the system leaves the mode. In the present case, the precession mode, after lasting for about five minutes decays spontaneously. The parameter changes are most prominently in the diagonal elements of both the model (left) and the controller (right panel) depicted by the red and purple lines (which almost coincide).
back to List of Videos

8.2.2  Creativity Using the Real Barrel

Video 8.14: Creativity in unexpected situations. (a) Robot with flat lower surface. The floor is very hard in comparison to the floor in the simulations above, such that the robot cannot get any feedback from the actions. If we push the robot by hand it starts to whip along the lower rim. (b) A small ring is added at the bottom of the robot, such that it can easier get into motion. The robot performs similarly to the simulated one. Note, the maximal amplitude of the moving weights was manually adjusted such that the robot does not flip over.
back to List of Videos

Chapter 9  Model Learning

Abstract: This chapter discusses several aspects concerning the simultaneous learning of controller and internal model. We start with discussing the bootstrapping dilemma arising in this context and the consequences of insufficient sampling. It appears that homeokinetic learning solves these problems naturally, which we illustrate in several examples. Further, we extend the implementation of the internal model by a sensor-branch. This is seen to increase the applicability of the homeokinetic controller because it allows for situations where the sensor values are subject to an action-independent dynamics. The extended model is prone to an ambiguity in the learning process, which can lead to instabilities. The problem can be resolved if the time-loop error is used as an additional objective for the model learning.

9.1  Cognitive Deprivation and Informative Actions

9.1.1  Demonstration by the TwoWheeled

Video 9.1: Deprivation of internal forward model and recovery shown with the TwoWheeled. The controller was first restricted to only straight driving. The model degenerates. After the learning is switched on (robot turns green) the robot mainly rotates at the spot for a while before driving around normally.
back to List of Videos

Chapter 10  High-Dimensional Robotic Systems

Abstract: This chapter contains many applications of homeokinetic learning to high-dimensional robotic systems. The examples chosen for investigation and proposed as experiments to the reader comprise various robots ranging from dog-like, to snake-like up to humanoid robots in different environmental situations. The aim of the experiments is to understand how the controller can learn to “feel” the specific physical properties of the body in its environment and manages to get in a kind of functional resonance with the physical system. In order to better bring out the characteristics of homeokinetic learning in these systems, we use a kind of physical scaffolding, for instance suspending the Humanoid like a bungee jumper, putting it in the Rhoenrad, or hanging it at the high bar. Interestingly, in all situations the robots develop whole-body motion patterns that seemingly are related to the specific environmental situation: the Dog starts playing with a barrier eventually jumping or climbing over it; the Snake develops coiling and jumping modes; we observe emerging climbing behaviors of a Humanoid like trying to get out of a pit; and wrestling like scenarios if a Humanoid is encountering a companion. Eventually, in our robotic zoo all kinds of robots are brought together so that homeokinesis can prove its robustness against heavy interactions with other robots or dynamical objects. Essentially this chapter provides a phenomenological overview and invites to play around with numerous simulations to see the “playful machine” in action.

Video 10.1: The challenge for self-organization. Emergence of embodiment-specific behavior in robots with different morphology but identical brains.
back to List of Videos

10.1  Underactuated and Compliant

Video 10.2: Swinging legs. The trunk of the Dog is fixed so that the legs can move freely. The motors are so weak that the controller must learn to excite a resonance mode. There are no physical cross couplings between the legs. Nevertheless, there are some frequency and phase correlations observable.
back to List of Videos

Video 10.3: Suspended Humanoid. The robot is pulled upwards by a simulated spring such that the robot can freely move, but still interacts with the ground. This is a kind of scaffolding situation in order to give the brain time to accustom to the body.
back to List of Videos

10.2  Dog and HippoDog

Video 10.4: Dog “playing” with barrier. The Dog video demonstrates the playful manner of exploring the world in the homeokinetic learning scenario. Note that the robot has nothing but its joint angle sensors as source of information about its body and the environment. The Dog is protected by an invisible box on its back preventing it from falling over.
back to List of Videos

Video 10.5: Dog climbing over a barrier. When encountering a barrier, the Dog can behave in many different ways since there is no goal prescribed. However, more often than not, the robot manages to climb over the barrier in a seemingly goal oriented way. In the video, the Dog has acquired a rather cautious behavior slowly probing different possibilities of interacting with the barrier. After some time the left hind leg is swung onto the barrier and after several minutes it climbs out completely.
back to List of Videos

Video 10.6: The HippoDog — The effect of the anatomy. The HippoDog with its spherical trunk is developing more active and jumpy motion patterns due to its specific anatomy.
back to List of Videos

10.3  Humanoid

Video 10.7: Initial development of the Humanoid (I). Top: From scratch. Bottom: 6 min later. If the robot is started initially on level ground, it develops rather smooth and slow motions, increasing steadily its behavioral spectrum.
back to List of Videos

Video 10.8: Initial development of the Humanoid (II). If started in the pit, the robot develops increasingly complex motion patterns related to that situation. Interestingly, these patterns keep repeating in the course of time, getting more pronounced and lasting over a longer time.
back to List of Videos

Video 10.9: Later developments. In the later development of the Humanoid on level ground we observe more active behaviors. Interestingly, one often observes patterns that look like the intention to get up, these patterns often repeating several times. The bottom video shows another motion pattern like they often emerge in the course of time. In the experiment the gravity was chosen half the earth gravity in order that the robot can move more freely. This is necessary in order to get enough feedback from the body.
back to List of Videos

High Bar and Rhönrad - Feeling the Body

Video 10.10: Humanoid exercise at high bar. Underactuated robot under physical constraints: motor forces are quite weak so that for instance the robot is not able to chin the bar. The observed motion patterns express the emerging cooperativity of body components.
back to List of Videos

Video 10.11: Humanoid in Rhönrad. The wheel can be moved by shifting the center of gravity. The only sensor information is from the joint angles so that the robot does not have any information on the physical coordinates and orientation of the wheel.
back to List of Videos


Video 10.12: Fight, Fight, Fight. If robots come into close contact they may excite each other to complex motion patterns. This is amazing since the only way of “feeling” each other is by the perturbations in the sensorimotor dynamics caused by the physical forces exerted in the interaction. Adherence is due to normal friction but essentially also by a failure of the ODE physics engine, which after heavy collisions produces an unrealistic interpenetration effect, which acts like a special gripping mechanism. So, the “fighting” is a truly emerging phenomenon not expected before we observed it.
back to List of Videos

Video 10.13: More Fighting. In the left video the robots come into contact by the narrowness of the arena, in the second one there is a small force attracting the robots towards the center.
back to List of Videos

10.4  Snakes — Adaptation and Spontaneity

Video 10.14: The Snake adapting to its environment. Under strong physical constraints, the homeokinetic learning finds a way to keeping the robot active while taking account of the geometry of the vessel.
back to List of Videos

Video 10.15: How the Snake may manage to get out of the pit. The first picture in the sequence shows the snake in a seemingly relaxed situation. However, as the video shows there are tensions building up in the body and after some time it suddenly crunches into a tight bundle of which it unrolls with very high velocity. By the inertia effects it manages to nearly jump out of this very deep pit despite the quite weak motor forces.
back to List of Videos

Video 10.16: Emergence and decay of collective modes. The video sequence demonstrates nicely the transient nature of the emerging modes. In the first part of the behavioral sequence (a) the Snake goes into a collective mode with the segments rotating in a coherent manner generating a kind of strangling effect by the interaction with the glassy cylinder. This mode, after prevailing for several minutes, is seen in (b) to break down rapidly giving way to the emergence of new motion patterns.
back to List of Videos

10.5  Self-Rescue Scenario

Video 10.17: Self-rescue scenario with the Humanoid. After falling into a narrow pit, the robot develops several alternative motion patterns adapted to the new situation. After some time one often observes the emergence of climbing like behavior patterns. The patterns are emerging without any goals as a consequence of the sensitive but active interplay between the robot and the specific environment. Nevertheless, they may help the robot to get out of this impasse.
back to List of Videos

10.6  World of Playful Machines

Video 10.18: The world of playful machines. The videos are an example of a robotic world, all robots being driven by homeokinetic learning. Such simulations can run over many hours, producing an unforeseeable sequence of motion patterns, even in the purely deterministic case. In the top video, the most interesting agent is the Armband that seemingly tries to get into a rolling mode. After heavy collisions it falls over, but in the later development it gets upright again by another collision. After that it starts rolling and jumping again. In the bottom video observe the Slider Armbands that are controlled in a slit control fashion. Later the fighters show also nice scenes. The chain of robots has infrared sensors which are coupled similarly as described in Section 3.5. The experiment “Play World” enables you to try this scene yourself.
back to List of Videos

Chapter 11  Facing the Unknown — Homeokinesis in a New Representation*

Abstract: Homeokinesis has been introduced and analyzed in the preceding chapters on the basis of the time-loop error (TLE). This chapter presents an alternative approach to the general homeokinetic objective by introducing a new representation of the sensorimotor dynamics. This new representation corrects the state dynamics for the predictable changes in the sensor values so that the transformed state is constant except for the interactions with the unknown part of the dynamics. The single-step interaction term will be seen to be identical to the TLE so that the learning dynamics is not altered. However, besides giving an additional motivation for the TLE, this chapter will extend the considerations to the case of several time steps and will eventually consider infinite time horizons making contact with the global Lyapunov exponents and chaos theory.

Chapter 12  Guided Self-Organization — A First Realization

Abstract: We introduce here an in the following chapters guided self-organization as the combination of specific goals with self-organizing control. As a first realization we propose in this chapter the guidance with supervised learning signals. First, we investigate how these signals can be incorporated into the learning dynamics and present then a simple scenario with direct motor teaching signals. We find that the homeokinetic controller explores around the given motor patterns and thus may find a more suitable behavior for the particular body. Second, we transfer this into a teaching at the level of sensor signals, which is very natural in our setup. This mechanism of guidance builds the basis for higher level guiding mechanisms as discussed in Chap. 13.

Chapter 13  Channeling Self-Organization

Abstract: Many desired behaviors are distinguished by a certain structure in the motor or sensor activity. In particular the phase relation between different motors or sensors capture a lot of this structure. We will now propose a way to embed these relations as soft constrains to the learning system, such that we break certain symmetries and let desired behaviors emerge. Starting from the guidance by teaching we introduce the concept of cross-motor teaching that allows to specify abstract relations between motor channels. First we study simple pairwise relations and shape the behavior of the TwoWheeled robot to drive mostly straight by a relation between both motor neurons. Then we will consider a high-dimensional robot—the Armband and demonstrate fast locomotion behaviors from scratch by guided self-organization.

Video 13.1: Armband learns to locomote — weakly guided. Behavior of the robot with cross-motor teaching and weak guidance (γ=0.001). A slow locomotive behavior with different velocities is exhibited. Explorative actions cause the posture of the robot to vary in the course of time.
back to List of Videos

Video 13.2: Armband quickly learns to locomote. Behavior of the robot with cross-motor teaching and medium guidance (γ=0.003). Comparable fast locomotive behavior emerges quickly and is persistent. Nevertheless the velocity varies. Only small exploratory actions are takes, such that the posture is mainly constant.
back to List of Videos

Video 13.3: Armband changing the direction of motion. The behavior of the robot with cross-motor teaching if the connections are changed. The video starts with a fast locomotive behavior to the left (k=1). At time 5:00 the couplings are changed (k=0) and the robot slowly stops. A period of probing actions follows until a reversed locomotion starts to show up.
back to List of Videos

Chapter 14  Reward-Driven Self-Organization

Abstract: In this chapter we investigate how to guide the self-organization process by providing an online reward or punishment. The starting point for the following considerations is that the homeokinetic controller explores the behavioral space of the controlled system and that those behaviors which are well predictable will persist longer than others. The idea we pursue in this chapter is to regulate the lifetimes of the transient according to the reward or punishment. The mechanism is applied to the Spherical with two goals, fast motion and curved rolling.

Chapter 15  Algorithmic Implementation

Abstract: This chapter presents a unified algorithm implementing the homeokinetic learning rules including a number of extensions partly discussed already in earlier chapters of this book. We continue with some guidelines and tips on how to use the homeokinetic “brain.” We discuss techniques and special methods to make the self-supervised learning of embodied systems more reliable from the practical point of view. This includes the regularization procedures for the singularities in the time-loop error and different norms of the error for the gradient descent. The internal complexity of the controller and the model is extended by the generalization to multilayer networks. Apart from that the computational complexity of the learning algorithm will be reduced essentially by easing non-trivial matrix inversions. This is important for truly autonomous hardware realizations.

Chapter 16  The LpzRobots Simulator

Abstract: In this chapter we describe our robot simulator. We start with the overall structure of the software package containing the controller framework, the physics simulator and external tools. The controller framework makes it very easy to develop and test our algorithms, be it in simulations or with real robots. The physics simulator can handle rigid bodies with fixed geometric representation that are connected by actuated joints. Particular efforts have been undertaken to develop an elaborated treatment of physical object interactions including friction, elasticity, and slip. The chapter also briefly discusses the generation of virtual creatures, the user interface and the most important features of the LpzRobots simulation environment.

Video 16.1: Illustration of the interaction of different materials. All spheres have the same size and mass. The substances are foam (yellow), rubber (dark brown), plastic (white), and metal (silver). Note the different penetrations (second picture) and bouncing heights.
back to List of Videos

Content: © Copyright 2011, Ralf Der and Georg Martius; Impressum. Parts of the page have been translated from LATEX by HEVEA.