Motor Learning

Latest News

Temporal Intelligence

Animal

Perceptual Learning

Perceptual Network

Memory

Motivation

Motor Learning

Motor System Architecture
  Effector Layer
  Effectors
  Activation Strength
  Feedback Signals
  Command Selection Layer
  Command Selection Neuron
Automatic Motor Control and Coordination
  The Principle of Motor Coordination
  Motor Command Timing vs. Action Selection
  There Is a Time For Everything
  Experimental Verification
Basal Ganglia and Cerebellum Hypothesis
  Basal Ganglia
  Cerebellum

Motor control in Animal consists of activating and deactivating the right effectors at the right time. Surprisingly, the timing of motor commands is not the responsibility of the motor control system. Every signal coming from the perceptual layers is assumed to have a unique temporal signature, a priori. The primary role of the motor control system is to connect command lines to effectors in a manner that maximizes motor coordination. As you read the following, note the role played by the Principle of Complementarity.

The effector layer contains all the neurons required for motor output. It receives its inputs from the command selection layer and sends its output to various actuators and back to the perceptual system via proprioceptive sensors. The number of effectors is fixed by design.

An effector is a special neuron that is attached to an actuator. There may be more than one effector attached to a given actuator, each with a specific duration and activation strength. During activation, an effector repeatedly fires at a fixed frequency and remains activated for a pre-programmed duration. Every effect has a beginning (onset) and an end (offset). Thus it takes two actions to control an effector, one to start the effect and another to stop it. Last but not least, for every effector there is a counter effector, e.g., agonist and antagonist muscles.

The activation strength of an effector is determined by its spike frequency. In Animal, all effectors have equal activation strengths. That is to say, when activated, they continually fire at the same frequency. However, a situated real-world robot will need to have a full assortment of weak and strong effectors from which to choose.

Every effector sends feedback signals to the perceptual network via proprioceptive sensors. This allows the system to compare its motor commands (its intentions) with what actually happens. Originally, I had thought that feedback signals needed to be sent only at the onset and offset of an action. But later I realized that signals should be sent back continually during activation. This is the only way the system can obtain information about the strength of an activation. It is also the only foolproof way to internally determine whether or not an effector is activated. This is important because oftentimes an agent needs to think about actions without actually executing them.

The command selection layer contains special cells called command neurons which are used to control the effectors in the effector layer. The CSL receives its inputs from the perceptual system and sends its outputs to the effector layer. The CSL is the opposite of the signal separation layer. The latter separates sensory streams into parallels pathways whereas the CSL fuses parallel pathways into single streams for output to motor effectors.

Every CSN has a single output which is connected permanently to a single effector neuron in the effector layer. The function of a CSN is to relay command signals from the perceptual system to its effector and to ensure that only input connections with properly coordinated signals survive. Every effector is controlled by a pair of CSNs, one for starting the effect and the other for stopping it. A CSN can have an unlimited number of input synapses of which there are two types, positive (excitatory) or negative (inhibitory).

Input synapses are slightly strengthened every time they fire. Unless inhibited, a CSN fires every time a positive input synapse fires. In addition to its command inputs, every CSN has a special input connection that carries a corrective or remedial signal. When this signal arrives, the most recently fired command connection is strongly weakened. This mechanism is used by the motor coordination system as I explain below.

There are two mechanisms of motor control. One is responsible for motor coordination and the other for the polarity of command inputs. The motor system's main function is to select command inputs whose signals are properly coordinated, regardless of polarity. As I mentioned earlier, motor timing, i.e., the arrival time of a motor command signal, is not the responsibility of the motor system. Signal timing emerges automatically in the perceptual network.

I mentioned in the perceptual learning page that the effector layer is the exact mirror image of the sensory layer. Just as sensory stimuli can have a range of durations and various intensity levels, motor actions can also have a range of durations and intensity levels. I wrote that no sensory phenomenon can start if it has already started or stop if it is already stopped. A similar principle applies to motor logic: no action can be started if it has already started or stopped if it is already stopped. This simple rule is what I call the Principle of Motor Coordination. In Animal, a remedial signal is sent back to every command neuron to suppress input connections that violate the PMC. When a command neuron receives a corrective signal, the most recently fired command input connection is weakened. Eventually, if the violation persists, the faulty connection is suppressed to a point where it no longer has any effect. I claim that the simple application of the PMC is sufficient to ensure automatic coordination. Its power is in its simplicity.

Robotics researchers and experts in neuroethology often talk of the action selection problem as one of the hardest problems in behavior control. We are told that the problem arises when an organism (or an autonomous software agent or robot) must choose between several behavioral alternatives that share the same motor mechanism. There seems to be a widespread assumption in the adaptive behavior community that humans and animals have self-contained motor programs that wait their turn to be executed and that the decision to select one of these programs for execution somehow must be done right before or at the moment of execution.

Needless to say, this is an erroneous assumption. So-called motor programs are really a legacy of the old discredited associative chaining hypothesis of serial behavior in disguise. Associative chaining was much in vogue among psychologists during most of the last century. Trouble is, it was decisively debunked by psychologist Karl Lashley in a 1951 article titled "The Problem of Serial Order in Behavior." It it is surprising to see that it is still being taken seriously in the twenty-first century under the guise of "sensorimotor programs." Lashley rightly concluded that the generation of sequential behavior necessitated, not the invocation of hierarchically organized motor programs, but the parallel activation of primitive motor actions. In other words, biological systems use no such things as motor programs in the computer sense of programs as algorithms.

The approach to motor control described here effectively bypasses the action selection problem for the reason that there are no motor programs in need of selection. Again, action timing decisions are made, not by the motor system, but by individual neurons working in parallel in the perceptual system. When a signal arrives at the command selection layer, it is assumed to have a precise and unique timing dictated by unique conditions. No signal arrives before or after its time. All motor signals that can potentially arrive at a command input connection are selected once and for all at the time the connection is first made. Thus the timing of every signal is preset prior to selection. The job of the motor control system is simply to make sure that the only command inputs that survive are those whose signals are temporally coordinated according to the Principle of Motor Coordination. This ensures that coherence is maintained even though control signals originate from separate regions of the system.

Although Animal is designed with only a few degrees of freedom for its eye and gripper, one can distinctly observe the PMC in action. It doesn't take long for Animal to learn to coordinate its movements so that both eye and gripper begin moving in unison. No explicit code was written to obtain this behavior. Motor coordination emerges automatically, thanks to the PMC.

The above hypothesis of movement generation and the Principle of Motor Coordination can be used to formulate a general theory of the operation of the basal ganglia and the cerebellum. At the heart of the theory is the idea that motor control is separated into two functionally distinct but complementary systems, one dedicated to starting motor actions and the other to stopping ongoing actions.

The basal ganglia (BG) are a complex assembly of cells whose primary role is to start and generate movements. The BG do not make start decisions on their own but rather, command signals arriving from the cortex are used to trigger a range of primitive movements. Each movement is generated by a tonically active neuron the duration of which is determined genetically by special circuitry forming feedback loops. The level of torque or force is determined by the discrete spike frequency of the neuron. Given the necessity for fine motor control, there is likely to be hundreds of effectors neurons for every muscle. One of the BG's main function is to ensure that movements are started in a coherent manner, i.e., the initiation of movements must obey the Principle of Motor Coordination as I explain above.

Once a movement is started, the BG are helpless to stop it. The movement must either run its genetically preprogrammed course or it can be stopped or inhibited by afferent connections originating elsewhere. In this theory, stopping an ongoing movement is the job of the cerebellum.

The cerebellum is a highly regular structure that contains from seven to eight million neurons called Purkinje neurons (PN). Unlike the basal ganglia, the cerebellum does not initiate movements. Its role is to stop ongoing movements under specific conditions. In this sense, the cerebellum is one of the two determinants of the duration of movements, the other being the intervals pre-programmed in the effectors themselves. It is known that people with cerebellar lesions regularly overreach when performing grasping movements. My hypothesis suggests that every PN controls a single motor neuron. Each PN receives about one two two hundred thousand inputs (the so-called parallel fibers) from various regions of the cortex and the spinal cord.

Many students of the cerebellum seem to be under the impression that the PN is some sort of coincidence detector and that it can fire only if most of its afferent inputs fire. My understanding of motor control leads me to believe that a PN can be caused to reach action potential by the firing of a single, sufficiently strong parallel fiber synapse. A parallel fiber is analogous to a stop command connection in Animal. The repeated firing of a parallel fiber should induce long term synaptic potentiation (LTP), making it strong enough to generate a post-synaptic potential in the PN.

It is known that the firing of the climbing or ascending fiber induces long term depression (LTD) in parallel fiber synapses that happen to fire at about the same time. The popular theory (Marr-Albus-Ito) proposes that the climbing fiber is used as part of a mechanism to correct PN activity that does not match the system's motor intentions. While I agree that the climbing fiber serves as an error signal, I think that the climbing fiber spike serves to depress any parallel fiber synapse that violates the Principle of Motor Coordination. To be effective as a remedial input, the climbing fiber must closely coincide with the firing of the offending parallel fibers.

It would seem that, if a PN does not fire, a remedial signal should be unnecessary. Lately, however, I have toyed with the idea that remedial signals can still be generated even if the PN does not fire. This is because the motor system can always detect whether or not a given movement is ongoing. Climbing fiber signals can be triggered as soon as the corresponding movement stops. This way, any new parallel fiber connections can be nipped in the bud if they misbehave.