Motivation (under revision) |
||
 
Foreword Note: Due to recent changes to the memory page, this page is no longer up-to-date. Stay tuned for a coming update.
ForewordIn this section I use the words pain and pleasure solely for their convenience as naming labels. In no way do these terms imply that Animal or any other artificial intelligence consciously feel pain or pleasure. I use pleasure to refer to appetitive stimuli and pain to aversive stimuli. There is a monumental chasm between consciously feeling pain or pleasure and reacting appetitively or aversively to certain stimuli. Reactive behavior is not consciousness, otherwise one should be ready to ascribe consciousness to thermostats--and I am sure many do. Many of the concepts used in this section (such as effectors, command neurons, etc..) are explained in the Motor Learning section. Note that I have not yet implemented the motivation mechanism in Animal. Stay tuned for an update.
AnticipationMotivation is the tendency of an intelligent system to behave either appetitively or aversively to certain stimuli. This tendency is due to the logic of the reinforcement mechanism. Behavior reinforcement is always anticipatory in nature, even in situations where the behaving system appears to react to a stimulus. For examples, we may spit out a bitter tasting fruit, not because we are reacting to the unpleasant taste, but because we anticipate that the unpleasantness will cease soon afterwards. Likewise, we may go to the dentist in anticipation of our toothache diminishing after treatment or because we expect that doing so will prevent toothaches in the future. These are all examples of aversive behavior but appetitive behavior, too, is anticipatory in nature. For instance, we decide to eat ice-cream because we anticipate that the taste will be pleasant. To anticipate means to participate ahead of time, i.e., to take advance action based on the likelihood of future events. In other words, to anticipate is to react to events before they actually happen. To adapt, an intelligent system must be able to anticipate the likelihood of future pain and pleasure stimuli. It can do so only through experience and learning. The motivational system itself neither learns nor anticipates anything. Learning to anticipate is the job of the memory subsystem. All inputs into the motivational system come from the memory layer which is the output layer of the perceptual system. The Animal network diagram is reproduced below for your convenience. Note: Due to recent changes and additions to the memory page, the network diagram does not yet incorporate motivation. Stay tuned for an update.
The Need for Intelligent ReinforcementThe problem of reinforcement learning is knowing what to reinforce. Motivation cannot rely on a blind mechanism that strengthens or weakens connections based on their temporal proximity to pain or pleasure stimuli. While temporal difference reinforcement may work well enough in small systems, it becomes prohibitive in large systems. At any given moment, the motor area receives a huge number of signals from afferent pathways. Many signals repeat and a blind mechanism is likely to associate the wrong motor commands with a given reinforcement stimulus. Also, there is a huge number of pain and pleasure sensors that must be associated with specific behaviors and sensory inputs. For examples: if I hurt my toe, I will not waste time attending to my fingers; an earache is not the same as a stomachache; thirst is not the same as hunger; etc... In sum, there is a need for an intelligent and associative response mechanism to pain and pleasure. To be Continued... Next: Memory
Last Update: 11/06/2002 Send all comments to: louis.savain@sbcglobal.net ©2002 Louis Savain |
||
