News

Temporal Intelligence

Animal

Perceptual Learning

Perceptual Network

Memory

Motivation

Motor Learning

The Silver Bullet

Project COSA

Software Composition

News
Latest Changes
Older News

Ok, it has been more than a month since my last news item. I've been stuck in a rut since last December. When that happens, my usual recourse is to concentrate my mind on other things. I've found in the past that I need to give my brain time to grow new synaptic branches. It's useless to try to figure something out if the required connections are not there.

It seems to me that the ability to focus one's attention on important phenomena is a prerequisite to any effective motivational system. My current thinking is that attention is the ability to maintain or run one or more trains of thought in short-term memory to the exclusion of others. A train of thought can be defined as a sequence of causally-connected neural firings or spikes. This ties in rather nicely with the temporal nature of intelligence and the anticipatory mechanism of short and long-term memory used in Animal.

I believe that one should view attention as part and parcel of action selection. There is only a limited number of output motor neurons. A motor neuron cannot possibly attend to more than one action at a time. Whenever a motor neuron receives more than one command simultaneously, it should be flagged as a conflict. This is the point at which the attention mechanism should decide which sequence should lock out the others.

But this approach raises an interesting question: can a sequence be preempted by another? At first, I did not think that the system should allow sequences to be preempted by other sequences. Lately though, I am coming to realize that it is all a matter of competition and survival of the fittest. The strongest connection wins. It is instructive to note that the amygdala (thought to be involved with attention and motivation) has two-way connections with the basal ganglia. The latter is known to be directly involved with volitional motor control.

As seen in the above diagram, the green lines transmit predecessor signals from short-term memory or STM neurons (blue). STM neurons fire repeatedly for a short time after being triggered by signals arriving from the perceptual network. This allows the LTM neurons (yellow) to form temporal associations over multiple time scales. An LTM neuron fires under two conditions. It sends out a single anticipatory spike as soon as it begins receiving predecessor input signals (green). It fires again if it receives a successor input signal (red) while the predecessor signals are still firing. The outputs of LTM neurons are fed to the motor command layer.

The anticipatory mechanism pictured above can be used to control which sequence is allowed to run at the exclusion of others. Looking at the diagram, it is apparent that a signal arriving on line A prepares for the next signal in the sequence to arrive on line B. To focus its attention on the sequence of signals that follow, the system only needs to inhibit all LTM cells that do not receive a predecessor input from cell A (blue).

Psychology teaches us that the human brain can maintain up to seven items at a time in short-term memory. At this point, I am not entirely sure how to interpret this finding. Is it due to an emergent property of the attention mechanism? Or should there be specific neural mechanisms whose fucntion is to limit the number items stored in short-term memory?

My plan for the next week is to implement the attention mechanism in Animal to see if it stabilizes its behavior. As it is, Animal is mightily distracted and cannot seem to concentrate on one thing at a time. Stay tuned for more.

I now think I fully understand how motivation works. I know, I have said the same thing about other things in the past that turned out to be wrong. All I can say in my defence is that building confidence is how I motivate myself to write C++ code (I really hate programming). The simplicity of the motivation mechanism will astound everybody. I am currently conducting experiments with Animal and I will have an update soon. Let me say for now that my original understanding was not too far off the mark. Let me also say that adding a motivation mechanism to Animal, although I believe it will improve Animal's behavior by an order of magnitude, it will not give it the ability to have deep thoughts, so to speak. For that, Animal needs to have a supervised layer (a cerebellum) which can take over routine sensorimotor tasks so as to let Animal use internal feedback to think about important things. More to come.

After experimenting with Animal over the last several days, I concluded that the memory problem I've been struggling with for the last few months is not a problem after all. I implemented the anticipatory mechanism in Animal and it did not work. Animal's behavior did not seem to change much whether with or without it. If anything, the network's behavior seemed even more stupid. It is never a good idea to stick to one's erroneous ways in the face of strong contrary evidence. So I decided to retrace my steps and try a simpler design. I firmly believe that simplicity should be one of the sacrosanct guiding principles of AI research along with the Principle of Complementarity. As I wrote in a recent message to the Temporal Intelligence discussion group at Yahoo!, intelligence is much simpler fundamentally than any of us in the AI business ever imagined. The simplicity will come as a big surprise to many, especially the experts in the GOFAI community.

At any rate, I got rid of the blue feedback lines and came up with the simpler design shown below:

STM neurons (blues cells) still work as they did before; they undergo a sustained oscillatory spiking activity for a short predetermined duration after being triggered by an incoming signal from the perceptual network. I made two significant changes. I got rid of the feedback lines and I changed the behavior of the LTM neurons (yellow cells). The new LTM neuron not only fires upon the concurrent arrival of a master (red line input) and slave (green line input) signals but-and this is the clincher-it also fires immediately upon the arrival of the leading spike of the spike train coming from the STM neuron. In other words, the LTM neuron fires an anticipatory spike before the arrival of the master signal. As I wrote elsewhere, to anticipate means to participate in advance. The end result is that LTM neurons have dual function: they generate two complementary types of signal. One is reactive and the other is proactive or anticipatory. I believe there is a need for both.

I made a few changes to Animal's code to reflect the new design, debugged the code and ran the program. Bingo! Within a short time, Animal was behaving much more intelligently than ever before. By the time the network had grown to about 10,000 cells, Animal was not only smoothly coordinating the movements of its eye and gripper but it had learned to generate rapid eye saccades continuously. Remember that Animal cannot "see" anything unless its eye is moving. This is not unlike the human eye which constantly moves in short jerky motions called saccades, even when we fixate our gaze on a single dot.

As it stands, Animal's STM layer is comparable to the brain's hippocampus, the cells of which are known to exhibit sustained oscillatory behavior for short periods of time. But is the hippocampus really the seat of our short-term memory? I am not so sure anymore. I am beginning to suspect that the hippocampus is just a part of the mechanism that allows the brain to form long-term associations over multiple time-scales. Patients with hippocampal lesions seem to retain their short-term recall capability. What they lose is the ability to form new long-term memories. With this in mind, I am slowly coming to the realization that another specialized layer is necessary to handle short-term memory. More to come.

The newest version of Animal still does not have a motivation mechanism. This means that Animal's behavior does not follow a purposeful pattern. Motivation is tied to our ability to anticipate future events. We choose our actions depending on their expected outcome. That is to say, motivation is action selection based on anticipated reward or punishment. Thus the motivation mechanism must be tightly integrated with the STM and LTM layers and especially with the system's ability to generate anticipatory spikes. I am presently conducting a few experiments with Animal and will report on my progress soon.

Animal (1.3) is now available for downloading. Please read the release note (ReadMe.doc) and stay tuned for a coming update. Also notice that Animal 1.3 uses far fewer cells than the previous version. For comparison purposes, you can still download version 1.2.

I think I may have found an elegant and biologically plausible solution to the problem of differentiating between the anticipatory and non-anticipatory signals generated by short-term memory neurons. Forget about doublets and dual synapses: they're not it. The solution has to do with spike frequency or firing rate. The solution is this: the two types of oscillatory signals simply have different firing rates. This may explain the discovery of pronounced and sudden variations in the theta (~12 Hz) oscillations of individual cells in the hippocampus of awake rats. The firing rate of certain cells changes in accordance with the rat's experience with a given place field (location) and the direction of the animal's motion through the field. The implication is that, once the animal has become familiar with a field, it is able to anticipate its location in the field and this is reflected in the anticipatory change in the firing rate of the so-called "place" cells.

Many neurobiologists have concluded on the basis of experiments with rats that the hippocampus is mainly a location mapping mechanism or "spatial memory" as they call it. However, it is my contention (an idea that has been gaining in acceptance recently) that the hippocampus is essential to all cognitive activities that involve sequences of events or episodes), which is pretty much every activity, in my opinion. This is an idea whose time has come, if for no other reason than the fact that the hippocampus is known to receive afferent connections from all sensory cortices, not just the visual cortex.

While the spike frequency solution is fine for biological neurons, a software network is not obligated to use it. The important thing is to be able to differentiate between anticipatory and non-anticipatory signals. In software, this can easily be accomplished with the use of a binary flag in each STM neuron. The algorithm responsible for learning in LTM neurons can check the flag to determine whether or not to modify the strength of slave synapses. I am going to try to implement this mechanism in Animal this weekend. This is exciting. Stay tuned.

In my last message, I described a two-layer memory mechanism (see diagram below) that uses a feedback loop to trigger anticipatory recall. The problem, (I mentioned this in a previous news article) is that the oscillatory anticipatory signals generated by an STM neuron are bound to form the wrong associations in the long-term memory or association layer. The reason has to do with timing. Anticipatory signals are really fake signals, in the sense that they always arrive before their time. They must not be allowed to influence associative learning, otherwise the system will become hopelessly confused. Thus the associative LTM neuron must have a way to distinguish between an anticipatory signal and a real signal.

My original hypothesis was that the solution to the problem involves the use of doublets. These are quick successions of two spikes which are observed in the hippocampus, the part of the brain thought to be involved with short term memory. I recently received an interesting email from gs_tech2000 (sorry, I have not yet replied) who suggested that the problem might be solved by using separate input synapses. Looking at the diagram above, it seems that LTM neurons should have two slave input synapses, one for past events and one for anticipated events. It would seem that this scheme requires dual STM neurons. One (as shown) would receive signals from the perceptual network and the second would get feedback signals from the LTM layer. At this point I am having trouble coming up with a biologically plausible solution that includes double slave synapses seeing that slave synapses are created on the fly during associative learning. 

I welcome feedback and suggestions from interested readers.

Ok, I had some time to work and meditate on the memory problem this week and I think am making excellent progress. I feel a breakthrough is right around the corner. I had previously made a distinction between reactive and anticipatory memory. Recently I came to the conclusion that long-term memory, whether associated with past events or anticipated future events, is fully reactive in nature. That is to say, the same reactive mechanism is used for both types of recall. As seen in the diagram below, afferent signals from the perceptual network react with both past and anticipated events in accordance with previously learned associations. As I explained elsewhere, STM neurons (light blue cells) repeatedly fire for a short time after receiving a signal from the perceptual network. Normally, these cells act as past-event generators but, as we shall see, they can just as easily be used to generate anticipated future events. Thus there are two types of associative recall, one for past events and one for future or anticipated events.

The key to understanding how the memory system works is to realize the close relationship that exists between short and long-term memory. For every cell in short-term memory, there is a corresponding cell (yellow) in long-term memory. An LTM cell learns by finding sequential or causal associations between a successor signal arriving at its master input (red line) and a predecessor signal arriving at its slave input (green line). The interval between the arrival of a predecessor signal and that of a successor signal does not matter. The only constraint is that it must be less than the duration of the system's short-term memory retention. This allows the system to find causal relationships over multiple time scales. An LTM neuron fires every time a predecessor signal is followed by a successor signal. The firing of an LTM neuron in association with a past event is known as a past event recall.

The same reactive master-slave mechanism can be used to generate a future event recall. This is accomplished with the help of a feedback loop (dark blue line). Here is how it works. Let us say a signal arrives at STM cell A. The cell immediately begins firing. As seen in the diagram, signal A is a precursor to a future signal arriving at LTM cell B'. In order to anticipate the arrival of a signal at B', the system must behave as if the signal did arrive. To do so, the system must fake the arrival by somehow getting the STM cell B to begin firing. This is where the one-to-one correspondence between STM and LTM neurons comes in handy. When cell A fires, if causes LTM cell B' to send a feedback signal (blue line) that triggers the firing of STM cell B. Subsequently, a signal arriving at LTM cell C' (which is associated with cell B) will cause cell C' to fire as if the signal had actually arrived at B.

At this point I am not sure whether or not the anticipatory firing of STM cell B should, in turn, trigger more anticipatory events. This could start a chain reaction that could quickly get out of control, resulting in spasms and seizures. I will have to conduct actual experiments with Animal to determine the advantages or disadvantages of cascading anticipatory feedback signals.

Note that the diagram shown above is simplified for clarity. In the actual network, an STM cell may be connected to multiple LTM cells. This makes it possible for the memory system to recall of multiple events concurrently. Also, this is not the way it is done in the brain exactly. The brain has biological imperatives it must attend to. For example, special neurons must be used to block the feedback signal from an LTM cell from traveling to the motor area. This can be easily done in software without using an extra neuron.

I am hoping to find some time to implement this new memory scheme in Animal over the coming weekend. I think it will make a world of a difference. I also plan to update the memory page with the new information. Stay tuned.

No, this is not a renunciation the Principle of Complementarity. It's just that I recently came to the conclusion that in order for the complementary-loops approach to work properly, the two sides would have to be exact mirror duplicates of each other. This means that all the connections would have to be exactly the same. Programmatically, this would be no problem but I don't see how it could happen in the brain. Every neuron on either side of memory would have to be psychic in order to make the exact same input connections as its complement on the other side. The solution is to use a single loop to handle both reactive and  anticipatory signal generation. Which takes me back to doublets.

A doublet is a pair of pulses or spikes that occur in quick succession. Doublets are observed to occur in the hippocampus of animals. I hypothesize that the first spike in a doublet is reactive while the second is anticipatory. The reactive spike causes an LTM neuron to fire in response to the arrival of a master signal. The anticipatory spike causes the neuron to fire in response to a slave signal. Only reactive spikes are transmitted to motor areas. Anticipatory spikes never get out of the memory system. Their role is to set and modify the system's goals and help it focus on no more than a few items at a time. The use of doublets means that the diagram above can be simplified into a single unified, albeit dual-purpose, loop memory as follows:

As I mention on the memory page, it is known that short-term memory can only hold up to seven items at a time. My hypothesis is that the memory system is divided into seven parallel branches or subsystems. In each subsystem there is a sort of preemptive competition going on between causal streams (or sequences) that prevents all but the strongest sequence from being active at a time. The resulting network diagram (I am working on it) will resemble a seven-branch candelabrum. I am not sure at this point how the seven branches are grouped. It seems that they should be grouped according to the various sensory modalities, i.e., visual, auditory, olfactory, pain/pleasure, etc... But then again, it could all be random. I will need to think a little bit more about this. In the meantime, I'll settle for just one branch as far as Animal is concerned. I don't think Animal needs more than one branch at this time since it has so few sensors compared to biological systems. The important thing is that Animal must have the ability to focus its attention on one thing at a time. Later, I may experiment with more branches to see if it makes a big difference. 

Lately I feel that I am on the verge of a major breakthrough in my research. Not that there has not been any breakthroughs already. I believe that the various principles on which Animal is based are conceptual breakthroughs in their own right. It is just that they have not resulted in something that others can readily assess. I now believe I have a good handle on the remaining problems of the project. I have a clear picture of the way short and long-term memory work together. I can now see how pain and pleasure inputs can be combined with anticipatory signals to form an effective motivational system that should behave according to what we know about classical and operant conditioning. Everything seems to be crystallizing rather quickly in my mind. It's rather exciting.

There is something I have been wanting to say to those of you who have been following the progress of Animal from the beginning. In researching this project, I draw upon several scientific sources of information such as neurobiology, psychology, discrete signal processing, etc... But, and this may sound weird, modern science is the least significant and the least reliable of my sources. I use it mostly to confirm my theories. There is another source that has guided me from the beginning and to which I attribute my most important discoveries. I cannot reveal the identity of my source at this time. Suffice it to say for now that it is rather ancient and that, when I do reveal its identity, many will be astounded and some may even take offense. Stay tuned.

I am fairly confident that my current approach to memory is the right one. Closely associated to memory and just as important, is the ability to focus. Here's what I am working on:

Psychology teaches us that the maximum number of items that can be stored in short-term memory is seven. How is that possible since, at any given time, there can be any number of signals arriving at the STM layer? One can only conclude that the brain has a way of suppressing all but seven of the most salient signal streams coming from the perceptual network. The mechanism of attention must not only be able to identify sequential signals as belonging to a given stream, it must decide which stream is salient and use this information somehow to directly suppress individual neurons in the STM layer. On top of it all, it must choose only seven streams or less to focus on.

The above paragraph is copied from the memory page. I have given the attention problem a lot of thought over the last few days and I think I now have a neural solution to the problem. I am working on ironing out the kinks and I will post an update to the memory page soon.

Take a look at the new memory page. I am still working on it but the gist of it is there. I am making relevant changes to Animal's code to reflect what I think is the final solution to the memory problem. I now realize that, in view of my current understanding of memory, the motivation page is no longer valid. There is more to come. Soon.

Older News