Temporal Intelligence

Latest News (2/16/2002)

Temporal Intelligence

Animal

Perceptual Learning

Perceptual Network

Memory

Motivation

Motor Learning

Software Reliability:
The Silver Bullet

Project COSA

AI Discussion Group

There Is a Time for Everything

Welcome
The Failure of GOFAI
The Real AI Problem
No Overarching Theory
Temporal Intelligence
Experimental Network

This site describes--among other things--an ongoing project to emulate biological intelligence in a computer. The chosen approach is based on the notion that animal intelligence is essentially a discrete, temporal signal processing phenomenon. The experimental setup used is a spiking neural network that learns to play chess through trial and error. The network starts out as a tabula rasa, i.e., it has no prior knowledge of chess or anything else. It bases its actions solely on the discrete temporal patterns of sensory and proprioceptive signals.

Most approaches to AI, especially the various knowledge representation schemes advanced by the GOFAI (good old-fashioned AI) community over the last fifty years, are missing the point about intelligence. Neither symbolic representation nor the current crop of artificial neural networks (ANN) have much to do with intelligence. The only intelligence we know is animal and human intelligence. Biological intelligence is what we should be trying to emulate in our machines. Yet it seems as if the GOFAI community has made it its mission to ignore every significant advance in neurobiology and psychology that has occurred over the last one hundred years. Even their ANNs bear little resemblance to biological neurons. Needless to say, they have failed to deliver on their original goal which was to create a machine with the intelligence of a human being.

By 1982, when GOFAI's failure to deliver on its promises could no longer be denied, Dr. Marvin Minsky of MIT, one of the leading luminaries of GOFAI from its very beginning, was saying that "The AI problem is one of the hardest science has ever undertaken." This has been the working assumption in GOFAI circles ever since. We are now regularly being warned by the artificial intelligentsia that progress in AI will be a slow incremental process and that it will most likely be another twenty years or so before true human-level AI becomes a reality. Is there any reason at this late time to take any pronouncement from the GOFAI crowd at face value? Fifty years of failure is not what most of us would call a good track record.

Amazingly, GOFAI proponents are trying to make a comeback. Just recently, MIT Technology Review published an article titled "AI Reboots" in which the author argues that "the focus of artificial intelligence today is no longer on psychology but on goals shared by the rest of computer science: the development of systems to augment human abilities." This begs the question, when was GOFAI ever focused on psychology? Science by redefinition is not progress. Intelligence, artificial or otherwise, is what psychology and neuroscience define it to be, period. The truth is that GOFAI scientists, having failed to deliver human-level intelligence and knowing all too well that they have no chance of ever doing so, are now trying to salvage what is left of their lost glory by pasting the AI label on every computer program that suits their agenda. This way they can claim successes (and secure more funding) even though none of what they are doing has anything to do with intelligence.

Certainly, solving the AI problem is hard if one has no clue as to what the problem is in the first place. If the assumption is that one must understand human cognition in order to develop human-level AI, then, of course, the problem is extremely hard. This is because the interconnectedness of human cognition is so astronomically complex as to be intractable to formal approaches. This realization immediately makes the use of symbolic knowledge representation approaches to creating human-like common sense in a machine look rather silly. Therefore the goal of the sensible AI researcher is not to develop a theory of cognition, but to discover the fundamental principles that give rise to intelligence. In other words, we must try to understand and replicate the basic neural mechanisms that will allow general intelligence to gradually emerge on its own. To do so, we must take a bottom-up approach and obtain as many clues as possible from neurobiology and from behavioral research in classical and operant conditioning. Above all, we must come to understand that fundamental principles are simple by virtue of being fundamental and that generality and complexity stem from simplicity. Those who doubt the power of simplicity should examine the work of Stephen Wolfram and Edward Fredkin.

Even though a lot is known about the detailed operation of many types of neurons and the architecture of various cell assemblies, there is no overarching theory to explain the brain's internal operation. Neurobiologists know that the brain processes signals but they cannot explain how signal processing gives rise to intelligent behavior. What does the brain really do? This is the question that I will attempt to answer in these pages.

I will introduce several general principles that an intelligent system designer can use to build a machine that uses sensors and effectors to learn from its environment and coordinate the selection of its actions. All the principles are simple, scalable and applicable to any learning task. Why not a single principle of intelligence? Because an intelligent system is not a homogeneous block with a single uniform architecture, but a tightly integrated collection of signal processing modules or layers, each with its own function and corresponding architecture.

Some AI researchers (e.g., Doug Lenat) are interested only in knowledge, the sort of knowledge that can be expressed linguistically. Their goal is to represent this knowledge in a machine using symbols and symbol manipulation algorithms. In my research I take the exact opposite approach. I am only interested in how the brain builds its knowledge, i.e., in how the evolution of sensory stimuli conspires to induce brain connectivity. I have no interest whatsoever in coming up with representational ways to store knowledge about the world (e.g., fruits, plants, animals, etc...) in a computer. There is a lot more to knowledge than the classification of namable objects and their relationships. There is a huge amount of knowledge that cannot be formalized with symbols. For examples, recognizing a subtle fragrance, a face or a musical tune, manual dexterity, finding one's way around an unfamiliar neighborhood, in other words, the sort of common sense knowledge that can only be acquired through direct sensory experience. I subscribe to the notion that all knowledge can be built automatically using a common underlying mechanism. The purpose of this site is to elucidate what this mechanism is and to find ways to implement it in a general learning machine.

One of the underlying premises of my research is that animal intelligence is essentially a temporal, discrete signal processing phenomenon. The biological evidence is clear on this issue: neurons generate and transmit discrete spikes or signals. The temporal nature of intelligence has been known for quite some time. Back in 1949, Donald Hebb proposed a temporal learning rule for neurons and cell assemblies that has exerted a strong influence on theories of neural learning. Psychologists have developed an entire science of operant behavior based on the timing of stimuli and responses. However, it was not until the latter part of the twentieth century, with the groundbreaking work of people like Terrence Sejnowski and Henry Markram, that neurobiologists began to appreciate the extent to which the brain's operation is dependent on the precise timing of neural spikes.

One of the most powerful aspects of the temporal approach to intelligence is that it is based on a single unifying concept: the relative arrival times of discrete signals. Its power is in its simplicity. At the heart of this approach is the claim that all knowledge, regardless of type, consists of patterns of discrete signals, a pattern being defined as a set of temporal relationships. Although the number of possible patterns is unlimited, all patterns can be expressed in terms of only two fundamental relationships: signals can be either concurrent or sequential. A second beneficial aspect of temporal intelligence is that it is domain-independent. That is to say, it makes no assumptions about either the origin (sensors) or the destination (effectors) of signals. Finally, a temporal system is ideally suited to the goal of emulating what is probably the most important attribute of biological intelligence: the ability to learn to anticipate or predict the evolution of various phenomena in one's environment. 

One often hears the phrase 'spatio-temporal intelligence' bandied about. The problem with characterizing intelligence as being anything other than temporal is that it overlooks the significance of one of the most important discoveries of neurobiology in the last century: all sensory phenomena are converted into discrete spikes prior to being processed by the brain. There is nothing in a spike that identifies its origin from the point of view of a receiving neuron. A spike is just a temporal marker that indicates that something just happened. All spikes look pretty much alike. In other words, there is no such thing as a spatial, auditory, visual or olfactory spike. The only thing that distinguishes one spike from another is its time of arrival. Thus spatiality and other modalities only have to do with the sensor and/or effector side of an intelligent system, i.e., with their physical type, distribution, position, etc..., not with the way the network processes signals.

This is not to say that actual physical connections do not have types. They do and, as I will show, this fact is used to excellent advantage as a way to minimize the proliferation of connections during sensory learning. The important thing to note is that connection types are irrelevant to the signal processing mechanisms of individual neurons. This is evidenced by the observation that the brain uses similar neurons to process signals originating from vastly different sensory organs. Indeed, one of the most striking things about the brain is the uniformity of its cell assemblies.

In the light of the universality that is inherent in the use of a single fundamental cognitive building block, it is not surprising that human beings are so adept at making analogies involving seemingly disparate concepts. Concepts are easy to compare if they are all expressed in a common "language." 

The experimental setup that I use for my research project is a chess learning program called Animal. Chess is a complex enough causal environment that can be easily simulated on a computer without much expense in time and money. A neural network that can learn to play chess from scratch through trial and error would certainly be proof of intelligence. Still, I would rather conduct my research using a multi-legged, spider-like robot. A real-world robot with many degrees of freedom and a full complement of sensors (visual, auditory, tactile, etc...) is an ideal platform with which to demonstrate true general intelligence. Learning to navigate through a changing environment while coordinating multiple legs in real time is a very complex problem that cannot be solved using a deterministic and/or symbolic approach. Unfortunately, such a project is beyond what I can afford at this time.

Animal is written in C++ for the MS Windows® operating system. Feel free to download the zipped executable and play with it. Eventually I will make the entire source code available for downloading. I suggest you read Animal's specifications before reading the theoretical sections. The section that describes the spiking network is especially important. Note that this is an ongoing research project, the ultimate goal of which is to build a general, scalable, adaptive, intelligent machine. I would appreciate sensible suggestions and comments from readers.