Sensor Fusion:

A Potential Application of the Ersatz Brain Architecture


One possible application of our Ersatz Brain architecture is to the important and difficult problem called sensor fusion.


Sensor fusion roughly means integration of information from different types of sensor into a unified interpretation.


Humans are so effective at doing information integration that it is not appreciated how hard it is to do.


This observation also has been made, usually after the futile expenditure of great amounts of money, for:


  • computer vision,
  • machine translation,
  • natural language processing, and
  • text comprehension.




The De-Interleaving Problem.


About a decade ago, I was involved in a project in collaboration with, at first, Texas Instruments and, later, with a local company called Distributed Data Systems, Inc.


The project was an attempt to solve the de-interleaving problem
in radar signal processing using a neural net.


This problem is from military electronics but in fact is a cognitive one.


In a complex radar environment, the problem is to determine how many radar emitters are present and whom they belong to.


Biologically, this corresponds to the behaviorally important question, “Who is looking at me?”


To be followed, of course, by “And what am I going to do about it?”



A receiver for radar pulses provide several kinds of quantitative data:


  • frequency,
  • intensity,
  • pulse width,
  • angle of arrival, and
  • time of arrival.



The user of the radar system wants to know qualitative information:


  • How many emitters?
  • What type are they?
  • Who owns them?
  • Has a new emitter appeared?


The way we proposed to solve the problem was by using a “concept forming” model from cognitive science.


Concepts are labels for a large class of members that may differ substantially from each other. (For example, birds, tables, furniture.)


We built a system where a nonlinear network developed an attractor structure where each attractor corresponded to an emitter.


That is, emitters were valid concepts.






A weakness is that it does not allow development of the kind of structures that characterize human concepts.


One of the most useful computational properties of human concepts is that they often show a hierarchical structure.



Examples might be:


animal > bird > canary > Tweetie




artifact > motor vehicle > car > Porsche > 911.





Sensor Fusion with the Ersatz Brain.


We can do a kind of sensor fusion in a simple and way in the Ersatz Brain.


Let us use a radar-like example.


The data representation we suggest is directly based on the topographic data representations used in the brain.


The appropriate software approach to the Ersatz Brain array it is topographic computation.


Spatializing the data, that is letting it find a natural topographic organization that reflects the relationships between data values, is a technique of great potential power.


It is a natural and effective way to compute for the two dimensional cortex.


And our two dimensional Ersatz Brain architecture.


Spatializing the problem also provides a way of “programming” a parallel computer.


(See our arithmetic application, Anderson, 2003).


Topographic Data Representation



We initially will use a simple bar code to code the value of a single parameter.




The precision of this coding is low.


This was one of the aspects of our approach that most disturbed traditional radar engineers: we deliberately threw out their hard won precision.






For our demo Ersatz Brain program, we will assume we have four parameters from the source.


An “object” is characterized by values of these four parameters, coded as bar codes on the edges of the array of CPUs.




Our architecture assumes local linear transmission of patterns from module to module.




The geometry we have used for the demonstration gives many two parameter interference patterns, the straight lines.


Each pair of input patterns gives rise to an interference pattern, a line perpendicular to the midpoint of the line between the pair of input locations.





Higher Level Features.



There are often places where three or even four features coincide at a module.


The higher-level combinations represent partial or whole aspects of relations of data in the input pattern. T


In this sense the higher level combinations have literally fused a number of spatial relations of the input pattern.


They have represented it as a new activity pattern at a specific location.





Formation of Ersatz Hierarchical Concepts.


This approach allows for the formation of what look like hierarchical concept representations.


Suppose we have a set of “objects”.


We have three parameter values that are fixed for each object and one value that varies widely from example to example.


The system develops two different types of spatial codes.


  • In the first, a number of high order feature combinations are fixed since the three input core patterns never change.


  • In the second, based on the additional spatial relations generated by the widely different examples, there is a varying set of feature combinations corresponding to the details of each specific example of the object.


Two kinds of pattern can potentially be learned.


  • The first learned pattern corresponds to the unchanging core and might correspond to an abstraction of the commonalities of many examples.


  • The second learned set of patterns corresponds to the core, plus the patterns associated with each specific learned example.


  • The specific examples are related by their common core pattern.

Core Representation


Three Input Patterns.



The group of coincidences in the center of the array is due to the three input values arranged around the left, top and bottom edges.


There is no right hand value.


This group of three constant values plus their feature combinations constitutes the core.



Below are two examples when there is a value on the right side of the array.


Examples Core


Note the presence of a number of new three or four
feature coincidences.


The original group based on the three core values is still present as we would expect it to be.


There are many more feature coincidences present with all four values. The coding rapidly becomes richer in terms of its possibilities for interaction as more information is added because there are more possible coincidences.

Development of A “Hierarchy” with Spatial Localization.



The coincidences due to the core (three values) and to the examples (all four values) are spatially separated.


We have the possibility of using the core as an abstraction and representation of the examples since it is present in all of them.


We could use it by itself as the descriptor of the entire set of examples.


It acts as the higher level in a simple hierarchy, that is, all examples contain the core.


The many-to-one relationship here – many low level examples, fewer high level examples — is typical of hierarchical semantic networks.





The Ersatz Brain Project has led us down an interesting path.


If we start to require software to use the general constraints that we feel characterize a real brain-like computer,


Then some new ways to tackle old problems emerge.


  • New “analog” control structures: Programming patterns to do arithmetic. Looks more like attention than anything else.


  • Spatializing problem data both as initial representations and to form feature combinations.


  • Potential emergence of hierarchical structure.


  • Disambiguation using context and semantic networks.


But these tasks are elements of some important software applications.


These ideas might be of value for current computers.


But I feel that their real domain of application will be to the computers of the future.

Published by

Naveen bhardwaj

The Author Naveen Bhardwaj is Chief Engineer R&D at Envirotech India, Completed M.Tech. Instrumentation from NIT Kurukshetra

Leave a Reply