published in: Proceedings on the Third Workshop on Neural Networks:
Academic/Industrial/NASA/Defense (WNN 92 Selected Papers)
Auburn University, Alabama 2/10-2/12 1992
Neural Network and Other Self Organizing System Simulations
Through
Container Classes in Object Oriented Programming
Deborah Vakas Duong Kevin D. Reilly
Computer & Information Sciences
University of Alabama, Birmingham
ABSTRACT
Object oriented programming with container classes is an excellent
tool for the simulation of self organizing systems because it represents
system entities themselves, is declarative, is modular, can be of
arbitrary detail, and facilitates micro/macro inquiries. With dynamical
allocation of container classes, a system's structure, such as a neural
network's topology, may be changed during simulation. A hierarchically
arranged topologically dynamic IAC neural network is described. An
example of its use in a dynamical sociological simulation is given.
KEYWORDS: Neural Net, Self-Organization, Object-Oriented
System, Container Class, Sociological Model
Self Organization and Object Orientation
The Brussels school approach to simulation of complex systems, originating
with Nobel Prize winning physicist Ilya Prigogine, is recognized to
be one of the most creative methods of simulation. These simulations,
based on the theory of self organization, capture the essence of the
unpredictable system change which occurs in biological and sociological
complex systems.
Yet some have objections to the methods used. Geographer Roger White
claimed that the differential equations used in most of these simulations
are an unnatural framework for dealing with structural evolution,
and that although convenient, these equations can not represent deep
change because they do not represent the actual entities of a simulation.
For example, a variable may represent the level of heavy industrial
activity, but nowhere are heavy industry employees represented. Changes
in functional organization can not be represented because it would
involve changes in the constituent entities of the system, the equations
themselves. Since models usually represent either the people or the
firm but not both, we can not ask questions such as "How do high
level structures depend on low level entities ?" and "How are
lower level entities influenced by high level structures?" (White
1989).
Kenyon B. DeGreene (1989, p.34) emphasized the importance of such
questions: One of the most fundamental problems, if not the
most fundamental problem, in science is the description and prediction
of how interrelations at a more microlevel of organization of matter,
energy, and information generate emergent, qualitatively new properties
at a more macrolevel."
Object-oriented programming with container classes provides a natural
framework for making these inquiries of systems which change dynamically
because it represents the entities themselves. Object oriented programming
provides the arbitrary detail of object features and object interactions
needed to provide a source of realistic fluctuation to drive self
organization. Such details are important in determining the system
state at bifurcation points. With dynamic function binding and dynamic
object allocation, whether an object comes into play or even exists
may depend on the dynamic state of the system, making for liquidity
in system outcome. The container class allows us to represent deep
structural change.
Container Classes in Simulation
When one thinks of hierarchy and object oriented programming, one
usually thinks of inheritance classes. The inheritance class is the
most popular and most supported type of class in the object oriented
paradigm.
Jeff Rothenberg at RAND corporation recognized that simulations
need to represent other types of relationships, such as container
relationships, as well. Wiener and Pinson (1990, p.110) defined a
container class to be a "class that contains one or more objects
or pointers to objects in its class protocol."
Inheritance classes represent the "Is A" relationship while container
classes represent the "Has A" relationship. As the representation
of the decomposition of an object rather than its taxonomy, container
classes are just as important as inheritance classes to simulation.
While inheritance classes provide a useful method of conceptualizing
objects in terms that the mind can grasp, container classes provide
a way of categorizing objects which is good not only for conceptual
purposes, but reflects true categories inherent in nature: whether
something is composed of another thing is more falsifiable than whether
it is similar to another thing. For example, it is of little controversy
that a society is composed of people and that people are composed
of cells, but the base class of a taxonomic scheme which would contain
traits common to all societies would be different for every social
scientist.
Because it is a more direct representation of nature, the use of container
classes facilitates our questions about nature, in particular our
inquiries on micro-macro level interrelations. Each container class
of a system represents one of the micro/macro levels in a hierarchy.
The descriptions of the objects at a level and the declarative rules
that govern their behavior at that level are encapsulated together
in a class. Having a different set of declarative rules tied to each
level is natural because "the forces in a field, the laws and
especially the variables are likely to be vastly different in the
different areas of science" (DeGreene p. 34).
Note that when you decompose something into its levels, you may also
transcend the various scientific disciplines. Societies are the
subject of social science while at other levels of their hierarchy,
cells are the subject matter of biology and dissipative processes
at the intracellular level are the subject matter of chemistry. Although
we may use biology to explain sociology, we do not reduce the laws
of sociology to the laws of biology. By encapsulating a level and
the laws that govern it together, there need be no reductionism involved,
but rather the creation of a hierarchical system where one level is
holistically composed of the next lower level of objects and their
behavior, each level being more than the sum of its parts.
The idea of a holistic hierarchy comes from the self organization
paradigm and is advocated by social scientists such as Erich Jantsch.
Jantsch (1980, p.243) claimed that although living systems (including
social systems) are hierarchically arranged, their dynamics are coordinated
not in a control hierarchy where orders go down a ladder and information
goes up, but in a stratified autonomy.
The declarative as opposed to procedural nature of object oriented
programming (Widman and Loparo p.6) facilitates distributed control
among classes/levels. Upper level system traits arise from the distributed
control of the lower level members acting upon each other in parallel.
The macro level structure unfolds naturally from the parts. In a
self-organizing hierarchical system, interactions only occur between
one level and the next, not across levels, because of their holistic
composition.
This organization is naturally implemented in container classes: it
is easier to design them so that interaction occurs only with contiguous
levels than it is to design them so that interactions may occur across
levels. Object oriented programming with container classes provides
a tool to approach the micro/macro problem, with which the scientist
may "work back and forth between the levels, performing analysis
and synthesis, and developing laws that explain and predict how one
level effects another" (DeGreene p.34).
Programming with container classes would facilitate Rothenberg's goal
of "building simulations whose level of aggregation can be varied
either statically or dynamically, as appropriate to the user's purpose"
(1989 p. 88). The modeler could change any level of interaction and
observe the effects on the entire system or just focus on a particular
level as he pleases.
The modeler could develop a tool box of container classes which would
grow as new classes are verified, and develop new theories based on
previously verified models, needing only to program the inquiry at
hand. Our object oriented neural network system
with container classes has just such dynamically variable and
modular properties.
ANN Modelling with Container Classes
The container class is an appropriate tool for representing neural
networks because their concept comes out of the self-organization
paradigm. Physicist John Hopfield re-ignited interest in neural networks
with a 1982 article entitled "Neural Networks and Physical Systems
with Emergent Collective Computational Abilities." Neural networks
have emergent properties resulting from the collective actions of
their constituent neurons: a neural network is more than the sum of
its parts.
Our object oriented IAC network is based on the PDP research group's
procedural IAC network (McClelland and Rummelhart 1988), with the
additional property of a changeable architecture: nodes can be added
and deleted as needed while the network is in use. The container class
structure facilitates changeable architecture because classes contain
not specific objects but pointers to dynamically allocated objects
and routines that create and delete them as necessary.
Dynamic connections in neural networks are important to biological
modelers (Feldman 1982). They were important to Roger White in his
use of neural networks to simulate urban structural evolution: "A
neural net model with an ability to evolve its topological structure
in addition to simply altering its quantitative structure as represented
by the weights would obviously be a much more powerful tool for investigating
structural evolution" (White, 1989).
In our group's work, the object-oriented IAC with changeable architecture
has been used in an expert system application (Krishnamraju et al.
1991; Reilly et al. 1991) and by the author to model how people's
associations of each other create emergent properties of societies
in a multilevel self-organizing system.
Hierarchy in the IAC Model
In our IAC model, the highest level of hierarchy is class iac_network
which is composed of pools and routines that act on the network as
a whole by manipulating the pools. Class pool, the next level, is
composed of nodes and procedures to manipulate the nodes. Class node
is composed of synapses and axonal connections and procedures that
effect the node by changing itself, its synapses, and its axons.
Axons contain another node's synapses and class synapse, the most
basic level, contains procedures that allow it to be manipulated.
Figure 1 provides an illustration of the hierarchical structure.
One level, in general, only effects the next: there is little manipulation
across levels. The emergent properties of the network as a whole
are the direct result of the autonomous interactions of the neurons:
if the rules of neuron (node) activation or the number of nodes in
a pool is changed, the emergent network state will be different.
The modeler knows exactly how to make such inquiries into micro/macro
interrelations by virtue of the system's container class organization.
For example, to ask how a change in a node's activation rule affects
the network's final outcome, it is obvious that a change to class
node must be made.
With respect to modularity, an inheritance class of the class node
may be used in any other neural network design and the class pool
may be used in any network which has mutually inhibitory nodes (such
as competitive learning). Inheritance classes may be used in conjunction
with any of the container classes for ease in the creation of different
taxonomies of neural networks. The classes of this IAC network thus
form the beginnings of a tool box: a tool box that will help build
and come to include various dynamic and static network architectures
as well as higher level psychological and sociological network classes.
Modular Architecture
The interactive activation and competition model is a descriptive
name for the IAC network's architecture. This network is divided
into several pools of nodes, each node having inhibitory connections
with other nodes in its pool. These inhibitions put the nodes in
competition with each other: the stronger one node is activated, the
more it is able to turn other nodes in the same pool off. Every node
in each pool is also connected to a node in a special pool with no
intra-pool inhibitions, the instance pool. The activation is interactive
because the bidirectionally excitatory connections provide the route
by which what is going on in one pool influences (while it is influenced
by) what is going on in the other pools.
This architecture serves as an inductive associative memory: it can
arrive at a general conclusion based on specific examples. The pools
represent "traits" : in the authors' sociological simulation they
represent traits such as type of suit, color of skin, and talent in
performing a particular job. Each instance node connects a single
set of associations in the pools: in the authors' simulation they
represent a person (or rather, they represent a cognitive representation
of a person). Each of the instance nodes has an excitatory connection
with the single trait in each pool that the person it represents has.
Figure 2 provides an illustration of an IAC network with sample individuals having
a variety of personal attributes.
Model Behavior
To ask a question of the network, you activate (clamp) a node. To
ask "What kind of people wear inexpensive suits?" you clamp node
inexpensive in the pool for suits, and let the activation circulate
through the network until each pool settles on a solution. The "solution"
will be the node that is most activated in a pool. The answer you
get will be a general, holistic answer, not only taking into account
what other traits most people with suits have, but more recursively,
"What traits do the people who have the traits that people who
have inexpensive suits have?" and even more recursively until every
correlation is taken into account. Figure 3 illustrates an IAC network
in C++ and its use in queries.
The holism in the network model is responsible for the network property
of graceful degradation: it can guess a missing trait given incomplete
data. This is an important property for the authors' simulation:
based on associations of observable traits and talent of employees
he has had, an employer tries to guess the amount of talent an applicant
has by observable traits alone. In an IAC network representing his
mind, he clamps traits for the applicant's suit and skin color to
see what talent comes up.
Even though he may never have seen a particular combination of traits
before, the holistic recursion of the network will come up with an
answer, a testimony to the superiority of neural networks to expert
systems in dealing with something new.
An Application in Sociological Dynamics
The authors' sociological simulation is a hierarchical system in which
the micro-level psychological decisions an employer makes about employees
and the decisions employees make about themselves give rise to macro
level sociological phenomena such as racial class and status symbols.
Class society contains class employer and class workforce, which contain
class worker, which contains class iac_network. While an employer
uses his IAC network to identify and hire more highly talented people
after dismissing mostly untalented ones, workers use their IAC networks
to change their traits, if they can, so that they will become employed.
They can not change their skin color, but they may buy a suit with
money obtained from employment, and may change a fad for free.
A neural network as opposed to an expert system is needed to allow
associations on new combinations of traits. An IAC network is useful
because it is a good representation of a generalized opinion - an
intuitive, irrational feeling about a person based on experience and
not the logic of expert systems. Dynamical topology is needed so
that the employer may add new associations to his network as he employs
new people and discovers their talent. Job applicants need to update
their network topology so as to keep track of the latest combination
of traits that will get them a job.
The dynamic self organization of the network at the micro (psychological)
level results in a dynamic simulation at the macro (sociological)
level which, like the micro level, dissipates to its own solution,
of racial class and status symbols. While employers and workers adapt
to each other, they change their minds, their minds change them,
and society unfolds.
C++ Source
#include <stream.h>
#include "iac.h"
main()
{
iac_network* iac = new iac_network(NUMPOOLS);
// Add to network
//Name Race Clothes Talent
//Janet Black Inexpensive High-Level
//Susan White Expensive Average
//Bill White Expensive High-Level
iac->read_file("second_employee_group");
iac->reset();
// Ask what very talented people are like by
// clamping the node for high-level (node 1)
// in the talent pool (pool 4).
iac->clamp_node ( 4,1, CLAMPVAL);
iac->cycle(NCYCLES);
iac->print_all_activations();
// Add to network
//Name Race Clothes Talent
//Tom White Expensive Average
//George White Inexpensive High-Level
//Joe Black Inexpensive High-Level
iac->read_file("first_employee_group");
iac->reset();
// Ask the same question again
// The answer to the inquiry is the node in each pool
// with the most activation in the output.
iac->clamp_node ( 4,1, CLAMPVAL);
iac->cycle(NCYCLES);
iac->print_all_activations();
delete iac;
}
Node Activations
First Inquiry Second Inquiry
Instance Pool (Pool 0)
Janet (Node 0): .50306 .631489
Susan (Node 1): .321854 .358361
Bill (Node 2): .591127 .579419
Tom (Node 3): .358361
George(Node 4): .651391
Joe (Node 5): .631489
Name Pool (Pool 1)
Janet (Node 0) : .163386 .158792
Susan (Node 1) : -.109998 -.116267
Bill (Node 2) : .212608 -.02101
Tom (Node 3) : -.116267
George(Node 4): .169784
Joe (Node 5) : .158792
Race Pool (Pool 2)
Black (Node 0): .089671 .368562
White (Node1): .361127 .555492
Clothes Pool (Pool 3)
Inexpensive (Node 0): .089671 .590033
Expensive (Node 1): .361127 .283094
Talent Pool (Pool 4)
Average (Node 0): -.135509 -.121552
High-Level (Node 1): .819476 .853136
Figure 3. A Program Using Class iac_network, in C++. This
program asks the network for the ideal "form" of the highly talented
person based on what it knows about three people. It then asks the
same question after three more people are added to the network. At
first, the iac thinks of a highly talented person as being white with
an expensive suit. This is due to the "ganging" effect: two out
of three people have the same set of traits (white and expensive)
and because they are in the majority, they have a greater impact on
the answer, while the highly talented minority are not considered.
When more people are added, white remains in the majority, but the
number of inexpensive suits becomes equal to the number of expensive
ones and correlated with high talent so that inexpensive suits are
now preferred. The network now imagines a highly talented person
to be white with an inexpensive suit. The network's impression is
of a "typical person" of high talent: it is intuitive but against
reason because every black person in the simulation has high talent,
but only half of the white people do.
References
DeGreene, Kenyon. "Micro-Macro Interrelations and the Order Parameter
Concept in the Field Theory of Societal Systems." Systems
Research, 6, 4, 277-288, 1989.
Feldman, Jerome A. "Dynamic Connections in Neural Networks," Biolo
gical Cybernetics, 46, 27-39, 1982.
Hopfield, J.J. "Neural Networks and Physical Systems with Emergent
Collective Computational Abilities," Proceedings of the National
Academy of Science, 79, 2554-2558, 1982.
Jantsch, Erich. The Self Organizing Universe. New York:
Pergamon Press, 1980.
Krishnamraju, P. V., Reilly, K. D. and Y. Hayashi. "Neural and
Conventional Expert Systems: Competitive and Cooperative Schemes."
Proceedings of Second Workshop on Neural Networks: Academic-Industrial-N
ASA-Defense 90, (WNN-AIND 91) Auburn, AL. February 11-13, 1991.
649-656.
McClelland, James L. and David Rummelhart. Explorations in
Parallel Distributed Processing. Cambridge: MIT Press, 1987.
Reilly, K. D., Hayashi, Y, Duong, D., and P. V. Krishnamraju. "Cooperative
Schemes for Conventional and Neural Expert Systems." Int. Joint
Conf. Neural Networks. Submitted.
Rothenberg, Jeff. "The Nature of Modelling." In: Widman, Loparo
and Nielson, eds., Artificial Intelligence, Simulation and
Modeling. New York: John Wiley and Sons, 1989.
White, Roger. "The Artificial Intelligence of Urban Dynamics:
Neural Network Modeling of Urban Structure," Papers of the
Regional Science Association, 67, 43-53, 1989.
White, Roger. "Structural Evolution in Urban Systems," Systems
Research, 6, 3, 245-253, 1989.
Widman, Lawrence E. and Kenneth A. Loparo. "Artificial Intelligence,
Simulation and Modeling: A Critical Survey." In: Widman, Loparo and
Nielson, eds. Artificial Intelligence, Simulation and Modeling.
New York:John Wiley and Sons, 1989.
Wiener, Richard S. and Lewis J. Pinson. The C++ Workbook.
Reading, Massachusetts:Addison-Wesley, 1990.