Biological Neural Systems

to
Artificial Neural Systems – An Evolution

V. Sree Hari Rao

Fellow of National Academy of Sciences, India
Fellow of Institution of Electronics & Telecommunication Engineers, India
Fellow of Forum D’Analystes
Professor of Mathematics
Jawaharlal Nehru Technological University
Hyderabad - 500 072, India

Email: vshrao@yahoo.com
URL: http://www.geocities.com/vshrao

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 1 / 30
 FUNDAMENTAL OBJECTIVE OF COMPUTING IS

to develop Intelligent Systems

Intelligent System:

learns during its existence

continually acts, mentally and externally, and by acting reaches its objectives

functions with high degrees of autonomy,

robustly handles data with large amounts of noise,

rapidly processes large amounts of data in a massively parallel fashion,

learns from the environment with minimal human intervention,

evolves to become better adapted.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 2 / 30
Nature’s Influence on Scientific Investigations

Nature is influencing researchers in many ways. History shows that many scientific
investigations were motivated by natural phenomena.

BIRDS AIR CRAFTS

INSECTS ROBOTIC
MOVEMENTS

ANTS SWARM BOTS

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 3 / 30
Nature’s Influence on Scientific Investigations (Contd…)

FISH SHIP

HORSE HORSE CART

RESISTANCE
SPIDER MATERIALS
WEB

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 4 / 30
Bio-inspired Computing:

focuses on set of techniques inspired by biological sciences. Biological organisms often exhibit
properties that would be desirable in computer systems. Some features of bio-inspired
computing include:

tackling complex problems using computational methods modeled after design principles
encountered in nature.

producing informatics tools with enhanced robustness, scalability, flexibility, and which
can interface more effectively with humans.

It is a multi disciplinary field strongly based on biology, mathematics, computer science,

informatics, cognitive science, control theory and robotics.

The development and progress of computer science, engineering and technology has greatly
contributed to the study of biological systems and sciences to gain a better understanding of
biological processes and functions through the modeling and simulation of natural systems.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 5 / 30
Bio-Inspired Techniques:
The Present day researcher generates new ideas by taking advantage of the examples provided
by the nature.

Biological Systems / Process Bio-Inspired Systems

EVOLUTION GENETIC ALGORITHMS

LIFE CELLULAR AUTOMATA
INSECT COLONIES, ANTS, BEES etc EMERGENT SYSTEMS
THE BRAIN NEURAL NETWORKS
LIFE ARTIFICIAL LIFE
IMMUNE SYSTEM ARTIFICIAL IMMUNE SYSTEMS
PLANT STRUCTURES LINDENMAYER SYSTEMS
INTRAMEMBRANE MOLECULAR PROCESSES MEMBRANE COMPUTERS
IN LIVING CELL
DNA and MOLECULAR BIOLOGY DNA COMPUTING
BIODEGRADATION BIODEGRADABILITY PREDICTION
FOREST FIRES EXCITABLE MEDIA

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 6 / 30
The Neuron:
Cells of nervous system or nerve cells are commonly called neurons. The purpose of the neuron
is to transmit messages through an electrochemical process throughout the nervous system.
These cells come in different shapes and sizes.

The neuron consists of CELL BODY / SOMA, the AXON, DENDRITES and the NEURONAL
MEMBRANE

The typical cell body of a neuron measures

about 20 micrometers in diameter.

The Axon is specialized for transferring

information over a certain distance in the
nervous hillock.

Dendrites are the branches that are coming

out of the neuron. These act as the antennae
of the neuron.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 7 / 30
The Neuron

Structure of the Biological Neuron

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 8 / 30
Connections between two Biological Neurons

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 9 / 30
The Biological Brain
The brain is a complex multi-processing system. To
simplify thinking process, regular activities are
routinized so that they require less brain activity
and we do not have to attend to them consciously:

“As a task to be learned is practiced, its performance

becomes more and more automatic; as this occurs, it fades
from consciousness, the number of brain regions involved in
the task becomes smaller.”
Edelman & Tononi

Thus, the routine processes are performed

subconsciously while the conscious “focus of
attention” or ‘short – term working memory’
attends to the important aspects.

The structure of the human brain differs from that of other animals in several important
ways. These differences allow for many abilities over and above those of other animals, such
as advanced cognitive skills. Much of human brain structure is similar to that of other
mammals. The human brain also has a massive number of synaptic connections allowing for
a great deal of PARALLEL PROCESSING

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 10 / 30
Biological Neural Nets:

Biological Neural Nets are made up of real

biological neurons that are connected or
functionally – related in the peripheral nervous
system or the central nervous system. In the
field of neuroscience, they are often identified
as groups of neurons that perform a specific
physiological function in laboratory analysis.

A single neuron can be connected to many

other neurons and the total number of neurons
and connections in a network can be extremely
large.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 11 / 30
Functioning of the Biological Neuron
The brain is a collection of about 10 billion interconnected neurons. Each neuron is a cell that
uses biochemical reactions to receive, process and transmit information.

A neuron’s dendritic tree is connected to a thousand neighboring neurons. When one of

those neurons fire, a positive or negative charge is received by one of the dendrites.

The strengths of all received charges are added together through a process of spatial and
temporal summation.

Spatial summation occurs when several weak signals are converted in to a single large
one, while temporal summation converts a rapid series of weak pulses from one source in
to one large signal.

The aggregate input is then passed to the soma or cell body. The soma and enclosed
nucleus don’t play a significant role in the processing of incoming and outgoing data.
Their primary function is to perform the continuous maintenance required to keep the
neuron functional.

The part of Soma that does concern itself with the signal is Axon hillock. If the aggregate
input is greater than the axon hillock’s threshold value, then the neuron fires, and an
output signal is transmitted down the axon.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 12 / 30
Functioning of the Biological Neuron (Contd..)
The strength of the output is constant; regard less of whether the input was just above the
threshold or a hundred times as great. The output strength is unaffected by the many
divisions in the axon; it reaches each terminal button with the same intensity it had at the
axon hillock.

Each terminal button is connected to the other neurons across a small gap called a
Synapse. The physical and neuro-chemical characteristics of each synapse determine the
strength and polarity of the new input signal.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 13 / 30
A Schematic diagram of a biological neuron

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 14 / 30
The Brain and the Computer
BRAIN COMPUTER
The brain uses chemicals to transmit information. The computer uses electricity.
Memories in the brain grow by stronger synaptic Computer memory grows by adding computer chips.
connections.
It is much easier and faster for the brain to learnThe computer can do many complex tasks at the same
new things. time that are difficult for the brain.
The human brain has weighed in at about 3 pounds Computers have evolved much faster than the human
for about the last 100,000 years. brain. The technological developments have made
computers faster, smaller and more powerful.
There are no new or used parts for the brain. Some It is easy to update computer with new parts.
work is being done with transplantation of nerve
cells for certain neurological disorders such as
Parkinson’s disease.
There are diseases that affect the brain. Computer can get a ‘VIRUS’.
The brain is always changing and being modified. The computer only changes when a new
There is no ‘OFF’ for the brain. Even when an hardware/software is added or some thing is saved in
animal is sleeping, its brain is still active and the memory. There is an ‘OFF’ for a computer when
working. the power is turned off.
The brain is better at interpreting the outside The computer is faster at doing logical things and
world and coming up with new ideas. computations.

‘There is more we do not know about the brain, than what we know’

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 15 / 30
Artificial Neural Networks (ANNs)

are inspired by neuro - scientific studies of the structure and function of human brain

are made up of interconnecting artificial neurons designed to mimic some properties of

biological neural networks.

use a mathematical or computational model for information processing based on

connectionist approach to computation.

are adaptive systems that changes their structure based on external or internal
information that flows through the network.

are similar to the biological neural networks in the sense that functions are performed
collectively and in parallel by the units.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 16 / 30
Artificial Neural Networks (Contd…)

Thus ANN is

an extremely simplified model of the brain.

essentially a function approximator.

Transforms inputs in to outputs to the best of its ability.

Inputs Outputs

Inputs Outputs
NN

composed of many neurons that co-operate to perform the desired function.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 17 / 30
Why use Neural Networks?

Neural networks, with their remarkable ability to derive meaning from complicated or
imprecise data, can be used to extract patterns and detect trends that are too complex to be
noticed by either humans or other computer techniques. A trained neural network can be
thought of as an "expert" in the category of information it has been given to analyze. This
expert can then be used to provide projections given new situations of interest and answer
"what if" questions.

Other Advantages
Adaptive learning: An ability to learn how to do tasks based on the data given for training
or initial experience.

Self-Organization: An ANN can create its own organization or representation of the

information it receives during learning time.

Real Time Operation: ANN computations may be carried out in parallel, and special
hardware devices are being designed and manufactured which take advantage of this
capability.

Fault Tolerance via Redundant Information Coding: Partial destruction of a network leads
to the corresponding degradation of performance. However, some network capabilities may
be retained even with major network damage.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 18 / 30
Mathematical Modeling

Basic nomenclature and facts:

System is the connected units or parts that form a whole and operate together.

Function of the system depends not only on the functions of the constituents but also on how
they connect to one another.

An open system is a system that is influenced by its environment, where as a closed system is
one isolated from its environment

The assumptions of a closed system under certain restrictions is a means to simplify the
analysis of system behavior.

System Theory is the study of system behavior. The behavior refers to external input /
output characteristics and internal state changes. The behavior can be described in terms of
qualitative or quantitative relationships among variables defined for the system. A system can
be imitated or simulated by something which we call a MODEL.
A system may be represented by a block diagram. In this representation, the system is treated
as a ‘black box’ whose internal behavior is opaque, and its behavior is described by external
input / output relationships.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 19 / 30
Mathematical Modeling in General
In scientific investigations, a model is constructed to explain a hypothesis or to simulate a
practical system (as shown in the fig. below)

Abstraction

Real Physical System Mathematical Model

Simulation

Fig. An overview of the problem of mathematical modeling

An experiment is performed to gather the relevant data for estimating model parameters or
evaluating the model. A model needs to be refined or replaced if it does not provide sufficiently
accurate results.

An Important Question is how to construct a model?

One can divide modeling strategies in to theory- driven, data – driven, and mixed.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 20 / 30
Steps involved in Mathematical Modeling
Define the problem for which a model is developed.

Specify the purpose of the model.

Define the boundaries of the model.

Postulate the structure of the model, for example, in terms of block diagram.

Define variables of interest.

Describe each element mathematically, based on physical laws.

Develop model equations and estimate model parameters from relevant data.

Test and analyze the model.

Modeling involves identification of:

Variables which can be neglected

Input variables (exogenous variables) which affect the system but whose behavior is not modeled.

Output variables (endogenous variables) whose behavior is modeled.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 21 / 30
Neural Networks and Mathematical Modeling

Given a physical system, a neural network can model it on the basis of a set of examples
encoding the input / output behavior of the system.

The neural network learns to map an input into a desired output by self-adaptation.

The modeling capability of the neural network can be ascribed to its ability to learn the
mathematical function underlying the system operation.

A neural network can be applied to mathematical modeling as

a preprocessor

a postprocessor

a mathematical model

a baseline control

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 22 / 30
Integration of neural networks with other modeling techniques

Preprocessor Modeling Post processor

Data Neural Network Techniques Neural Network Mathematical
Model

The integration of neural networks with other modeling techniques

A difficult part of building a model is to determine which variables should be included in

the model. Introducing irrelevant variables and missing relevant ones are both
deleterious to the model.

A common practice is to select a tentative set of variables and refine the set successively.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 23 / 30
Correspondence between neural networks and mathematical models

Mathematical Models Neural Networks

Linear models Single-layer networks
Nonlinear models Multilayer networks
Static models Static networks
Dynamic models Temporal networks
Deterministic models Deterministic networks
(e.g., back propagation network)
Stochastic models Stochastic networks
(e.g., the Boltzmann machine)
Discrete models Discrete networks
Continuous models Continuous networks
The basis of using neural networks as mathematical models is “MAPPING”.

Y = f ( X 1,.... Xn). Y Output layer

………
Hidden layer

….…….
Input layer

Fig. A mapping neural network.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 24 / 30
Applications of ANNs:
Neural nets have broad applicability to real world problems. Some areas include

Classification
Noise Reduction
Sales forecasting
Industrial process control
Customer research
Data validation
Risk management
Target marketing

Some typical examples also include

Pattern recognition
Diagnosis of hepatitis
Undersea mine detection
Texture analysis
3-D object recognition
Hand-written word recognition
Estimation of the rate of spread of an epidemic
Facial recognition

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 25 / 30
Are there any limits to Neural Networks?
The major issues of concern today are the scalability problem, testing, verification, and
integration of neural network systems into the modern environment.

Neural network programs sometimes become unstable when applied to larger problems. The
defense, nuclear and space industries are concerned about the issue of testing and verification.

The mathematical theories used to guarantee the performance of an applied neural network
are still under development. The solution for the time being may be to train and test these
intelligent systems much as we do for humans.

Also there are some more practical problems like:

the operational problem encountered when attempting to simulate the parallelism of

neural networks. Since the majority of neural networks are simulated on sequential
machines, giving rise to a very rapid increase in processing time requirements as size of
the problem expands.

instability to explain any results that they obtain. Networks function as "black boxes”
whose rules of operation are completely unknown.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 26 / 30
The Future

Because gazing into the future is somewhat like gazing into a crystal ball, so it is better to
quote some "predictions". Each prediction rests on some sort of evidence or established trend
which, with extrapolation, clearly takes us into a new realm.

Prediction 1:
Neural Networks will fascinate user-specific systems for education, information processing,
and entertainment. "Alternative realities", produced by comprehensive environments, are
attractive in terms of their potential for systems control, education, and entertainment. This is
not just a far-out research trend, but is something which is becoming an increasing part of our
daily existence, as witnessed by the growing interest in comprehensive "entertainment centers"
in each home.

This "programming" would require feedback from the user in order to be effective but simple
and "passive" sensors (e.g. fingertip sensors, gloves, or wristbands to sense pulse, blood
pressure, skin ionization, and so on), could provide effective feedback into a neural control
system. This could be achieved, for example, with sensors that would detect pulse, blood
pressure, skin ionization, and other variables which the system could learn to correlate with a
person's response state.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 27 / 30
The Future (Contd...)

Prediction 2:

Neural networks, integrated with other artificial intelligence technologies, methods for direct
culture of nervous tissue, and other exotic technologies such as genetic engineering, will allow
us to develop radical and exotic life-forms whether man, machine, or hybrid.

Prediction 3:

Neural networks will allow us to explore new realms of human capability realms previously
available only with extensive training and personal discipline. So a specific state of consciously
induced neuro-physiologically observable awareness is necessary in order to facilitate a man
machine system interface.

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 28 / 30
John von Neumann

ALL STABLE PROCESSES WE

CAN PREDICT.

ALL UNSTABLE PROCESSES

WE CAN CONTROL

Von Neumann with the first computer

V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 29 / 30
V. Sree Hari Rao, Jawaharlal Nehru Technological University, Hyderabad – 500 072 Page
Email: vshrao@yahoo.com; URL: http://www.geocities.com/vshrao 30 / 30