Guest editors introduction

Intelligent Monitoring of
Complex Environments

Christian Micheloni, University of Udine, Italy

Paolo Remagnino, Kingston University, UK
How-Lung Eng, Institute for Infocomm Research, Singapore
Jason Geng, Galax Technologies

any countries around the world have implemented or are in the process of implementing tighter security measures in public and pri-

vate places. Such measures are becoming widespread and are applied not
only at government, military, and corporate facilities, but also in civilian
infrastructures such as railway and bus
stations, concourses, hospitals, nursing
homes, playgrounds, and so forth. Stricter
monitoring implies the use of a larger
number of sensors, surveillance networks,
and platforms, all of which generate large
volumes of data that must be processed.
A decade ago, surveillance systems had to
deal with terabytes of video information;
now we must talk of petabytes and soon
12

higher orders of magnitude. Therefore,

it is of utmost importance that we devise
intelligent monitoring systems that can
fi lter data and extract only relevant information and knowledge about the monitored
scene. Such systems must implement algorithms that can automatically understand
events, detect anomalous activity, generate an immediate response in emergency
cases, and intelligently archive information

1541-1672/10/$26.00 2010 IEEE

Published by the IEEE Computer Society

Ieee InTeLLIGenT SySTeMS

Audio
arrays

for post-event analysis

and system update.

Camera
networks

Sensors

classify as explicit event

recognition and anomaly detection. In the explicit event recognition
approach, the system has
an explicit knowledge
of the events that must
be identified, whereas in
the anomaly detection,
the system does not require a prior knowledge
base about the events that
should be recognized. Hybrid approaches that combine these techniques are
currently under study.

May/june 2010

Recognition

Tracking

Monitoring System
Architectures
Modern monitoring systems should cover a range
of applications, thus a
Human-computer interaction
Surveillance
modular architecture will
Feature
help us adapt systems to
extraction
specific tasks. Figure 1
shows a typical logical
architecture.
We can think of a modern surveillance system
as consisting of different
modules.
Storage
Human-computer inA sensor layer usually
teraction
is fundamenconsists of a network of
Event
tal to providing relevant,
cameras, audio arrays,
analysis
timely information. This
physical perimeter senis a research field that still
sors, and other types of
is not studied as deeply
information feeders that
as the other levels of the
are opportunely orga- Figure 1. Typical processes involved in intelligent monitoring
systems.
monitoring systems. This
nized in a centralized or
aspect will get increasdistributed fashion.
tracking objects in the entire moni- ingly more consideration, however,
A surveillance layer represents the
tored area, from centralized track- because taking human decision makcore of the intelligent system. Its role
ing mechanism to distributed ones, ing out of the loop is no longer posis to process all the data generated by
the tracking problem is one of the sible. Thus, providing operators with
the sensor layer. This module is typimost studied techniques in surveil- the most important information in a
cally complex and itself modular so
way that is easily understandable is a
lance systems.
it can adapt its functionality to the
system requirements. Generally, this To have a complete picture of what mandatory level for intelligent assisis happening, the system must tance systems in the security field.
module includes several components:
The large amount of data that
know which people and objects are
active in the monitored environ- modern surveillance systems generate
A feature extraction step is responment. Hence, recognition processes requires the use of efficient storage
sible for reducing the enormous
at different granularity levels mechanisms. Both the analysis and
amount of data that the network
from coarse (such as the object the human interface need to store and
generates. It extracts only the most
might be a car, human, or bike) to retrieve information.
significant information in an apAll these components represent the
fine (such as a persons identity, a
propriate form and delivers it to the
vehicles make and model, or a li- focus of the current research in defurther modules. This step reprecense plate)must help extract veloping the future monitoring syssents the lowest level of the intellitems. Data will be collected from a
such information.
gent system and usually operates at
The module in charge of the event huge number of sensors, and relevant
the signal level.
analysis is at the highest process- information will be extracted by raw
Time properties are commonly deing level of modern monitoring signals and processed in real time
rived by keeping track of any acsystems. In this context, research- to reach accurate situation awaretion within the monitored environers have developed different ap- ness. Timely information will be proment. From tracking objects within
proaches that we can broadly vided to operators and key data will
a single sensors field of view to
www.computer.org/intelligent

13

Guest Editors Introduction

The Authors
Christian Micheloni is an assistant professor at the University of Udine, Italy. His

research interests include active vision for the understanding of the scene by images
acquired with moving cameras, neural networks for the classification, recognition of
objects moving within the scene, and pattern recognition techniques for both the automatic tuning of the camera parameters for improved image acquisition and face detection. He is member of the International Association of Pattern Recognition (IAPR) and
IEEE. Micheloni has a PhD in computer science from the University of Udine. Contact
him at christian.micheloni@uniud.it.

Paolo Remagnino is a reader in the Faculty of Computing, Information Systems and

Mathematics at Kingston University, UK. His research interests include image processing, computer vision, machine learning, artificial intelligence, and robotics. Remagnino
has a PhD in computer vision from the University of Surrey, UK. He is a member of IEEE.
Contact him at p.remagnino@kingston.ac.uk.
How-Lung Eng is a research scientist with the Institute for Infocomm Research, Singapore. His research interests includes real-time vision, pattern classification, and machine
learning for abnormal event detection. He is a member of IEEE. Eng has a PhD in electrical and electronic engineering from Nanyang Technological University, Singapore. Contact him at hleng@i2r.a-star.edu.sg.
Jason Geng founded Genex Technologies, a Maryland-based company that specializes in
developing and commercializing advanced 3D and 360-degree imaging technologies and
products. He is currently the vice president of the IEEE Intelligent Transportation Systems Society and leads the ITS standards effort as the chairman of the IEEE ITSS standards committee. Contact him at jason.geng@ieee.org.

be stored in appropriate form (such

as security, forensic, and privacy) for
further needs.

In this Issue
The articles in this special issue consider different aspects of the development of modern and intelligent
systems for monitoring complex
environments.
In Monitoring Complex Environments Using a KnowledgeDriven Approach Based on Intelligent Agents, Javier Albusac, David
Vallejo, J.J. Castro-Schez, Paolo Remagnino, Carlos Glez-Morcillo, and
Luis Jimenez present a methodology to design self-contained analysis
modules to deal with specific events.
They show how to divide the monitoring problem into subproblems
that surveillance components can
more easily tackle. These subproblems include a surveillance concept,

14

a set of algorithms to instantiate the

concept, and an interface with other
components.
Horst Bischof, Martin Godec,
Christian Leistner, Andreas Starzacher, and Bernhard Rinner propose
a multisource based surveillance
system in Autonomous AudioSupported Learning of Visual Classifiers for Traffic Monitoring. More
specifically, they investigate the necessity of detecting and classifying
vehicles by exploiting audio and
video sensors. They study a framework for a complementary training
to use a priori trained classifier for filtering low confidence data out from
a training set for a second classifier
that is trained online. During the
classification phase, both classifiers
outputs are combined to improve the
classification accuracy.
Moving toward the highest processing level of monitoring systems,

www.computer.org/intelligent

the human behavior analysis certainly represents an interesting issue.

In Interaction Analysis with a Bayesian Trajectory Model, Alessio Dore
and Carlo S. Regazzoni propose a
Dynamic Bayesian Network (DBN)
to recognize human interactions
by using their patterns of movement. They introduce a bioi nspired
model working on an appropriate representation of human trajectories to statistically describe the
interactions.
Brendan Tran Morris and Mohan
Manubhai Trivedi, in Contextual
Activity Visualization from LongTerm Video Observations, propose
a user-interaction interface for instantaneous feedback of contextual processing units. This represents a methodology for developing an assistance
system that involves adopting an advanced monitoring technique to provide useful information for decision
making and planning.
Finally, Douglas C. Derrick,
Aaron C. Elkins, Judee K. Burgoon,
Jay F. Nunamaker Jr., and Daniel
Dajun Zengs article Border Security Credibility Assessments via Heterogeneous Sensor Fusion presents
a system for assisting customs personnel with border security. In particular, they propose a heterogenous
sensor network to accurately perform credibility assessments on the
basis of infrared cameras, audio recording, thermal imaging, video
recordings, and Laser Doppler
Vibrometer data.

Selected CS articles and columns

are also available for free at
http://ComputingNow.computer.org.

IEEE INTELLIGENT SYSTEMS