1.

Introduction to CBIR
As processors become increasingly powerful, and memories become increasingly
cheaper, the deployment of large image databases for a variety of applications have now
become realisable. Databases of art works, satellite and medical imagery have been attracting
more and more users in various professional fields for example, geography, medicine,
architecture, advertising, design, fashion, and publishing. Effectively and efficiently
accessing desired images from large and varied image databases is now a necessity.
1.1 Definition
CBIR or Content Based Image Retrieval is the retrieval of images
based on visual features such as colour, texture and shape . Reasons for
its development are that in many large image databases, traditional
methods of image indexing have proven to be insufcient, laborious, and
extremely time consuming. These old methods of image indexing, ranging
from storing an image in the database and associating it with a keyword
or number, to associating it with a categorized description, have become
obsolete. This is not CBIR. In !IR, each image that is stored in the
database has its features extracted and compared to the features of the
"uery image. It involves two steps #
Feature Extraction: The $rst step in the process is extracting
image features to a distinguishable extent.
Matching: The second step involves matching these features to
yield a result that is visually similar.
1.2 Applications of CBIR
%xamples of !IR applications are#
Crime prevention: &utomatic face recognition systems, used by
police forces.
Security Check: 'inger print or retina scanning for access
privileges.
Medical Diagnosis: (sing !IR in a medical database of medical
images to aid diagnosis by identifying similar past cases.
Intellectual Property: Trademark image registration, where a new
candidate mark is compared with existing marks to ensure no risk of
confusing property ownership.
1.3 CBIR Systems
)everal !IR systems currently existing are#
QBIC or Query by Image Content was developed by I!*,
&lmaden Research entre, to allow users to graphically pose and
re$ne "ueries based on multiple visual properties such as colour,
texture and shape. It supports "ueries based on input images, user+
constructed sketches, and selected colour and texture patterns.
VIR Image Engine by ,irage Inc., like -!I, enables image
retrieval based on primitive attributes such as colour, texture and
structure. It examines the pixels in the image and performs an
analysis process, deriving image characterization features.
MARS or Multimedia Analysis and Retrieval System was
developed by the !eckman Institute for &dvanced )cience and
Technology, (niversity of Illinois. It supports colour, spatial layout,
texture and shape matching.
Viper or Visual Information Processing for Enanced
Retrieval was developed at the omputer ,ision .roup, (niversity
of .eneva. It supports colour and texture matching.
The process of digitisation does not in itself make image collections easier to
manage. )ome form of cataloguing and indexing is still necessary / the only
di0erence being that much of the re"uired information can now potentially be
derived automatically from the images themselves. The extent to which this
potential is currently being realized is discussed below.
The need for efcient storage and retrieval of images / recognized by managers
of large image collections such as picture libraries and design archives for many
years / was reinforced by a workshop sponsored by the ()&1s 2ational )cience
'oundation in 3445 67ain, 34489. &fter examining the issues involved in managing
visual information in some depth, the participants concluded that images were
indeed likely to play an increasingly important role in electronically+mediated
communication. :owever, signi$cant research advances, involving collaboration
between a number of disciplines, would be needed before image providers could
take full advantage of the opportunities o0ered. They identi$ed a number of
critical areas where research was needed, including data representation, feature
extractions and indexing, image "uery matching and user interfacing.
;ne of the main problems they highlighted was the difculty of locating a desired
image in a large and varied collection. <hile it is perfectly feasible to identify a
desired image from a small collection simply by browsing, more e0ective
techni"ues are needed with collections containing thousands of items. 7ournalists
re"uesting photographs of a particular type of event, designers looking for
materials with a particular colour or texture, and engineers looking for drawings
of a particular type of part, all need some form of access by image content.
Caracteristics of image !ueries
<hat kinds of "uery are users likely to put to an image database= To answer this
"uestion in depth re"uires a detailed knowledge of user needs / why users seek
images, what use they make of them, and how they >udge the utility of the
images they retrieve. &s we show in section %rror# Reference source not found
below, not enough research has yet been reported to answer these "uestions
with any certainty. ommon sense evidence suggests that still images are
re"uired for a variety of reasons, including#
illustration of text articles, conveying information or emotions difcult to
describe in words,
display of detailed data ?such as radiology images@ for analysis,
formal recording of design data ?such as architectural plans@ for later use.
&ccess to a desired image from a repository might thus involve a search for
images depicting speci$c types of ob>ect or scene, evoking a particular mood, or
simply containing a speci$c texture or pattern. Aotentially, images have many
types of attribute which could be used for retrieval, including#
the presence of a particular combination of colour, texture or shape features
?e.g. green stars@B
the presence or arrangement of speci$c types of ob>ect ?e.g. chairs around a
table@B
the depiction of a particular type of event ?e.g. a football match@B
the presence of named individuals, locations, or events ?e.g. the -ueen
greeting a crowd@B
sub>ective emotions one might associate with the image ?e.g. happiness@B
metadata such as who created the image, where and when.
%ach listed "uery type ?with the exception of the last@ represents a higher level
of abstraction than its predecessor, and each is more difcult to answer without
reference to some body of external knowledge. This leads naturally on to a
classi$cation of "uery types into three levels of increasing complexity 6%akins,
344CB %akins, 344D9#
"evel # comprises retrieval by primitive features such as colour, texture, shape
or the spatial location of image elements. %xamples of such "ueries might
include E$nd pictures with long thin dark ob>ects in the top left+hand cornerF,
E$nd images containing yellow stars arranged in a ringF / or most commonly
E$nd me more pictures that look like thisF. This level of retrieval uses features
?such as a given shade of yellow@ which are both ob>ective, and directly derivable
from the images themselves, without the need to refer to any external
knowledge base. Its use is largely limited to specialist applications such as
trademark registration, identi$cation of drawings in a design archive, or colour
matching of fashion accessories.
"evel $ comprises retrieval by derived ?sometimes known as logical@ features,
involving some degree of logical inference about the identity of the ob>ects
depicted in the image. It can usefully be divided further into#
a@ retrieval of ob>ects of a given type ?e.g. E$nd pictures of a double+decker
busF@B
b@ retrieval of individual ob>ects or persons ?E$nd a picture of the %i0el
towerF@.
To answer "ueries at this level, reference to some outside store of knowledge is
normally re"uired / particularly for the more speci$c "ueries at level 5?b@. In the
$rst example above, some prior understanding is necessary to identify an ob>ect
as a bus rather than a lorryB in the second example, one needs the knowledge
that a given individual structure has been given the name Ethe %i0el towerF.
)earch criteria at this level, particularly at level 5?b@, are usually still reasonably
ob>ective. This level of "uery is more generally encountered than level 3 / for
example, most "ueries received by newspaper picture libraries appear to fall into
this overall category 6%nser, 344G9.
"evel % comprises retrieval by abstract attributes, involving a signi$cant amount
of high+level reasoning about the meaning and purpose of the ob>ects or scenes
depicted. &gain, this level of retrieval can usefully be subdivided into#
a@ retrieval of named events or types of activity ?e.g. E$nd pictures of )cottish
folk dancingF@B
b@ retrieval of pictures with emotional or religious signi$cance ?E$nd a picture
depicting su0eringF@.
)uccess in answering "ueries at this level can re"uire some sophistication on the
part of the searcher. omplex reasoning, and often sub>ective >udgement, can be
re"uired to make the link between image content and the abstract concepts it is
re"uired to illustrate. -ueries at this level, though perhaps less common than
level 5, are often encountered in both newspaper and art libraries.
&s we shall see later, this classi$cation of "uery types can be useful in
illustrating the strengths and limitations of di0erent image retrieval techni"ues.