IEEE - INTERNATIONAL CONFERENCE ON ADVANCES IN ENGINEERING & TECHNOLOGY (ICAET-2014) | ISBN No.

: 978-1-4799-4949-6

Firefly Algorithm Approach for the

Optimization of Feature Selection to
Perform Classification
P.Shunmugapriya1, S.Kanmani2, P.Sindhuja3, V.Yasaswini3, G.Koperundevi3

1 Associate Professor, Department of Information Technology,

Dr.SJS Pauls College of Engg. & Tech.,
2 Professor, Department of Information Technology,
3 Final Year, Department of Information Technology,
Pondicherry Engineering College,
Puducherry,
India.
1
pshunmugapriya@gmail.com

Abstract—Classification is an important task of Data classification [3]. FS is seen as an optimization

Mining and its performance is highly affected by the
redundant, irrelevant and noisy features present in
the dataset related to any domain. So, Feature
Selection (FS) and optimization of FS are very
important in order to filter out these features so as to
obtain optimal feature subset. Firefly Algorithm (FA)
is one of the latest metaheuristic search algorithms
that is found to be applicable for a number of
optimization problems. In this paper, FS optimization
has been attempted using FA and hence FA-FS
algorithm has been proposed. The proposed algorithm
has been evaluated on 10 datasets from UCI
(University of California, Irvine) and the results prove
the effectiveness of the proposed FA-FS in optimizing
FS.
Keywords—Firefly Algorithm; Feature Selection;
Classification; Optimization
I. INTRODUCTION AND BACKGROUND
Pattern Classification or simply ‘Classification’
is an important step in Data Mining and Machine
Learning. Classification has achieved remarkable
success in solving complex real-world problems
like credit card approval, medical diagnosis, fraud
detection, web page categorization etc. that are not
easily solvable by humans or would take months to
provide solutions. In real-world, objects are
characterized by a set of measurements called
‘attributes’ or ‘features’. Classification is the action
of assigning an object to a category according to the
characteristics of the object [1 and 2].
Feature Selection (FS) is an important pre-
processing step for the task of classification and it
is highly effective in improving the performance of
problem, because obtaining the optimal subset of
features is very important. This problem of
optimization has been widely addressed by
evolutionary and Swarm Intelligent (SI) algorithms
and in this regard, we have extensive ongoing
research [4-9].
Motivated by the usage of SI algorithms for FS
optimization, we have previously experimented
with Ant Colony Optimization (ACO) and
Artificial Bee Colony (ABC) algorithms for
optimizing the selection of features [10 – 14].
These two attempts have yielded optimal feature
subsets showing improvements compared to the
existing literature. Apart from our proposals, in
literature there exists a huge volume of papers for
FS optimization using ACO, ABC, Genetic
Algorithm and Particle Swarm Optimization [4-9].
The scenario is that, out of these proposals, one
shows better results than the other and none has
claimed to be a consistent performer for the entire
application domain. In this regard, we have our
next attempt for optimizing FS using the Firefly
algorithm (FA-FS). FA-FS algorithm has been
proposed, experimented using UCI datasets and
evaluated in comparison with ABC based FS
optimization (ABC-FS) [12].
Firefly Algorithm (FA) is a relatively new
stochastic search algorithm proposed by Xin-She
Yang in 2007 [15] and since then has been used
widely for different optimization problems [16 -
18]. Inspired by the results obtained from FA for
other optimization problems, it has been utilized to
optimize the selection of feature subsets in this
paper.

Organized by CSE & IT

 IEEE - INTERNATIONAL CONFERENCE ON ADVANCES IN ENGINEERING & TECHNOLOGY (ICAET-2014) | ISBN No.: 978-1-4799-4949-6
This paper is organized as follows: Section II
gives a brief description about feature selection and
classification related to feature selection. The
concept of Firefly algorithm is explained in Section
III. Details of the proposed methods are given in
Section IV. Computations and results are discussed
in Section V and Section VI concludes the paper.
II. FEATURE SELECTION AND
CLASSIFICATION
A. Feature Selection
Feature Selection (FS) is a commonly
used pre processing step in data mining, especially
when dealing with high dimensional space of
features. The main objective is to choose a subset
of features from the original set of features as a
representation of the entire domain [1]. The process
of FS is extensive and it spreads throughout many
fields which includes text categorization, machine
learning, pattern recognition, and signal processing.
Considering the entire features may slowdown the
learning process and may reduce the performance
of the classifier because of redundant and irrelevant
features. Thus it is essential to reduce the number
of features by selecting the most relevant features
to represent a dataset. FS allows the reduction of
feature space, which is crucial in reducing the training
time and improving the prediction accuracy. This is
achieved by removing irrelevant, redundant, and noisy
features [3].
B. Classification
A classifier works on the entire feature set of
an application to perform the task of classification.
Features of the application domain affect the
performance of the classifier in different manners.
So, there are higher chances for the irrelevant and
noisy features to degrade the performance of the
classifier and hence they should be removed [1 and
2]. Not only the irrelevant, the redundancies in the
features should also be removed for effective
classification [6].
III. FIREFLY ALGORITHM
It is a wonderful sight to view the flashing of
lights from the fireflies in the summer sky. There
are a number of species of fireflies and most of the
species have a sort of unique pattern in the flash
behavior. The fundamental principle behind this
flashing behavior is either to attract other fireflies
either for mating or as a prey [19 and 20]. This
principle is exploited and variants of FA have been
proposed.
The FA employed in the proposed work
depends on the variation of light intensity and the
formulation of attractiveness [adopted from [21]]. It
depends on three idealized rules and uses them in
framing the algorithm. The rules are:
Organized by CSE & IT
(i) All fireflies within a population are
unisex, so that one firefly will be attracted to other
fireflies irrespective of their sex;
(ii) Attractiveness between fireflies is
proportional to their brightness, implying that for
any two flashing fireflies, the less bright one will
move towards the brighter one. Attractiveness and
brightness both decrease as the distance between
fireflies increases. If there is no brighter firefly
within its visible vicinity, then a particular firefly
will move randomly;
(iii) The brightness of a firefly is
determined by the landscape of the objective
function.
IV. OPTIMIZATION OF FEATURE
SELECTION USING FIREFLY
ALGORITHM (FA-FS)
In FA-FS algorithm, FA searches through the
feature space and generates possible feature subset
combinations; each time the feature subsets are
generated, they are evaluated by the Prediction
Accuracy given by the classifier. The detailed
description of FA-FS algorithm is as follows:
The number of fireflies is set equal to the
number of features in the dataset and each firefly is
assigned a single feature from the dataset. A binary
string (of length equal to the number of features) is
assigned to each firefly to represent the selection of
features. The support to classification extended by
each feature (i.e. Predictive Accuracy (PA)) is
considered as the brightness ( ) of the firefly
holding that particular feature. PA is the percentage
of instances that have been correctly classified as
instances of their original category [22]. The firefly
with the highest PA is termed as the “BRIGHTEST
FIREFLY” (BF) of the iteration.
When the search procedure begins, each firefly
selects the feature pointed by the BF for
combination. This is because of the principle of the
FA, “fireflies get attracted towards the brightest
firefly in its vicinity”. So the feature subset
combinations that may result will definitely have
the feature with the highest accuracy. Thus the
principle of the FA refines the search concentrating
only on more promising features than the others.
The feature of the BF has only been considered
and before forming the feature subset combinations,
the new value of needs to be computed by using
the equation (1) as follows:
= + exp − ( − ) + (1)
where, is the solution pointed by the current
firefly (Classification Accuracy) and is the
solution pointed by the BF. is the attractiveness
measure between 0 and 1. γ is the variation of
attractiveness whose value is chosen between 0.1
and 10. Distance is set to 1. α is a randomization
 IEEE - INTERNATIONAL CONFERENCE ON ADVANCES IN ENGINEERING & TECHNOLOGY (ICAET-2014) | ISBN No.: 978-1-4799-4949-6

parameter normally selected within the range [0,1] TABLE I DATASETS DESCRIPTION
and is a vector of random numbers drawn from
either a Gaussian or uniform (generally [-0.5,0.5]) Dataset Instances Features Classes
distribution [21]. Heart-C 303 14 2
If the newly found is greater than the Dermatology 366 34 6
previous value, the feature of the BF and the Hepatitis 155 19 2
present firefly are combined, passed on to the Lung Cancer 32 56 2
classifier for evaluation and the PA extended by the Pima Indian Diabetes 768 8 2
combined features is now set as the brightness of Iris 150 4 3
the firefly; otherwise nothing is done and the firefly Wisconsin 699 9 2
will just be holding the feature it was previously Lymphography 148 18 4
Diabetes 768 9 2
holding. Once, all the fireflies have completed one
Heart-Stalog 270 13 2
iteration, the value of BF is reset based on the
highest value of PA computed. The above
As the algorithm proceeds, the possible
procedure is repeated for a pre-determined number
feature subsets that are generated are passed on to
of times or until an optimal configuration of feature
the J48 algorithm for evaluation. After every
subset with a reasonably good accuracy is reached.
iteration, the optimal feature subsets are chosen and
As we have considered the number of fireflies
the firefly is made to point to them. When the
equivalent to the number of features, they are
algorithm comes to a halt, FA-FS algorithm would
distributed throughout the available space and
have yielded optimal feature subset along with
hence the FA has an extremely quicker
maximum PA.
convergence [21]. The steps of the proposed FA-FS
The results obtained from FA-FS
are summarized and given in Fig. 1.
algorithm are presented in comparison with ABC-
1.Cycle =1
FS [12] in Fig. 2 and Fig. 3. The comparison of size
2. Initialize FA parameters of the feature subset obtained for each of the dataset
3. Evaluate the fitness of each individual feature from both FA-FS and ABC-FS is given in Fig. 2. It
4. Repeat could be seen from Fig.2 that, for some datasets,
5. Construct solutions by the fireflies ( Xi ) ABC-FS holds good and for the others FA-FS holds
 Select the BRIGHTEST FIREFLY good in substantially reducing the size of the
 Assign feature subset configurations (binary bit feature set. The PA obtained for all the datasets
string) to each Firefly from both ABC-FS and FA-FS are compared in
 Produce new feature subsets
Fig. 3. Except for Hepatitis and Lung Cancer
 Pass the produced feature subset to the classifier
 Evaluate the fitnes of the feature subset by datasets, FA-FS has performed better than ABC-FS
yielding more PA. Thus, the proposed FA-FS
computing the new value of Xi algorithm has promising behavior and has selected
 Reset the value of Xi based on either consideration the features in a way so as to maximize the
of rejection of the BRIGHTEST FIREFLY classification accuracy.
6. Calculate the best feature subset of the cycle
7. Cycle = Cycle + 1
8. Until pre-determined number of cycles is reached 80
9. Employ the same searching procedure of fireflies to generate 70
the optimal feature subset configurations 60
Fig.1 Steps of FA-FS algorithm 50 No.of
40 Features ABC
V. EXPERIMENTAL RESULTS AND 30
DISCUSSION No.of
20
Features FA
10
The performance of the proposed FA-FS
0 Total
algorithm has been tested with 10 different UCI
Dermatol…

Heart…

Lung…

Features
Lymph

Wisconsin
Pima
Diabetes

Hepatitis
Iris

Sonar
Heart C
Audiology

datasets [23] and their description is given in Table

1. Since, we wanted to have a comparison with our
previous proposals on FS optimization, we have
adapted the same 10 datasets that had been
exploited in all our previous proposals [11-14].
Classification is implemented using Decision Tree
(J48 algorithm) from WEKA (Waikato
Environment for Knowledge Analysis) tool [24].
To start with, the parameters of the FA are
initialized to values in the permitted range. Each
feature is evaluated by using the J48 algorithm and
the concerned fireflies are assigned brightness.
Fig. 2 Chart representing the comparison of features selected by
ABC-FS and FA-FS
In order to prove the efficiency of the proposed
algorithm, the spread over of features of a one of
the datasets is shown in Fig. 4. Fig. 4 provides the
visualization of the Pima dataset from which its
Feature A is along the X axis and Class is along the
Y axis.

Organized by CSE & IT

 IEEE - INTERNATIONAL CONFERENCE ON ADVANCES IN ENGINEERING & TECHNOLOGY (ICAET-2014) | ISBN No.: 978-1-4799-4949-6
Fig.3 Chart representing the comparison of PA obtained from
ABC-FS and FA-FS
Fig. 4 Distribution of features for Pima Dataset
Blue indicates C1 and red indicate C0 which
indicates either positive or negative. The clustered
format of the features is due to the presence to
indicate either C1 or C0. The proposed FA-FS
algorithm has given a good increase in PA for the
Pima dataset with such a mixed distribution of
features belonging to different class categories.
VI. CONCLUSION
Nature inspired algorithms being powerful
optimization algorithms, have been applied in a
large number of domains for problem solving.
Previously having attempted on ACO and ABC for
FS optimization and since the results were
remarkably good, we attempted on using FA - a
relatively new meta-heuristic algorithm. The
proposed FA-FS algorithm is found to be more
promising and has performance comparable to that
of ABC-FS. The concept of “BRIGHTEST
FIREFLY” that we exploited makes the algorithm
to have a good convergence rate and hence a robust
and speedy performance.
REFERENCES
[1] R.O. Duda, P.E. Hart and D.G. Stork, Pattern Recognition,
John Wiley & Sons, Inc 2nd edition, 2001.
[2] L.I.Kuncheva, Combining Pattern Classifiers, Methods and
Algorithms, Wiley Interscience, 2005.
[3] Dash.M and Liu.H., “Feature Selection for Classification”,
Intelligent Data Analysis, Vol.39, No.1, pp.131–156, 1997.
[4] Ahmed Al-Ani: Ant Colony Optimization for Feature Subset
Selection. WEC (2) 2005: 35-38.
Organized by CSE & IT
[5] Nadia Abd-Alsabour and Marcus Randall, Feature Selection
for Classification Using an Ant Colony System, Proc. Sixth
IEEE International Conference on e–Science Workshops,
2010, pp 86- 91.
[6] Kabir M.M., Shahjahan, M. and Murase, K. (2008) ‘A new
hybrid ant colony optimization algorithm for feature
selection’, Journal of Applied Soft Computing, Vol. 8, pp.
687–697.
[7] Tu, C.J., Chuang, L., Chang, J. and Yang, C. (2007) ‘Feature
Selection using PSO-SVM’, IAENG International Journal
of Computer Science, Vol. 33, No. 1, pp. 18 – 23.
[8] N.Suguna and K.G.Thanushkodi, “An Independent Rough
Set Approach Hybrid with Artificial Bee Colony Algorithm
for Dimensionality Reduction”, American Journal of
Applied Sciences 8 (3): 261 – 266, 2011.
[9] N.Suguna and K.G.Thanushkodi, “A novel Rough Set
Reduct Algorithm for Medical Domain based on Bee
Colony Optimization”, Journal of Computing, Vol. 2(6),
2010, 49 -54.
[10] P. Shunmugapriya and S. Kanmani, “Designing Classifier
Ensemble through Multiple-Pheromone Ant Colony based
Feature Selection”, Journal of Computing, Vol.4, No.5,
pp.39-44, May 2012.
[11] P. Shunmugapriya and S. Kanmani, “Classifier Ensemble
Design using Artificial Bee Colony based Feature
Selection”, International Journal of Computer Science
Issues, Vol.9, No.3, pp.522-529, May 2012.
[12] P. Shunmugapriya and S. Kanmani, “Artificial Bee Colony
Approach for Feature Selection”, International Journal of
Computer Science Issues, Vol.9, No.3, pp.432-438, May
2012.
[13] P. Shunmugapriya, S. Kanmani, Devipriya. S, Pushpa. J
and Archana. J, “Investigation on the Effects of ACO
Parameters for Feature Selection and Classification”, in the
proceedings of Springer - Third International Conference
on Advances in Communications, Networks and
Computing, CNC 2012, LNICST, Chennai, India,
February 22-23, pp.136-145, 2012.
[14] P. Shunmugapriya, S. Kanmani, R. Supraja, K. Saranya and
Hemalatha, “Feature Selection Optimization through
Enhanced Artificial Bee Colony Algorithm”, IEEE
Proceedings of Third International Conference on Recent
Trends in Information Technology, pp. 56-61, 2013.
[15] Iztok Fister , Iztok Fister Jr. , Xin-She Yang , and Janez
Brest, “A Comprehensive review of Firefly algorithms”
Elsevier – Swarm and Evolutionary Computation, Vol. 13,
pp. 34–46, 2013.
[16] Theofanis Apostolopoulos and Aristidis Vlachos,
“Application of the Firefly Algorithm for Solving the
Economic Emissions Load Dispatch Problem”,
International Journal of Combinatorics, vol. 2011, Article
ID 523806, 23 pages, 2011. doi:10.1155/2011/523806.
[17] J. Kwiecień and B. Filipowicz, “Firefly algorithm in
optimization of queueing systems”, Bulletin of the Polish
Academy of Sciences: Technical Sciences, Vol. 60, Issue 2,
pp. 363–368, Oct. 2012.
[18] Firefly Algorithms for Multimodal Optimization”, Springer
–Verlag Proceedings of the 5th International Conference
on Stochastic Algorithms: Foundations and Applications,
pp.169-178, 2009.
[19] Xin-She Yang, Nature-Inspired Metaheuristic Algorithms,
Luniver Press, Second Edition, 2010.
[20] K.N. Krishnanand and D. Ghose, “Glowworm swarm based
optimization algorithm for multi-modal functions with
collective robotics applications,”Multi-agent and Grid
Systems, Issue 3, Volume 2, 2006, pp. 209 - 222.
[21] RahaImanirad, Xin-She Yang and Julian Scott Yeomans,
“Modelling To Generate Alternatives Via the Firefly
Algorithm”, Journal Of Applied Operational Research,
2013.
[22] Han, J., and Kamber, M.: Data mining concepts and
techniques, Academic Press, 2001.
[23] A.Frank, A. Asuncion, UCI Machine Learning Repository,
(http://archive.ics.uci.edu/ml. Irvine, CA: University of
California, School of Information and Computer Science
(2010))
[24] WEKA: A Java Machine Learning Package,
http://www.cs.waikato.ac.nz/˜m l/weka/.