2011 10th International Conference on Machine Learning and Applications
Developing an appropriate data normalization method
Balemir Uragun and Ramesh Rajan
Monash University
Physiology Department
Clayton, Victoria, Australia
b.uragun@hotmail.com and Ramesh.Rajan@monash.edu
Abstract — The issue of data normalization has been
extensively studied with respect to many different applications.
However, before implementing any data normalization, it is
necessary to establish whether the data requires normalization,
and this can be determined by a simple test-bench test. Our
test-bench compared seven well-known normalization
techniques for nine possible sensitivity functions recorded from
brain neurons to variations in Interaural Level Differences
(ILD), a cue to azimuthal location of a sound source. These
nine realistic ILD functions were systematically modified to be
delivered at the most suitable normalization technique. This
method then helped to select a coherent normalization
technique for the data before applying the statistical technique
of Cluster Analysis which can be explored in future studies.
Keywords - Data Normalization; Interaural Level Difference
I. INTRODUCTION TO DATA NORMALIZATION
Data normalization is a scaling process for numbers in a
data array and is used where there is the heterogeneity in the
numbers renders difficult any standard statistical analysis.
The data is then often normalized before any application
process is applied and therefore data normalization is usually
termed as data pre-processing. Many different data
normalization techniques have been developed in diverse
applications such as in diagnostic circuits in electronics [1],
temporal coding in vision [2], predictive control systems in
seismic activities [3], modeling labor market activity [4],
pattern recognition [5], and extensively in microarray data
analysis in genetics, [6], [7], [8], [9], [10], [11], [12], [13].
The purpose of data normalization depends on the
proposed application, and hence data normalization includes
use of linear scaling to compress a large dynamic range [1],
scaling of values to correct for variation in laser intensity
[11], handling obscure variation [6] or removing systematic
errors in data [5], [10], [13], or efficiently removing
redundancy in a non-linear model as an optimal
transformation for temporal processing [2]. Although the
benefits of data normalization depend on data type, data size
and normalization method, generally the advantages of data
normalization are (a) to give a more meaningful range of
scaled numbers for use, (b) to rearrange the data array in a
more regular distribution, (c) to enhance the correctness of
subsequent calculations, and (d) to increase the significance
or importance of the most descriptive numbers in a non-
normally distributed data set.
Here we examine the use of data normalization methods
to model responses of brain neurons to variations in an
important parameter for localization of the azimuthal
location of high-frequency sounds, Interaural Level
Differences (ILDs) [14]. ILDs are the difference in sound
978-0-7695-4607-0/11 $26.00 © 2011 IEEE
DOI 10.1109/ICMLA.2011.53
levels at the two ears as a sound source moves about an
animal and are created by head and body shadowing effects
which affect high frequency sounds more than low frequency
sounds [15]. There is a vast literature on the importance of
ILDs and how neurons at various brain levels respond to
ILDs that cover a wide azimuthal range across frontal space,
from opposite one ear across to opposite the other. We focus
on application of data normalization techniques to the
response heterogeneity in ILD functions (plots of the
strength of neuronal responses to variations in ILDs)
recorded from neurons in an obligatory midbrain auditory
relay structure, the Inferior Colliculus. To examine the effect
of different normalization methods, we created a test bench
of theoretical ILD functions. These functions included
variations in different features of ILD functions to simulate
all the ILD variants reported in the literature. Use of this
standard simulated set of ILD functions allowed direct
comparison between different normalization methods and we
carried out a three-step procedure to find the best ‘tailored’
data normalization for prototypical ILD functions.
II. GENERATING PROTOTYPICAL ILD FUNCTIONS
Our own database and an extensive literature review
showed that four prototypical ILD functions can be recorded
in [different] neurons at all levels of the brain above the
brainstem. These four functions (Figure 1) consist of (a) two
Sigmoid functions where neuronal responses vary
sigmoidally across a wide range of ILDs with a plateau of
responses in the ILD range favoring either one ear or the
other, (b) a Peaked function where neuronal responses are
peaked at some ILD within the range encompassing frontal
space, and (c) Insensitive functions where neuronal
responses vary very little with ILDs. Each of these four
broad response categories encompasses functions that can
vary in metrics defining the features of the ILD function, e.g.
position along the ILD axis of the peak of responses or of the
slope from maximum to minimum responses, the steepness
of the slope– features that have been variously discussed to
be defining information-bearing elements that may be used
to derive azimuthal location of a sound source [16].
In the simulated ILD sensitivity functions, 13 ILD values
were used ranging from +30 dB (30 dB louder in one ear) to
-30 dB (30 dB louder in the other ear) as detailed in Section
V below. Neuronal responses were represented as spikes /
stimulus on a scale from ‘0’ to ‘100’ (“m”, maximum
response). This normalized scale allowed us to simulate ILD
functions in absolute values across all normalization tests to
allow comparison of effects across the tests.
195
 III. GENERATING VARIANTS OF ILD FUNCTION
To appropriately model the range of ILD functions seen
experimentally, we generated the four prototypical ILD
functions (i.e. Sigmoidal-EI, Sigmoidal-IE, Peaked, and
Insensitive) and then added in variants in feature elements
of each class to produce a data set of nine possible variants
in ILD functions. For Sigmoidal functions, three important
features likely to be critical for coding ILDs were (a)
position along the ILD axis of the cut-off of the slope (the
point where the ILD function starts to turn over), (b)
steepness of slope, and (c) ILD at which the maximum
firing rate occurs. For Peaked functions, the three important
features were (a) position of peak, (b) ILD at which the
maximum firing rate occurs, and (c) position of cut-off (tip
of the function) with varying steepness of slope. For
insensitive function, the sole feature was ILD at which the
maximum firing rate occurs. Additionally, to model
experimentally observed variants between prototypical
classes, we identified two transitional ILD types. In one a
peaked function transitioned to a Sigmoidal function as one
arm of the peak function moved gradually to increasing
spike counts (“vector” v#1 through v#4 in Fig. 1G). In the
second transitional type, spike counts on both descending
arms of a peaked function increased so that it transitioned to
becoming insensitive ILD function (Fig. 1H).
Thus the nine different test bench ILD functions were:
(i) Three categories of variants in Sigmoidal ILD
functions with variations in (a) the maximum firing rate
occurs (Fig. 1A), (b) location of cut-off along the ILD axis
(Fig. 1B), and (c) steepness of slope of cut-off (Fig. 1C).
(ii) Three categories of variants in peaked ILD functions
with variations in: (a) maximum firing rate occurs (Fig.
1D), (b) location of cut-off along the ILD axis (Fig. 1E), and
(c) steepness of the slope of cut-off (Fig. 1F).
(iii) For the two types of transitional ILDs (transitioning
from Peaked to Sigmoid by increasing firing rate
unilaterally in one arm of the peaked function, or from
Peaked to Insensitive by increasing firing rate bilaterally in
both arms of the peaked function): (a) changes in one limb
only to change from peaked to Sigmoidal (Fig. 1G), (b)
changes in both limbs to change from peaked to insensitive
(Fig. 1H), and (c) variations in the maximum firing rate
occurs of insensitive response functions (Fig. 1I).
IV. APPLYING SPIKE COUNT PERTURBATIONS
To produce more realistic looking ILD functions we
applied a small and statistically-insignificant perturbation to
deform the shape of the nine ILD functions. All points in the
data groups (viz., numbers of spike counts) were arbitrarily
perturbed within ± 6% of the original values. The ± 6%
perturbation range was determined from initial visual
inspection of different test ranges which showed that
perturbation by < 6% made the ILD functions still look too
ideal and perturbation by > 6% made it too easy to confuse
different ILD patterns. For example, a 7% increase in one
part (and 7% decrease in another part) of the insensitive ILD
196
function (v#1s of Fig. 1A and Fig. 1I) made it look like a
Sigmoidal ILD function.
Data perturbation was carried out in three-steps: (i)
generate 13 random numbers varying between -0.06 and
+0.06 for every unit (i.e. ±6 for the maximum of 100 spike
counts), (ii) add the 13 random numbers arbitrarily to each
ILD pattern (each ILD pattern contains 13 numbers and
each number presents the number of spike counts), and (iii)
monitor the previous steps to verify that perturbed data
validly fit in the range from minimum to maximum (e.g.,
log normalization may produce errors for some values if
divided by zero).The ±6% perturbation was applied to the
nine ILD functions to produce the final test bench.
V. GENERATING THE FINAL TEST BENCH ILD FUNCTIONS
Simulated ILD functions were generated using MATLAB
Statistical toolbox (v5.0.2) to generate a matrix of ILD data
which could be tested with seven common normalization
techniques, and to graphically display results. In theory, the
nine possible ILD variants can be classified using
mathematical models. This approach has been used to
classify neuronal Inter Spike Intervals (ISI) distributions
where around 200 ISI histograms from various neurons
were classed using 10 different mathematical models [17].
However, our data has not been categorized yet to allow
classification by a suitable mathematical model. Therefore,
the following method was applied.
For the ILD axis, we used ILDs from +30dB through 0dB
to -30dB, in 5 dB increments as used in many
electrophysiological studies [18], [19], [20], [21], including
our own studies that provided the neuronal data here. In our
studies, ILDs were generated by varying the levels in both
ears symmetrically about some average level (the Average
Binaural Intensity (ABI) constant method [22]). At any one
ABI level, the 13 ILD levels were represented on a scale of
1 to 13, where 1= +30dB and 13= -30dB. We initially
mathematically modeled four types of ILD sensitivity
functions to mimic the four general forms of ILD sensitivity
(see Section II) [23], [24], [25], [21]. Then, to mimic the
variations in neuronal response functions that are classed
experimentally as being of the same general form, the basic
prototypes were intentionally varied systematically (v#4s of
Fig. 1A, 1G, 1D and 1I), as explained in detail later.
VI. DATA NORMALIZATION: RESULTS AND DISCUSSION
Since the appropriate normalization technique for ILD
data is unknown, we tested seven widely-used techniques
against our data set of nine ideal ILD function variants.
A. Normalization by mean correction
The covariance matrix, or mean-corrected data, removes
mean values from each vector’s variables one by one (Table
1.1). In this method “µ” is the scalar mean value of “Xn”
data, which has “n” number of ILD patterns. Data
normalization by the covariance matrix is data scaling that
has a zero mean [26], [27]. The covariance matrix is set by
each row of observations with each column of variables that
measures the linear relationship between variables. The
covariance matrix method is not often used and the more
preferred method is use of correlation matrices for reduction
of data dimension (such as in Principal Component
Analyses) because the correlation matrix is a normalized
measure of linear relationship between variables.[26]. The
result of this data normalization technique is (1) Maximum
and minimum values of all normalized data variables are
spread between negative and positive values, which
misrepresent the original values of the ILD data set, and (2)
variations of the four Insensitive ILD patterns are
exaggerated by normalization.
B. Data normalization by a single maximum value
All unnormalized data are scaled to range between 0 and
a single maximum value (Table 1.2). This maximum value
is the largest of the maxima across all unnormalized data
sets and is used to divide each matrix component to generate
a “Vn(i, j)” normalized matrix, i.e. the maximum is always
=1 and the minimum always = 0 (where, ∀ Xn(i) ≠0). This
technique is often used for microarray data [13], [28].
However for our data it resulted in value changes that were
so small that it was not possible to distinguish very small
changes among similar types of ILD patterns.
C. Data normalization by each vector’s maximum value
Data points in each ILD are divided by the maximum
spike count in that function, i.e., each vector’s maximum
“Xn (i, j)” is used to normalize the “Xn” raw matrix (Table
1.3). It has the advantage that all functions are scaled to a
maximum of “1” (i.e. the maximum spike count for that
function) and other values are effectively expressed as a
proportion of this maximum spike count for that function.
The disadvantage of this procedure was that the shapes of
ILD functions with small spike counts became distorted
because each vector’s maximum spike counts were all
normalized to the same maximum of 1. Thus, this technique
was not suitable for all ILD functions, especially those with
small spike count changes with slight perturbations.
D. Data normalization by each vector’s standard deviation
An uncommon technique is where each data point is
normalized by its standard deviation (Table 1.4: “σ” = SD,
“X” = raw matrix, and “V” = normalized matrix). This
produces better data spread, especially for asymmetric
functions (e.g., our Sigmoidal functions) [29]. However, it
was unsuitable for our data because it did not preserve the
different number of spike counts in a proportionally scaled
manner for similar types of nonlinear ILDs.
E. Logarithmic normalization
Logarithmic normalization is widely used especially
when data analysis involves large number of data [30]. It is
a nonlinear procedure [10] that is suitable to deal with
nonlinear data [12]. Each data point “Xn” is divided by the
mean value “µn” for that function. Then the logarithm to
base 2 (log2) “Vn” is calculated (Table 1.5). With this
transformation there is decreased variance [8], as large spike
counts are reduced more than low ones. However, this
197
technique was also unsuitable for the ILD functions. The
number of spike counts for all nine ILD functions originally
varied from nearly 0 to 100 (spikes/stimulus). Logarithmic
normalization scaled down the spike count range by log2 but
the outcome was an exaggerated perturbation of ILD
functions, which deformed the original functions. Moreover,
some irregular transformations occurred, especially that
transitions from Peaked to Sigmoidal and to Insensitive ILD
functions were not smooth as in the original data sets.
F. Unit Total Probability Mass normalization
The unit total probability mass (UTPM) normalization or
total intensity normalization [9] is achieved by dividing
each vector’s element by the sum of that vector’s variables,
and multiplying by the mean (Table 1.6, where “Xn”= raw
data, “µn”= mean value, and “Vn”= normalized data) [3]
[31] [4] [32]. It has been used in the cumulative distribution
function [33] as a discrete type of data normalization and in
normalizing input for cluster analysis [8]. It was well suited
to our test bench because: (1) The perturbation in variations
of Sigmoidal, Peaked, and Insensitive ILD functions with
varying spike counts were not distorted; (2) The positions of
varying cut-off and varying slopes of the ILD functions
were kept the same as in the original functions; and (3) The
irregularity transformation (transitions from Peaked to
Sigmoidal and to Insensitive ILD functions) were not too
smooth as in the original data set. The maximum values of
each ILD functions are all scaled down by ~13% without
losing their original shapes.
G. Normalization by data standardization
Data standardization is achieved by dividing the mean
subtracted data (mean-corrected) by its standard deviation,
(Table 1.7: mean “µn” value is subtracted from each data
point “Xn” and then is divided by its SD “σn”). The
variances of standardized variables are 1 and therefore
covariance of standardized variables always ranges between
‘-1’ and ‘+1’ [28]. An advantage of this normalization is
that data are expressed in comparable units. On some
occasions, data have been standardized by zero mean and
unity SD (like σ = 1 in [34], towards a standard procedure
for Principal Component Analysis). This normalization can
also make a more efficient front-end application for neural
network training [1]. However, this technique was not
suitable for the insensitive type of ILD functions as
perturbations of the ILD functions were adversely affected
by this technique, causing ILD function distortion. The
maxima and minima values in this normalization method are
also spread between negative and positive values to keep the
mean = 0 and SD = 1 for more nonlinear type of functions
i.e. not for insensitive type ILD functions.
VII. CONCLUSION AND DISCUSSION
In general terms normalization is signal intensity divided
by a reference value, to reduce systematic errors in data
variables [30]. Data normalization also maximizes variance
[34], which is especially important before applying data
dimension reduction technique for ILD type data. Data
normalization is often a prerequisite for statistical data
analysis, and finding a suitable scaling technique for the
data is an important task. In a novel approach for the field,
we developed a test bench of prototypical ILD functions to
investigate appropriate normalization techniques. We found
that the unit total probability mass normalization method
was the best for ILD response functions.
Other data normalization techniques can be generated by
variation of existing ones, such as dividing by sum of all
signals or by the standard error signals after corrected mean
values [12]. Also slight variations of normalization
techniques used here could give a procedure useful for a
non-linear feature for the data [10]. For our data this could
be achieved by multiplication of the mean value of data for
our unit total probability mass normalization technique for
each ILD functions.
In addition to visual comparisons of the result of a
selected normalization technique against the raw data, other
methods are also available for better selection of the correct
normalization technique. The quality of the normalization
technique can be estimated by: (i) calculating the sum of
squares of differences between the model and normalization
histogram, (ii) using Pearson correlation coefficients
between the values before and after data normalization [35].
Such a quantification method for normalization selection is
worth investigation, but is beyond the scope of this study.
REFERENCES
[1] M. Aminian and F. Aminian, "Neural-network based analog-circuit
fault diagnosis using wavelet transform as preprocessor," IEEE
Transactions on Circuits and Systems II: Analog and Digital Signal
Processing, v. 47, pp. 151-6, 2000.
[2] M. Buiatti and C. van Vreeswijk, "Variance normalisation: a key
mechanism for temporal adaptation in natural vision?," Vision
Research, v. 43, pp. 1895-906, 2003.
[3] Y. Kosugi, M. Sase, H. Kuwatani, N. Kinoshita, T. Momose, J.
Nishikawa, and T. Watanabe, "Neural network mapping for nonlinear
stereotactic normalization of brain MR images," Journal of Computer
Assisted Tomography, v. 17, pp. 455-60, 1993.
[4] G. Skoog and J. Ciecka, "Probability mass functions for additional
years of labor market activity induced by the Markov (increment-
decrement) model," Economics Letters, v. 77, pp. 425-31, 2002.
[5] H. Kim, D. Kim, and S. Bang, "Face recognition using the mixture-of-
eigenfaces method," Pattern Recognition Letters, v. 23, pp. 1549-58,
2002.
[6] B. Bolstad, R. Irizarry, M. Astrand, and T. Speed, "A comparison of
normalization methods for high density oligonucleotide array data
based on variance and bias," Bioinformatics (Oxford,UK), v. 19, pp.
185-93, 2003.
[7] E. Dougherty, J. Barrera, M. Brun, S. Kim, R. Cesar, Y. Chen, M.
Bittner, and J. Trent, "Inference from clustering with application to
gene-expression microarrays," Journal of Computational Biology, v.
9, pp. 105-26, 2002.
[8] J. Kasturi, R. Acharya, and M. Ramanathan, "An information
theoretic approach for analyzing temporal patterns of gene
expression," Bioinformatics (Oxford,UK), v. 19, pp. 449-58, 2003.
[9] J. Quackenbush, "Microarray data normalization and transformation,"
Nature Genetics, v. 32, pp. 496-501, 2002.
[10] G. Tseng, M. Oh, L. Rohlin, J. Liao, and W. Wong, "Issues in cDNA
microarray analysis: quality filtering, channel normalization, models
of variations and assessment of gene effects," Nucleic Acids Research,
v. 29, pp. 2549-57, 2001.
198
[11] D. Venet, "MatArray: a Matlab toolbox for microarray data,"
Bioinformatics, v. 19, pp. 659-60, 2003.
[12] J. Weiner III, C. Zimmerman, H. Gohlmann, and R. Herrmann,
"Transcription profiles of the bacterium Mycoplasma pneumoniae
grown at different temperatures," Nucleic Acids Research, v. 31, pp.
6306-20, 2003.
[13] C. Workman, L. Jensen, H. Jarmer, R. Berka, L. Gautier, H. Nielser,
and et al, "A new non-linear normalization method for reducing
variability in DNA microarray experiments," Genome Biology, v. 3,
2002.
[14] D. Irvine, "IID in the cat: changes in sound pressure level at the two
ears associated with azimuthal displacements in the frontal horizontal
plane," Hearing Research, v. 26, pp. 267-86, 1987.
[15] W. Hartmann and B. Rakerd, "ILD: Diffraction and localization by
human listeners," The Journal of the Acoustical Society of America, v.
129, p. 2622, 2011.
[16] B. Grothe, M. Pecka, and D. McAlpine, "Mechanisms of Sound
Localization in Mammals," Physiological Reviews, v.90, pp. 983-
1012, 2010.
[17] H. Tuckwell, Stochastic processes in the neurosciences. Philadelphia,
PA, USA: Society for Industrial and Applied Mathematics, 1989.
[18] D. Irvine, The Auditory Brainstem: A Review of the Structure and
Function of Auditory Brainstem Processing Mechanisms, 1st ed. N.Y.:
Springer-Verlag, 1986.
[19] D. Irvine, V. Park, and L. McCormick, "Mechanisms underlying the
sensitivity of neurons in the lateral superior olive to IID," Journal of
Neurophysiology, v. 86, pp. 2647-66, 2001.
[20] T. Lohuis and Z. Fuzessery, "Neuronal sensitivity to interaural time
differences in the sound envelope in the auditory cortex of the pallid
bat," Hearing Research, v. 143, pp. 43-57, 2000.
[21] D. Phillips and D. Irvine, "Responses of single neurons in
physiologically defined area AI of cat cerebral cortex: sensitivity to
IID," Hearing Research, v. 4, pp. 299-307, 1981.
[22] L. Aitkin, The auditory midbrain: structure and function in the
central auditory pathway. Clifton, N.J.: Humana Press, 1986.
[23] L. Aitkin, D. Irvine, J. Nelson, M. Merzenich, and J. Clarey,
"Frequency representation in the auditory midbrain and forebrain of a
marsupial, the northern native cat," Brain Behaviour and Evoluation,
v. 29, pp. 17-28, 1986.
[24] L. Aitkin, The auditory cortex: structural and functional bases of
auditory perception, 1st ed. London: Chapman & Hall, 1990.
[25] L. Aitkin, M. Merzenich, D. Irvine, J. Clarey, and J. Nelson,
"Frequency representation in auditory cortex of the common
marmoset," Journal of Comparative Neurology, v. 252, pp. 175-85,
1986.
[26] J. Jackson, A user's guide to principal components. N. Y.: Wiley &
Sons, Inc., 1991.
[27] H. Moghaddam and K. Zadeh, "Fast adaptive algorithms and
networks for class-separability features," Pattern Recognition, v. 36,
pp. 1695-702, 2003.
[28] S. Sharma, Applied multivariate techniques. N.Y.: John Wiley, 1996.
[29] A. Zaknich, Neural networks for intelligent signal processing vol. 4.
River Edge, NJ, USA: World Scientific, 2003.
[30] D. Geschwind and J. Gregg, Microarrays for the neurosciences: an
essential guide. Cambridge, MA, USA: MIT Press, 2002.
[31] Z. Ahmad, L. Balsamo, B. Sachs, B. Xu, and W. Gaillard, "Auditory
comprehension of language in young children: neural networks
identified with fMRI," Neurology, v.60, pp. 1598-605, 2003.
[32] S. Patra and R. Misra, "Evaluation of probability mass function of
flow in a communication network considering a multistate model of
network links," Microelectronics and Reliability, v. 36, pp. 415-21,
1996.
[33] C. C. Abnet and D. M. Freeman, "Deformations of the isolated mouse
tectorial membrane produced by oscillatory forces," Hearing
Research, vol. 144, pp. 29-46, June 2000.
[34] J. Lattin, P. Green, and J. Carroll, Analyzing multivariate data. Pacific
Grove, CA, USA: Thomson Brooks/Cole, 2003.
[35] I. Sidorov, D. Hosack, D. Gee, J. Yang, M. Cam, R. Lempicki, and D.
Dimitrov, "Oligonucleotide microarray data distribution and
normalization," Information Sciences, v. 146, pp. 67-73, 2002.
TABLE 1: SEVEN DATA NORMALIZATION METHODS (WITH THE EQUATIONS) WERE APPLIED TO NINE (FROM “A” TO “I”)
PROTOTYPICAL ILD FUNCTIONS. THE RESULT WAS PRESENTED IN THIS TABLE WITH THE MINIMUM OF MINIMA (FOUR VECTORS)/
MAXIMUM OF MAXIMA (FOUR VECTORS) VALUES WERE ALL SHOWN IN SPIKE COUNTS.

Fig. 1: The four ILD patterns, Sigmoidal-EI (A4), Sigmoidal-IE (G4), Peak (D4), and Insensitive (I2) are all described in numbers of spike
counts “#sp.c.” (spikes/ stimulus) which varied between maxima of ‘100’ and minima of ‘0’ zero units, within -30dB to +30dB ILD level
differences. Nine possible ILD functions are generated from those four typical ILD sensitive function variations. These are; Sigmoidals with
varying number of spike count (# sp.c.) spikes/stimulus (A) position of the cutoff (B), the steepness of the slope (C), and four Peaked with
varying number of spikes/stimulus (D), the cutoff (E), the cutoff & slope (F), and Peaked with unilateral transition to Sigmoidal (G), and Peaked
with bilateral transition to Insensitive (H), and four Insensitive with varying the number of spike count spikes/stimulus (I).

199