Authors Accepted Manuscript

A New Pre-Whitening Transform Domain LMS

Algorithm and its Application to Speech Denoising

Laid Chergui, Saad Bouguezel

www.elsevier.com/locate/sigpro

PII: S0165-1684(16)30132-3
DOI: http://dx.doi.org/10.1016/j.sigpro.2016.06.021
Reference: SIGPRO6181
To appear in: Signal Processing
Received date: 21 February 2016
Revised date: 23 May 2016
Accepted date: 22 June 2016
Cite this article as: Laid Chergui and Saad Bouguezel, A New Pre-Whitening
Transform Domain LMS Algorithm and its Application to Speech Denoising,
Signal Processing, http://dx.doi.org/10.1016/j.sigpro.2016.06.021
This is a PDF file of an unedited manuscript that has been accepted for
publication. As a service to our customers we are providing this early version of
the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting galley proof before it is published in its final citable form.
Please note that during the production process errors may be discovered which
could affect the content, and all legal disclaimers that apply to the journal pertain.
 A New Pre-Whitening Transform Domain LMS
Algorithm and its Application to Speech Denoising

Laid Chergui, Saad Bouguezel

Laboratoire de Croissance et Caractrisation de Nouveaux Semiconducteurs, Dpartement dElectronique, Facult de

Technologie, Universit Ferhat Abbas Stif-1, Stif 19000, Algeria
E-mail adresses : chergui.laid@gmail.com (L. Chergui), bouguezel_saad@yahoo.com (S. Bouguezel).

Abstract

In this paper, we propose a new pre-whitening transform domain LMS algorithm. The main idea is to

introduce a pre-whitening using a simple finite impulse response decorrelation filter of order one before

applying the transform to reinforce its decorrelation. The resulting algorithm has the advantage of using any

transform even with low decorrelation. This advantage can be exploited to consider transforms having lower

computational and structural complexities than those of the classical transforms. For this purpose, we also

investigate the use of other transforms, namely the parametric Fourier and Hartley transforms. This

investigation is accomplished by studying the eigenvalue spreads obtained by a given parametric transform

and then finding the value of the parameter corresponding to the minimum eigenvalue spread, which is

equivalent to the best mean square error (MSE) convergence behaviour. This approach provides new

attractive transforms for the proposed algorithm. Moreover, we consider the adaptive speech denoising as an

application to evaluate the performance of the proposed algorithm. The comparisons between the proposed

and conventional algorithms for different transforms are perform in terms of the computational complexity,

MSE convergence speed, reached steady state level, residual noise in the denoised signal, steady state

excess MSE, misadjustment and output SNR.

Keywords: Adaptive filtering, parametric transforms, pre-whitening, speech denoising, transform domain LMS algorithm.
1. Introduction
The least mean square (LMS) algorithm is the mostly used in the adaptive filtering for its simplicity and

robustness [1]. However, it has a slow convergence in the case of highly correlated input signals [2,3]. This

is due to the fact that the autocorrelation matrix of the input signal has a large eigenvalue spread. To

overcome this problem by reducing the eigenvalue spread, whitening or decorrelated adaptive algorithms

have been proposed for time-domain LMS [4-6]. In [4], the authors proposed a joint decorrelation of both

the input and error signals. The decorrelation is achieved in the time domain using an adaptive decorrelation

filter based on the concept of prediction. The resulting decorrelated normalised LMS (NLMS) structure has

the advantage of improving the mean square error (MSE) convergence speed and steady state compared to

the conventional LMS and NLMS algorithms. The decorrelation of the input signal can also be achieved by

an orthogonal transformation followed by power normalization [7]. These two operations have led to a new

configuration named transform-domain LMS (TDLMS) adaptive filters, which outperform the time-domain

LMS adaptive filters in terms of the MSE convergence speed and steady state [8-10]. The orthogonal

transforms such as the discrete Fourier transform (DFT), the discrete Hartley transform (DHT) and the

discrete cosine transform (DCT) have been used in the TDLMS [7]. The resulting adaptive filters have been

named as DFT-LMS, DHT-LMS and DCT-LMS, respectively [11]. The convergence speed of these filters

depends on the used transform [8,11,12]. In general, the DCT-LMS filter presents a convergence

performance better than those of the DFT-LMS and DHT-LMS filters [11]. This is mainly due to the fact

that the DCT is suboptimal in terms of decorrelation [13,14]. Therefore, it is highly desirable to use in the

TDLMS adaptive filters a transform having better decorrelation. However, the existing transforms are fixed

and hence it is not possible to increase their decorrelation ability. Since the introduction of a decorrelation in

the time-domain LMS has brought interesting improvements, it is important to investigate the introduction

of such a decorrelation in the TDLMS. However, to the best of authors knowledge, this investigation has

not been reported in the literature.

 One of the applications of LMS adaptive filters is speech denoising. It is a very crucial operation to

enhance the quality and intelligibility of the voice and reduce communication fatigue in modern

communication systems such as mobile phones, hands-free telephony and voice-controlled systems, which

are generally used in noisy environments [15]. Most of the audio frequency acoustic noises, such as

computer fan noise and noise from people and cars, have low-frequency spectra and hence are colored

[15,16]. These noises often corrupt the speech signal, which is also colored. Therefore, it is important to

consider colored noises in designing speech denoising techniques.

In this paper, we propose a new pre-whitening TDLMS (PW-TDLMS) algorithm. It maintains the

structure of the conventional TDLMS algorithm and introduces a pre-whitenning before applying the

transform. This is a novel and interesting strategy to reinforce the decorrelation of the used fixed transform

in the TDLMS. The proposed pre-whitening is achieved by using a simple finite impulse response (FIR)

decorrelation filter of order one based on a fixed prediction concept. The resulting PW-TDLMS has the

advantage of using any transform even with low decorrelation. For this purpose, we investigate the use of

other transforms such as the parametric DFT and DHT transforms [17] and study the performance of the

proposed PW-TDLMS algorithm in terms of MSE convergence speed and steady state. We carry out this

comparative study by considering adaptive speech denoising as an application of the proposed algorithm

and show the simulation results of the proposed PW-TDLMS and TDLMS for different transforms.

Moreover, we compute the eigenvalue spread of the autocorrelation matrix obtained after applying the

parametric DFT or DHT transform and power normalization in the case of highly correlated Markov-1

noise, which is equivalent to the first order autoregressive (AR) process [11], for different values of the

independent parameter of the transform. The transform with the value of the parameter corresponding to a

good compromise between the eigenvalue spread and computational complexity is selected for the proposed

PW-TDLMS. It should be noted that the eigenvalue spreads in the case of the DCT and classical DFT and

DHT transforms are known in the literature [9,11,18].

 The reminder of this paper is organized as follows. Section 2 briefly presents the parametric DFT and

DHT. In Section 3, we analyze the TDLMS adaptive filter by considering the stability, steady state and

convergence performances in the case of a first order AR process. For the convergence analysis, we review

the eigenvalue spreads in the cases of the DFT, DHT and DCT, and then find the eigenvalue spreads in the

cases of the parametric DFT and DHT. The proposed PW-TDLMS adaptive filter is developed and

compared with the conventional TDLMS adaptive filter in Section 4 in terms of the eigenvalue spreads in

the cases of the DCT, DFT, DHT, parametric DFT and DHT transforms. Section 5 presents the

computational complexities of the DCT-LMS and proposed parametric DHT-based PW-LMS algorithms. In

order to compare the performance of the proposed PW-TDLMS with that of the TDLMS for different

transforms, we consider in Section 6 the speech denoising application for the cases of speech-like and real

speech signals. The simulation results and comparisons are given therein in terms of the MSE convergence

speed, reached steady state level, residual noise in the denoised signal, steady state excess MSE,

misadjustment and output SNR. Some concluding remarks are given in Section 7.

2. Parametric discrete Fourier and Hartley transforms

The three-parameter DFT transform of a complex sequence of order , , is defined in

[17] as

where , , are the entries of the parametric kernel vector given by

[ ] (2)

with the vector

[ ]

, , denotes the complex conjugate transpose, and , and are three

nonzero parameters. The matrix form of (1) can be written as

 (3)

where the entries of the three-parameter matrix are ,

. Since the parametric transform defined by (1) or (3) possesses three parameters that can arbitrarily be

chosen from the complex plane, a large number of new transforms with different features can be obtained. If

the three parameters are chosen as , and , then the parametric transform

reduces to the classical DFT. If the values of the three parameters are arbitrarily chosen from the unit circle,

then the parametric transform reduces to a three-parameter unitary transform. An interesting special case of

this transform is obtained in [17] when , with being an arbitrary real-valued parameter,

and . Let be the matrix operator of this unitary transform, which is nothing but the classical

DFT parameterized by one parameter and then denoted by . The matrix operator of the one-

parameter DHT is defined in [17] as

(4)

The corresponding transform denoted by is involutory, i.e., , with is an identity

matrix. It is obvious that is the classical DFT matrix and thus, is the classical DHT matrix, and

hence, the classical DFT and DHT correspond to and , respectively.

3. Transform domain LMS adaptive filter analysis

LMS

Fig. 1. Block diagram of the TDLMS adaptive filter.

In the TDLMS adaptive filter presented in Fig. 1, the correlated tap delayed input vector

[ ] is transformed into the vector using a fixed data-independent orthogonal

transform matrix , where N is the filter length. The resulting vector is less correlated then the

vector . The transformed vector is then power normalized using the diagonal matrix defined

in [9,11,19] as

[ ] (5)

where is the power estimate of the ith input , which can be computed using the recursive

equation given by

| | (6)
with the smoothing factor being a constant value between 0 and 1. The power normalization at each time

sample k leads to a time varying step size . The weight vector is then adapted as

(7)

where is a constant small-value introduced to avoid the overflow when the diagonal elements of are

close to zeros, is the step size parameter and denotes the complex conjugate of . The output error is

given by

where is the desired signal sample.

3.1. Stability performance with a first order AR process

We use the stability condition found in [11] for which the authors assume that the form of (7) is a mean-

square stabilized with the condition

(9)

It is clear that this stability condition on the step size depends only on the filter length N and therefore, the

input signal does not have any influence on the stability.

3.2. Steady state performance

The excess mean square error (EMSE) and the misadjustment (M) are among the mostly used adaptive

filtering performance measures. In [20], the is defined at each instant sample k as

| |

where, in the case of the adaptive noise cancellation, is the residual error, which is the difference

between the original speech signal sample and the filtered speech signal sample , and is the number

of samples used in the estimation of the . The steady state excess mean square error is the
result of computing the average of for k after the algorithm reaches the steady state. Then, the

is defined as

( )

where is the total number of samples and P the number of samples after which the algorithm reaches the

steady-state. The misadjustment is defined as

(12)

where is defined as

( ) | |

The noise reduction performance is evaluated by the estimate of the output signal to noise ratio (SNR)

computed in decibel (dB) as

(| | | | )

3.3. Convergence analysis with a first order AR process

As discussed earlier, the LMS algorithm suffers from a slow convergence in the case of a highly

correlated input signal, i.e., in the case where the autocorrelation matrix of the input signal has a large

eigenvalue spread. Therefore, we analyse in this subsection the effect of different transforms on reducing

the eigenvalue spread of the autocorrelation matrix of the transformed and power normalised signal, which

has a straight impact on the convergence of a TDLMS algorithm. This analysis is performed by modeling

the different noise sources by Markov 1 process, which is a simple class of signals, very general and

practical [4,5,9,17,21]. Markov 1 process, which is equivalent to the first order AR process [11], can be

generated by passing a white Gaussian noise through a single pole low-pass filter. The
autocorrelation matrix of the input signal obtained from the first order AR process of correlation

coefficient [ is given in [9] as

( )

A. Eigenvalue spreads in the cases of the DFT, DHT and DCT

In [9,22,23], the authors show that, for a very large value of (theoretically, for N tending to infinity),

the eigenvalue of are the values of the power spectrum of evaluated at uniformly distributed points on

the frequency axis. The asymptotic eigenvalue spread of is thus given by

( ) ( )

A high value of increases the correlation of the signal and therefore, it increases the eigenvalue spread

of the autocorrelation matrix . The autocorrelation matrix obtained after transformation and power

normalization of the input signal is not Toeplitz, and the above theory cannot be used to find the

eigenvalue spread. The asymptotic distribution of the eigenvalues of can be derived by solving the

following equation for

where denotes the matrix determinant , and

denotes the complex conjugate transpose operation. In [9], the problem of solving (17) is simplified to

an easier problem to deal with

where is the DFT or DCT matrix and is an identity matrix. The eigenvalue spread of obtained after

the DFT and power normalization is given by

 ( ) ( )

For the case of the DCT, the eigenvalue spread is

( )

In [11,18], the authors showed that the eigenvalue spread of achieved by the DHT-LMS tends to

| | | | for , where | | denotes the absolute value. It is clear that, for [ , the

eigenvalue spread of achieved by the DHT-LMS is the same as that achieved by the DFT-LMS. In [11],

the authors showed that, for ], the eigenvalue spreads of achieved by the DFT-LMS and the

DCT-LMS tends to and , respectively. The convergence performance of the

DFT-LMS and the DHT-LMS is not deteriorated in the case of negative values of , whereas that of the

DCT-LMS is deteriorated.

B. Eigenvalue spreads in the cases of the and

In this subsection, we study the eigenvalue spreads of for the cases of the and . We show

that these eigenvalue spreads depend on the parameter and can similarly be computed using (18) for

being the or matrix. This interesting propriety is exploited in this work to analyze these

eigenvalue spreads by numerical calculations for different values of and = 16 and then find the value of

that leads to the minimum eigenvalue spread that can be achieved by the -LMS and -LMS,

which is equivalent to the best MSE convergence behaviour. Fig. 2 shows the eigenvalue spreads of

achieved by the -LMS with a first order AR process for different values of the coefficient and filter

length It is clear from this figure that the minimum eigenvalue spread can be achieved for the

parameter in the interval [ ] Fig. 3 shows the eigenvalue spreads of achieved by the -

LMS and -LMS for = 16 and . It is seen from this figure that the eigenvalue spread of

achieved by the -LMS is slightly lower than that achieved by the -LMS. Some other results

obtained from numerical calculations are given in Table 1 for the case of and different values of .
 It is clear from this table that the value of the parameter corresponding to the lowest eigenvalue spread

depends on the value of the correlation coefficient . For = 0.9 or 0.8, the parameter is . For

= 0.7, 0.6 and 0.5, the parameter is . It can also be seen from Table 1 that the difference

between the minimum eigenvalue spreads, for a specific value of and a specific transform corresponding

to the various values of the parameter , is not meaningful and hence, the MSEs of the TDLMS algorithm

based on the corresponding transforms would have similar convergence behavior. Moreover, in [17],

Bouguezel et al found that the and lead in Wiener filtering to MSEs similar to that of the

classical DFT and DHT, respectively, and showed that the former reduce the computational complexity

compared to the latter. However, the transforms corresponding to and have

higher computational complexities. Therefore, it is highly desirable to consider the and

in the TDLMS algorithm.

70 = 0.9 70 DFT-LMS
= 0.8 DHT-LMS
60 = 0.7
60
= 0.6
= 0.5
Eigenvalue Spreads of SN

50 50
Eigenvalue Spreads of SN

40 40

30 30

20 20

10 10

0 0
2 - 0 2 -

Fig. 2. Eigenvalue spreads of SN achieved by DFT-LMS algorithm for N = 16, Fig. 3. Eigenvalue spreads of SN achieved by DFT-LMS or DHT-LMS
and some values of . algorithm for N = 16, and .
Table 1
Minimum eigenvalue spreads of for the various transforms with =16 and some values of .

0.9 0.8 0.7 0.6 0.5

DFT 10.7394 7.0308 4.8819 3.6048 2.7820
-2/15
DFT 10.6452 7.0044 4.8558 3.5986 2.7729
-3/20
DFT 10.9039 7.0111 4.8471 3.5870 2.7732
-/6
DFT 11.5244 7.1844 4.9091 3.6083 2.7881
DHT 9.4325 6.1360 4.3770 3.3075 2.6029
-2/15
DHT 9.4108 6.0630 4.3494 3.2814 2.5820
-3/20
DHT 9.6327 6.1030 4.3410 3.2771 2.5818
-/6
DHT 10.1632 6.3014 4.3802 3.2911 2.5968

4. Proposed pre-whitening transform domain LMS adaptive filter

Noise path

Pre-whitening
TDLMS
Filter g

Proposed PW-TDLMS

Fig. 4. Proposed pre-whitening TDLMS adaptive noise canceller.

In this section, we propose a new pre-whitening TDLMS (PW-TDLMS) adaptive noise canceller by

introducing a pre-whitening filter at the input of the TDLMS as presented in Fig. 4. The purpose of this

filter is to obtain a decorrelated version of the signal at the input of TDLMS. This has a direct impact

on further reducing the eigenvalue spread of the autocorrelation matrix obtained after transformation and

power-normalization of the signal in TDLMS and consequently on accelerating the convergence speed

and minimizing the reached steady state of the MSE of the TDLMS. The enhancement in the performance

of the TDLMS improves the quality of the output signal sample , which is the estimation of the original

signal sample to be recovered. The decorrelated signal at each time sample k can be obtained as


where [ ] , [ ] , L is the length of the pre-whitening

filter and [ ] is a finite impulse response (FIR) prediction filter. Thus, the impulse

response of the pre-whitening filter is [ ] .

Without loss of generality, we consider the case of and real-valued coefficients, i.e., [ ]

and [ ] . Therefore, (21) becomes

(22)

Let us now generate the input signal of the system given by Fig. 4 and used in (22) from a first order

AR process with a correlation coefficient that ensures a high eigenvalue spread of given by (15) as

(23)

where is a zeros mean random white Gaussian noise. Equation (23) can be rearranged as

( )

By substituting (24) in (22), we obtain

( )

It is clear from (25) that the signal can become completely white when leads to zero, i.e.,

tends to . In the case where the input signal is assumed to be highly correlated, the correlation

coefficient of the corresponding autocorrelation matrix would be close to unity. Therefore, perfect

pre-whitening can asymptotically be achieved by choosing the value of close to unity. For other values of

, , and unknown values of , some decorrelation of the input signal can be achieved, which
can then be exploited to reinforce the decorrelation that can be obtained by the considered transform in the

proposed PW-TDLMS.

Since the above condition on the filter coefficient depends on the coefficient of the input, which is in

practice not known, it is important to find an appropriate interval of possible values of that

asymptotically satisfy the condition without a prior knowledge on the exact value of , and thus ensure

acceptable decorrelation. For this purpose, we study the eigenvalue spreads of achieved by the DFT-

LMS, DHT-LMS, DFT-/6-LMS, DHT-/6-LMS and DCT-LMS without and with a pre-whitening filter in

terms of . The results of this study are given in Figs. 5, 6, 7, 8 and 9 for = 16, [ ] and 0.5,

0.6, 0.7, 0.8 and 0.9. Note that we have used the notation DFT-LMS, DHT-LMS, DFT-/6-LMS, DHT-/6-

LMS and DCT-LMS for the case of the conventional TDLMS and PW-DFT-LMS, PW-DHT-LMS, PW-

DFT-/6-LMS, PW-DHT-/6-LMS and PW-DCT-LMS for the proposed PW-TDLMS. Figs. 5, 6, 7, 8 and 9

show that the eigenvalue spreads of achieved by the proposed PW-TDLMS for different transforms and

for the values of the coefficient in the intervals [ ], [ ], [ ], [ ] and

[ ] decrease below the corresponding eigenvalue spreads achieved by the DCT-LMS in the cases

of , 0.8, 0.7, 0.6, 0.5, respectively, and approach to 1 when the coefficient leads to , i.e., the

input signal becomes completely decorrelated. In order to find the interval for that is appropriate to all

the cases, we adopt an averaging scheme and hence suggest the average interval [ ]. Therefore, the

middle value in this interval can allow the proposed PW-TDLMS to achieve a good reduction in

the eigenvalue spreads of for different transforms and for any value of [ .
 DFT-/6-LMS DFT-LMS DFT-/6-LMS DFT-LMS
Proposed PW- DFT-/6-LMS Proposed PW- DFT-LMS Proposed PW- DFT-/6-LMS Proposed PW- DFT-LMS
DHT-/6-LMS DHT-LMS DHT-/6-LMS DHT-LMS
15 Proposed PW- DHT-/6-LMS Proposed PW- DHT-LMS Proposed PW- DHT-/6-LMS Proposed PW- DHT-LMS
DCT-LMS Proposed PW- DCT-LMS DCT-LMS Proposed PW- DCT-LMS
8

Eigenvalue Spread of SN
Eigenvalue Spread of SN

10 6

0 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Different values of a1 Different values of a1

Fig. 5. Eigenvalue spreads of achieved by the TDLMS and proposed PW- Fig. 6. Eigenvalue spreads of achieved by the TDLMS and proposed PW-
TDLMS for different transforms and for N =16, = 0.9 and different values of TDLMS for different transforms and for N =16, = 0.8 and different values of
. .

DFT-/6-LMS DFT-LMS DFT-/6-LMS DFT-LMS

Proposed PW- DFT-/6-LMS Proposed PW- DFT-LMS Proposed PW- DFT-/6-LMS Proposed PW- DFT-LMS
DHT-/6-LMS DHT-LMS DHT-/6-LMS DHT-LMS
6 Proposed PW- DHT-/6-LMS
5 Proposed PW- DHT-/6-LMS Proposed PW- DHT-LMS
Proposed PW- DHT-LMS
DCT-LMS Proposed PW- DCT-LMS DCT-LMS Proposed PW- DCT-LMS

4
Eigenvalue Spread of SN
Eigenvalue Spread of SN

2
2

0 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Different values of a1 Different values of a1

Fig. 7. Eigenvalue spreads of achieved by the TDLMS and proposed PW- Fig. 8. Eigenvalue spreads of achieved by the TDLMS and proposed PW-
TDLMS for different transforms and for N =16, = 0.7 and different values of TDLMS for different transforms and for N =16, = 0.6 and different values of
. .
 DFT-/6-LMS DFT-LMS
Proposed PW- DFT-/6-LMS Proposed PW- DFT-LMS
DHT-/6-LMS DHT-LMS
Proposed PW- DHT-/6-LMS Proposed PW- DHT-LMS
DCT-LMS Proposed PW- DCT-LMS

Eigenvalue Spread of SN
4

0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Different values of a1
Fig. 9. Eigenvalue spreads of achieved by the TDLMS and proposed PW-
TDLMS for different transforms and for N =16, = 0.5 and different values of
.

In order to show that the average interval [ ] is appropriate for a random selection of the values

of regardless the values of the coefficient of the input signal, we present in Table 2 the eigenvalue

spreads of achieved by the TDLMS and the proposed PW-TDLMS for the various transforms. It is clear

from this table that, for a given transform including the DCT, the eigenvalue spreads of obtained in this

interval by the proposed PW-TDLMS are less than the eigenvalue spreads obtained by the corresponding

conventional TDLMS. This table also shows the importance of the proposed pre-whitening in rendering the

performance of the DHT-LMS, DFT-LMS, DFT-/6-LMS or DHT-/6-LMS closer to that of the DCT-LMS,

which is known for its high efficiency. Therefore, the proposed PW-TDLMS based on the DHT-/6, which is

a real-to-real transform and has reduced computational and structural complexities compared to the DCT

[17], would be an excellent alternative to the DCT-LMS algorithm. Table 2 also confirms that the suggested

value is convenient for all the considered cases and ensures a good performance for the pre-

whitening filter independently of the degree of the correlation of the input signal. Therefore, we suggest a

pre-whitening filter with the impulse response [ ] for the proposed PW-TDLMS.
Table 2
Some values of the eigenvalue spreads of achieved by the TDLMS and the proposed PW-TDLMS algorithms for various
transforms, N = 16, some values of and [ ].
0.9 0.8 0.7 0.6 0.5
Eigenvalue spreads of 186 60 27 14.5 8.5
with a pre-whitening filter [ ]
[ ] [ ] [ ] [ ] [ ]
PW-DCT-LMS [ ] [ ] [ ] [ ] [ ]
PW-DHT-LMS [ ] [ ] [ ] [ ] [ ]
PW-DFT-LMS [ ] [ ] [ ] [ ] [ ]
achieved by

PW-DHT-/6-LMS [ ] [ ] [ ] [ ] [ ]
PW-DFT-/6-LMS [ ] [ ] [ ] [ ] [ ]
with a pre-whitening filter
PW-DCT-LMS 1.44 1.29 1.09 1.09 1.31
Eigenvalue Spreads of

PW-DHT-LMS 2.50 1.64 1.16 1.14 1.46

PW-DFT-LMS 2.60 1.67 1.16 1.15 1.49
PW-DHT-/6-LMS 2.47 1.62 1.15 1.14 1.44
PW-DFT-/6-LMS 2.57 1.66 1.16 1.14 1.47
without a pre-whitening filter (classical algorithm)
DCT-LMS 1.87 1.88 1.78 1.69 1.57
DHT-LMS 9.40 6.10 4.37 3.31 2.60
DFT-LMS 10.67 7.01 4.87 3.62 2.78
DHT-/6-LMS 10.13 6.27 4.38 3.30 2.59
DFT-/6-LMS 11.48 7.15 4.90 3.61 2.79

5. Computational complexity

According to [24], the computational complexity of the conventional TDLMS (without including the
complexity of the used transform) is multiplications and additions for each iteration. The
computational complexity of the proposed PW-TDLMS is increased by only one addition and one
multiplication due to the first order pre-whitening filter. Therefore, by including the complexity of the
transform, the computational complexities of the DCT-LMS and proposed PW- -LMS are given in
Table 3 for one iteration and different values of , where , and are the numbers of
multiplications, bit-shifts and additions, respectively. It should be mentioned that we have used for the DCT
the complexity required by the fast algorithm developed in [25] and for the the complexity
required by the algorithm reported in [17]. It is seen from Table 3 that the proposed PW- -LMS
outperforms the DCT-LMS in terms of the computational complexity. Moreover, it is well-known that the
structural complexity of fast DHT is lower than that of fast DCT. Therefore, the structural complexity of the
proposed PW- -LMS, which is based on the that has a structure similar to that of the
DHT, is lower than that of the DCT-LMS.

Table 3
Computational complexities of the DCT-LMS and proposed - -LMS.

DCT-LMS - -LMS

16 129 129 106 0 111

32 273 305 222 1 280

64 577 705 474 2 649

128 1217 1601 1006 3 1466

256 2561 3585 2130 8 3343

512 5377 7937 4518 17 7464

1024 11265 17409 9546 30 16389

6. Simulation results

In this section, we present a set of simulations to test and compare the performance of the proposed PW-

TDLMS with that of the TDLMS using the DCT, DFT, DHT, DFT -/6 and DHT-/6 transforms. The

algorithms are implemented as adaptive noise cancellers as shown in Fig. 4.

6.1 Test on Speech-Like signal

Let us first test and evaluate the performance of the TDLMS and proposed PW-TDLMS algorithms by

taking the signal as a stationary speech-like signal obtained by passing a white Gaussian noise with a

variance through an 11th order AR filter. The input signal of both the noise path and the

adaptive filter is considered as a colored noise generated from a first order AR process (AR (1)) with

and in order to evaluate the efficiency of the proposed algorithm for AR (1) input noise signals
with different eigenvalue spreads of as shown in Table 2. The signal obtained at the output of is

then used for corrupting with an SNR equals to 0 dB. The length, step size, smoothing factor and

regularization parameter of the adaptive filter are N = 16, , = 0.9 and = 2.510-2, respectively.

The proposed PW-TDLMS algorithm is used for the various transforms with the pre-whitening filter

[ ] . The considered noise path is modeled as

(26)

whose impulse response is shown in Fig. 10.

Figs. 11 and 12 show the results of this test in terms of the MSE convergence of the TDLMS and the

proposed PW-TDLMS algorithms for the various transforms with and , respectively. It can be

seen from Fig. 11, which corresponds to a relatively lower eigenvalue spread of of the AR(1) input noise

signal, that the proposed PW-TDLMS algorithm performs well with the suggested pre-whitening filter

[ ] and outperforms the TDLMS algorithm for the various transforms in terms of the MSE

convergence speeds and reached steady state levels. Similar conclusion can be made from Fig. 12, which

corresponds to an AR (1) input noise with a high eigenvalue spread of . Figs. 11 and 12 show also that

the performance obtained using the suggested pre-whitening filter [ ] approaches that obtained

using the optimal filters [ ] and [ ] .

It has been shown in [4] that the decorrelated NLMS (DNLMS-NLMS) algorithm is better than the

NLMS algorithm. Thus, we compare in Fig. 13 the MSE convergence of the proposed PW-DCT-LMS

algorithm with that of the former by considering the speech-like signal corrupted by AR(1) colored noise.

The experiments are performed for the following three different cases:

Case 1: The weight vector adaptation step size is the same for both the algorithms.

Case 2: The adaptation step sizes of the DNLMS-NLMS are fixed to achieve an MSE steady state similar to

that of the proposed PW- DCT-LMS.

 Case 3: The adaptation step sizes of the DNLMS-NLMS are fixed to have an initial MSE convergence rate

similar to that of the proposed PW- DCT-LMS.

It is seen from Fig. 13 that the proposed PW-DCT-LMS algorithm significantly outperforms the

decorrelated NLMS for all the considered cases.

-30
-30
DCT-LMS
0.6 DFT-LMS
DHT-LMS
-35
-35
DFT-/6-LMS
-35 DHT-/6-LMS
0.4 PW-DCT-LMS (a1=0.65)
PW-DFT-LMS (a1=0.65)
PW-DHT-LMS (a1=0.65)
0.2 -35 PW-DFT-/6-LMS (a1=0.65)

MSE (dB)
-40
-40
-40 PW-DHT-/6-LMS (a1=0.65)
Tap weights

PW-DCT-LMS (a1=0.5)
PW-DFT-LMS (a1=0.5)
0 PW-DHT-LMS (a1=0.5)
-45
-45
-40
-35 PW-DFT-/6-LMS (a1=0.5)
-45 PW-DHT-/6-LMS (a1=0.5)
-0.2
-50
-50
-35
-45
-40
-50
-0.4

-55
-55
-35
-40
-50
-45 0 1000 2000 3000 4000 5000 6 000
1 3 5 7 9 11 13 15 -55
1000 2000 3000 4000 5000 6000
Tap Number
Iterations
Fig. 10. Impulse response of the noise path . -35
-40
-45
-55
-50
Fig. 11. MSE convergence of the TDLMS and proposed PW-TDLMS with
suggested and optimal values of for various transforms in the case of a
speech-like signal, N = 16 and AR (1) colored noise with .
-40
-45
-50
-55

-45
-50
-55
-30
-30
-20
-20
DCT-LMS
DFT-LMS ProposedPW-DCT-LMS
Proposed PW-DCT-LMS
DHT-LMS
DFT-/6-LMS -50 -25
-55 -25 DNLMS-NLMS( (
DNLMS-NLMS
-35
-35
-35 DHT-/6-LMS
PW-DCT-LMS (a1=0.65)
-35 DNLMS-NLMS( (
DNLMS-NLMS
PW-DFT-LMS (a1=0.65) -30
-30
DNLMS-NLMS( (
PW-DHT-LMS (a1=0.65) DNLMS-NLMS
PW-DFT-/6-LMS (a1=0.65) -55 -35
-35
-40
-40
-40 PW-DHT-/6-LMS (a1=0.65) -40
-35
-35
MSE (dB)

PW-DCT-LMS (a1=0.9)
MSE (dB)

MSE (dB)

PW-DFT-LMS (a1=0.9) -35

PW-DHT-LMS (a1=0.9)
PW-DFT-/6-LMS (a1=0.9) -40
-40
-40
-45
-40
-45
-35
-45 PW-DHT-/6-LMS (a1=0.9) -45
-35
-40
-45
-45

-50
-50 -45
-35
-35
-45
-40
-50 -50
-40
-50
-50

-45
-35
-55
-55
-35 -50
-40
-40
-50
-45
-55 1000 2000 3000 4000 5000 6000 7000 8000 -55
-55
1000 2000 3000 4000 5000 6000 7000 8000
1000 2000 3000 4000
Iterations
5000 6000 7000 8000 -55
-45
-35 1000 2000 3000 4000 5000 6000 7000 8000
-50
-40
Iterations Iterations
-35
-40 -55
-45
-45
-55
-50
Fig. 12. MSE convergence of the TDLMS and proposed PW-TDLMS with -35
Fig. -40
13.
-50 MSE convergence of the DNLMS-NLMS and proposed PW-DCT-
suggested and optimal values of for various transforms in the case of a LMS-55 algorithms for a speech-like signal corrupted by AR (1) colored noise
speech-like signal, N = 16 and AR (1) colored noise with . -45
with and N = 16.
-40
-45
-50
-55 -50
-40
-55
-45
-50
-45
-50
-55 -55
-45
-50
-55
-50
-55 -50
-55

-55 -55
 6.2 Test on real speech signal

The results obtained in Subsection 6.1 for the case of a speech-like signal also confirm that the proposed

PW-TDLMS algorithm, for the various transforms including the DHT-/6, outperforms the DCT-LMS

algorithm in terms of the convergence of the MSE. Moreover, as shown in Section 5, the proposed PW-

DHT-/6-LMS has reduced computational and structural complexities compared to the DCT-LMS algorithm.

Therefore, we carry out in this subsection a complete comparison between the performance of the proposed

PW-DHT-/6-LMS and that of the DCT-LMS by considering a clean speech signal taken from the noisy

speech corpus (NOIZEUS) described in [26]. The clean speech signal used corresponds to the sentence 24

A cruise in warm waters in a sleek yacht is fun. The speech signal is recycled four times in order to get a

signal with sufficient duration for the simulations and then corrupted by a test noise. The test noises, namely

Babble, Street, Train, and Airport, are taken from the LABROSA web site [27]. These noises are passed

through the noise path given by (26) and added to the clean speech signal with different SNR levels: 0, 5,

10 and 15 dB. A portion of the clean speech signal is presented in Fig. 14(a), whereas its corresponding

noisy speech is given in Fig. 14(b) for Street noise with SNR=0 dB. The noise cancellation in the cases of

the DCT-LMS and proposed PW-DHT-/6-LMS is carried out by considering the filter length N = 16,

, and the step sizes for the babble noise and for the other noises. A

portion of the resulting enhanced speech after convergence is presented in Fig. 15 for each of the two

algorithms.

The results corresponding to this test are shown in Figs. 16-21 and Table 4. It is clear from Fig.16 that the

proposed PW-DHT-/6-LMS algorithm outperforms the DCT-LMS algorithm in terms of the convergence

speed and steady state level achieved by the EMSE. Fig. 17 shows the residual noises in the denoised

signals at the outputs of the proposed PW-DHT-/6-LMS and DCT-LMS algorithms. It is clear from this

figure that the residual noise in the signal at the output of the proposed PW-DHT-/6-LMS algorithm is

weaker than that in the signal at the output of the DCT-LMS algorithm and the quality of the corresponding

denoised signal is better than that of the DCT-LMS, which is also confirmed by listening tests. Figs. 18, 19,
 20 and 21 present the output SNRs evaluated at the outputs of the proposed PW-DHT-/6-LMS and DCT-

LMS algorithms for different input SNR levels and different noises. It is seen from these figures that the

proposed PW-DHT-/6-LMS algorithm achieves noticeably higher SNRs compared to the DCT-LMS

algorithm. Table 4 shows that the proposed PW-DHT-/6-LMS algorithm outperforms the DCT-LMS

algorithm in terms of the steady state excess mean square error EMSEss, Misadjustment M and output SNRs

achieved for all the considered conditions.

0.4 0.4

Amplitude
0.2
Amplitude

0.2

(a)
(a)

0 0
-0.2 -0.2

-0.4 -0.4
7 8 9 10 11 12 13 7 8 9 10 11 12 13
Time (Seconds) Time (Seconds)

0.4 0.4
0.2
Amplitude
0.2
Amplitude

(b)
(b)

0
0
-0.2 -0.2

-0.4 -0.4
7 8 9 10 11 12 13 7 8 9 10 11 12 13
Time (Seconds) Time (Seconds)

Fig. 14. A portion of the: (a) clean speech signal, (b) noisy speech signal Fig. 15. A portion (after convergence) of the enhanced speech signal, in the
corrupted by Street noise with SNR = 0 dB. case of Street noise and SNR = 0 dB, at the output of the: (a) DCT-LMS
adaptive filter, (b) proposed PW-DHT-/6-LMS adaptive filter.

-25 0.4
DCT-LMS
-30 0.2
Amplitude

PW-DHT-/6-LMS
(a)

-35 0

-40 -0.2
EMSE (dB)

-45 -0.4
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Time (Seconds)
-50
0.4
-55
0.2
Amplitude

-60
(b)

0
-65
-0.2
-70 -0.4
0 1 2 3 4 5 6 7 8 9 10 11 12 13 0 1 2 3 4 5 6 7 8 9 10 11 12 13
Time (Seconds) Time (Seconds)
Fig. 16. EMSE of the proposed PW-DHT-/6-LMS and DCT-LMS
algorithms for speech signal and Street noise with SNR = 0 dB. Fig. 17. Residual noise in the signal at the output, with Street noise and SNR
= 0 dB, of (a) the DCT-LMS, (b) the proposed PW-DHT-/6-LMS.
 30 30
DCT-LMS DCT-LMS
Proposed PW-DHT-/6 -LMS Proposed PW-DHT-/6 -LMS
25 LMS 25 LMS

20 20

Output SNR (dB)

Output SNR (dB)

15 15

10 10

5 5

0 0
0 5 10 15 0 5 10 15
Input SNR (dB) Input SNR (dB)

Fig. 18. Output SNRs for the proposed PW-DHT-/6-LMS and DCT-LMS Fig. 19. Output SNRs for the proposed PW-DHT-/6-LMS and DCT-LMS
algorithms with Airport noise and different input SNR levels. algorithms with Street noise and different input SNR levels.

30 35
DCT-LMS DCT-LMS
Proposed PW-DHT-/6 -LMS Proposed PW-DHT-/6 -LMS
25 LMS 30 LMS

25
20
Output SNR (dB)
Output SNR (dB)

20
15
15
10
10

5
5

0 0
0 5 10 15 0 5 10 15
Input SNR (dB)
Input SNR (dB)
Fig. 20. Output SNRs for the proposed PW-DHT-/6-LMS and DCT-LMS Fig. 21. Output SNRs for the proposed PW-DHT-/6-LMS and DCT-LMS
algorithms with Babble noise and different input SNR levels. algorithms with Train noise and different input SNR levels.
Table 4
Steady state excess mean square error , Misadjustment and output SNR obtained by the DCT-LMS and proposed PW-
DHT-/6-LMS algorithms for P = 60000 and J = 200.

DCT-LMS Proposed PW-DHT-/6-LMS

Improvement
in the output
Input Output Output
Noises SNR (dB)
SNR (dB) SNR (dB) SNR (dB)
0 -47.88 1.25 12.15 -54.71 0.26 14.44 2.29
5 -49.45 0.87 14.92 -55.61 0.21 18.63 3.71
Street
10 -50.08 0.75 16.35 -55.91 0.20 21.70 5.35
15 -50.29 0.71 16.92 -56.00 0.19 23.38 6.46
0 -46.17 1.86 10.91 -52.82 0.40 12.08 1.17
5 -48.90 0.99 15.30 -54.93 0.25 16.85 1.55
Airport
10 -50.33 0.71 18.80 -55.94 0.20 21.18 2.38
15 -50.92 0.62 20.94 -56.35 0.18 24.56 3.62
0 -46.99 1.54 14.13 -54.98 0.24 15.81 1.68
5 -48.99 0.97 17.64 -57.07 0.15 20.43 2.79
Train
10 -49.86 0.79 19.76 -57.96 0.12 24.25 4.49
15 -50.16 0.74 20.67 -58.26 0.11 26.67 6
0 -50.20 0.73 13.12 -54.64 0.26 14.79 1.67
5 -50.46 0.69 16.95 -54.67 0.26 19.18 2.23
Babble
10 -50.55 0.68 19.54 -54.68 0.26 22.70 3.16
15 -50.58 0.67 20.83 -54.69 0.26 24.89 4.06

7. Conclusion

In this paper, a new pre-whitening transform domain LMS (PW-TDLMS) algorithm has been developed

by introducing a pre-whitening using a simple finite impulse response decorrelation filter of order one

before applying the transform to reinforce its decorrelation. It has been shown that the proposed PW-

TDLMS algorithm significantly decreases the eigenvalue spreads compared to the conventional TDLMS for

all considered transforms, namely the DCT, DFT, DHT, and parametric DFT and DHT. In both the

algorithms, the DCT has been found to be the best. In addition, we have studied the eigenvalue spreads in

the cases of the parametric DFT and DHT for different values of the parameters and found that the DHT-/6

presents a good compromise between the eigenvalue spreads and computational complexity. In order to

evaluated the performance of the proposed PW-TDLMS algorithm and compare it to that of the

conventional TDLMS, we have considered the speech denoising application and used the DHT-/6 in the

former and the DCT in the latter. All the obtained simulation results confirm the high efficiency and

superiority of the proposed PW-TDLMS algorithm compared to the conventional TDLMS algorithm in

terms of the mean square error convergence speed, reached steady state level, residual noise in the denoised
signal, steady state excess mean square error, misadjustment and output SNR. Moreover, we have shown

that the computational and structural complexities of the proposed PW-DHT-/6-LMS are lower than those

of the conventional DCT-LMS. Therefore, the former would be an excellent alternative to the latter for

adaptive filtering and speech denoising.

References
[1] S. Haykin, Adaptive Filter Theory, 3 rd ed., Prentice-Hall, 1996.

[2] D. I. Kim, P. D. Wilde, Performance analysis of signed self-orthogonalizing adaptive lattice filter, IEEE Transactions on
Circuits and Systems II: Analog and Digital Signal Processing 47 (11) (2000) 1227-1237. doi: 10.1109/82.885130.

[3] E.V. Kuhn, J. E. Kolodziej, R. Seara, Analysis of the TDLMS algorithm operating in a nonstationary environment, Digital
Signal Processing 45 (2015) 69-83. doi: 10.1016/j.dsp.2015.05.013.

[4] S. Gazor, T. Liu, Adaptive filtering with decorrelation for coloured AR environments, IEE Proceedings - Vision, Image and
Signal Processing 152 (6) (2005) 806-818. doi: 10.1049/ip-vis:20045260.

[5] S. C. Douglas, A. Cichocki, S. Amari, Self-whitening algorithms for adaptive equalization and deconvolution, IEEE
Transactions on Signal Processing 47 (4) (1999) 1161-1165. doi: 10.1109/78.752617.

[6] M. Mboup, M. Bonnet, N. Bershad, LMS coupled adaptive prediction and system identification: a statistical model and
transient mean analysis, IEEE Transactions on Signal Processing 42 (10) (1994) 2607-2615. doi: 10.1109/78.324727.

[7] J. Lu, X. Qiu, H. Zou, A modified frequency-domain block LMS algorithm with guaranteed optimal steady-state
performance, Signal Processing 104 (2014) 27-32. doi: 10.1016/j.sigpro.2014.03.029.

[8] S. Narayan, A. Peterson, M. Narasimha, Transform domain LMS algorithm, IEEE Transactions on Acoustics, Speech, and
Signal Processing 31 (3) (1983) 609-615. doi: 10.1109/TASSP.1983.1164121.

[9] F. Beaufays, Transform-domain adaptive filters: an analytical approach, IEEE Transactions on Signal Processing 43 (2)
(1995) 422-431. doi: 10.1109/78.348125.

[10] K. Mayyas, New transform-domain adaptive algorithms for acoustic echo cancellation, Digital Signal Processing 13 (3)
(2003) 415-432. doi:10.1016/S1051-2004(03)00004-6.

[11] S. Zhao, Z. Man, S. Khoo, H. R. Wu, Stability and Convergence Analysis of Transform-Domain LMS Adaptive Filters With
Second-Order Autoregressive Process, IEEE Transactions on Signal Processing 57 (1) (2009) 119-130. doi:
10.1109/TSP.2008.2007618.

[12] B. F. Boroujeny, S. Gazor, Selection of orthonormal transforms for improving the performance of the transform domain
normalised LMS algorithm, IEE Proceedings F - Radar and Signal Processing 139 (5) (1992) 327-335. doi: 10.1049/ip-f-
2.1992.0046.

[13] N. Ahmed, T. Natarajan, K. R. Rao, Discrete Cosine Transform, IEEE Transactions on Computers C-23 (1) (1974) 90-93.
doi: 10.1109/T-C.1974.223784.
[14] D.I. Kim, P.D. Wilde, Performance analysis of the DCT-LMS adaptive filtering algorithm, Signal Processing 80 (8) (2000)
1629-1654. doi:10.1016/S0165-1684(00)00098-0.

[15] S.V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction, 3rd ed., Wiley, 2006.

[16] K. Ngo, T.V. Waterschoot, M.G. Christensen, M. Moonen, S.H. Jensen, Improved prediction error filters for adaptive
feedback cancellation in hearing aids, Signal Processing 93 (11) (2013) 3062-3075. doi:10.1016/j.sigpro.2013.03.042.

[17] S. Bouguezel, M. O. Ahmad, M. N. S. Swamy, New Parametric Discrete Fourier and Hartley Transforms, and Algorithms for
Fast Computation, IEEE Transactions on Circuits and Systems I: Regular Papers 58 (3) (2011) 562-575. doi:
10.1109/TCSI.2010.2072151.

[18] M. Kamenetsky, Accelerating the convergence of the LMS adaptive algorithm, Ph.D. dissertation, Stanford Univ., Stanford,
CA, 2005.

[19] J. J. Shynk, Frequency-domain and multirate adaptive filtering, IEEE Signal Processing Magazine 9 (1) (1992) 14-37. doi:
10.1109/79.109205.

[20] Z. Ramadan, A. Poularikas, An adaptive noise canceller using error Nonlinearities in the LMS adaptation, Proceedings of the
IEEE Southeast Con. (2004) 359-364. doi:10.1109/SECON.2004.1287943.

[21] C.H. Chen, Signal and Image Processing for Remote Sensing, Taylor & Francis, 2007.

[22] U. Grenander, G. Sego, Toeplitz Forms and Their Applications, 2nd ed., New York: Chelsea. 1984.

[23] R. M. Gray, Toeplitz and circulant matrices: II, Tech. Rep. 6504-1, Inform. Syst. Lab., Stanford Univ., 1977.

[24] R. C. Bilcu, P. Kuosmanen, K. Egiazarian, A transform domain LMS adaptive filter with variable step-size, IEEE Signal
Processing Letters 9 (2) (2002) 51-53. doi: 10.1109/97.991136.

[25] M. Puschel, Cooley-Tukey FFT like algorithms for the DCT, Acoustics, Speech, and Signal Processing, IEEE International
Conference on Acoustics, Speech, and signal processing 2 (2003) 501-504. doi: 10.1109/ICASSP.2003.12024133

[26] Y. Hu, P.C. Loizou, Subjective comparison and evaluation of speech enhancement algorithms, Speech Communication 49
(78) (2007) 588-601. doi:10.1016/j.specom.2006.12.0063

[27] Lab Rosa, Columbia University. http://www.ee.columbia.edu/ln/rosa/sounds/noise/ (accessed 17/02/2016).

Highlights
A novel pre-whitening transform domain LMS algorithm is proposed.
This pre-whitening is introduced before applying the transform to reinforce its decorrelation using a
simple finite impulse response decorrelation filter of order one.
In adaptive speech denoising, the proposed algorithm outperforms the conventional algorithms for
different transforms in terms of the computational complexity, mean square error convergence
speed, reached steady state level, residual noise in the denoised signal, steady state excess mean
square error, misadjustment and output SNR.