SPECTRAL MOMENTS AND THE ANALYSIS OF CHAOTIC SYSTEMS

Nick T. Koussoulas

Laboratory for Automation and Robotics

Electrical and Computer Engineering Department
University of Patras
Rio Patras 26500, GREECE
+30.61.997-296 Tel

+30.61.997-309 FAX

E-mail: ntk@ee.upatras.gr

KEYWORDS: chaotic dynamics, spectral analysis, identification, stochastic systems

ABSTRACT: The simplicity of structure of chaotic systems, combined with the richness of
their output, inspires their use in modeling efforts. On the other hand, the difficulty of their
analysis warrants approximation methods, especially since the absence, by definition, of welldefined limit sets prohibits, in general, a meaningful linearization. In this work we present some
results, which can support a methodology founded on spectral analysis for approximating
chaotic systems via stochastic linear systems. The main contribution is the use of spectral
moments for identifying the location of embedded limit cycles and the spectrum-based validation
of approximations.

1. INTRODUCTION
The newly appreciated field of chaotic nonlinear dynamic systems receives a great deal of
attention by scientists and recently engineers as well. There exist many textbooks and
monographs discussing matters related to the concepts and mathematics of chaos. For the
engineer this new field has not provided any spectacular applications, in the sense that no
long-standing problems have been solved yet. Nevertheless, many researchers are busy trying
to apply the ideas in an attempt of different, and hopefully more successful, approach to
problems, especially modeling natural phenomena. In this line of thought there are some
outstanding problems that must be tackled before any application-oriented methodology can
be developed and/or used. The two major problems are the detection of chaos in a system and
the control of a chaotic system.

Chaotic behavior can be claimed when there is sensitive dependence on initial conditions and
the orbits of the system come arbitrarily close to each other as time increases, provided some
technical conditions also hold (sect. 4.11 of [Wiggins, 1990]). Some measures of these
conditions can be derived through power spectrum analysis, the Lyapunov characteristic
exponents and K-entropy, and time series analysis (Ch. 9 of [Rasband, 1990]). Of these
measures, the power spectrum analysis seems most natural and familiar to the engineer.
Besides, chaotic signals may very well appear just like noise to an electrical engineer. Hence,
the idea is to work mostly in the frequency domain. There is a novelty, however, in this work:
we introduce the use of spectral moments for quantifying characteristics of the spectrum.

The next, perhaps natural, step toward approximate control of a chaotic system is its
approximation by a stochastic linear system, given that the relevant theory is well developed
and known. The methodology we follow can be considered as another type of shaping filter
design. It simply means that the colored noise (non-flat spectrum), or "noise" in the case of
chaotic signals, is approximated by filtering white noise through a linear system of appropriate order and parameters. To identify the linear system, stochastic or otherwise, whose output
approximates the chaotic signal we use the following approaches:

Spectral moment matching

Spectral factorization/stochastic realization approach

ARMA identification

Inverse frequency model identification

Gauss-Markov model identification

Heuristic identification of a linear system

In the rest of this paper, section 2 contains a review of spectral moments and discusses
expressions for spectral moments related to some simple linear dynamical systems. Section 3
presents the application of spectral moments to nonlinear systems. Section 4 presents the
approach and results of the stochastic linear system approximation to chaotic dynamics.
Throughout, we do not dwell much in the theory of chaotic systems as such, assuming the
reader has some familiarity with the technical aspects of the subject, e.g. with [Parker &
Chua, 1989; Rasband, 1990; or Wiggins, 1990]. Also, we do not get into the problem of
controlling a chaotic system, making the assumption that either the approach of [Ott et al.,
1990] will be used (guidance to and "lock" onto a limit cycle), or that the control policy will
be based on the approximating linear system.

2. A REVIEW OF SPECTRAL MOMENTS

The concept of spectral moments has been known for many years and various definitions and
properties have existed for a long time but do not appear to have been used in control theory
contexts. A significant contribution to the study and use of spectral moments has taken place
in the area of random fields. The most important source is [Vanmarcke, 1984], where an
extensive survey of the subject is performed. From that source we reproduce some
introductory material right below.
The kth spectral moment, k, of a stationary random process x(t) is defined as follows:

k = k G( )d = S ( )d ,
0

k = 0 ,1,2 ,...

(1)

where G() and S() are the one-sided and double-sided spectral density functions of the
process, respectively. Note that 0 is equal to the variance of the process. Characteristic frequencies are defined so that for each moment

k = k
0

1k

k = 1,2,

(2)

and it holds that k k +1 , for all k. It is natural to draw a parallel with the probability
density function (pdf) and its moments. A spectral density function, normalized so that it has
an area equal to unity, is totally analogous to a pdf. Given this relation, it is very helpful to
interpret the characteristic frequencies 1 and 2 as the "mean" and the "root mean square"
frequencies, respectively. We can also define, in perfect analogy with a pdf, the square root of
the difference of the squared second and first characteristic frequencies

s = 22 12

12

(3)

which can be called the frequency "standard deviation." Further, we can introduce two
dimensionless spectral parameters

12

= 1
0 2

22
= 1
0 4

12

12
= 1 2
2

24
= 1 4
4

12

s
2

(4)

and
12

12

(5)

lies between 0 and 1 and measures the degree of dispersion, i.e. the bandwidth, of the
spectrum. is a less direct measure of dispersion and also lies between 0 and 1. The use of the
fourth moment reminds of the kurtosis measure in a pdf. Higher order spectral moments tend
to be extremely sensitive to details of the high frequency content of G(). When that part of
the spectrum does not contain much or reliable information, the use of higher order moments
is not recommended.

The analogy to the probability density function is established when we consider the
normalized spectral density function g(). One interesting aspect of this analogy is the
calculation of the contribution to the total variance of frequencies above a given value.
Defining the normalized one-sided spectral distribution function

F ( ) = g ( )d

(6)

then, for > 2 we have

F IJ
F ( ) 1 G
H K
s

2 22
= 1
1

F 1IJ
1 G
H K
g

(7)

This inequality provides a bound on the fractional contribution to the total variance from
frequencies above a given value. For example, given that = 0.2 and = 32 we find that
F (3 2 ) 0.99 and consequently that F (32 ) 0.99 2 , which means that the contribution
of components with frequency above 32 to the total variance is less than 1%. A known or
estimated maximum value of will establish the minimum contribution. The upper bound for
= 1 gives F (3 2 ) |

=1

0.75 . If only 2 is known and the spectral density function S() is

symmetrical, the following stronger inequality holds:

F IJ
F ( ) 1 G
HK

(8)

which for = 32 yields F (3 2 ) 0.889 . In case S() is unimodal (at = 0, of course) then
it can be shown that a sharper bound is
F ( ) 1

FG IJ
H K

4 2
9

(9)

In time domain, spectral moments are equivalent to the derivatives of the autocorrelation
function, as it is easy to see from elementary properties of the Fourier transform.

Based on the above concepts and properties, many interesting features or characteristics of
stochastic processes can be quantified. The following is just a sample:
Mean threshold crossing rate
Average rate of occurrence of local maxima
Local maxima above high levels
Mean length of stay above or below a threshold
Envelope crossing statistics
Clustering of threshold crossings and local maxima
Probability distribution of maximum values

For details and many results concerning Gaussian processes the reader is referred to
[Vanmarcke, 1984]. Furthermore, the first spectral moment is widely used in time-frequency
signal analysis [Boashash, 1991], although it is hardly ever explicitly mentioned by name. It
can be said that although the concept has appeared occasionally in the literature, its
applications to date are rather limited. For example, spectral moments have been used in the
past for failure diagnosis and analysis [p. 312 in Pau, 1981] and more recently for
electromyogram analysis [Merletti et al., 1994].

For the convenience of the reader, the following table summarizes the spectral moments and
other relevant quantities for various simple models that are used extensively in practice. Most
of the results have been taken from [Vanmarcke, 1984] with some minor errors corrected.

MODEL

SPECTRUM

SPECTRAL MOMENTS k

Low-pass white
noise

Ib

Ideal white noise

II

Sinusoid with
random phase angle
0 =

Damped oscillator
excited by white
noise

G ( ) =

G0
2

+ 4 2 2 2

1 =

( 0 and 1)
2 =

Markov correlation
(approximate white
noise)
G( ) =

G0
2

+2

1/2

2/3

--

--

1k 2

R|1 1
|S 1
||ML1 1 tan FG 2 1
TMN H 1 2

G0
,
8

4 1
2

LM
F 2 1
tan G
MN 2
H 2 1

I OP
JK P
Q

2
2 =
=
0

3 = 4 = =

1 3 5 2 m 3 G0
2 4 6 2 m 2 2 2 m 1

2
2m 3

1 =
2 =

--

2 4 6 ( 2m 2)
m 1 1 3 5 (2m 3)

b2m 3g

m = 2,3,...

U|
|
I OP V|
JK P |
QW

1/2

G0

2 =

1
G0
1 =
2 m 2 2 m2
m=2

1/ k

, k = 1,2,3,...

G0
8 3

0 =
IV

F 1I
H k + 1K

1
1k +1G0
k +1

Ia

III

CHAR. FREQUENCIES k

1/ 2

LM1 F I OP
MN GH JK PQ

2 1/ 2

--

Table 1.

3. APPLICATION OF SPECTRAL MOMENTS TO NONLINEAR SYSTEMS

For nonlinear systems it is not possible, in general, to arrive at explicit forms for the spectral
moments. However, it is possible (and easy) to calculate the moments from experimental or
simulation data. For this purpose, the author has written a MatlabTM program. The interesting issue
is how can one utilize spectral moments to detect and/or analyze properties of nonlinear systems.
We proceed now into showing one example of the potential of spectral moments.

It is enlightening to follow the spectrum of the output or a state of a nonlinear system as the key
parameter drifts through chaotic and non-chaotic regions. Consider the logistic equation

x n +1 = ax n (x n 1)

(10)

The power density spectra for four different values of the key parameter a in are shown in Figure 1.
A similar presentation of spectra for different values of a key parameter of the Duffing equation can
be found in the first Chapter of [Parker & Chua, 1989]. Figures 2a, 2b, and 2c give the variability

Figure 1. Power spectra of the logistic map.

indices and the characteristic frequencies for the logistic map as its parameter a varies between 3.30
and 4.00. Note that characteristic frequencies are a more convenient way to investigate spectral
properties because of their more compact range of numerical values (cf. Eq. (2)).

1
0.9

0.8
0.7

0.6
0.5
0.4
0.3
0.2

0.1
0
3.2

3.3

3.4

3.5

3.6

3.7

3.8

3.9

Figure 2a. , , and s vs. the parameter a for the logistic map.
0.7

0.6

0.5

0.4

0.3

0.2

0.1

a
0
3.2

3.3

3.4

3.5

3.6

3.7

3.8

3.9

Figure 2b: Characteristic frequencies 2 (lower curve) through 11 (higher curve) vs. the
parameter a for the logistic map.

0.9

0.85

0.8

0.75

0.7
0.65

0.6
0.55

0.5
3.2

a
3.3

3.4

3.5

3.6

3.7

3.8

3.9

Figure 2c: Characteristic frequencies 12 (lower curve) through 21 (higher curve) vs.
the parameter a for the logistic map.
The drastic changes of the characteristic frequencies at the points of occurrence of chaos (near a =
3.65) and at the area covered by the embedded limit cycles (near a = 3.73 and near a = 3.85)
indicate the change of behavior. Moreover, the whole non-chaotic area from a 3.83 to a 3.86
can be clearly seen as well as the non-chaotic zone near a 3.73. The changes are more
pronounced for the higher frequencies and moments. The correlation between spectrum complexity
(which is typical for chaotic systems) and high values of the frequencies is clearly seen. Therefore,
the reliability of prediction of embedded limit cycles increases through the use of a higher moment.
It can also be seen that contrary to the power of high spectral moments and frequencies, the
variability indices (, , and s) cannot convey reliable information in the case of the logistic map.

The analytical power of the spectral moments and characteristic frequencies is still valid for
narrower parameter ranges. In Fig. 3, the range for a [3.67, 3.77] is explored and, again, the
higher-order characteristic frequencies identify very accurately the regions where embedded limit
cycles are present. Therefore, it is again evident that spectral moments can reliably detect the region
where a limit cycle is embedded. Furthermore, the straightforwardness of computation of spectral
moments and, of course, characteristic frequencies, must be pointed out, especially when compared
to other popular indicators of chaotic behavior, such as the Lyapunov exponents [Rangarajan,
10

1998]. It is clear that for a system with unknown behavior depending on the values of some
parameters, one can easily and quickly scan a range of interest of these parameters and find out
whether there are embedded limit cycles or in which area to investigate further for notable behavior.

0.9

0.85

0.8

0.75

0.7
0.65

0.6
0.55

0.5
3.7

3.75

3.8

Figure 3. Characteristic frequencies 12 (lower curve) through 21 (higher curve)

for a limited range of the logistic map parameter.

To explore the applicability of the spectral moments analysis for continuous dynamical systems, we
investigated the Roessler system in the following form:

x = y z
y = x + ay
z = b + z ( x c)

(11)

where a, b, and c are parameters. In our experiments we took a = b = 0.2 and we let c vary. The
bifurcation (Feigenbaum) diagram of the Roessler system is quite rich in embedded limit cycles. In
Fig. 4 we present the results of an exploration of the range c [7.00, 8.50] with a resolution of
0.05.
It is worth noting that the dips in the characteristic frequency curve indicate the presence of
embedded limit cycles as verified by checking Fig. 12.30 in p. 693 of [Peitgen et al., 1992].
Depending on the resolution used, it is possible, naturally, to miss an embedded limit cycle. Also, to
11

have such unambiguous results, it is necessary to apply the spectral moments analysis to the part of
the system trajectory that belongs to the attractor. In other words, one must wait for the transient
phase to subside.

0.7
0.68
0.66
0.64
0.62
0.6
0.58
0.56
0.54

c
0.52
7

7.5

8.5

Figure 4. Characteristic frequency 25 vs. the parameter c for the Roessler system.

Finally, in case of additive noise contamination, the dips get subdued and even though this effect is
in general analogous to the noise power (which means that for low enough noise power it is still
possible to reliably predict an underlying limit cycle), the results cannot be generally interpreted
without risk of error. The presence of noise, additive, multiplicative or otherwise, represents an
interesting direction for further investigations, to be followed in the future.

4. SPECTRAL MOMENTS IN THE APPROXIMATION OF CHAOTIC DYNAMICS

The typical approach toward the control of a nonlinear system is to focus the effort onto its
linearized version, with the linearization taking place around a specific operating point or attractor.
However, this approach cannot be used for chaotic systems due to the presence of the strange
attractor. Instead, one viable alternative for approximation could exploit the resemblance between
chaotic and stochastic signals. Spectrum-based techniques are especially suited for this purpose.
12

This idea is similar to the approximation of colored noise via the filtering of white noise through an
appropriate linear system (the "shaping filter" technique). To this end, several approaches have
been tried with varying success. They are concisely described below.

4.1 Spectral moment matching. As mentioned in section 2, spectral moments can characterize the
spectrum of a signal in the same way probabilistic moments can represent or summarize the
corresponding probability density. To exploit this fact, one has to calculate the spectral moments of
a linear (or nonlinear, for that matter) system and match them to those of the chaotic signal. This is
straightforward for low-order linear systems but for higher orders analytic calculation becomes
extremely tedious, if at all possible (cf. Table 1 and [Vanmarcke, 1984]). The complexity of the
inverse problem, namely determining the parameters of a linear system from its spectral moments,
increases at the same rate. Here, we used the second-order linear oscillator excited by white noise
(case III in Table 1).

It is worth noting that an alternative way would be to consider directly the probability density that
can be associated to the chaotic signal. Stochastic approximations of chaotic signals have been
known for some time [Young, 1983] and in a more generalized way in [Lasota & Mackey, 1994].
As shown in [Wu & Yao, 1995], a relation can be established between a chaotic system and the
probability density function (pdf) of an It stochastic differential equation. The moments of this pdf
can be matched to the spectral moments computed experimentally. This approach has not been
taken up here.

4.2 Spectral factorization/Stochastic realization approach. Based on the theoretical results of

[Caines, 1984] and the algorithms of [Ch. 9 in Aoki, 1984], it is possible to match a linear system
structure to a given chaotic signal. The standard method is to determine the covariance coefficients
of the signal and then form a succession of Hankel matrices of increasing order and check their
rank. If the rank remains unchanged through a step, then the order of the system equals this rank.
For complex signals, such as those corresponding to chaotic systems, this is realized by judging
when a singular value is much smaller than the preceding one. Then, simple formulae (for example,
pp. 119-121 in [Aoki, 1984]) yield the system matrices.

4.3 ARMA identification. The signal is considered as a time series and an input-output model is
matched to it in the classical way based on the MatlabTM functions arxstruc and armax. These
and other similar routines belong to the MatlabTM System Identification Toolbox and serve for
extracting the parameters of ARMA (Auto-Regressive Moving Average) models from input-output
13

data. The order of the model is determined in the same way as for the stochastic realization. Other
options for arriving at the correct order include Akaike's information criterion (so-called AIC).

4.4 Inverse frequency model identification. A very simple method for identifying a linear system
from frequency data is provided by Levis method and improvements thereof, better known as
inverse frequency identification. This method is an equation-error approach implemented as the
invfreqs function in the MatlabTM Digital Signal Processing toolbox, whose manual contains
also a brief but comprehensive description of the technique and the relevant algorithm.

4.5 Gauss-Markov model identification. Spectral moment matching can also be attempted for the
mth-order Gauss-Markov process, i.e. a band-limited white noise (as opposed to regular white noise
used in 4.1 above) admitting an exponential correlation model such as the one appearing in row IV
of Table I.

4.6 Heuristic identification of a second order linear system. It is observed that the spectra of
many chaotic signals contain a peak. This is also true for the frequency response of a second order
linear system (driven by white noise). Thus, the parameters of a second order linear system are determined so that its resonant frequency coincides with the chaotic peak.

As an example of application of the above methods, we use a chaotic Roessler system described by
the set of ordinary differential equations
x = y z
y = x + 0.398 y

(12)

z = 2 + z ( x 4)

Our mission is to approximate one of the states, say the second state x2, which we consider as
output of the system. The performed experiments indicated that the quality of approximation
depends on the form of the chaotic signal. Just to give an idea, results of approximating the second
state of (12) are summarized in Table 2. We used all the approaches mentioned above, but we
limited the search to second order systems only. The distances mentioned in Table 2 measure the
separation between the spectra of the chaotic system and that of each linear model in the Euclidean
and the maximum in absolute value sense. The values of the first spectral moment with its deviation
and the dispersion measures and are also compared. The values of and help in distinguishing
between two approximations of similar quality.

14

The results of Table 2 strongly suggest that the Gauss-Markov approach is inadequate, something
that perhaps could have been predicted since the resulting spectrum will always tend to be flat over
a large part in the low frequency section. Spectral factorization also looks weak, while increasing
the order of the model does not help much. The heuristic approach gave the best results overall. In
summary, we believe that with the help of the distance and dispersion measures, reliable
conclusions can be reached. With respect to spectra, an indicative outcome (corresponding to Table
2) appears in Fig. 5.

l2 distance

max. abs.
distance

1 s

Spectral moments

0.64394

0.75101

0.0959

0.3668

0.0623 0.0995

Spectral factorization

0.11781

0.27269

0.6558

0.9741

0.0806 1.1610

ARMA

0.10868

0.28808

0.3892

0.6757

0.0540 0.3779

Inverse frequency

0.45055

0.61345

0.1233

0.4530

0.0622 0.1280

Gauss-Markov

0.11895

0.33599

0.6568

0.8613

0.2210 3.1908

Heuristic 2nd order

0.09655

0.23967

0.3317

0.8230

0.0570 0.3318

0.4308

0.8073

0.0565 0.4471

Method

CHAOTIC

Table 2.

10

10

10

G (w)

10

10

10

10

10

-1

-2

Gauss-Markov
-3

Spectral factorization
-4

Heuristic
-5

Inverse frequency

ARMA

-6

Spectral moment
hi
-7

0.1

0.2

0.3
0.4
Frequency

0.5

0.6

0.7

Figure 5. Relative performance of various second-order linearization methods. The gray

curly curve is the power spectral density of the chaotic system.

15

5. CONCLUSIONS

We presented a short review of spectral moments and their expressions for simple systems. Then
we proposed a method based on these rather little used objects for the detection of embedded limit
cycles in a chaotic attractor. The analysis is based on the characteristic frequencies, which are a
kind of normalized spectral moments with tame arithmetic variation and easily computable. The
key fact is that a significant drop in the high-order characteristic frequencies of signals from a
chaotic system, whose parameter varies, indicates the presence of a limit cycle. We also set forth
some ideas about how linear stochastic systems can serve as approximations to chaotic systems and
how the identification of such linear systems can be realized via spectral moments. There are
clearly plenty of further issues to be explored, including mainly the theoretical justification of the
above technique as well as for practical matters, including the case of systems with noise
contamination. For example, it would be interesting to investigate the role of cross-correlation
between noises driving linear systems approximating different components of a chaotic system.
Another important topic is the analysis and design of control policies for chaotic systems based on
their surrogate linear stochastic systems.
ACKNOWLEDGMENT
The author gratefully acknowledges discussions with Prof. T. Bountis.
REFERENCES
Aoki, M. [1984] State-Space Modeling of Time Series (Springer, New York).
Boashash, B. [1991] Time-Frequency Signal Analysis, pp. 418-517 in Haykin, S. (Editor) [1991]
Advances in Spectrum Analysis and Array Processing (Prentice Hall, Englewood Cliffs, New Jersey).
Caines, P. E. [1984] Stochastic Linear Systems (Wiley, New York).
Lasota A., and M. C. Mackey [1994] Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics,
(Springer, New York).
Merletti, R., Gulisashvili, A., & Lo Conte, R. L. [1994] Estimation of EMG Spectral Moments from
Time Domain Data, Proc. 16th IEEE-EMBS Conference.
Ott, E., Grebogi, C., & Yorke, Y. A. [1990] Controlling Chaos, Phys. Rev. Lett., 64(11), pp. 1196-1199.
Parker, T. S. & Chua, L. O. [1989] Practical Numerical Algorithms for Chaotic Systems (Springer, New
York).
Peitgen, H.-O., Juergens, H., & Saupe, D. [1992] Chaos and FractalsNew Frontiers of Science,
(Springer Verlag, New York).
Pau, L. F. [1981] Failure Diagnosis and Performance Monitoring (Dekker, New York).
16

Rangarajan, G., Habib, S., & Ryne, R. D. [1998] Lyapunov Exponents without Rescaling and
Reorthogonalization, Phys. Rev. Lett., 80, p. 3747.
Rasband, S. N. [1990] Chaotic Dynamics of Nonlinear Systems (Wiley, New York).
Vanmarcke, E. [1984] Random Fields (MIT Press, Cambridge, MA).
Wiggins, S. [1990] Introduction to Applied Nonlinear Dynamical Systems and Chaos (Springer, New
York).
Wu, Q. H. & Cao, Y. J. [1995] An Equivalent Stochastic System Model for Control of Chaotic
Dynamics, Proc. of the 34th Conf. on Decision & Control (New Orleans, LA), pp. 2898-2903.
Young, L. S. [1983] Entropy, Lyapunov Exponents, and Hausdorff Dimension in Differentiable
Dynamical Systems, IEEE Trans. on Circuits and Systems, 30(8), pp. 599-607.

17