Reconstruction of

Phase Space:
Embedding Dimensions

AMATH 575
Kyle Mandli
June 2nd, 2005
 Introduction

• Chaotic data is observed often in experiments in many

fields

• Traditional techniques such as Fourier analysis are not

effective when dealing with chaotic signals

• Need a similar systematic technique to linear signal

analysis for non-linear dynamical systems
 Outline

• Basic Steps in analysis of a time series

• Reconstruction of phase space

• Embedding Dimensions

• Conclusions
Analysis of Measured Signals
• Separating signal from background noise or signal
contamination

• Determine the appropriate space to analyze the signal

• Classification of the signal

• Make predictions or models of the system

• A signal is some scalar quantity that we are sampling at

regular time intervals τs which we will assume are
constant
s(n) = s(t0 + nτs )
 Reconstructing Phase Space
Two steps to reconstructing phase space from a signal:

• Choosing a time delay

• Choosing a embedding dimension

Based on work by Mañé and Takens in 1981 and Sauer et al.

in 1991.

Any smooth nonlinear change of variables will act as a

coordinate basis for the dynamics which will be
independent of the time lag T we choose.
The theory provides us with a sufficient embedding
dimension for our attractor.
 Reconstructing Phase Space
Time-Lagged Coordinates
Use time lagged coordinates in the signal to construct a
new set of coordinates
s(n + T ) = s(to + (n + T )τs )

We can then construct a vector in d dimensions that we

can examine the dynamics of the signal directly on

y(n) = [s(n), s(n + T ), s(n + 2T ), ..., s(n + (d − 1)T )]

New set of coordinates for phase space, different then

physical coordinates but just as good.
Reconstructing Phase Space
Time-Lagged Coordinates

• Embedding dimension independent of dynamics

• Two points close to each other should be property of

the set, not of geometry

• Dynamics in too small of a dimension leads to orbits

being close that should not

• When a proper dimension has been found,

provides a phase space for the analysis for the
dynamics
 Example: Henon Map
x(n + 1) = 1 + y(n) − 1.4x(n)2

y(n + 1) = 0.3x(n)

Signal we receive: s(n) = x(n) + y(n)

Time delayed coordinates

S(n) = [s(n), s(n + 1)]

 ')*+,-.-*/01
'"(

'

!"(

!!"(

!'

!'"(
! !"# !"$ !"% !"& ' '"# '"$ '"% '"& #
23-4
 %&'()*+*',-.
!"#

$"#
/0*231

!$"#

!!

!!"#
!!"# !! !$"# $ $"# ! !"#
/0*1
 -./01'2'/345

!"#

$"#
%&')+,*(

!$"#

!!

!!"#
!!"# !"#
!! !
!$"# $"#
$ $
$"# !$"#
! !!
!"# !!"#
%&')*(
%&'(
 Example: Goodwin Equations
dx y(t)2 − 1 3
= ρ sin ωt − c1 2
x(t) + c2 y(t) − c3 y(t)
dt y(t) + 1
dy
= x(t)
dt
ω=1 ρ = 14
c1 = 3/4 c2 = 1/2 c3 = 1/2
Signal we receive: s(n) = x(t0 + nτs )
 '()*++,-./)01234.+/5
%

!#

!$

!%
! !"# !"$ !"% !"& ' '"# '"$ '"% '"& #
56/7
 #%&'(()*+,&-./01+(,2
!

"

#
23,564

!#

!"

!!
!! !" !# $ # " !
23,4
 ,-./00123'.4567830'%

"

#
%&')#+*(

!#

!"

!!
!! !
!" "
!# #
$ $
# !#
" !"
! !!
%&')*(
%&'(
 Example: Lorenz Attractor
ẋ(t) = σ(y(t) − x(t))
σ = 16
ẏ(t) = −x(t)z(t) + rx(t) − y(t) b=4
ż(t) = x(t)y(t) − bz(t) r = 45.92

Signal we receive: s(n) = x(t0 + nτs )

 ')*+,-./0*123456,/7
$!

(!

#!

'!

!'!

!#!

!(!
! !"# !"$ !"% !"& ' '"# '"$ '"% '"& #
78/9
 #&'()*+,-'./0123),4
%"

!"

#"

$"
45,786

"

!$"

!#"

!!"
!!" !#" !$" " $" #" !" %"
45,6
 $-./012(3.4567890(&

!"

$"

#"

%"
&'(*#,+)

"

!%"

!#"

!$"
!!"

!#" !"
$"
" #"
%"
"
#" !%"
!#"
!" !$"
&'(*+)
&'()
 Reconstructing Phase Space
Choosing an Embedding Dimension
In theory, as long as we pick a d high enough, we are fine
but there are other considerations

• Computational costs increases quickly with increasing d

• If noise is present in our signal, the higher d will be

populated by this extra noise instead of the meaningful
dynamics of the system

We then want a systematic way to choose as small of a d

as possible but still have unfolded the dynamics of the
system
 Reconstructing Phase Space
Choosing an Embedding Dimension: Invariants of the System

Idea: Any property of the system that is dependent on the

distance between two points will stop changing when we
reach a sufficient d

• Many different ways to choose a function that is

invariant under increase of dimensions

• Indirect method using the dynamics of the system to

compute a geometrical property of the system
 Reconstructing Phase Space
Choosing an Embedding Dimension: False Nearest Neighbors
Idea: Measure the distances between a point and its
nearest neighbor, as this dimension increases, this distance
should not change if the points are really nearest neighbors

Define the distance between a point and its nearest

neighbor using a Euclidean distance

Rd (k)2 = [s(k)−sN N (k)]2 +[s(k+T )−sN N (k+T )]2 +. . . +[s(k+T (d−1))−sN N (k+T (d−1))]2

and the change in distance by adding one more dimension is

Rd+1 (k)2 = Rd (k)2 + [s(k + dT ) − sN N (k + dT )]2
we can now look at the relative change in the distance Rd+1
as a way to see if our points were not really close together
but a projection form a higher phase space
 Reconstructing Phase Space
Choosing an Embedding Dimension: False Nearest Neighbors

Using a threshold RT we can then write a criteria for false

neighbors
|s(k + T d) − sN N (k + T d)|
> RT
Rd (k)
Using this criterion we can then test our sequence of points
and, as d increases, find where the percentage of nearest
neighbors goes to 0
In practice values of RT in the range 10 ≤ RT ≤ 50 work
well for most situations.
 Reconstructing Phase Space
Choosing an Embedding Dimension: False Nearest Neighbors

Apply the criterion to a data from a random-number

generator. We find that embedding dimension is small.

Need another criterion taking into account the distances as

measured with respect to the size of the attractor RA
If Rd (k) ≈ RA then Rd+1 (k) ≈ 2Rd (k) =⇒
Rd+1 (k)
≥2
RA
as another test for false nearest neighbors, a common way
to estimate RA is by using the rms value of observations
1 !N
1 !N
2 2
RA = [s(k) − s̄] s̄ = s(k)
N N
k=1 k=1
 Henon Map False Nearest Neighbors
100

90

80

70
Percent False Neighbors

60

50

40

30

20

10

0
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
Dimension
 Lorenz Equations False Nearest Neighbors
100

90

80

70
Percent False Neighbors

60

50

40

30

20

10

0
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
Dimension
Reconstructing Phase Space

• One of our original problems was noise and how it is

often indistinguishable from our signal in terms of
traditional fourier techniques if our signal is chaotic

• Noise appears as high-dimensional chaos

• We can develop a requirement that if our required

dimension goes beyond 20, our signal is too noisy and
statistical techniques are the better choice for practical
analysis

• We can also use the false nearest neighbors test to

find the relative contamination of noise in the signal
 Noisy Lorenz Data False Nearest Neighbors
100
! = 1.0
! = 0.5
90 ! = 0.1
! = 0.05
80 ! = 0.01

70
Percent False Nearest Neighbors

60

50

40

30

20

10

0
1 2 3 4 5 6 7 8 9 10
Dimension
 Conclusions

• Finding an embedding dimension turns scalar time data

into a multivariable system

• False Nearest Neighbors provides a robust way to

determine necessary embedding dimensions

• Noise appears as high dimensional chaos and can be

examined using the false nearest neighbors technique
 References

• Abarbanel et al., 1993, “Analysis of Observed Chaotic

Data,” Rev. Mod. Phys.,Vol. 65, 4

• Sauer, T., A.Yorke, and M. Casdagli, 1991, Phys. Lett. A

160, 411

• Kennel, M. B., R. Brown, and H. D. I. Abarbanel, 1992,

Phys. Rev. A 45, 3403