Stochastic Hybrid Systems With Renewal Transitions: Duarte Antunes Jo Ao Hespanha Carlos Silvestre

Stochastic Hybrid Systems with
Renewal Transitions
Duarte Antunes† João Hespanha‡ Carlos Silvestre†
† Institute for Systems and Robotics, Instituto Superior Técnico

‡ Dep. of Electrical and Computer Eng., UC Santa Barbara
July 1, 2010
2010 American Control Conference

Baltimore, Maryland, USA
Stochastic Hybrid Systems with Renewal Transitions 1/19

Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis
Example and Conclusions

Networked Control Example
Conclusions and Future Work

Outline
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis


Stochastic Hybrid Systems (SHS) Motivation
Networked Control
• Control loops closed through shared network.

Networked Control

• For simplicity: Focus on single loop & Only the control law is sent.

Networked Control

• For simplicity: Focus on single loop & Only the control law is sent.
• Need network abstraction.
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.

Network abstraction
Example- CSMA-type protocol

Network abstraction

Network abstraction

Network abstraction
Key: model time intervals tk+1 − tk as i.i.d.

Network abstraction
Prob. Distribution of the access times
• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
Network abstraction
Prob. Distribution of the access times
• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)

SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)
Case II
controller (re)samples x = (xP , u)
sensor, computes J models sample and control
control, and transmits update
most recent data. u(tk ) = KxP (t−
k)
Stochastic Hybrid Systems (SHS) SHS definition
SHS definition
Differential Eqs. Reset maps Transition distributions

ẋ(t) = Aq(t) x(t) (q(tk ), x(tk )) = (ξl (q(t− −
k )), Jq(t− ),l x(tk )) Fi,l
k
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq

SHS definition

k )), Jq(t− ),l x(tk )) Fi,l
k
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq
Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.

SHS definition

k )), Jq(t− ),l x(tk )) Fi,l
k
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq
Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.
See paper for assumptions on atom points of the trans. distributions.
Stochastic Hybrid Systems (SHS) Literature Review
Literature Review
Wiener Stochastic Stochastic
Dynamics
Process Intervals Transitions
SHS1 Non-linear Yes Guards No
SHS2 Non-linear Yes Yes-Exponential No
SHS3 Non-linear No/Yes* Yes-SD No/Yes*
MJLS Linear No/Yes* Yes-Exponential No/Yes*
PDPs Non-linear No/Yes* Yes-SD Yes-SD
Our work Linear No/Yes* Yes No/Yes*
SD-state dependent
*See [SHS3]
SHS1 -Hu et all, Towards a Theory of Stochastic Hybrid Systems, Workshop on Hybrid Systems: Computation and
Control,2000.
SHS2 -Ghosh et all, Ergodic Control Of Switching Diffusions, SIAM Journal of Control and Optimization,1997.
SHS3 -Hespanha, A Model for Stochastic Hybrid Systems with Application to Communication Networks,Special Issue on
Hybrid Systems, 2005.
MJLS -Mariton, Jump Linear Systems in Automatic Control, 1990, Marcel Dekker Inc.
PDPs -Davis, Markov Models and Optimization, Chapman & Hall, 1993.

Main results
Outline
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis


Main results Transitory analysis
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1
where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0
Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Moment analysis
The following holds
nq
X X
m1 m2 mr
i=1
where Z t
0
Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−

k ), tk+1 − tk ∼ F

Moment analysis
The following holds
nq
X X
m1 m2 mr
i=1
where Z t
0
Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−

k ), tk+1 − tk ∼ F
E[x(t)| x(t)] = x|0 Z(t)x0 , where
Z t
|
Z(t) = (JeAs )| Z(t − s)(JeAs )F (ds) + eA t eAt (1 − F (t)).
0
Why moment analysis
Computing E[a(t)], a(t) = xi1 (t)m1 xi2 (t)m2 . . . xir (t)mr allows to
• Provide probability bounds at time t. Chebychev’s inequality:
E[a(t)]
Prob[|b(t)| > ] ≤ , b(t)2 = a(t)
2
• Compute probability distribution of Markov process at time t
(moment problem).

Main results Asymptotic analysis
Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if
det(I − K̂(z)) 6= 0, <[z] ≥ 0.
K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].
1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if
det(I − K̂(z)) 6= 0, <[z] ≥ 0.
K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].

Nyquist Criterion! Example taken from our previous work2
nl = 1, nq = 1, A, J
given in2
F ∼ Unif([0, τ ])
1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Equivalent stability conditions

Theorem
The following are equivalent
(A) det(I − K̂(z)) 6= 0, <[z] ≥ 0,
(B) σ(M ) < 1, M := K̂(0) and σ denotes the spectral radius.
(C) There exists a set of matrices P := {Pi > 0, i ∈ Q} such that for every i ∈ Q
Li (P )−Pi < 0,
where Li (P ) is a finite-dimension operator.3

(D) For every set of matrices {Qi ≥ 0,i ∈ Q} the solution to
Li (P ) − Pi = −Qi , i ∈ Q
is unique and satisfies {Pi ≥ 0,i ∈ Q}

3
See the paper for the expression.
Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.
Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).

Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.
Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).
Theorem
Under easy to test technical conditions, if SHS is not MSS then there exists one
unstable pole z1 of det(I − K̂(zi )) = 0, ∀i = 1, . . . , nz ;, which is real and
satisfies <[zi ] ≤ z1 , ∀i ;
Recall that E[x(t)| x(t)] = x|0 Z(t)x|0 and from results of Volterra equation
nz m
X Xi −1
Z(t) = Ri,j tj ezi t + ζ̃(t), ζ̃(t) → 0.

i=1 j=0
Positive kernel
Special property of Volterra Eq. lies at the heart of the results.

Example nq = 1, nl = 1
Z t
|
Z(t) = (JeAs )| Z(t − s)(JeAs )F (ds) + eA t eAt (1 − F (t))
0
The kernel is positive (leaves a cone invariant) since
∀s , (JeAs )| X(JeAs ) ≥ 0, if X ≥ 0
The results can be generalized to Volterra Eqs. with positive operators:4
4
Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages.
Outline
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis


Example and Conclusions Networked Control Example
Example
Recall
• Plant connected to controller though a shared network.
• Only control law is sent through the network.
• i.i.d. random access time to the network.
• Consider two cases:
Case I: controller transmits data collected at time it initially tried to
transmit data.
Case II:controller (re)samples sensor, computes control, and transmits
most recent data.
Moreover
• Plant: Inverted pendulum. Network distribution- uniform with
support T .
• LQR control law minimizes
e(t) = xP (t)| xP (t) + û(t)2
and yields rate of convergence α = 2 ((x(t)| x(t) ≤ ce−2t )).
Results
Transitory (support T = 0.1)

Results
Transitory (support T = 0.1)
Asymptotic
Table: Exponential decay rates E[e(t)] ≤ ce−αt
(a) Case I
T 0.1 0.2 0.3 0.4 0.5 > 0.521
α 2.000 2.000 2.000 0.849 0.118 NOT MSS
(b) Case II
T 0.4 0.6 0.8 1.0 1.2 > 1.211
α 2.000 2.000 1.969 0.477 7.63 × 10−5 NOT MSS
Conclusion: Better to transmit most recent data.

Example and Conclusions Conclusions and Future Work

Key ideas:
• Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
• Volterra analysis to specific class of SHS with renewal transition provides
different results.
• Volterra equation with positive kernel structure.
• Networked control modeled by SHSs.


Key ideas:
different results.
Future work:
• Compare method of obtaining the probability density function thought a
large number of momemnts with solution methods to Fokker-Plank equation.
• Asymptotic for other higher order functions (other than E[x| (t)x(t)])-
Volterra equations has positive kernel in the sense that leaves the cone of
positive definite tensors (instead of positive semi-definite matrices) invariant.


Key ideas:
different results.
Future work:
• Compare method of obtaining the probability density function thought a
large number of momemnts with solution methods to Fokker-Plank equation.
• Asymptotic for other higher order functions (other than E[x| (t)x(t)])-
Volterra equations has positive kernel in the sense that leaves the cone of
positive definite tensors (instead of positive semi-definite matrices) invariant.
Thank you.

Stochastic Hybrid Systems With Renewal Transitions: Duarte Antunes Jo Ao Hespanha Carlos Silvestre

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stochastic Hybrid Systems With Renewal Transitions: Duarte Antunes Jo Ao Hespanha Carlos Silvestre

Uploaded by

Copyright:

Available Formats

Stochastic Hybrid Systems with

† Institute for Systems and Robotics, Instituto Superior Técnico

2010 American Control Conference

Stochastic Hybrid Systems with Renewal Transitions 1/19

Example and Conclusions

Stochastic Hybrid Systems with Renewal Transitions 2/19

Example and Conclusions

Stochastic Hybrid Systems with Renewal Transitions 3/19

• Control loops closed through shared network.

Stochastic Hybrid Systems with Renewal Transitions 4/19

• Control loops closed through shared network.

Stochastic Hybrid Systems with Renewal Transitions 4/19

• Control loops closed through shared network.

Stochastic Hybrid Systems with Renewal Transitions 5/19

Stochastic Hybrid Systems with Renewal Transitions 5/19

Stochastic Hybrid Systems with Renewal Transitions 5/19

Stochastic Hybrid Systems with Renewal Transitions 5/19

Key: model time intervals tk+1 − tk as i.i.d.

Stochastic Hybrid Systems with Renewal Transitions 5/19

Key: model time intervals tk+1 − tk as i.i.d.

Prob. Distribution of the access times

Key: model time intervals tk+1 − tk as i.i.d.

Prob. Distribution of the access times

Stochastic Hybrid Systems with Renewal Transitions 6/19

Differential Eqs. Reset maps Transition distributions

Stochastic Hybrid Systems with Renewal Transitions 7/19

Differential Eqs. Reset maps Transition distributions

Stochastic Hybrid Systems with Renewal Transitions 7/19

Differential Eqs. Reset maps Transition distributions

Stochastic Hybrid Systems with Renewal Transitions 8/19

Example and Conclusions

Stochastic Hybrid Systems with Renewal Transitions 9/19

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Stochastic Hybrid Systems with Renewal Transitions 10/19

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−

Stochastic Hybrid Systems with Renewal Transitions 10/19

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−

Why moment analysis

• Provide probability bounds at time t. Chebychev’s inequality:

Stochastic Hybrid Systems with Renewal Transitions 11/19

det(I − K̂(z)) 6= 0, <[z] ≥ 0.

K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].

det(I − K̂(z)) 6= 0, <[z] ≥ 0.

K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].

Equivalent stability conditions

where Li (P ) is a finite-dimension operator.3

is unique and satisfies {Pi ≥ 0,i ∈ Q}

Stochastic Hybrid Systems with Renewal Transitions 14/19

Z(t) = Ri,j tj ezi t + ζ̃(t), ζ̃(t) → 0.

Special property of Volterra Eq. lies at the heart of the results.

The results can be generalized to Volterra Eqs. with positive operators:4

Example and Conclusions

Stochastic Hybrid Systems with Renewal Transitions 16/19

Stochastic Hybrid Systems with Renewal Transitions 18/19

Conclusion: Better to transmit most recent data.

Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 19/19

Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 19/19

Conclusions and Future Work