You are on page 1of 37

Stochastic Hybrid Systems with

Renewal Transitions
Duarte Antunes† João Hespanha‡ Carlos Silvestre†

† Institute for Systems and Robotics, Instituto Superior Técnico


‡ Dep. of Electrical and Computer Eng., UC Santa Barbara

July 1, 2010

2010 American Control Conference


Baltimore, Maryland, USA

Stochastic Hybrid Systems with Renewal Transitions 1/19


Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review

Main results
Transitory analysis
Asymptotic analysis

Example and Conclusions


Networked Control Example
Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 2/19


Stochastic Hybrid Systems (SHS)

Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review

Main results
Transitory analysis
Asymptotic analysis

Example and Conclusions


Networked Control Example
Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 3/19


Stochastic Hybrid Systems (SHS) Motivation

Networked Control

• Control loops closed through shared network.

Stochastic Hybrid Systems with Renewal Transitions 4/19


Stochastic Hybrid Systems (SHS) Motivation

Networked Control

• Control loops closed through shared network.


• For simplicity: Focus on single loop & Only the control law is sent.

Stochastic Hybrid Systems with Renewal Transitions 4/19


Stochastic Hybrid Systems (SHS) Motivation

Networked Control

• Control loops closed through shared network.


• For simplicity: Focus on single loop & Only the control law is sent.
• Need network abstraction.
Stochastic Hybrid Systems with Renewal Transitions 4/19
Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.

Stochastic Hybrid Systems with Renewal Transitions 5/19


Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Stochastic Hybrid Systems with Renewal Transitions 5/19


Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Stochastic Hybrid Systems with Renewal Transitions 5/19


Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Stochastic Hybrid Systems with Renewal Transitions 5/19


Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Key: model time intervals tk+1 − tk as i.i.d.

Stochastic Hybrid Systems with Renewal Transitions 5/19


Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Key: model time intervals tk+1 − tk as i.i.d.

Prob. Distribution of the access times

• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation

Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol

Key: model time intervals tk+1 − tk as i.i.d.

Prob. Distribution of the access times

• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation

SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)

Stochastic Hybrid Systems with Renewal Transitions 6/19


Stochastic Hybrid Systems (SHS) Motivation

SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)

Case II
controller (re)samples x = (xP , u)
sensor, computes J models sample and control
control, and transmits update
most recent data. u(tk ) = KxP (t−
k)
Stochastic Hybrid Systems with Renewal Transitions 6/19
Stochastic Hybrid Systems (SHS) SHS definition

SHS definition

Differential Eqs. Reset maps Transition distributions


ẋ(t) = Aq(t) x(t) (q(tk ), x(tk )) = (ξl (q(t− −
k )), Jq(t− ),l x(tk )) Fi,l
k

1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq

Stochastic Hybrid Systems with Renewal Transitions 7/19


Stochastic Hybrid Systems (SHS) SHS definition

SHS definition

Differential Eqs. Reset maps Transition distributions


ẋ(t) = Aq(t) x(t) (q(tk ), x(tk )) = (ξl (q(t− −
k )), Jq(t− ),l x(tk )) Fi,l
k

1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq

Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.

Stochastic Hybrid Systems with Renewal Transitions 7/19


Stochastic Hybrid Systems (SHS) SHS definition

SHS definition

Differential Eqs. Reset maps Transition distributions


ẋ(t) = Aq(t) x(t) (q(tk ), x(tk )) = (ξl (q(t− −
k )), Jq(t− ),l x(tk )) Fi,l
k

1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq

Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.
See paper for assumptions on atom points of the trans. distributions.
Stochastic Hybrid Systems with Renewal Transitions 7/19
Stochastic Hybrid Systems (SHS) Literature Review

Literature Review
Wiener Stochastic Stochastic
Dynamics
Process Intervals Transitions
SHS1 Non-linear Yes Guards No
SHS2 Non-linear Yes Yes-Exponential No
SHS3 Non-linear No/Yes* Yes-SD No/Yes*
MJLS Linear No/Yes* Yes-Exponential No/Yes*
PDPs Non-linear No/Yes* Yes-SD Yes-SD
Our work Linear No/Yes* Yes No/Yes*
SD-state dependent
*See [SHS3]
SHS1 -Hu et all, Towards a Theory of Stochastic Hybrid Systems, Workshop on Hybrid Systems: Computation and
Control,2000.
SHS2 -Ghosh et all, Ergodic Control Of Switching Diffusions, SIAM Journal of Control and Optimization,1997.
SHS3 -Hespanha, A Model for Stochastic Hybrid Systems with Application to Communication Networks,Special Issue on
Hybrid Systems, 2005.
MJLS -Mariton, Jump Linear Systems in Automatic Control, 1990, Marcel Dekker Inc.
PDPs -Davis, Markov Models and Optimization, Chapman & Hall, 1993.

Stochastic Hybrid Systems with Renewal Transitions 8/19


Main results

Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review

Main results
Transitory analysis
Asymptotic analysis

Example and Conclusions


Networked Control Example
Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 9/19


Main results Transitory analysis

Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1

where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Stochastic Hybrid Systems with Renewal Transitions 10/19


Main results Transitory analysis

Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1

where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−


k ), tk+1 − tk ∼ F

Stochastic Hybrid Systems with Renewal Transitions 10/19


Main results Transitory analysis

Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1

where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0

Φi (ds) is a matrix-measure. For AC distributions Φi (ds) = Ki (s)ds.

Example: ẋ(t) = Ax(t), x(tk ) = Jx(t−


k ), tk+1 − tk ∼ F
E[x(t)| x(t)] = x|0 Z(t)x0 , where
Z t
|
Z(t) = (JeAs )| Z(t − s)(JeAs )F (ds) + eA t eAt (1 − F (t)).
0
Stochastic Hybrid Systems with Renewal Transitions 10/19
Main results Transitory analysis

Why moment analysis

Computing E[a(t)], a(t) = xi1 (t)m1 xi2 (t)m2 . . . xir (t)mr allows to

• Provide probability bounds at time t. Chebychev’s inequality:

E[a(t)]
Prob[|b(t)| > ] ≤ , b(t)2 = a(t)
2
• Compute probability distribution of Markov process at time t
(moment problem).

Stochastic Hybrid Systems with Renewal Transitions 11/19


Main results Asymptotic analysis

Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if

det(I − K̂(z)) 6= 0, <[z] ≥ 0.

K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].

1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis

Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if

det(I − K̂(z)) 6= 0, <[z] ≥ 0.

K̂(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)|x(t)].


Nyquist Criterion! Example taken from our previous work2

nl = 1, nq = 1, A, J
given in2
F ∼ Unif([0, τ ])

1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis

Equivalent stability conditions


Theorem
The following are equivalent
(A) det(I − K̂(z)) 6= 0, <[z] ≥ 0,
(B) σ(M ) < 1, M := K̂(0) and σ denotes the spectral radius.
(C) There exists a set of matrices P := {Pi > 0, i ∈ Q} such that for every i ∈ Q

Li (P )−Pi < 0,

where Li (P ) is a finite-dimension operator.3


(D) For every set of matrices {Qi ≥ 0,i ∈ Q} the solution to

Li (P ) − Pi = −Qi , i ∈ Q

is unique and satisfies {Pi ≥ 0,i ∈ Q}


3
See the paper for the expression.
Stochastic Hybrid Systems with Renewal Transitions 13/19
Main results Asymptotic analysis

Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.

Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).

Stochastic Hybrid Systems with Renewal Transitions 14/19


Main results Asymptotic analysis

Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.

Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).
Theorem
Under easy to test technical conditions, if SHS is not MSS then there exists one
unstable pole z1 of det(I − K̂(zi )) = 0, ∀i = 1, . . . , nz ;, which is real and
satisfies <[zi ] ≤ z1 , ∀i ;
Recall that E[x(t)| x(t)] = x|0 Z(t)x|0 and from results of Volterra equation
nz m
X Xi −1

Z(t) = Ri,j tj ezi t + ζ̃(t), ζ̃(t) → 0.


i=1 j=0
Stochastic Hybrid Systems with Renewal Transitions 14/19
Main results Asymptotic analysis

Positive kernel

Special property of Volterra Eq. lies at the heart of the results.


Example nq = 1, nl = 1
Z t
|
Z(t) = (JeAs )| Z(t − s)(JeAs )F (ds) + eA t eAt (1 − F (t))
0
The kernel is positive (leaves a cone invariant) since
∀s , (JeAs )| X(JeAs ) ≥ 0, if X ≥ 0

The results can be generalized to Volterra Eqs. with positive operators:4

4
Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages.
Stochastic Hybrid Systems with Renewal Transitions 15/19
Example and Conclusions

Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review

Main results
Transitory analysis
Asymptotic analysis

Example and Conclusions


Networked Control Example
Conclusions and Future Work

Stochastic Hybrid Systems with Renewal Transitions 16/19


Example and Conclusions Networked Control Example

Example
Recall
• Plant connected to controller though a shared network.
• Only control law is sent through the network.
• i.i.d. random access time to the network.
• Consider two cases:
Case I: controller transmits data collected at time it initially tried to
transmit data.
Case II:controller (re)samples sensor, computes control, and transmits
most recent data.
Moreover
• Plant: Inverted pendulum. Network distribution- uniform with
support T .
• LQR control law minimizes
e(t) = xP (t)| xP (t) + û(t)2
and yields rate of convergence α = 2 ((x(t)| x(t) ≤ ce−2t )).
Stochastic Hybrid Systems with Renewal Transitions 17/19
Example and Conclusions Networked Control Example

Results
Transitory (support T = 0.1)

Stochastic Hybrid Systems with Renewal Transitions 18/19


Example and Conclusions Networked Control Example

Results
Transitory (support T = 0.1)

Asymptotic
Table: Exponential decay rates E[e(t)] ≤ ce−αt
(a) Case I
T 0.1 0.2 0.3 0.4 0.5 > 0.521
α 2.000 2.000 2.000 0.849 0.118 NOT MSS
(b) Case II
T 0.4 0.6 0.8 1.0 1.2 > 1.211
α 2.000 2.000 1.969 0.477 7.63 × 10−5 NOT MSS

Conclusion: Better to transmit most recent data.


Stochastic Hybrid Systems with Renewal Transitions 18/19
Example and Conclusions Conclusions and Future Work

Conclusions and Future Work


Key ideas:
• Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
• Volterra analysis to specific class of SHS with renewal transition provides
different results.
• Volterra equation with positive kernel structure.
• Networked control modeled by SHSs.

Stochastic Hybrid Systems with Renewal Transitions 19/19


Example and Conclusions Conclusions and Future Work

Conclusions and Future Work


Key ideas:
• Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
• Volterra analysis to specific class of SHS with renewal transition provides
different results.
• Volterra equation with positive kernel structure.
• Networked control modeled by SHSs.
Future work:
• Compare method of obtaining the probability density function thought a
large number of momemnts with solution methods to Fokker-Plank equation.
• Asymptotic for other higher order functions (other than E[x| (t)x(t)])-
Volterra equations has positive kernel in the sense that leaves the cone of
positive definite tensors (instead of positive semi-definite matrices) invariant.

Stochastic Hybrid Systems with Renewal Transitions 19/19


Example and Conclusions Conclusions and Future Work

Conclusions and Future Work


Key ideas:
• Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
• Volterra analysis to specific class of SHS with renewal transition provides
different results.
• Volterra equation with positive kernel structure.
• Networked control modeled by SHSs.
Future work:
• Compare method of obtaining the probability density function thought a
large number of momemnts with solution methods to Fokker-Plank equation.
• Asymptotic for other higher order functions (other than E[x| (t)x(t)])-
Volterra equations has positive kernel in the sense that leaves the cone of
positive definite tensors (instead of positive semi-definite matrices) invariant.

Thank you.
Stochastic Hybrid Systems with Renewal Transitions 19/19