Renewal Transitions
Duarte Antunes† João Hespanha‡ Carlos Silvestre†
July 1, 2010
Main results
Transitory analysis
Asymptotic analysis
Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis
Networked Control
Networked Control
Networked Control
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
Network abstraction
User (sensor, controller,...) waits independent and identically distributed
(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
• N pairs of nodes
implement above
protocol.
• Back-offs ∼
Uniform([0, hs ])
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)
SHS model
After waiting i.i.d. time to obtain access to the network:
Case I
x = (xP , u, v)
A models system and
controller transmits controller dynamics
data collected at time J1 models sampling
it initially tried to v(rk ) = KxP (rk− )
transmit data. J2 models control update
u(sk ) = v(s−k)
Case II
controller (re)samples x = (xP , u)
sensor, computes J models sample and control
control, and transmits update
most recent data. u(tk ) = KxP (t−
k)
Stochastic Hybrid Systems with Renewal Transitions 6/19
Stochastic Hybrid Systems (SHS) SHS definition
SHS definition
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq
SHS definition
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq
Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.
SHS definition
1 ≤ l ≤ nl , 1 ≤ q(t), i ≤ nq
Execution:
Transition times tk+1 determined by first hq(tk )j ∼ Fq(tk )j to trigger.
See paper for assumptions on atom points of the trans. distributions.
Stochastic Hybrid Systems with Renewal Transitions 7/19
Stochastic Hybrid Systems (SHS) Literature Review
Literature Review
Wiener Stochastic Stochastic
Dynamics
Process Intervals Transitions
SHS1 Non-linear Yes Guards No
SHS2 Non-linear Yes Yes-Exponential No
SHS3 Non-linear No/Yes* Yes-SD No/Yes*
MJLS Linear No/Yes* Yes-Exponential No/Yes*
PDPs Non-linear No/Yes* Yes-SD Yes-SD
Our work Linear No/Yes* Yes No/Yes*
SD-state dependent
*See [SHS3]
SHS1 -Hu et all, Towards a Theory of Stochastic Hybrid Systems, Workshop on Hybrid Systems: Computation and
Control,2000.
SHS2 -Ghosh et all, Ergodic Control Of Switching Diffusions, SIAM Journal of Control and Optimization,1997.
SHS3 -Hespanha, A Model for Stochastic Hybrid Systems with Application to Communication Networks,Special Issue on
Hybrid Systems, 2005.
MJLS -Mariton, Jump Linear Systems in Automatic Control, 1990, Marcel Dekker Inc.
PDPs -Davis, Markov Models and Optimization, Chapman & Hall, 1993.
Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1
where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1
where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
nq
X X
m1 m2 mr
E[xi1 (t) xi2 (t) . . . xir (t) ]= c| zim (t), for mj = m, mj > 0,
i=1
where Z t
zim (t) = Φi (ds)zim (t − s) + hm
i (t) (Volterra equation), (1)
0
Computing E[a(t)], a(t) = xi1 (t)m1 xi2 (t)m2 . . . xir (t)mr allows to
E[a(t)]
Prob[|b(t)| > ] ≤ , b(t)2 = a(t)
2
• Compute probability distribution of Markov process at time t
(moment problem).
Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if
1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis
Stability
Theorem
Suppose that transition distributions have finite support1 . Then, under easy to
test technical conditions, the SHS is MSS (E[x(t)| x(t)] → 0) if and only if
nl = 1, nq = 1, A, J
given in2
F ∼ Unif([0, τ ])
1
See the paper for the infinite support case
2
[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis
Li (P )−Pi < 0,
Li (P ) − Pi = −Qi , i ∈ Q
Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.
Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).
Performance
Theorem
If SHS is MSS and trans. distribution have finite support, then for α ∈ [0, αmax )
E[x(t)| x(t)] ≤ ce−αt x|0 x0 ,
(
∞, if det(I − K̂(a)) 6= 0, a ∈ (−∞, 0),
where αmax =
max{a < 0 : det(I − K̂(a)) = 0}, otherwise.
Recall that convergence rate of E[x(t)| x(t)] implies convergence rate for
probabilities bounds (performance).
Theorem
Under easy to test technical conditions, if SHS is not MSS then there exists one
unstable pole z1 of det(I − K̂(zi )) = 0, ∀i = 1, . . . , nz ;, which is real and
satisfies <[zi ] ≤ z1 , ∀i ;
Recall that E[x(t)| x(t)] = x|0 Z(t)x|0 and from results of Volterra equation
nz m
X Xi −1
Positive kernel
4
Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors’
webpages.
Stochastic Hybrid Systems with Renewal Transitions 15/19
Example and Conclusions
Outline
Stochastic Hybrid Systems (SHS)
Motivation
SHS definition
Literature Review
Main results
Transitory analysis
Asymptotic analysis
Example
Recall
• Plant connected to controller though a shared network.
• Only control law is sent through the network.
• i.i.d. random access time to the network.
• Consider two cases:
Case I: controller transmits data collected at time it initially tried to
transmit data.
Case II:controller (re)samples sensor, computes control, and transmits
most recent data.
Moreover
• Plant: Inverted pendulum. Network distribution- uniform with
support T .
• LQR control law minimizes
e(t) = xP (t)| xP (t) + û(t)2
and yields rate of convergence α = 2 ((x(t)| x(t) ≤ ce−2t )).
Stochastic Hybrid Systems with Renewal Transitions 17/19
Example and Conclusions Networked Control Example
Results
Transitory (support T = 0.1)
Results
Transitory (support T = 0.1)
Asymptotic
Table: Exponential decay rates E[e(t)] ≤ ce−αt
(a) Case I
T 0.1 0.2 0.3 0.4 0.5 > 0.521
α 2.000 2.000 2.000 0.849 0.118 NOT MSS
(b) Case II
T 0.4 0.6 0.8 1.0 1.2 > 1.211
α 2.000 2.000 1.969 0.477 7.63 × 10−5 NOT MSS
Thank you.
Stochastic Hybrid Systems with Renewal Transitions 19/19