You are on page 1of 49

NORTH-HOLLAND SERIES IN

SYSTEM SCIENCE
AND ENGINEERING
Andrew P. Sage, Editor
WiSIIIl'J' alld Chattergy Introduction to Nonlinear Optimization:
A Prohlem Solving Approach
2 Suthl'J'land Societal Systems:
3 Siljak
4 Porter et a1.
5 Boroush et a1.
6 Rouse
7 Scngupta
Methodology, Modeling, and Management
Large-Scale Dynamic Systems
A Guidebook ror Technology Assessment
and Impact Analysis
Tech nology Assessmen t :
Creative Futures
Systems Engineering Models
or Human- Machine Interaction
Decision Models in Stochastic
Programming
8 Chankong and Haimcs Mulliobjective Decision Making:
Theory and Methodology
9 Jamshidi Large-Scale Sys tems:
Modeling and Control
Large-Scale Systems
Modeling and Control
Series Volu111e 9
Mohanuuad J atushidi
Department or Electrical and Computer Eng,incc'ring,
The University or New Mexico, Albuquerque
i i ") (. I'. j '
\ " " \ '. '
Z-" I ~ - ; ' " ""',
() < " ~ J (--;
NORTH-HOLLAND,
New York' Amsterdam' Oxrord
/
I
Elsevier Science Publishing Co .. Inc.
52 Vanderbilt Avenue, New York, New York 10017
Sole distributors outside the United States and Canada:
Elsevier Science Publishers B.V.
P.O. Box 211. 1000 AE Amsterdam. The Netherlands
1983 by Elsevier Science Publishing Co., Inc.
Library of Congress Cataloging in Publication Data
Jamshidi, Mohammad.
Large-scale systems.
(North-Holland series in system science and engineering: 9)
Includes bibliographies and indexes.
I. Large scale systems. 1. Title. II. Series.
QA402.J34 1983 003 82-815 II
ISBN 0-444-00706-7 AACR2
Manufactured in the United States of America
'i! . :.'
' ..
1
I
1 ~
l'
~
"
.
. ,
~
'j
~ j
1
j
"
;!
I
t
j'
To three
who meant the most in my life
My Mother
My Wife lila
My Brother Ahmad
! i
I' ;
;, '
~ ) .
1
,j,
~ I
! :
. \.
6
Problem 1.2. (continued)
d. A home heating system.
e. A radar control system.
Introduction to Large-Scale Systems
1.3. The allocation of water resources in any state is commonly the responsibility
of the state engineers' office which checks for overall system feasibility by
overseeing municipalities and conservancy districts, which work independently
and report their programs to the state engineers' office. Consider a two-muni-
cipality and three-district state and draw a block diagram representing the
water resources system.
References
Cruz. J. B., Jf. 1978. Leader-follower strategies for multilevel systems. IEEE TrailS .
Aut. Cont . AC-23:244-255.
Dantzig, G., and Wolfe, P. 1960. Decomposition principle for linear programs.
Oper. Res. 8:101-111.
Haimes, Y. Y. 1977. Hierarchical Analysis of Water Resources Systems. McGraw-Hili,
New York.
Ho. Y. c., and Mitter, S. K., eds. 1976. Directions in Large-Scale Systems, pp. v-x.
Plenum, New York.
Mahmoud, M. S. 1977. Multilevel systems control and applications: A survey. IEEE
Trans . Sys. Man. Cyb. SMC-7:125- 143.
Mayne, D. Q. 1976. Decentralized control of large-scale systems, in Y. C. Ho and
S. K. Mitter, eds., Directions in Large Scale Systems , pp. 17-23. Plenum, New
York.
Mesarovic, M. D.; Macko, D.; and Takahara, Y. 1970. Theory of Hierarchical
Multilevel Systems. Academic Press, New York.
Saeks, R. , and DeCarlo, R. A. 1981. Interconnected Dynamical Systems. Marcel
Dekker, New York.
Sandell, N. R., Jf.; Varaiya, P. ; Athans, M.; and Safonov, M. G. 1978. Survey of
decentralized control methods for large-scale systems, IEEE Trans . Aut. Cont .
AC-23 : 108- 128 (special issue on large-scale systems).
Singh, M. G., 1980. Dynamical Hierarchical Control, rev. ed. North Holland,
Amsterdam, The Netherlands.
Varaiya, P. 1972. Book Review of Mesarovie et aI. , Theory of hierarchical multi-level
systems. IEEE Trans . Aut. Cont. AC-17:280-28I.
'.
Chapter 2
Large-Scale Systems Modeling:
Tilne Domain
2.1 Introduction
Scientists and engineers are often confronted with the analysis, design, and
synthesis of real-life problems. The first step in such studies is the develop-
ment of a "mathematical model" which can be a substitute for the real
problem.
In any modeling task, two often conflicting factors prevail - " simplicity"
and "accuracy." On one hand, if a system model is oversimplified, presum-
ably for computational effectiveness, incorrect conclusions may be drawn
from it in representing an actual system. On the other hand, a highly
detailed model would lead to a great deal of unnecessary complications and
should a feasible solution be attainable, the extent of resulting details may
become so vast that further investigations on the system behavior would
become impossible with questionable practical values (Sage, 1977; Siljak,
1978). Clearly a mechanism by which a compromise can be made between a
complex, more accurate model and a simple, less accurate model is needed.
Such a mechanism is not a simple undertaking. The key to a valid modeling
philosophy is to set forth the following outline (Brogan, 1974):
1. The purpose of the model must be clearly defined ; no single model can
be appropriate for all purposes.
2. The system's boundary separating the system and the outside world
must be defined.
3. A structural relationship among different system components which
would best represent desired or observed effects must be defined.
4. Based on the physical structure of the model, a set of system variables
of interest must be defined. If a quantity of important significance
cannot be labeled, step (3) must be modified accordingly.
lntroclucuon to Large-Scale Systems
DYNAMIC SYSTEM
1/1 YI liZ yz
CONTROLLER 1 CONTROLLER 2
1 VI f "z
Figure 1.2 A two-controller decentralized system
controllers (stations), which observe only local system outputs. In other
words, this approach, called decentralized control, attempts to avoid difficul-
ties in data gathering, storage requirements, computer program debuggings,
and geographical separation of system components. Figure 1.2 shows a
two-controller decentralized system. The basic characteristic of any de-
centralized system is that the transfer of information from one group of
sensors or actuators to others is quite restricted. For example, in the system
of Figure 1.2, only the output YI and external input VI are used to find the
control U I' and likewise the control U
2
is obtained through only the output Y
and external input V
2
. 2
The determination of control signals U
I
and U
2
based on the output
signals YI and Yl, respectively, is nothing but two independent output
feedback problems which can be used for stabilization or pole placement
purposes. It is therefore clear that the decentralized control scheme is of
feedback form, indicating that this method is very useful for large-scale
linear systems. This is a clear distinction from the hierarchical control
scheme, which was mainly intended to be an open-loop structure. Further
discussion of decentralized control and its applications for stabilization,
robust controllers, etc., will be considered in Chapters 5 and 6.
In this and the previous two sections the concept of a large-scale system
and two basic hierarchical and decentralized control structures were briefly
introduced. Although there is no universal definition of a large-scale system,
it is commonly accepted that such systems possess the following characteris-
tics (Ho and Mitter, 1976):
I. Large-scale systems are often controlled by more than one controller
or decision maker involving "decentralized" computations.
2. The controllers have different but correlated "information" available
to them, possible at different times.
3. Large-scale systems can also be controlled by local controllers at one
level whose control actions are being coordinated at another level in a
"hierarchical" (multilevel) structure.
"
: ~ I
',r"
':J
j.
"
J :'
f
r
Problems 5
4.
5.
6.
Large-scale systems are usually represented by imprecise" aggregate"
models.
Controllers may operate in a group as a "team" or in a "conflicting"
manner with single- or multiple-objective or even conflicting-objective
functions.
Large-scale systems may be satisfactorily optimized by means of sub-
optimal or near-optimum controls, sometimes termed a "satisfying"
strategy.
1.4 Scope
Since the subject of large-scale systems is rather new, and since the literature
is still growing, it is difficult and pointless to attempt to cite every reference.
On the other hand, if we were to confine the discussion to one or two
subtopics and use only immediate references, it would hardly reflect the
importance of the subject. In this text an attempt is made to consider
primarily modeling and control of iarge-scale systems. Other important
topics, such as stability, controllability, and observability, are discussed
briefly. Most of our discussions are focused on large-scale linear, continu-
ous-time, stationary, and deterministic systems. However, other classes of
systems, such as discrete-time, time-delay, nonlinear, and stochastic large-
scale systems, are also considered. Among control strategies, the main focus
has been on hierarchical (multilevel) and decentralized controls. On the
modeling side, aggregation and perturbation (regular and singular) are
among the primary topics discussed. Other topics such as identification and
estimation as well as large-scale systems control and modeling schemes,
such as the Stackelberg approach (Cruz, 1978), component connection
model (Saeks and DeCarlo, 1981), multilayer and multiechelon structures,
and Nash games, are either considered very briefly or have not been
discussed. The emphasis has been on the use of the subject matter in the
classroom for students of large-scale systems in a simple and understand-
able language. Most important theorems are proved, and many easily
implement able algorithms support the theory and ample numerical exam-
ples demonstrate their use.
Problems
1.1. Develop a multilevel (hierarchical) structure for a business organization with a
board of directors, a chairman of the board, a president, three vice presidents
(marketing-sales, research, technology), etc.
1.2. Explain whether the concept of "centrality" holds for each of the following
systems. State your reasons in a sentence.
a. An autopilot aircraft control system.
b. A three-synchronous machine power system.
c. A computer system involving a host computer and five terminals.
2
Introduction to Large-Scale Systems
The underlying assumption for all such control and system procedures
has been "centrality" (Sandell et aI., 1978), i.e., all the calculations based
information (be it a priori or sensor information) and the
Itself are localized at a given center, very often a geographical
pOSItlOn. .
A notable of most large-scale systems is that centrality fails
to hoi? to the lack of centralized computing capability or
centrahzed InformatlOn. Needless to say, many real problems are considered
large-scale by nature and not by choice. The important points regarding
large-scale systems are that they often model real-life systems and that their
(mul.tilevel) and decentralized structures depict systems dealing
wlth soclety, bUSIness, management, the economy, the environment, energy,
data networks, power networks, space structures, transportation, aerospace,
water resources, ecology, and flexible manufacturing networks, to name a
few. These systems are often separated geographically, and their treatment
requires consideration of not only economic costs, as is common in central-
ized systems, but also such important issues as reliability of communication
links, value of information, etc. It is for the decentralized and hierarchical
control properties and potential applications that many researchers
the world have devoted a great deal of effort to large-scale
systems III recent years.
1.2 Hierarchical Structures
One of the earlier attempts in dealing with large-scale systems was to
. a given system into a number of subsystems for computa-
efflclency and design simplification. The idea of "decomposition"
was first treated theoretically in mathematical programming by Dantzig and
(1960) by treating large linear programming problems possessing
speCIal structure.s. The coefficient matrices of such large linear programs
often have relatlvely few nonzero elements, i.e., they are sparse matrices.
There are two basic approaches for dealing with such problems: "coupled"
"decoupled." The coupled approach keeps the problem's structure
and takes advantage of the structure to perform efficient computa-
tions. The "compact basis triangularization" and "generalized upper bound-
ing" are methods (Ho and Mitter, 1976). The "decoupled"
dIVIdes the ongInal system into a number of subsystems involving
c.ertam values of parameters. Each subsystem is solved independently for a
fIxed . value of the so-called decoupling parameter, whose value is subse-
quently adjusted by a coordinator in an appropriate fashion so that the
subsystems resolve their problems and the solution to the original system is
obtained.
Perhaps the most active group in axiomatizing the decoupled approach
has been Mesarovic, Lefkowitz, and their colleagues at the Case Western
I
f ..:
I '
i
I
i
Decentralized Control
SECOND LEVEL
FIRST LEVEL SUJ3SYS1' EM N
Figure 1.1 Schematic of a two-level hierarchical system
Reserve University, who have termed it the "multilevel" or "hierarchical"
approach (Mesarovic et aI., 1970). Consider a two-level system shown in
Figure 1.1. At the first level, N subsystems of the original large-scale system
are shown. At the second level a coordinator receives the local solutions of
the N subsystems, s;, i = 1,2, ... ,N, and then provides a new set of "interac-
tion" parameters, a;, i = 1,2, ... ,N. The goal of the coordinator is to arrange
the activities of the subsystems to provide a feasible solution to the overall
system.
The success of the hierarchical multilevel approach has been primarily in
social systems (Mayne, 1976) and water resources systems (Haimes, 1977).
The multilevel structure, according to Mesarovic et a1. (1970), has five
advantages: (i) the decomposition of systems with fixed designs at one level
and coordination at another is often the only alternative available; (ii)
systems are commonly described only on a stratified basis; (iii) available
decision units have limited capabilities, hence the problem is formulated in
a multilayer hierarchy of subproblems; (iv) the overall system resources are
better utilized through this structure; and (v) there will be an increase in
system reliability and flexibility. There has been some disagreement among
system-and control specialists regarding these points. For example, Varaiya
(1972) has mentioned that the first three advantages are a matter of opinion,
and there is no evidence in justifying the vther two. One shortcoming of
most multilevel structures is that they are inherently open-loop structures,
although closed-loop structures have been proposed (Singh 1980). Detailed
discussion on the hierarchical (multilevel) method will be given in
Chapter 4.
1.3 Decentralized Control
Most large-scale systems are characterized by a great multiplicity of mea-
sured outputs and inputs. For example, an electric power system has several
control substations, each being responsible for the operation of a portion of
the overall system. This situation arising in a control system design is often
referred to as decentralization. The designer for such systems determines a
structure for control which assigns system inputs to a given set of local
Chapter 1
Introduction to Large-Scale SystenlS
1.1 Historical Background
A great number of today's problems are brought about by present-day
technology and societal and environmental processes which are highl y
complex, "large" in dimension, and stochast;c by nature. The notion of
"large-scale" is a very subjective one in that one may ask: How large is
large? There has been no accepted definition [or what constitutes a "large-
scale system." Many viewpoints have been presented on thi s issue. One
viewpoint has been that a system is considered large-scale if it can be
decoupled or partitioned into a number of interconnected subsystems or
"small-scale" systems for either computational or practical reasons (J-Io and
Mitter, 1976). Another viewpoint is that a system is large-scale when its
dimensions are so large that conventional techniques of modeling, analysis,
control, design, and computation fail to give reasonable solutions with
reasonable computational efforts. In other words, a system is large when it
requires more than one controller (Mahmoud, 1977).
Since the early 1950s, when classical control theory was being established.
engineers have devised several procedures, both within the classical and
modern control contexts, which analyze or design a given sys tem. These
procedures can be summarized as follows:
1. Modeling procedures which consist of differential equations,
input- output transfer functions, and state-space formulations.
2. Behavioral procedures of systems such as controllability, observability,
and stability tests, and application of such criteria as Routh-Hurwitz,
Nyquist, Lyapunov's second method, etc.
3. Control procedures such as series compensation, pole placement, opti-
mal control, etc.
Marshall, S. A. 1074. Continlll:d-fracti on model-reduction technique for l11ultivari-
ahle sys tems. Proc. lEE 12 I : 1032.
Nagarajan, R. 197 \. Optimum reduction of large dynamic sys tem. 11l1. 1. COlltr.
14:1164- 11 74.
PaciC, l-I. 1 xn. Sur la represenl ation approachec d'une function par dcs fractions
rationelks. Annalt's S'ci('nlijiques de l" Ecole Normale S lIpiL'lIre, Ser. :1 (SuPP!. )
0: 1-'n
Parthasarathy, R., and Singh, H. 1975. Comments on linear system reduction by
continued fraction expansior about a general point. Elect Letters. II: 102.
Paynter, H. M. 1956. On an between stochastic process and monotone
dynamic sys tems, in G. Muller, ed., Regelullgstechllik iv[oc/ern Theoriell wul illre
Venvendbarkeit. R. Oldenbourg-Verlag, Munich.
Rao, S. V., and Lunba, S. S. 1974. A new frequency domai n technique for the
simplification of linear dynamic systems. lilt . 1. COlltr. 20:7 1-79.
Rao, A. S., S. S., and Rao, S. V. 1978. On simplifi cation of unstable sys tems
using Routh approximation technique. I EEE TrailS. Aut. COllt. AC-23:943 - 944.
Reddy, A. S. S. R. 1976. A method for frequency domain simplification of transfer
functions. lilt. 1. COlltr. 23:403-408.
Shamash, Y. 1974. Stable reduced-order models using Pade-type approximations.
TrailS. Aut. COlli . AC- 19:615-617.
Shamash, Y. 1975a. Model reduction using the Routh stahili ty criterion and the
Pade approximation technique. lilt . 1. COlltr. 21 :475-4l\4.
Sham ash, Y. 1975b. Linear system reduction using Pade approximation to allow
retention of domi nant modes Jilt. 1. COlltr. 2 1 :257- 272.
Shamash, Y. 1975e. Multivari:lble system reduction via modal methods and Pade
approximation. TrailS. Alit. COlil . AC-20:XI5- X17.
Shich, L. S., and Guadiano, r. F. 1974. Matrix continucd fraction cxpansion and
inversion by thc g,eneralizcci matrix algori thm. lilt . J. COllII' . 20:727- 737 .
Shieh, L. S., and Goldman, M. J. 1974. A mixed Caller form for linear sys tem
reduction. I TrailS. Sl's . Mall. Cyl!. SMC-4:584- 5HX.
Shieh, L. S., and Wei, Y. J. 1975. A mixed method for multi-vari ahle systcm
reduction. J EEL' TrailS. A III. COIlt. AC-20:429-432.
Singh, V. 1979. Nonuniqueness of model reduction using the Routh approach. I EEE
TrailS. A ul. COllI. AC-24:65'J- 65 1.
Sinha, N. K., and Pille, W. 19'7 1 a. A new method for rcduction of dynamic sys tcms.
JIII..!. COlli. 14:111 -- IIH.
Sinha, N. K., ,md Berczani , (;. T. 197 1 b. Opti mum approximation of high-order
;,ysteI1ls by low-order modcis. Jill . J. Contr. 14:951 - 959.
Towill, D. R. , and Mehdi, Z. 1970. Prediction of transient response sensi tivity of
hi gh order linear systems using low order models. Meas urelllelli Control 3:TI - T9.
Wall, H. S. 1948. Analytic Thcory of Continued Fractions. Van Nostrand, Ncw
York.
Wright , D. J. 1973. The continued fraction representation of transfer functions and
model simpli fication. fnt . J . COlllr. 18:449-454.
Zakian, V. 1973. Simpli fication of linear time-invariant systems by moment ap-
proximations. Jill. J. COlll r. 18:455-460.
Chapter 4
I-lierarchical Control
of Large-Scale Systems
4.1 Introduction
The notion of a large-scal e sys tem, as it was hriefly discussed in Chapter I,
may be described as a complex system composed o f a numher o f C\ln-
stituents or smaller subsystems serving particular functions and shared
resources and governed by interrelat ed goa ls and constraint s (Mahmoud,
1977). Although interaction among subsystems can take on many forms, ()[11!
of the most common one is hi erarchi cal, which appears somewhat natural in
economi c, management , organi zational , and complex industrial
sll ch as steel, oil , and paper. Within thi s hierarchi ca l st ructure, the
terns are positioned on levcls with different degrees of hi erarchy. A s lIhs\ ls-
tem at a given level control s or "coordinates" the unit s on the it
and is, in turn, controll cd or coordinat ed by the unit on the h:vel illl -
mediat ely above it. Figure 4. 1 shows a typic;t1 hierarchical (" multil evel" )
system. The hi ghest level coordinator, sometimes ca lled the sliprellllll coordi-
nator , can be thought o f as the hoard of directors of a corporati on. whih:
another level's coordinators may he the president , vice-presi dent s, direcl\) J' s,
etc. The lower level s can be occupi ed by plant managas, shop mana gers,
etc., whil e the large-sca le sys tem is the corporati on itself. In spite of thi s
seemi ngly natural represen ta ti on of a hi erarchical struc ture, it s exac t hehav-
ior has not been well understood mainl y due to the fact that very litlh:
quantitative work has been done on these large-scale sys tems and
Si mon, 1958). Mesarovic et al. (1970) presented one of the earliest formal
quantitative treatments of hi erarchical (multilevel) systems. Since then, a
great deal of work has been done in the fi eld (Schoeffler and Lasdon 1966'
Beneveniste et aI. , 1976; Smi th and Sage, 1973; Geoffri on, 1970;
197 1; Pearson, 197 1; Cohen and lolland, 1976; Sandell et aI., 1978; Singh,
104
CONTROL
ACTION
Hierarchical Control of Large-Scale Systems
A
/ '\
/ '\
/ "-
/ "-
/ \
/ \
\
\
\
\
\
\
\\ DECISION-MAKING UNIT

\ PERFORMANCE
RESULTS
'\
\
'\
\


INPUTS L-_ ________ ___ _ _ _ OUTPUTS
Figure 4.1 A hierarchical (multilevel) control strategy for a large-scale system.
1980). For a relatively exhaustive survey on the multil evel systems control
and applications, the interested reader may see the work of Mahmoud
(1977). .
In thi s section, a further interpretation and insight of the notIon of
hierarchy, the properties and types of hierarchical .and
reasons for their existence are given. An overall evaluatIOn of hIerarchical
methods is presented in Section 4.6. . .
There is no uniquely or universally accepted set of properties aSSOCiated
with the hierarchical systems. However, the following are some of the key
properties:
I. A hierarchical system consists of deci sion making components struc-
tured in a pyramid shape (Figure 4.1).
2. The system has an overall goal which may (or may not) be in harmony
with all its individual components.
3. The various levels of hierarchy in the system exchange informa tion
(usually vertically) among themselves iterati vely.
4. As the level of hierarchy goes up, the time horizon increases ; i.e., the
lower-level components are faster than the higher-level ones.
There are three basic structures in hi erarchical (multilevel) systems de-
pending on the model parameters, decision variables, behavioral and en-
Introduction
lOS
vironmental aspects, uncert ainties, and the existence of many conflicting
goals or objectives.
1. Multistrata Hierarchical Structure: In this multilevel system in which
levels are called strata, lower-level subsystems are assigned more
specialized descriptions and details of the large-scale complex system
than the higher levels.
2. Multilayer Hierarchical Structure: This st ructure is a direct outcome of
the complexities involved in a decision making process. The control
tasks are distributed in a vertical division (Singh and Titli, 1978), as
shown in Figure 4.2. For the multilayer structure shown here, regula-
tion (first layer) acts as a direct control action, followed by optimi za-
tion (calculation of the regulators' set points), adaptation (direct
adaptation of the control law and model) and self-organization (model
selection and control as a function of environmental parameters).
3. Multiechelon Hierarchical Structure: This is the most general structure
of the three and consists of a number of subsys tems situated in levels
such that each one, as discussed earlier, c::. n coordinate lower-level
units and be coordinated by a higher-level oae. This structure, shown
Figure 4.2 A multil ayer control strategy for a large-scde system.
SELF-ORGANIZATION
STRUCTURE
ADAPTATION
PARAMETERS
OPTIMIZATION
SET POINTS
REGULATION
CONTROL MEASUREMENTS
LARGE-SCALE SYSTEM
INPUTS
OUTPUTS
10('
Hierarchical Control of Large-Scale Systems
in Pigure 4. L consitl ers conflicting goals and objectives between deci-
sion suhprobl ems. The higher-level echelons, in other words, resolve
the conflict goals while relaxing interactions alllong lower echelons.
The distrihution of control task in contrast to the multilayer s tructure
descrihed in Figure 4.2 is horizontal.
111 add iti(1n to the vertical (multilayer) and horizontal (multiechelon)
division of cnntwl task, a third division called a time or functional divi sion
is possihle (Singh and Titli. 1978). In this divi sion a given subsys tem's
functiollal optimization prohlcm is decoll1poseJ into a finite numher of
singlc-pmallleter optimi7,ation problems at a lower level and results in a
c(\llsidcrahlc reduction in computational e ffort. This seheme will be dls-
cllssed in Section 4.5 In connection with the hi erarchical control of
dislTl,tc-tilllc sys tcms.
Before th e of the present chapter is given, based on the above
disl'lISsinl1. one can make a tentative conclusion that a successful operation
(11' hierarchical systems is best described by two processes known as deco/1/-
[,!lsitinll alJd ('()(;rriillatioll. 1he coordination process as applicable to most
hi erarchi cal systems is descrihed in Section 4.2. Section 4.3 IS concerned
\\ilh the control nr continuous-time hierarchical systems where
cO(1rdination betwcen two levels are considered. The closed-loop hierarchi-
Cl i C(1 nlrol of large-scale is di scussed in Section 4.4, including the
of " intnacti()n predict inn" and "structural perturhation." [n Scc-
ti(1n 4.5. the structural perturbation and interaction prediction approaches
are extended t(1 take 011 the discrete-time case with and without delay.
;\ mono t he hi erarchical con tml strategies disclissed here are a three-l eve l
l:'
time- dcl;l\' coordination algorithm hy Tamura (1974, 1975) and a "cos tate
c(1ordinaiion" (1r "costate prediction" scheme due to Mahmoud et al. (1977)
and and Singh (1976a, 1977;). A number of numerical examples
illustra te the various techniques presented. The near-optimllm design or
linear and nonlinear hierarchical systems are di scussed in Chapter 6. Section
4.6 is devoted to further di scus-sion and the evaiuation of hi erarchi cal
COil t rol tec h niqucs.
4.2 Coordination of Hierarchical Structures
[t was mentioned in the previous section that a large-scale sys tem can be
hi erarchically controlled by decomposing it into a number of subsystems
and then c;)ordinating the resulting subproblcms to transform a given
intergrated system into a multilevel one. This trans fonnation can be achieved
by a hos t of dirferent ways. However, most of these schemes are essentially
a comhination of two distinct approaches: the model-coordinatiofllllethod (or
Illethod) and gO(l /-coorriil1(lliol1l11cl/rod (or "dual-feasible" method)
(Mesarovic et aI., 1(69). In the next two scctions, these methods are
Coordination of Hierarchical Structures
107
described for a two-subsystem s tatic optimizatic!l (nonlinear programming)
problem.
4.2.1 Model Coordination Method
Consider the following static optimization problem (Schoeffler, 1971) :
minimizeJ(x, u, y)
subject to f( x, Lt, y) = 0
(4.2. I)
(4.2.2)
whcre x = vector of sys tcm (state) variables, II = vector of manipulated
(control) variables, and y = vector of interaction variables between subsys-
tems. Let thc problem and its objective fun cti on be decomposed into two
subsystems, i.e.,
and
(4.2.3)
(4.2.4)
where Xi, 1/ i, and f arc vectors of sys tcm, m;1I1ipulated, and interac ti on
variables for ith subsystem, respectively. This decomposition has produced
a pcrformance function for each subsys tem. However. through the vectors
.1'1, i = 1,2, the subsys tems are still interconnected. The objective of tile
Ill odel coordination method is to convcrt the intcgrated problem
(4.2.1)- (4.2.2) into a two-level problem by fixing the interacti on va riables /
and)'2 at some value, say Wi, i = 1,2, i.e.,
Constrain/=w
i
, i=I,2 (4.2.5)
Under this situation the problem (4.2.1 )-( 4.2.2) may be divided into the
following two sequential problems:
First - Level Problem - Subsystem i
Find KI(w) = minl,(xi. u
i
Wi)
x', u'
(4.2.6 )
(4.2.7)
Second - Level Problem
minimize K( w) = KI (IV)+ K2 (w)
IV
(4.2.8)
The above minimi za tions arc to be done, respectively, over the follO\ving
feasible sets:
i = I,2
= {Wi: K
I
( Wi) exists}, i = \. 2
(4 .2. 9)
(4.2.10)
Figure 4.3 shows a two-level slnwtllre rnr Ihf'. nl().!f' l ('()nrrlin"linn nwth,,,,l
Hierarchical Control of Large-Scale Systems

\I'
1,(11') = "I(W) + A'
2
(\\, )
J2---

'111111 i l(\I. II I.j


,
l)
, ,/II
<;; 1I!' \ed to
,I (\ 1, /11 , H, I, 1\ 1 ) -:=- U
------- ----
r'llt\
A
2
(\I') = mill i
l
( X
2
. Ill, \\,1)
,,2, 11
2
s. ub,;d \0 1 .,
/1 (\ ' , 11 _. \\ ' . \\ ,- ) ()

Figllre 4.3 A two-levcl soluti on of a sLatic optimization probl cm using model
l' tltlrdin:ltitln .
In this coordination procedure the variables Wi which fix interaction
va riahles ri are termed coordillatillg variables. Moreover, since certain
int ernal are fi xed hy adding a constraint to the mathematical
Ill ode\. this procedure is called model coordination. In other words, due to
the fact that all intermediate variables x, /1, and yare present, it is
alt ernati vely termed " feasible decomposition method." Therefore, a system
can nperate with these intermediate values with a near-optimal performance.
The fir st-level prohlems are constructed by fixing certain interacting var-
i;, h1es in the original optimization prohlem, while assigning the task of
detellllining these coordinating variables to the second level.
4.2.2 (;0(// Coordination Method
('onsidn the stati c optimi za tion problem (4.2.1) - (4.2.2). In the goal coordi-
nati oll method the int eractions are literall y removed by cutting all the links
;\I11 ong the su hsys lems. Let .I ,i be the outgoing variable from the ith
subsvs tClll. while it s incoming variable is dCllot ed by z'. Duc to the remova l
pi' a,'1 lillks het\veen subsys tems. it is clcar that Vi =1= Z i. Under thi s condition.
:' acts as an arhitrary variable and should be chosen by the
nptillli zing subsystellls like .Y. 11 . and y . Moreover, the optimization problem
cOllsidered in the previous section is completely decoupled into two subsys-
telll s due to the fact that thei r interactions are cut and their objcctive
fUllction s were already separated. III order to make sure the individual
suhprohlems yield a solution to the original problem. it is necessa ry that the
illlcmclioll -halollce prillciple be satisfied, i.e., the independently selected y'
and Zi actually become equal (Mesarnvic et aI., 1969; Schoeffler, 1971).
Here again, the procedure is to decompose the problem into a number of
denlllpkd subprohlems which constitute the firs l-Ieycl prohlem. The sec-
olld-Ievel nrohlcm is to force the first -level subproblems to a for
Coordination of Hierarchical Structures 109
which the interaction-balance principle holds. Mathematically, thi s multi-
level formulation can be set up by introducing a weighting parameter 0:
which penali zes the performance of the system when the interactions do not
balance. Hence, to the objective function (4.2.3) a penalty term is added:
1 (x, U, y, z, a) = 11 (x I, U I, y' ) + 12 ( X 2 , Ii 2 , Y 2) + a'Tv - z)
(4,2.11)
where a is a vector of weighting parameters (positive or negative) which
causes any interaction unbalance (y - z) to affect the objective function. By
introducing the z variables, the system's equations an.: given by
fl(XI , Ul, y', Z2) = ()
f 2(X 2, u
2
, y 2, Zl) = 0
The sct of allowable system variables is defined by
(4.2.12)
(4.2.13)
(4.2. 14)
Once the objective function (4.2.11) is minimi zcd over the sct So it rcsu lts a
function,
K(a) = min 1( x,u,y,z ,a) (4.2.15)
.,", II .y.ZES
o
After expanding the penalty term a
T
(y -z ) = a;()lI- z l)+Ci;e y 2 -.:;2)
and considering the relations e 4.2. 11 )- ( 4.2.13). the first-level probl em is
formulated as
Subsystem 1: min 1,exl, /1
1
, y', Z2)+ aryl -
xl, u
1
.rl ,z2 -
Subsystem 2:
( 4,2.16)
(4.2.17)
(4.2.1 8)
( 4.2. 19)
The second-level probl em is to manipulate the coordinating variable 0: III
order to derive the two-s ubsys tems interaction error to zero, i.e.,
min e = min (y - z) (4.2.20)
a a
It is clear frol11 the second-level problem ( Equation (4.2.20)) that the
coordinating variablc a is manipulated until the error e approaches zcro;
i.e., the interaction balance is held by manipulating the objective functions
of the first-Icvel problems (4.2.16) and e4.2. 18) through variable 0:: hence.
the name "goal coordination." Figure 4.4 shows the two-level solution via
goal coordination. Thc rcader should compare the two structurcs in Fi gures
4.3 and 4.4.
'I"
' . . - . .
.,.1'"
Hierarchical Control of Large-Scale Systems
Choose Cc' to achi eve
int eraction balance
l11inJI(x l . 1/I .yl. Z2)+ "' i y l _ ",rZ2
subject 10
fl(x
l
, 1/
1
, y l . z2) = 0
l11in h(x
2
, 1/ 2. y 2. z 1) - ", r zl +"'f /
subjecl 10
f2(x
2
. 1/
2
, y2. zl ) = 0
4.4 A two-level solution of a static optimization problem using goal coordi-
nation.
It will be seen later that the coordinating variable IX can be interpreted as
a vector of Lagrange multipliers and the second-level problem can be solved
through well-known iterative search methods, such as the gradient, Newton' s,
or conjugate gradient methods.
4.3 Open-wop Hierarchical Control of Continuous-Time Systems
In this section the goal coordination formulation of multilevel systems is
applied to large-scale linear continuous-time systems within the context of
open-loop control. In addition to the interaction-balance approach another
scheme known as the interaction prediction method is also discussed.
Let a large-scale dynamic interconnected system be represented by the
following state equation:
(4.3.1)
x and u are n X I and m X 1 state and control vectors, respectively. It
IS assumed that the system consists of N interconnected subsystems s
i = 1,2, ... ,N, and the ith subsystem's state equation is given by "
x;=/;(x;.u;.t)+g;(x, t) , x;(tJ=x;n (4 .3.2)
where x, u, x;, u; are respectively n-, m-, n ;-, and m ;-dimensional, g; (.)
represents the ith subsystem interaction and
xT(t) = (xnt),xi(t), ...
uT(t) = (unt), unt) , . ..
(4.3 .3)
(4.3.4)
The objective, in an optimal control sense, is to find control vectors
u
1
' u
2
,,,,,U
N
such that a cost function
J=G(x(t
l
))+ F 1h(x(t),u(t),t)dt
10
(4.3.5)
Open-Loop Hierarchical Control of Continuous-Time Systems
III
is minimized subject to (4.3.1) and a feasible domain
u(t) E U( x (t) , t) = {ul v ( x (t) , u, t) O}
(4.3.6)
Through the assumed decomposition of system (4.3.1) into N intercon-
nected subsystems (4.3.2), a similar decomposition can be assumed to hold
for the cost function constraint (4.3.6) and the interaction g;( x, t) in (4.3.2),
l.e.,
J = LJ; = L { G;( x;( t
l
)) + F 1h;(x;(t) , z;(t) , u;( t) , t) dt }
I . Ito
v(x , u, t) = LVj ( Xj , uj , t)
j
g;(x , t) = Lg;j (Xj , t)
j
(4.3.7)
(4.3.8)
(4.3 .9)
where z;(t) is a vector consisting of a linear (or nonlinear) combination of
the states of the N subsystems. Under the above assumption of separation,
the large-scale system' s optimal control problem (4.3.1), (4.3.5), and (4.3.6)
can be rewritten as
minimize
LJ;=L{G;(X;(tl ))+{lh;( X; (t) ,Z; (t) , U;(t), t )dt } (4 .3.10)
I I ' 0
subject to
x;(t)=!;( x;(t) , u;(t) , t)+t;(Zj(t) , t) , Xj(t o)= x jo, i=I ,2, .. . ,N
(4.3.11)
t;(Zj(t) , t)=Lgij( X/t) , t) , i =l ,l ,oo . ,N
j
L Vj (x
j
, uj (t) , t ) 0
j
(4.3.12)
(4.3.13)
The above problem, known as a hierarchical (multilevel) control, was
demonstrated for a two-level optimization of a static problem in the
previous section. The application of two-level goal-coordination to large-
scale linear systems is given next.
4.3.1 Linear System Two - Level Coordination
Consider a large-scale linear time-invariant system:
x(t)=Ax(t)+Bu(t), x(O) =xo
It is assumed that (4.3.14) can be decomposed into
Xj(t) = Ajx j(t)+ Bjuj(t)+ CjZj( t ), Xj(O) = Xjo
(4.3.14)
(4.3. 15)
11 2 II icra rchical Control of Large-Scale Systcms
and the h, X I intcraCli()ll vector
.V
,:, (1) = L G;; ."
/ - ' I
(4.3 . I 6)
is a lincar l"Olllhination of the states of the other N -- I subsystems, and G
' /
<I n ", X " , matri x (Singh. I The original sys tem's optimal control
prohlem i, reduced to the optimization of N subsystems which collectively
S:I ti sfy (4 . .1 . 15) .. (4.3. 16) while minimil.ing
+ .:,'(f),':), .::, (I)] ill} (4 .3. 17)
\\' here <l, <l IT " , X " , positive semidefinite matri ces, R; and S; are Ill, X III;
and " , X ", positi ve definite matrices with
,v
11 = L II ,.
r "" I
,v
III = L Ill; .
I
.'1'
k = Lie;, le
r
Il ,
; = I
ThL' phvs ic:li int erpretati on of the last term in the integrand of (4.3.17) is
diffi cu lt at this point. In fact. the introduction of thi s term, as will be seen
Liter. i, to avoid singu lar control s. The "goal coordination" or "interaction
h: li ;l ll L"l'" approach of Mesarvic et al. (1970) as applied to the " linear-
LJlladratic" prohlem by Pearson (1971) and report ed by Singh (1980) is now
11IL'SL'llt cd.
In thi s decompositi on of a large interconnected linea r system the COllllllon
l"\)upling factors among it s N subsys tems are the" interacti on" variables
:, ( I). \\ hi ch. ahmg with (4.3. 15)- (4.3.16). constitut e the "coupling" COI1-
st r:li llts . Thi , rormul ation has been ca lled "global" by Benveniste et al.
( 1l) 7(,) :llld is dl'll\)ted hy .\.(;. The foll owi ng assumption considered to
Il(lld . The g. 1()h, d prohlem .1"(; is replaced by a family of N subprohlems
('('nplcd tog.et her through a parameter vector a=(a
l
.a
2
, ... . a
N
)"1 and de-
Il(lt t.' d hv S, ( (v). i = 1. 2 ..... N. In other words, the globa I system problem .I
r
;
is " illlhcdtkd" int(1 a L\lnil y of subsystem pn)blcllls .1", ( (.\') through an
illlkddillg l'ar:\llleler IY (S<J1H1e1l et ,d .. 197R ) in such a way that for :1
,);lIlil 'III ;II' valut' pf <y '". the slIhsys tems .\( (\" * ). i 1,2, ... , N, y,jeld the
';(, hlli OIl 10 Sr . ' In terms of hi erarchi ca l control Ilotation. this imbedding
("ll ll l.'!: !)1 is nOlhing hut the notion of c(l() rdination . hut in mathema ti cal
j1I(1gr:lIllining pmblell1 tCl"minology, it is dellot ed as the " master" prohlem
('eoi"fri( ln. 1970). Figure 4.5 shows a two-level control structure or a
l:iq.!.e-s(": d<.' system. Under this strategy, e:leh loca l controller i recei vcs
flll])1 thl' ("()()rd inator (second-level hierarchy). solves .\', <), and trall Slll it s
Opcn-Loop Hi erarchi cal Control of Continuous-Time Systcms
SECOND
LEVEl..
Figure 4.5 Thc two-level goal-coordinati on structure for dynami c systems.
11 3
I: II(ST
t.l'VEI..
(report s) some functi on y/ of its soluti on to the coordinator. The coordina-
tor, in turn, evaluates the next updatcd value or IX, i.e. ,
(4.3.19)
wherc 1/ is thc fth iterat ion step sizc, and the upda te ter m iI'. as wil l he seen
shortl y, is commonl y taken as a functi on of "interaction error":
N
e;(a(/), / )=z,(a(/) ,t)- L G,/x,(a(/). I)

(4.3.20)
The imbedded interaction variable Zr ( . ) in (4.3.20) can be considered as part
of the control variable available to controller i. in which case the pa ramet er
vector a( t) scrves as a set of "dual" variabl es or Langrange Illul ti
corresponding to in teraction equality constrain ts (4.3.16). The fund ament al
concept behind thi s approach is to convert the original system's minimi za-
ti on problem into an easier maximization problem whose soluti on can be
obtained in the two-l evel iterative scheme di scussed above.
Let us in troduce a dual function
q(a) = min {L(x. u, z, u)} (4.3 .21 )
X. Ii, Z
su bject to (4.3.15), where the Lagrangian L ( .) is defineu by
L(x,u,z,o:) = t X/(f)Qr \(t) v ( [
, = 1 0
+ l/ ;"( f ) R ,11 , (I ) + .: ;r( f ) S,.:, ( f ) (4.3.22)
+2a;'(z,( f) - Gr j"Y/I))l cit)
.I _. I J
where the parameter vector (\' consists of k Lagr:lI1 ge Illultipliers. In Ihis way
the original eOllstrained (subsystems interacti ons) op timi zation problem is
! 11 I Iicra rcliica! ( '() l1lm! or I,argl'-Scak Systems
t' il ,111 f', n ! (n ;111 lI11CollstrOlincd one. In pliler words, the cOllstraint (4.3. 1(1) is
q li .'; i'i l' 11 hy rkterllllning a set of Lagrange Illuitiplicrs (Xi' i = 1,2, ... ,k.
l ! nil e, .'lIch C;J':es, \\'liell the cOll straint s are convex, Geoffrion ( 1971 a, h) and
Sil1?-h (llJXO) ha ve showil that
rV(;ni mi ze q( <t} ==:0 Minill1il.cJ (4.3.23)
I' II
in<iic;lling Ihat ll1inillli l;lti(l1l or J in (4.3. 17) subject to (4,3,15) - (4,3. 16) is
l'qu i\ ',llcnt to m;1'\imi7.ing the dual function q(iX) in (4,3,21) with respect to
n, Tn f:ll'ilitale the solution of this problem, it is observed that for a given
set or I. agrange Illultipliers a = a*, the Lagrangian (4,3,22) can be rewritten
<IS
(4.3.24)
whi ch reveals thaI the (kcc1l1lpositioll is carried on to the Lagrangian in sllch
;1 wav that a sub-Lagrangian exists for each subsystem, Each sll hsysle1l1
\\'oliid intend til Ill inimilt' its own Li as defined by (4.3,24)
suh ject to (4,3. 15) and using the Lagrange lllulti pli Crs_a* which are trented
as known funet iPlls at the first level of hinarc!J.y, The result of C<J ch s\ll' h
1l1illil11i 7.a tion would allow one to determine the rl ll al rllDc!iL1l1 q(x*) ilL
At the second level. where the solutions of all first-level subsystems
are known, the value of q( a*) would be improved by a typi cal tllleop-
strained nptil11i za tion such ii'StTi e Newton's method, the gradient method or
the cpnjllg;ill' gradiellt l11c1hod, The rcason I'm a gradient-type l11c1hod is
dil l' t(l thl' fact that the gradient of <f((t) is defined by
N
"7 n)I ",._ , ... Z,. _ \", b.
\'" " L V,r\; = <'i'
i = L 2,,,, (4 ,3.25 )

is l1o!hing hut the suhsvstems' interaction errors, which are known through
first kycl soilltions, and \, f defines the gradient of f with respect tn .\'. At
the sl'l'(lll d lc\'elthe vector H is updated as indicated by (4,3,19) and Figure
4.), If a gr<1 di ent (steepes t descent) method is employed, the vector d
l
in
(4.3. ILJ) is simply the Ith iteration interaction error (,1(1), However, a
superior techniqlle from a computational point of view is the conjugate
Open-Loop Hierarchi cal Control of Continuous-Timc Systems 11 5
gradient defined by
dl 'l (f)=el l t(t)+/ ' Idl(f), (4,3,26)
where
[I( elt 1 (I) )f'e
I 1
' ( t) cll
1+1 -"- 0 __ _._----- --
Y = [ /(el)Teld(
o
(4 ,3.27)
and d O = eO, Once the error vector e(t) approaches zero, the optimum
hierarchical control is resulted, Below, a step-by-step computational proce-
dure for the goal coordination method of hierarchical control is given,
A 19oritlzm 4.1. Goal Coordination Method
Step 1: For each first-level subsystem, minimi ze each sub- Lagrangian
L; using a known Lagrange multiplier 0: = 0:*, Since the
subsystems arc linear, a Riccati equation formulationi' can be
used here, Store solu tions,
Step 2: At the second level, a conjugate gradient iterative method similar
to (4,3,26)- (4,3,27) is used to update 0:*(/) trajectories like
(4.3,19), Once the total system interaction error i!1 normali zed
form
Error = ( [ /{ Z; Gi/X
j
If T {z; - G,jX
j
} dt) //:::' (
, = \ 0 J=\ J = \
(4,3,28)
is sufficiently small, an optimum solution has been obtained for
the system, Here D.t is the step size of integration,
Two examples follow that help to illustrate the goal coordination or
interaction balance approach, The first example. which was first suggested
by Pearson (1971) and further treated by Singh (1980), is used in a modified
form, The second example represents the model of a multi-reach river
pollution problem (Beck, 1974; Singh, 1975), The overall evaluation of the
multilevel methods is deferred until Section 4,6, and the treatment of
nonlinear multilevel systems is given in parts in Section 4,5 and Chapter 6,
'i'Readers unfamiliar with the Riceati formulation can com,ult Section 4.3.2 on Interacti,)n
Predict ion Method.
11(' I1i crarchical Control of Large-Sca le Systel1ls
-.-.. -.--- ---]
II
/,'-'----... 1
r---+ l - 1
I ' , I :., - q - -, - ; - , : I: /, -- ---.-- I
I I " 1..:.... -- - - ----- ---- ---' I I
I L ____ __________ I
L ________________________________ I
Figure .t.() A hlock diagralll [or :.;ys lcm of Example 4.3. 1.
V" :lIl1Pil' .I.J . I. ( 'oll sidn:t 12Ih-(lrdcr sy.-;lc ll1 illtroduCl:d by Pearson (lIn I)
and showlI in h gure 4.() with a stale equation
\ _.
Il
Il
- .1
1\
II
Il
1\
()
Il
I)
I
I I
(\
II
1\
II
Il
Il
II
II
(\
I)
Il
1\
II
I)
I)
Il
I
I I
II
(\
Il
2
tl
()
II
II
()
I)
1
1/
Il I
1
, 1
t) I I)
() ()
() - I
t) I
I) I
()
()
i\
()
an d a ljll :ldra li c cosl fllncl ion
o
1\
. -'
t)
()
1
(\
o I
() I 0
1 I ()
I
- 2 I ()
() I ()
() t)
I
() L. -: 1_
()
I )
1
()
()
()
I
()
- 2
()
()
()
t) 0 0
I () 1 ()
- .. - - - - - - - - -
o I
() I 0
I
() I
- - -- - - - - - - - -
o I
1 I 0
I
_--. 3 _ 1 _ _
() I () ()
() I () (\ I
() 1 _ 3 - 2 - I
(4 .3.2l1)
.I ("{ I'(t )( ) I(/) I 1I'(t)!<II(/ )} dt
'11
( i U .ll))
Open-Loop Hierarchical Control of Continuous-Time Systems
11 7
with
where
The sys tem out pu t veclor is given by
I 0 0 0 (l
0 1 0
0 I
t "
v = Cx=
()
I 0
()
o I (l ()
() I 0
0 0 0
0
0
()
I 0
0 I
o
()
(4.3.3 1)
It is desired to find a hi erarcllical control strategy through the interacti on
balance (goal coordination) approach.
SOLUTION: From the schematic of the system shown in Figure 4.6 (clott ed
lines) and the s tate matrix in (4.3.29), it is clcar that there are fou r
third-order subsys tems coupled together through six equality (number of
dott ed lines in Fi gure 4.6) constraints given by
= [( ZI - XI I) ,(Z2 -XI) , (Z3-X7 ),(Z4 -X5 ), (ZS- XX) ,(Z(,- X2)]
(4 .3.32)
where e;, i = J, 2, ... ,6, represents the interaction errors between the four
subsys tcms. The first- Icvel subsystcm problcms were solved through a set o f
four third- order matri x Riccati equat ions
K;(t) =A;K,(t) + K,(t)A, - K;(t)V;K,(t) +Q;, K,( IO)=O
(4.3.33)
where K,(/) is a n 11 / X 11 ; positi ve-ddinite symmetri c Rieea ti ma tri x and
= B, R;-I B/'. T he " integration-free," or " doubling," method of solving the
differen t ial matri x Ri ccati equalion proposed hy Davison and Maki ( 1973)
and ovcrvicwccl by Jamshidi (1980) was used on an HP-45 comput cr and a
BASI C source program. The subsys tcms' stat e equations were solved by a
sl :llld:JI'lI fllurl h-ortlcr Runge-- Kulta method. while thc Sl'Cll IHI-le\'cl ilL'ra-
...... 1 _ .. _ .. _.. _ .. . .. . _l ; "._ . ""I .............. . . 1..1 '1 11) \
II X llierarchi cal Conlrnl of Large-Scale Systems
1.0
1(1
I[)
If) '1 ____ L ___. 1 _____ .1 ___ 1._.__. __ 1 __ .._ L __________ .--.l ___ J
,1 (, 7 R l) 1 ()
("f IN.1l )(;,', II' ( ; RII 1>11 ; N J liT RATIONS
Jiil'.lJl"( 1.7 NOflll aii rnl intt'r;l cti ol) \'IT\lr vs cOl1gugate g. radi ent iteratio1ls for LX;1111 -
I'k 4 .1. 1.
(:L\.2(,) (4 .. LJ.7 ) utili zing the cubic interpolation (Hewlett-Packard,
1979 ) t(1 cvnlllnte appwpriate nUllleri cal integrals. The step size was chosen
10 he (\{ ,- n. 1 as in earlier treatments of this example (Pearson, 1971: Singh,
llJ XO) . The conjugate gradient algorithm resulted in a decrease in error from
1 to about 10 ) in six iterations, as shown in Figure 4.7, which was in close
agreemcnt with previously report ed result s of a modified version of system
(4.J,29) bv Singh (1980). Now let us consider the second example.
F,\<lmple 4.3.2. COllsider a two-reach model of a rivcr poilu lion control
pmh1em.
[
.
-- 1 .J2 II I 0 0
, -- (U2 - 1.2 I 0 0
x =
(J ,90 0 I - 1.32 0
() (J .9 I -- 0,J2 - 1.2 l
o.1 I 0 j
x+ u
o I 0.1
_0 I 0
(4. J.34)
\\ hl'll' c; I\ 'h f'l'<1ch (suhsys tl'm) of the rlycr h<l s Iwo 5t,lte5 ---'\"1 is the
w il een I r,1 tion of hiochell1i c<1 l oxygCll demand (BO \)) , * and x 2 is the COIl -
c(' ntrati(lll (If di ssolved oxygell (DO) -- and its control III is the BOD of the
dllucnt di sch<1rge into the river. For a quadratic cost function
(4.J.35)
with (J -- diag.{2.4,2.4) alld N c= dia g. {2. 2}, it is desired to find an uptilllal
cOlltrol whicl! lIlinimil',cs (4.J, 35) subject lo (4,J.34) alld\(O) = (I I - I 1)1.
Open-Loop lIierarchieal Control of Continuous-Time Systems 11 Y
SOLUTION: The two first -level problems are identical. amI a scconu-oruer
matri x Riecati equation is solved by integrating (4.3.33) using a fourth-oruer
RUl1ge- Kutta method for 6.t = 0.1. The interaction error for thi s example
reduced to about 10 - 5 in 15 iterations, as shown in Figure 4.8. The
optimum BOD and DO concentrations of thc two reaches of the river are
shown in Figure 4.9.
4.3.2 Int eractio/l Prediction Method
An alternative approach in optimal control or hierarchical systcms which
has both open- and closed-loop forms is the interacti on predi cti on method
hased on the initial work of Takahara (1 965), which avoids second-level
gradient-type iterations. Consider a large-scale linear int erconnected sys lem
which is decomposed in to N subsystems, each of which is described by
5:; (t) =A, \' ,(t)+B;II ;(t) +C;z Jt), x; (O) ='\;0' i = l,2,. . ., N (4.3 .36)
where the interaction vector z; is
N
z; (t) = L G;r\"j (t )
./ = 1
(4 .3.37)
The optimal control problem at the first level is to find a control [/ , (1) which
Figure 4.8 Int eraction error behavior [or river pollution system of Exalllple 4,3,2,
\ 50
o __ l __ __ __ __ __
I 3 4 5 r. 7 x <) 10 \1 12 13 14 \)
ITERATIONS
120
o
Vl
/
o
f=
..-
c,;
f-
/
.:J
L:
/ .

L
.'.
;.:,
;:
'--
1.0
()
.. I
6 -.(' -
x /
1.0 f
/
/
/
IIicrarchical Control of Large-Scale Systems
-- --.
-- t .. - -: . TIME ( Sl' e)
2. --.. 3 4 5
X I HOI> REACII I
Xl DO RFAU I I
\") HOIl RI'I\CII 2
\" .1 Il() RI'A(,II 2
Fi[!IJr(' ,1.9 Optilnlllll !lUI) and I)U concClltrati()ns fpr thc two-reach Jl1udel (If a
rin'r p!lllllli()n L'(mtrnl prohlem in Example 4J.L
s:l li si'i cs (4,3,36) -(4,3.37 ) while lllinil11i7jng a usual quadratic cost [ullction
Thi s prnhlel11 C:l1I he s()ln'd hy first introducing a set of Lagmnge Illulti-
pliers n,(t) and costate vec tors p,(I) to augment the "interaction" equality
c<l llstr,lint (4.3.37) and suhsystcm dynamic constraint (4,3,36) to the cost
fUIll,tinn's integrand: i.c .. Ihc ith subsys!cm Ilallliltonian is defined by
(4,3.39)
Open-Loop l-uerarehicaJ Control of Continllolls-Time Systems
Then the following set of necessary conditioJ<s can be written:
N
jJ
i
= - iJllilax
i
= - Q,x
i
- A;p, + I: G/-ai(r)

p, ( t j ) = 0 (i x;" ( r j ) Q i Xi ( I j ) ) I iJx;( r j ) = Q i X, ( 1 j )
,\ ,( /) = (lll,/op, = Aixi(r)+ Billi(t) +C,::j(/). x,(O) = Xjo
0 = 8Hj8u, = Rju;{t)+ B/jJ,(/)
121
(4.3.40)
(4.3.41)
(4.3.42)
(4,3.43)
where the vectors u,( t) and Z,( t) arc no longer considered as un k nowll s at
the second level. and in fact z,(t) is augmellted with nj(t) to constitute a
higher-dimensional "coordination vector." which will be obtained shortly.
For the purpose of solving the first-level problem, it suffices to assume
(IY:-(/) : z7(/)) as known. Note that IIj(/) can be eliminated from (4.:1.4:\),
u, ( I) = - R, - 1 n /jl I ( 1 ) ( 4.3.44)
and substituted into (4.3.40)- (4.3.42) to obtain
x,(r) = A,x,(t) - SjP,(t)+Cjzj(t), xi(O) =xjn
(4.3.45)
N
jJ,(r)=-Qjx,(/)- /t;p,(/)+ I: Gjj(X,(t), pj(rj)=Qjx,(l
j
) (4.3.46)

which constitute a linear two-point bounda;'y- value (TPBV) problem <Jlld
Sj BjR j- 1B/- It can he seen that this TPl3V problem can be decoupled by
introducing a matrix Riccati formulation. Here it is assumed that
(4.3.47)
where gj(/) is an II,-dimensional open-Ioo]) 'adjoint," or "compensation,"
vector. [f both sides of (4,3.47) are differentiated and iJ/r) and .\-,(/) from
(4.3.46) and (4,3.45) are substituted into it, makicg repeated usc of (4.3.47)
and equating coeffici ents of the first and zeroth powers of x
i
(/), the
following matrix and vector differential equations result :
N
id!) =- (A,-SjKj( t))Tg,(/) -Kj (/)CjZ j( t)+ I: G/n;(r) (4,3.49)
/ = 1
whose final conditions K,(t j ) and g;(l
f
) foliow froJll (4.3.41) and (4.3.47),
I.e ..
(4.3.50)
Following this formulation, the first -lewl oplimal c(>I1lml (4.3.44) hccomcs
(4.3.5 I)
I Iicrarcilical C(lllirol of Large-Scale Svslems
\\ hich 11<ls ;\ partial feed hack (closed-h,op) tcrm and a feed forward (open-
loop) terill. Two points arc made here. First. the solution of the differential
IIl<1lrix Riccati equatipn which involves 11
/
(11
1
+ 1)/2 llonlinear scalar equa-
tions is independent of the initial state X;(O). The second point is that unlike
K./(t), ,1; /(1) in (4.3.49), hv virtue of ZI(t), is dependent on x;(O). This
property ,viII he used in Section 4.4 to obtain a completely closed-loop
conlrol in a hierarchical structure.
The second level problem is essentially updating the new coordination
\ l'dor (II : (/) : ::1 (1)'- For Ihis purpose, define the additivcly separable
1.; l).', rant'- i;1lI
I
2 /I: ( / ) /{ I II I ( /) I (l: ( r ) Z I ( r ) . L 1< ( I ) (i
ll
\ I ( r )
, - 1
I f1/(/)i - i,(t) I AI. Y,(r) I /J,/I;(t)-lC, Z/(I)1)dr)
II\(' P\" IY
/
( t) ;lllt! .:/ ( t) can he obt<lincd hy
\\ hich plll\idc
() = ill ., ( . ) / iI Z I ( t ) = It, ( t ) -I (,/ f! ; ( I )
v
ill ., ( . )/ (/H/(r) = Z/(t) -- L U'I " ,{r)
I I
N
(X;(t) =- C'/p;(t), z;(t)= L Gj;.,)t)

(4.3 .52)
(4.3.S3)
(4.3.54)
(4.3.SS)
The sClolld -bcl ('()ordinatioil procedure at the (I + I )th iteration is simply
I.
<.Y I ( I ) ]1 I 1
:: I ( r )
.. C/'I), ( r )
.v
L (il;\' (I)
1 = 1
(4.3.56)
J'll\' intl'1';lcli('n prediction method is formulated by the following algorithm.
AIRor;,fllll 4. L, Interaction Prcdiction Method for Continllolls-Time Systems
,\/cP I: Snhe N illdcpcndcn t di ffcrential Ill;) trix RilTa li equations (4.3.4X)
with filial condition (4.3.50) and store K,(t), i = 1,2" . "N.
,\rlp }. For indial ( r), z: ( t) solve the "adjoint" equation (4.3.49) with
fin,11 condition 14. 1')0\ FV;tiW11P ;l1lel p( t) i = J? N
Opcn-Loop llil'l'archical Control of Continuous-Timc Systems 123
Stcp 3: Solve the stale equation
.x;(t) = (AI - S;KI(t))xl(t)-Slg;(t)+C;z;(t) , .\;(0) = .\1"
(4.3.57)
and storex;(t), i = 1,2, .. . ,N.
Stcp 4: At the second level, use the results of Steps 2 and 3 and (4.3.56)
to update the coordination vector
Step 5: Check for the convergence at the second level by evaluating the
overall interaction error
T

(4.J. SX )
It must be noted that depending on the type of digital computer amI
operating system, subsystem calculations may be clone in parallel and that
the N matrix Riccati equations at Step 1 are independent of x/(O), and
hence they need to be computed once regardless of the number of seL'\llJd-
level iterations ill the interaction prediction algoritbm (4.3.56). It is further
noled that unlike the goal coordination methods, no ZI(t) term is needed in
tbe cost function, which was intended, as is di scusscd in next section, to
avoid singularities.
The interaction prediction method, originated by Takahara (196S), has
been considered by many researchers who have made significant contribu-
tions to it. Among them are Titli (1972), who called it the" mixed method"
(Singh 1980), and Cohen et al. (1974), who have presented more refined
proofs of convergence than those originally suggested. Smith and Sage
(197J) have extcnded the scheme to nonlinear systems which will be
considered in Chapter 6. A genuine comparison of the interaetion prediction
and goal coordination methods has been considered by Singh et al. (1975)
which will be discussed in Section 4.6. The following two examples illustrate
the interaction prediction method.
Example 4.3.3. Consider a fourth-order system
x= __ _ __ -jx+ (4.3.59)
0.05 0. 15 I 1 0.05 0 I O.S
o - 0.2 I - 0.25 - 1.2 0 1 0.25
124 Hierarchical Control of Large-Scale Systems
\\ilb \"(0) =, ( -- l,O. l, I.O, -- O.S)/" and a quadratic cost functioll with (j =
diag(2, 1. 1. 2), R = diag( 1. 2) and no terminal penalty. It is desired to use the
interaction prediction method to find an optimal control for (j = I.
SOl tll JON: The system was divided into two second-order subsystems, and
steps oUllined in Algorithm 4.2 were applied. At the first step, two indepen-
dent differcntial matrix Riccati equations were solved hy using hoth the
douhling ;dg.()rithlll of Davisoll and Maki (1973) and the standard
Runge K utta methods. The clements of the Riccati matrix were fitted in by
qlladratil' polynomial in the Chebyschev sense (Newhouse, 1962) for com-
putational con\enience:
/\ I (I) == I 4.44 + lU21 --- 1.261
2
O.(}9+0J)(17I -- 0.0271
2
A , (I) = I 2.X7 - 5 .Hllt
.- _ - 0 . 1+0.161 - 0.0541
2
0 .09+0.0071 - 0.0271
2
]
0.5 -Hl.0341 --- 0.141 t 2
- 0.1 +0.1(11 .- 0.0541
1
]
O.73+0.IIXI -- O.X31 2
(4 .3 .60)
;\ t the fir st level. a set of two secoJld-order adjoint equations of the form
(4_3.49) and two subsystem state equations as in Step 3 of Algorithm 4.2
the fourth-order Runge- Kutta method and initial values
(\" (I) cc \ - () .5]
I ().5 '
(4.3.61)
\
1l.75]
(\,(1 ) = 0.75 ' z2(1) = G2I X I (O) = l
\\ere '(l ln'd. At the s<:cond level. the interaction vectors [0:
11
(1),
and w<:r<: predicted using the
reCllrsi\'l' rdations (4.]. 56). and at <:ach infurmation exchange it<:ration the
t('t :J1 interactiull error (4.3.5X) was eva luated for 6.1 = 0. 1 and a cubic spline
inteq)(\iator program. Th<: interaction error was reduc<:d to 3.5113456 X 10 (>
ill six it erations. as shown in Figure 4. 10. The optimum outputs for
( ', = (i I) control signals \vere obtained. Next, for the sake of compari-
S(ln , the original system (4.].59) was optimi7.ed by solving a fourth-order
tillll' -V; Hving nwtrix Riccati equat ion by hack ward int egration and solved
fnrY, (I), i = l.2.],4: and lI , (t), ;=1.2. The output s and control
sign;J1s f(lr buth hierarchic;J1 and exact centrali zed cascs arc shown in Figure
4. 11 , Note the relativel y close correspondence betwecn the output s for the
(lriginal C\)upled and hierarchical decouplcd systems. However, as one
\voldd ,'-"pccl. the two control s are different. When the decoupled the
COlltwl signals are partially dosed loop and partially open loop,
(4.3.62)
For the overall system, a complete feedback structure results :
u(I) =- F\(I) (4.H13)
10
10" _
c:r:
10-
2
c:r:
w
:.<:
o
f:
'-.J
..,'

10-
3
6:
10--
4
\
\
\
____ ____ IIL-__ -L _____ 11____ -L __ _
234 6
7
ITERATIONS
125
Figure 4.10 Interaction error vs iterations for the inter;'etion prediction in Example
4.3.3.
j"b Hlcrarclucal Control of Large-Scale Systems
(/]
w
(/]
z
1.0
o
-- 1.0 ,-
(/]
w
0::
f-
:::>
r::
:::>
o
-2.0
- 3.0
',",

----------
--.." --
-- "- .
- .. - .. .. - .. --
.
)' I
.\'1
,
.\'2
- -Y2
__ ... _L _ .._L ---.l .. __ __
o 0. 1 0.2 0,3 0.4 0.5 0.6 0.7 0,8 0.9 1.0
TIME
(a)
Figure 4.11 The optimal (centralized) and suhoptimal (interaction prediction) rc-
sponscs for Example 4,3.3: (a) outputs, (0) controls.
Now let us consider the second example.
Example 4.3.4. Consider an eighth-order system
- 5 0 0 0 0.1 - 0.5 - 0.009 3
() - 2 0 0 - 0.29 0 - 0.3 0.48
-- 0.11 - 3.99 - 0.93 0 0.1 0 0
.X =
0 ()
0 ()
1.32 - 1.39 - I - 0.4 0 0
- 0.1 - 0.4 - 0,2 0 ()
0
x
()
0 0 0 0 - 0,17 0 0
()
0 0 0 0 0 0.5 0
0 0 0 0 0 0.01 0 - 0,11
() 0
0 0
10 0
+
0 ()
() 4
II (4.3.64)
0 0
0 0
0 0
Open-Loop Hierarchical Control of Continuous-Time Systems 127
4
3
2
' - .- . ------
--
(/)
Z
o - , _
'-------

-- .. --
-l
1
f-
Z
o
u
2
3 /
/
4
-
-
___
/
.. _ . . -lIt
---. - - 112
.-.-112
5'---__ -'- 1 _ _ -'---_---'- 1 _ _ __ __'_ I __ --,- I _ _ -'
o 0.1 0.2 0. 3 0.4 0. 5 0.6 0. 7 0, 8 0,9 1.0
TIME
(0)
It is desired to use interaction prediction approach to find u*.
SOLUTION: The sys tem was decoupled into two fourth-order sub-
systems and (/=2, t1t=O.I, QI=Q2= i
4
, RI=R2 =1 were chosen. The
initial values of a;'(t), i = 1,2, and state x(O) were assumed to be
ar(t) = (0.5 I -I ol, a
2
(t)=(1 0 -\ O)T, and x(O)=
( - I - 0.5 0.5 I - I 0.5 0.5 )T. The convergence was very
rapid, as shown in Figure 4.1 2. In just four second-level itera ti ons the
interaction error reduced to 2x 10 -
4
. In fact there, was excellent conver-
gence for a variety of x(O) and afU).
More applications of the interaction predicti(;ll method arc given in the
problems section.
4.3.3 Goal Coordination alld Singularities
When the goal coordination method was discussed earlier in
(4.3. 15)-( 4.3.17), it was mentioned that the posi live dcfini te matrices S; 'were
'/
o

lJ
-r:
0::
I Ii crarcllical COlltrn\ of Large-Scale Systems
L'-l
r-
::::: III
" -


() .___________________.1 =====--- L
I 2 J 4
ITI'.RAT1()NS
Fi!!l1H' U 2 The in Icract inn eIT(lr liS i tera t inns fnr (he eig, h ( order sys telll of Exalll
"il- ,U .I\
illl1(H.llIl"l'd ill th e C(lst fun ction (4.3.17) to avoid singularities. To see thai
thi s is ill fact the case, let us reconsider the problem of minimi l.ing L, 111
(4.3. 24) suhj ec t to (4.:1. 15). Lel the ith subsys [clll's Hamiltonian he
I I, (' " II" p" n; ) =, 1/ 2\: ( I) Q,-'", (I)
1 1/ 211 : (I)R,II , (I)-1- 1/ 2:' /(I)S, Z,(I)
N
-I IY;\ -- L 1\ ; 1(;"X, I I),/( ; I,x, + n,lI , +

Opcn-Loop Hicrarchical Control of Continuous-Timc Systcms 129
As one or the necessary equations for the sol LI tion of the i th su bsystel11
problem at the first lcvel, we have
(4.3.66)
or
Z; (t) = - S; - I ( C/p; (() + (x: (t)) (4.3.67)
whcrc a singular solution arises if the Z/C /)S;Z; (t) term docs not appear in
the cost function. Here two alternative approaches are given to avoid
singularities at the first leve l. The following example illustrates the two
approaches.
Example 4.3.5. Consider the following system:
;'1 = - XI + x
2
-I- /II' xl(O) = X
ln
i: 2 = - X 2 + u 2 ' X 2 ( 0) = X 20
(4.3.68)
It is desired to find (u
l
, u
2
) such that (4.3.68) is sati sfied while a q uadratic
cost function
J = 1/2 t ( x + xi + II f + /I n dt
o
(4.3.69)
is minimi zed via thc goal coordination method.
SOLUTION : From (4.3.68)- (4.3.69) it is seen that the system can be decolll-
posed int o two first-order subsys tems,
5; 1 = - XI + III + Z I' .\1(0) = X
IO
}; 2 = - X 2 + u 2' X 2 (0) = .\ 20
with interaction constraint
The problem in its present form has the following Hamilt onian :
II = (-ixf -I- ;lI f + lYZ
I
- (U
2
- PI XI + PIUI + PI:'I)
-' (I . 2 -I- I 2 . I- . )
T 2\ 2 2 II 2 - P 2 ' \ 2 - P 211 2
(4.3.70)
(4.3.7 1)
(4.3.73)
in which the intera ct ion variable appears linearl y. Thus, th e appli cation of
goal coordination fo r the present formulati on would lead to a si ngu lar
problem, since 2 1 appears linearl y in (4.3.73). The followin g system reformu-
lations of the problem would avoid singularities.
4. ::I.::I .a Reformulation I
Bauman (1968) suggests rewriting the interact ion constraint (4.3.72) in
ljuadratic fo rm,
1 \' ) lIicrarchi cal Control of Larg,c-Scalt: SyS ll' Ill S
\yi1ich \\'( )\lIt1 givc the foll (l wing necessa ry conditions for optimalit v <II tl,e
fir sl k n' l:
t) lI lIl / nll l II, 1 PI ' () = iJ lIl / (I :' 1 = 2HZ
I
+ /) 1
-, j) I ill 'I / il x I co .\ , - PI ' /) I ( I ) = ()
\' , iIJJI / ()p, = - XI + li
l
+ " I ' .\1(0) = x II)
I' ll ' fir st sUhSYS tl" 1l
() = = 11 2 1- /) 2
- jJ
2
= (J ll,/ilx
2
=x2 - P2' P2(1) = 0
-' 2(O) =.X20
(4.3. 75 )
(4.3.76)
1'(1(' tlie sl.'l'Ond suhsys tem. Arl er thc introduction of the Ri cca ti formulation
(4 .. 1 75 ) alld (4.3.7(,) will lead to
\ , ( 1 ) - { I -I (If I / It ) k I ( 1 ) } .\' I ( 1 ) , \' I (0) = \. )()
:. 1 ( I ) = - (x) k , ( 1 ) .\ I ( 1 )
11 , (1) = - " 1( /)\ ,(1)
.\'2 (/) = - (I +k
2
(/)) X
2
(1), X
2
(0) = X
20
( 1) -- k ( 1 )\' 2 ( 1 )
(4 .3.77 )
(4 .3.n)
" ' I! nl' k, ( 1) is the ilil time-varying Ri cca ti Illatri x. The
('( l(lrdin;lli PIl at the sccolld level is achi eved through thc following iteratiol1s:
H' I I ( 1 ) = ttl ( 1 ) I- fill I ( e( 1 ))
<'(I ) =
(4.3 .79 )
(hi s rdorillul at ioll w(luld avoid singul ariti es hut makes the second-level
it erati (l l1 s Cl1 nvcrgcnce vcry slow.
4 .. Reformulation 2
Singh cl ;d, (I CI7(,) sug.gcst ,111 alternative formulati on whi ch not onl y avoids
sill gll\ :,!ilics hilt a\sll gives gO(ld Cl1n vergcllcl'. The procedure is hased Oil
<;f )"'ill g tn! \' ill terms (11' the int eraction vector z and it into the
C(ls t fllllL'ti(ll1 : i.e .. Z G ill hc written in general as
(4 .. UO)
\\ here (; ,Slimed to be nOllsingular and the rdormulated Hamiltonian is
11() = 1,,' (/)V\2I1z(I) + \1I
1
(/)RII(I)+ly' i:
-- (rIG.\' + p'( ;Ix -I- fill + Cz ) (4.3.RI)
I'l l! Ill c exa mple under considerati on. the matri x. C; is singular, but "
Open-Loop Ili cra rchical Control o f Continuous-Tilllc SyslClll s
solution ca n still he found. J-Jere the Hamiltonian is
1I( . ) = + + az , - IXX
2
- PI" ; -I- PIli , + p, ",
+ lz;- + - P2X2 + P211 2
with first-I evcl subsystem problems bcing
0= aH/Oll
l
= lI, + PI' 0 = fJ H/r7:.
I
=" , +a + p,
I ] I
- jl , = r11f /iJx , = x, - p" p,(I) = O (4 .3.83)
and
0 = OH/Oll
2
= lI 2 + P2
-- /1
2
= a H / (h 2 = - a - P2, P 2 ( I) = 0 (4 .3.R4)
.\: 2 = () If /0 P 2 = - X 2 -I- U 2 ' X 2 ( 0) = .\ 20
The second subsystem can be solved immediately, since the P2-costate
equa tion is decouplcd from X l and can be solved backward in time and
substituted into the X
2
equati on, which would, in erkct , mcan that Lh e
solution of a Ri eca ti equation has been avoided for thi s particular example.
For the first subsystem, however, following the formul ati on of first-I cvel
problems in interaction prediction (4.3.40) - (4.3.5 1), both a Riecati and an
opcn-loop adjoint (compensa ti on) vee tor equati on sll ch as (4.3.48) and
(4.3.49) need to be evaluated. For thi s exa mple, thc first subsystem prob-
lem is
.\ I ( 1 ) = _. ( I -I- 2k I ( 1 ) ) X 1 ( 1 ) - 2 g, ( t ) - n ( 1 ) , .Y , (0) = .\ 10
k,(I) = O
gl( /) = (I +2k
l
(t))g,(t)+ k,(I) n(/). gl ( l )=O (4 .3.85)
UI(I) = - kl(I)X,(t)+ g,(t)
where two differential equati ons for ",(I) and ;:;1 (1) must be solved back-
ward in time. Thus, whil e no auxiliary equati on needs to be solvcd for the
second subsys tem. two such equati ons should be solved for the first subsys-
tem. In general , this reformul at ion would requirc thc solution of
.k= Ax -Sp - GQ - '(p +n), .,(O) =x
o
jl = -- C/'a + AijJ , p(lj ) = O
II = - R - 'B Tp , z = - G Q - '( p + IX)
(4 .3 .86)
which indicates tilat thc cos tat c vector p equation is deeouplcd fr0111 .\ and
ca n hc solved backward in time (eliminating a Ri cea ti equati on) and
subSt ituted in the lop equat ion to find x. Since the iI. }3, Q, and R matri ces
hlnck -di ;H>nnal nrnhIPrn (41 X6) (' ;111 11<' ti ('('( ', mnnSt, d inln N sllh<: v<: l " lll
IIierarchical Control of Large-Scale Systems
I' r(1h1clIlS. prp\'idcd that the terlll ( X' (/)V FQ Vz (l is separabl e in z where
I ' '7 (; I,
(1.4 ( 'Iosell -I ,001' II i<'rarchkHI Contwl of COlllillllous-Timc Systellls
I il l' 1,Ist se,: tiPIl dealt with 0pcll -Ill(lp lli era rchi cal control of continuoll s- timc
, \', tCI11' ill ,,''' ich tli t.: control depended Oil the sys tem's initial cond iti nll
,I ( 1,, ). 11 11,' SI: h': IlIl' Iwn lTs t tn a closed-loop structure was the int eraction
l" ('" Ii di(l n Ill ,., th(ld whi ch n:slllted a partia ll y closed-loop strll ctlll' e wit h all
opell -I(l(l !1 cOlll!1Pllent ",hich still depended nn ,Y(lII)' Although one may
,,1 " <lYS hl' ahlc to ll11:aSIIIT ,\,(1
0
), hy the time all open-loop con trol is
c: tl clIi alcd alld <lpplit.:d to tht.: system. the inili ,d stat e has IllOSt likel y
,h;1I1 )2,nl. thll s resulting ill lillpredictahle and undesirahle responses, I t is
Ihcn: rore \\'orthwhile to C(lllstrul'l closed-loop control la ws whi ch an.: indc-
IWlldent ('I' Ihe initial stat c (Singh. 19XO). The 1110st likely case in whi ch such
a c(l lltnli structure is p(1ssihl e. as in nonhierarchi cal systelll s. is thc lincar
quadratic regulator probl em. Many auth()rs have considered the prohlem of
dosed-loop cOlltrol of hierarchic;d sys tems. Mesarovic et al. (I nO)
gL's ted a sllil uptill1al structure whi ch had an adaptive feature as the sys tem
Sage and hi s associates (Smith and Sage, 1973; Arafch and Sage,
11.) 74:1. il) 1I< lye used a similar suboptimal technique for the [ilter problem.
01 hers (Cil ene\'caux. 1972: Cohen et al. . 1972, 1974) have considered the
l'f(lhlcm alld sugges ted either a " partial" feedback controll er or "complete"
r" l'( lh;ll'k cnntrnls wh ich invol ve off-lin e calculations of the overall coupled
I"rge,sl',ll c sys tem. An atlr;lct ive extension of the partial [eedhack algorithm
"I' ('oil cll l' t al. (1 974) whi ch provides " complet e" closed-loop struct ure
hased on nrf-line c,llculations or feedback gains and their on-l ine implemen-
tati on i , due to Singh (19RO). Another attemp t along thi s line, due to Siljak
(Jild SUlldareshan (Silj ,lk and Sundarcshan, 1974, 1976a, b; Sundares han,
19'77: Sil,iak. 197R). is bascd on (1htai ning local feedback controllers for each
StlhS\,s t"lll (Ill the first level hy igllnring the int eractions: then a glohal
l'olltr(llkr ;11 the sCl'(l nd lew l is applied tll minimi 7.e the interaction errors
"" d i 1111'1 (l\\: the (1nrllrll1;lIl Ce. "'"" is l11 et hod elll phasi l.cs the st ru ct u 1' ,11
perturhations cat cgnri 7cd as " benefi cial. " " nonbendicial," and " neutral"
illlCl' c(1 I1IllTti ons and their corn.:sponding loss of perrnrmance.
III Ihi s sectinn Singh's (19XO) extension o[ the interaction predi ction
apIHl l<1 ch and the feedhack C(1ntrol scheme of Sil jak and SUlldareshan (1974.
I
l
17 (' a. il) alld Sundaresllan (1977) hased 011 structural perturhation along
" 'ith Il\I1l1l'J'i cal exampl es are prese nt ed.
4.4.1 ('/o.l' (' d - / ,(}(}fI Co//trol oia ///I emcfi()// Predicti u//
111 II1\' I)f('vi(lli s sl'l'l i(lll . till' i ntcr,ll'lioll predict i(lll ill it s gcncra I se ns,' was
;"" ",(II '".! ", ;Ib :I n!lrl; :1) ,In,,.. d-)nnn (' l1 lltroll er ;lI1d an onen- Ioon C( " "I)O-
CloscJ -Loop Hierarchical Control or Continuous-Ti me Systems
133
mentioned ea rlier, that thc open-loop component depcnds on the initial
state of the system and there is no apparent possibility [or on-line imple-
mentation. To see this dependency of the open-loop component. let us
consider the differential equation for thc adjoint vector g;(t) in (4.3.49) and
li se (4.3 .55) to eliminate D:,(t) and Z; (I ); i. e.,

. T '
k.(I) = - (A, - S;K;(l)) g;(I) - K;{I )C, L G,;Xj (l)
j = 1
,\'
L Gj;C/( K
j
( I ) Xj ( I ) - gi ( I ) ) (4.4. 1)
; = 1
which indicates that the vector K; (t) depends on the states of all the other
suhsystems and hence the initial statc x (I,,) . Thc foll owi ng theorcm due to
Si ngh (1 \)1)0) relat es the open-loop component
(4.4 .2)
to the state x ( I) for the overall system which ean be used in a hi erarchical
structure of a regu lator problem.
Theorem 4.1. Tize open-lool' adjoin 1 vecl ur g(l) rlnel slal e x ( I) ar e refuled
Ihrough llze followi ng tranl/orlllation:
g(I) = M(tj ,t)x(t) (4.4.3)
PROOF: Rewriting the adjoint equations (4.4.1), the overa ll system's adjoin t
eq uation becomcs
g( t) = - (A - SK( I) + CG ) T g( t) - (K( t) CG + GrClK (t )) x (I)
g(l j ) = O (4.4.4)
which can be represented in terms of its homogeneous and particular
solutions.
g ( t) = (I) I ( / , /" ) g ( 1 () ) - 1'<1) I ( t , T ) ( K ( T ) C G 1- G 'e Tf{ ( T ) X ( T ) ) d T
(0
(4.4 .5)
where ({)I(I, (
0
) is the "state transition" matrix or (11 - SK( 1) - CG)T. Note
al so that K(t) is a block-diagonal matri x consi sting of subsystems Ri cca ti
matrices. i.e. , K(I) = cliag{ KI(I) , ... ,K;(I), .... K,v (I) }. However. for the
composite sys tem,
.\:(1) = I1x(I) + BII(/)
(4.4.6)
with stalldard quadrati c cost
.1 = . . tty/(1 )Qx(t) + 1I1( I) NU(I)] ell ( 4.4.7)
1"llcran.:JucaJ Control of Largc-Scale Systems
the dosed-loop optimal control system is well known:
x(t) = (A - SP(t))x(t)
or
(
(4.4.9)
where P( t) is the n X n-dimensional time-varying Riccati matrix for the
composite system and <1>2(1, to) is the state transition matrix corresponding
to the feedhack system matrix (A - SK) and S = BR - IBT. Now if we
substitute x(t) from (4.4.9) in (4.4.5),
g(t)=<lll(t,tO)g(to)- 1'<I>I(t,T)(K(T)CG
to
(4.4.10)
U si ng the fi nal condi tion g(t I) in (4.4.4) at t = t ( and maki ng use of the
properties of the state transition matrix, (4.4. 10) can be used to solve for
g(to ):
g ( to) = <I> I ( I (), t I) 1'1 [<I> I ( t I' T) ( K ( T) CG
to
(4.4.11)
By moving the term <I> I (to' t I) inside the integral sign and taking advantage
of product property of transition matrices, (4.4. 11) is rewritten
g(t o) = M(t
l
, to)x(t
o
) = {{'<I>I(l
o
, T)(K( T)CG
to
(4.4.12)
or
(4.4. 13)
which gives the desired relation. Q. E.D.
A corollary of the above theorem can be stated as follows. For the
time-invariant case as II -400, constant A, B, C, G, Q, and R matrices and
time-invariant g and K, M hecomes a constant transformation matrix (Sage,
1968). This corollary is used to find an approximate feedback law for the
open-loop component.
The relation (4.4.13) is a sound theoretical property, but from an imple-
mentation point of view, it is not very desirable for a large-scale system
because the overall system's Riccati differential equation must be solved
and that defeats the original purpose of finding a control via hierarchical
control. Singh has raised the point that for the time-invariant case, M
can be computed easily, since near t = 0, M is constant while x and g are
Closed-Loop Hierarchical Control of Continuous-Time Systems 135
not. Therefore, it is suggested that if the first 11 values of x(t,,) and
g( t k) for k = 0, 1, ... ,11 are evaluated and recorded, M can be approximately
given by
M=GX -
I
(4.4.14)
where
G = [g ( (
0
) : g ( I I): .. . : g ( I II) ] , X = [ x ( lo) : x ( I I) : .. , : x ( t ,J ]
(4.4.15)
and the inversion of X is done off-line. Note that if a time-varying M is
desirable, it is possible to solve the problem wi th fl initial condi tions, i.e ..
x(to),X(tO+I)"'" and form nXn time-dependent matrices G(t) and XCt)
to find M(t) for each integration step. In summary, the resulting control for
the composite system can be formulated by
u = - R - I B T Kx - R - I
= - R - IBT(K + M)x = - Fx (4.4.16)
It is noted that the above gains are all independent of x(t o), and the
matrices R, B, K, and M are obtained from decentralized calculations. The
following example describes the above feedback law.
Example 4.4.1. Let us reconsider the system in Example 4.3.3 with II = 4.
G 12' and G
2I
matrices switched. It is desired to find a feedback gain
matrix F.
SOLUTION: The decomposed system Riccati matrices at t = 0 are
K =[4.440 0.093] K2=[ 2.9067 - 0.1010] (4.4.17)
I 0.093 0.498' -0.1010 0.7522
The problem was simulated on an HP-9845 cOinputer using pASIC. After
six iterations of interaction prediction at the second level, G and X were
determined to be
[
-0.78
G=IO - I -0.73
-0.69
-0.66
-2.13

-2.20
- 2.16
-1 .38
- .20
-1.02
-0.85
1.241
1.28
1.29
1.24
[
1.0 -0.500 -1.00
0.100 J
X= 0.77
0.59
0.45
-0.510
- 0.516
-0.516
the partial feedback matrix M to be
- 1.08
- I.I7
-1.28
-210.14
-0.117
0.127
0.135
(4.4.18)
(4.4.19)
r
63.35
M=GX -
I
= 66.72
77.32
- 221.3
- 258.4
229.97
242.26
285.52
.66
-87.40 (4420)
- 82.881
-104.2 ..
65.70 - 218.4 - 86 .70
1_\ (, I licrarchi eal Conlrol or Larg,c-Scak SySll' IlI S
"nd Ihe (l\Tr,, 1\ apprnxi111'1k feedba ck gam matri x hased on hi erarchical
u lnl rill l(l he
N IB/( K -I- M) = [74.S
27.5
- 232 .1
-- 9I. R9
254.2
102.0
-- 91.6 ]
- 36 .X2
(4.4 .2 1)
4.-1.l ()nsl' d - Loo{l COll frn/I.' iu Sfrucfural Pertur /}(f(ioll
TI l\' sCl'I 'IHl1l1elhod of c(ll1tml or a hierarchi ca l sys tem di scussed
here is 0 11 the fe ed hack control suggested hy Siljak and
SI11HI :ll l'S\i;1I1 (\474. \Y7(' ;1.h) ;1I1d ex tensions by Sl1llllaresh;\1l (19Tl ) h;lscd
(' 11 <;1 111 1."1'1 1;11 j1er tlilhati llll s due to the inl eractions ;lInong suhsystel1l s in ;J
l;l1 12.<'-- s,, ;1I <_' Sy<; lelll . Slich 1'l'rlurhatill1\S 1ll ,lY ve ry well occur d,uring lhc
Pl'n:l1illll pf tl1<.' svs lelll , <llld the h:1Sil' i1\ilial issue addn:ssl' d hy Silj,d, al1d
SI111l1 ;l1 l's h:J 11 (197(1:1.h) is that assul1ling each suhsys k11l ha s its own inde-
pendent l(l cal controller. how reliabl e the overall system performance will be
in the 11I l'Sl' 1\ Ce of structmnl perturhati oll s. /\ classical example is a "power
1',,"1" ill \y hidl each pOlVer C\lllipany (suhsys tem) is for the load,
In'q uell l' Y rcgulati(1ll , and power generati on within its own regiun while
tIHo\! ' ,h li c lines it can exch:mge power with other companies (subsystems)
rcslIltill tc in va riations in l)(1wer among the companies which would affect
tlie entire sys tem' s (wer:"1 performance. Each suhsystcm has a feedback
C( 'I11TOI consisting of a " local" component obtained Ihrough
sllhs\'stClll ca1culati(1ns :1I1c1 a "global" component which minimi zes int erac-
ti (11l -c lIPrs all d possihl v improves the (lVerall syslcm performance.
( 'ol1 sid','r a large-sca le linear time-invariant syst em described by N su hsys-
tems:
.,,-
\,(I) o=;l ,\,(/) -I- IJ,l(,(I)+ L (; ,/X/ (I), x,(fo) =XjO (4.4.22)
/ ,- I
1' (' 1 i I . . : .. __ , N. where all matrices aTC dcfi11l'd earlier. Each suhsys tem ha s
:In illl l1l<.' .] iatc gLla l (If fil1ding a " local" controller {( ,'(I) which minimi zes an
as>(Wi :l1 cd lju:Hlrat ic ("(lS i
.I, (I " . . 1, (/
0
), {( , ( 1
0
)) = J OY; (.\ ; ( I )Q,", (I) + U;(f )R,llj( ()) cit (4.4. 23 )
I n
\\ hilt- s: 11i s fvi ll g (4.4.22) ", ilh the intnactiol1 111:1lrin:s (',/ set to I nn. 11 is
" SSl JlIl"" IIi :ll :dtlinll gh (,:ll' h slIhsVs tc' 111 is il1dcpcl1dcllt , they have :1 "go:d
h;lII1HlI1 Y" in mini11li;.ing the oVl'1all system cost functioll
N
.I ( I () .\ ( 10 ) , {( ( I (l )) = L .I, ( I () . . \ j ( I () ) . II j ( III ) )
I
(4.4. 24)
Closed-Loop II ie rarchical Conlrol of Continuous,Time Sys tems 137
Furthermore, it is assumed that the system possesses a complete decentral-
ized informati on structure and each subsystem pair (A j' B,) is completel y
controllable. For the decoupl ed subsystem,
x ;C/) =Aj-'dt ) + B,uj(t) , i = 1,2, ... ,N (4.4.25)
and cost (4.4.23), the decentrali zed optimal control is given by
uj'( /) = - R,- iBTKjx,( t) = - PjXj (t) (4.4.26)
where K, is the symmetric positive-definite soluti on of the al gebraic matri x
Riccati equation (AMRE)
(4.4.27)
where V; = I3J?" j- i/Jr, the va lue of the corresponding optimal cost, is given
by
(4.4.20)
for i = 1,2, .. . , N. Let the optimal performance function of the centrali zed
sys tem he denoted by J(t IJ' x (lo' The decoupled system' s cos t
N
J*( /o' X( IO)) = L J /t. (lo, Xj(IIJ)) (4.4 .29)
i = i
may, in general, be greater or less than 1"(to' x(r,,), depending on the type
of interconnection among subsystems. In fact, as Sundarcshan (1977 ) point s
out and as we will see shor tl y through the followi ng theorem, there is a
useful class of interconnections for which J *( .) == 1"( '). l3efnre we introduce
the theorem. the following definitions are given:
Definition 4,1, For a large-scale system consist ing of N subsystems with
overall and decoupled performances 1"(.) and J*(.) , respectivel y, an
interacti on G={g'i}' i,j=I,2, ... ,N, is said to be "nonbeneficial" if
J*(- ) < J
O
( .). -
Definition 4.2. An interacti on G is said to be ir J*(.) > 1"(.).
Definition 4.3, The interaction G is said to bc " neutral " if J*(.) = .f 0(' ) .
Theorem 4,2, Let (Ill II X 11 block-diagollal matrix K = block-diag
{ K I' K 2 '"'' K,y}, where 1\" i = L 2,. '" N, is the positioe-defilli te srllllll etri c
sO/lIl iOIl of ilh slIhs)"slClI1 A/,l'iRE (4.4.27). Theil tli c (iDem/! S\'stCIII 'S Olitilll!ll
I ,cr/iil'l)/(/I/ ('c i I/ric. , J" (lll d the dccollplei/ SI 'stelil optilll(l/ pcr/i
'
nJ11lI/cC .1 * (Ire
('1//((// il (ll/ri Ol/Z)" if the inleraction IIl!Ifri_\ C; = {g,,) , i, j = 1, 2, . . . , N, ('({/I he
jilctori:.cc/ liy -
G=SK (4.4.30)
II'/zere S is a skelV-,IYlIIlII ctri c matrix, i .c., S = - ST.
I
Ilw l'I'<,,,f PI' til l'pr CIll. dill' to SlInd:lresi1:11l (1977) , is givl' ll he low.
"I(( lor : I.l'l II " , f1 matri,\ ,. ' he the solution of the following /\MRE:
(A + (j) IF + 1-'( A + Ci) - FIlF + (l = 0 (4.4 .31)
wherc , . . ,. liR 1(11, " j Block-diag(A
I
, A
2
, ... ,A
N
), n = Block-
di:l g(Il
I
,II:, .... /I\. ), J( = Block-diag(R
I
, J( l , .... R
N
) , Q = Block-
di ag(() I'(> " "" (!\) illld .... VN)' It is clear th;lt
.f" ( '11' \( '0)) .-- 1/ 2 \ I ( t () ) /-'.\( II) ) i r F is positive-ddini te and sYlllmetric. I I' (;
s: llislies (4.4.J()). (4.4.31) rcduccs to
(A'F + r >l- FVF +Q )+(FSK+KS
1
f') = O (4.4. 32)
Si nce 1\ ,. i = I. 2. .... N. arc positivc .. definite and symmctric solutions or
(,I An). tit l' n 1\ is thc symmetri c positive-definite suilition of AMRE:
(4.4.33 )
Np\\, if (4.4 . .12.) and (4.4.33) arc comparcd, it fnlh1ws that J(
(4A. .'n ) ;l!ld is nllci sYlllmetric, Therdorc. (4.4.3())
i Ill!, li es tilnt .1"( . ) = 1/2\ / (, (I) Kx ( Ill) =, J " ( . ).
' [ he n.'Vt' rsl' is also true, If F is the symmetric positive-definit e sollition of
Ihcll by comparing (4.4.31) and (4,4,33), F = K onl y if KG +
( i' () : hellce, /\(i = _. (,"/\' = ,I.;. a skew-symmetric nwtrix. If we choose
,<; -c, /\ ,\'/\, til l' desirable condition (4,4,30) follows. Q.E.D. rn
Ihi s thl' oITI11. as will hc seell by the numerical examples. Illay be used to
find :1 l' l;JsS of int cractioll matrix patterns for which the optimum overall
SystC!ll C:tll hc achicved by applying decentrali/.ed controls ut (l), i.e"
N
.\; (I) = (:1 ; - B,[';)X,(I) + L G;ixi (t)
, I
(4.4.34)
1'(11' . . . ,N. It is Iwtcd that under condition (4.4.30) the local con-
tr"ller s Il"t (ll1ly optimize the overall sys tem but also stabilize it. II must be
l? 111ph:t silCd that if the structural perturbation of the systcm is limited to
(4 .4.31)), one ca n ignore the interactions Hlld solve N independent small-
'l': lIc l'whlcllls which pnlVitic both optimal and stahili7.ing control. llo\\, -
ncr. in gencral. (; docs not satisfy (4.4.30). and the application of u;"(1) to
thc ovnall sys tcm will result in a performance
N
)(10' X(lo)) = L ),(I
n
, x,(t
o
)) (4.4.35)

whcrc )'(/0' .\,(1
0
= J,UI)' X,(l
o
)' ut Uo)) for the composite system (4.4.34)
is greater than or less than .1* , depenuing on whether the intcraction matrix
(i is 11(lllhcndicial or hClldieial in the sensc of Definitions 4.1 and 4.2. SiIjak .
:t ntl SIIIIlI:trcsl1an (llJ7(l<l .h) have dcterlllineu \)()unds on the resulting
Closed-Loop IIicrarchical Control of Continuous-Time Systems IJ'J
pcrfollnancc suboptimality based on the interactions and have established
Illultilevel schemes fOf rninil1liz-i ng interaction errors and improving (In the
performance. This is achieved by a so-called "corn:ctive" control 11;( t) in
addi lion to the" local" decentralized control lIi( I), i.e.,
u;(t) = u;(t)+ u; (t) (4.4.36)
where /1; (1) is given by (4.4.26), while ur(t) is assul1leu to be
11'
ur(t) = - L HiiXi(t) ( 4.4.37)

where Il; i is an 111 ; X n i gain matrix for the feedback signals from thel th
subsystem to the ith subsystem. The application of ui(t) in (4.4.36) to
(4.4.22) using (4.4.26) and (4.4.37) yields
N
,\:;(1) = (A, - B;P;)x;(t)+ L (G;I - FJJJiJ' j (t)

(4.4.38)
for i = 1,2, ... ,N. The corrective gain matrices fI'l arc obtained through
BlI = B"P (4.4.3LJ)
where P = diag{ P
I
,P
2
, ... ,PN)' f-! ={ f-!; ), i,.l = 1, 2, .. . ,N. and lJl' is an
11 X /JI perturbatioIl in B, Notc that matrix lJ is block-diagonaL i.e" B =
diag{B
I
, [J2, ... ,13
N
}, whilc [JP is not. By virtue of the ahove fOfmulation of a
structu ral perturbation, (4.4.38) can be rcconstructed :
( t ) = ( A + G) x ( t ) - ( B + B I' ) Px ( t ) (4.4.40)
with A = diag {AI' A
2
" . ,A
N
} , The closed-loop sys tem (4.4.40) is influenced
through the two components, i.e., "local" BP.\ (I) and "corrector." or
"global," BPpx(t). Figure 4.13 shows a schematic for the proposed multi-
level control method. The local-global feedback control system (4.4.40) can
be utilized to give an estimate on the performance index; i,e .. once the
perturbation matrix BI' is determined, a bound on performance J rcsults.
The following theorem, uue to Sundarcshan (1977), provides the mechanism
to achieve thi s.
Thcorcm 4.3, For a nOllsingular (A + G) lI1atri.\, let a SkC1F-,ITlI1l11clric /J/(/Irix
S hc thc solution o/Ihc /ollowing lIIalrix LyaplillmHype equation:
(4AAI)
K = (S - KG)(A +G) - I ( 4.4.42.)
bc such Ihat (K + K) is a positive-de/inile matrix. Then all inpul /Ilalrix
perlllrhll/io/J J]I' gillen hI'
(4A.43)
14()
I Iierarchical Conlrol of Large- Scale
LOCAL CON! HOI..LI :({ I LOCAL CONTROLLI'.R N
Fil!Il!'p <I.U 1\ closed -loop Illultil evel sta lc regul ation sl rul'lure.
1
'
l'Ol'ir/cs (/ l,cr(orl1/(Tllcc il1(/C.\
. _ I . 1' . '. .
.1(1
0
.\(1
0
)) - :;.\ (t
o
)( K + k. ) .\(t
o
) (4.4.44 )
fill' fh e III ' C/'{/ 1/ .ITs/ell l (4.4.40) . fl.foreovcr. lir e /lpper hO/lnd 01/ lir e {Ie,!or-
/11,/1/(',' r/n' i(/Iion
fl = (.i -- .1*) j.I' (4.4.45)
il ,l! /J ' CI/ 11.\' \I'hl'/'(, A",( ' ) I/I/d A
M
(') represent Ihe
I1lil/IIl1ll111 (Jlld l1/(/xil1ll1l1l oj lire ei,e.cl/llll/ues (1/ tlr eir associated lIIatri.\
argll/)/ clll s.
PROO! ': Consider the following perturbed system:
.\( I) = ( .1 + (;)X(/) + (fl + IJI')I/(I) (4.4.46 )
:Ind kt /I X II posi tive-definite and symllletric lll ;ltrix F he the solution of
i\ 1\1 R 1 : ..
( / f l (i) /r + r( II I- G) - F/l I' 1-' I Q = () ( 4.4.47)
whne 1'1' ( If -I- n l' ) N I( H + 13 " )r. Clearly, the closed-loop control l/( 1)
..- R I( n -I res ults in a minimulll cost iXJ(/o)Fx(to) for the
fn' dhad c(l iltrol sys te!n
i ( I ) ( A + (i) \' ( I ) .. V I' h ( I ) (4.4.4X)
Ilowl'\'er. since the desired closed-loop system is of the form (4.4.40) , it is
seen that for to take on that form the following relation SllOUld
hold :
R I ( 11 + 1J ,, ) Jp = I' = R In/ K
(4.4.49)
Closed-Loop I J icrarchical Control of ContinLioLis-Time SystcIlIs
141
Since K is a positive-definite and symmetric solution of (4.4. 33), (4.4.49) can
be used twice to rewri te (4.4.47):
(4.4.50)
Thc relatioll (4.4.50 ) can hc rcwrittcn as
KG + (F-K)(A +G) = S
(4.4.51)
wherc S is a skcw-symmctric matri x which sati sfics (4.4.41), sincc (F - K)
= (S - KG)( II + G) . I is a symmetric matrix. Now if we let K = ( p - K),
then. through direct substitution. it can be seen that /J I' defined ill (4.4.43)
in fact satisfies (4.4.49); therefore. thc minimum cost is J(tn. x(tn)) =
1/2x
J
(I O )Fx(/o) = 1/2x
T
(/o)(K+ K) x (to). which is the desired cost
(4.4.44) for the overall composite system (4.4.40). Moreover. it follows that
the suboptimalit y index p satisfies (4.4.45) and is given by
p A
M
(K) / A
II1
(K)
for al1 t o and x (t o) . Q.E.D.
(4.4.52)
An immediate corollary of thi s theorem is that for a perturbation matri x
/]" given by (4.4.43), the control law
u(t) = - (p + H )x(tl
(4.4.53)
is a stabilizing control for the original large-scale system. For tbe proof of
thi s corollary. see Probl em 4. 5.
This combined global-local controll er extends the earlier work of Siljak
and Sundareshan (1974) in the sense that (4.4.53) t,lkes advantage of
possible beneficial aspect s of interconnection. The foll owing examplcs il-
lus trat e the structural perturbation method. [7urther discussion on the
method will be given in Section 4.6.
Example 4.4.2. Considcr a simple second-order system,
'\:1= 2x
1
+1I
1
, x l(O) = i
x
2
= 4X2 + 11
2
, .\' 2(0) = 0.5
with a 2x2 interacti on matri x
(4.4.54)
(4.4.55)
whose el ements arc kcpt variable for illustrative purposes. For a quadratic
cos t function
J = to ( x (t) + x ( ( ) + u ( I ) + u H t ) ) cll
o
(4.4.56)
we would like to investigate thc effects of various interconnections.
"/' ,
Hierarchical Control of Large-Scale Systems
SOLUTION: To begin with, assume that the system is completely decoupled,
i.e., G = 0, 'and hence the solutions of two independent scalar AMRE's
provide KI = 4.24 and K2 = S. 123 and optimal local controllers
[
ur(t)] = [-4.24
ui(t) 0
( 4.4.57)
with optimal decoupled performance index
2 I
J* = E 1;* = 2{K
l
x?(0)+ = 3.1353
;=1
(4.4.58)
Next we consider the case of "neutral" interaction in which J= J*. Using
K = diag{4.24,S.123} and an arbitrary skew-symmetric matrix
S = [_Ob b
O
]
(4.4.30) implies that
gil = g22 = 0, gl2 = -1.915g
21
Thus, for any interaction matrix,
G = - 6
915b
]
(4.4.59)
(4.4.60)
(4.4.6 1)
where b is a nonzero constant, the performance index of the overall system
does not improve any more. To see this we let b = 1 and solve a second-order
AMRE for matrices,
(A+G)=[i R=I2 (4.4.62)
with a solution
K O = [4.23671007
0.00143700
0.00143700]
8.12243780
(4.4.63)
resulting in a performance J=txT(O)KOx(O) = 3.13438, which corresponds
to the value of J* in (4.4.58) after rounding.
Next let us consider a case of "beneficial" interaction by assuming an
interaction matrix
G = [5.5
5.5
6.2]
12
(4.4.64)
Using G, A, and K = diag{4.24, 8.123), the linear-matrix equation (4.4.41) is
solved for S,
[
0 00.62044]
S= -0.62044
(4.4.65)
and the corresponding K, K + K matrices follow from (4.4.42):
K=[=;:i (4.4.66)
Closed-Loop Hierarchical Control of Continuous-Time Systems 143
which are negative- and positive-definite, respectively. The value of J is
J = 1X T(O)( K + k )x(O) = 0.4409 (4.4.67)
which is much smaller than J* in (4.4.58). The suboptimality index for this
case is p = - 0.85933, with upper bound p - 1.334. The control for this
case is
, u(t) =u*(t)+uC(t) = u*(t)-BPPx
_ [ - 4.24 0 ] [x I (t) ] + [ - 12.34
o -8.123 x
2
(t) -12.64
- 11.88 ] [ x I ( t) ]
-23.82 x
2
(t)
( 4.4.68)
whose first part is local (decentralized) control component and the second
part is the corrective control component.
The remaining possible structural perturbation to be discussed is "non be-
neficial." Let us take an interaction matrix (or structural perturbation)
-6] (4.4.69)
Using this perturbation, the skew-symmetric matrix S from (4.4.41) becomes
S = [0 -4.64] (4.4.70)
4.64 0
and the resulting K and K matrices become
K- [-0.1337 -0.1337] K+K= [ 4. 106
- 13.92 3.48' 13.92
-0.1337]
11.604
(4.4.71)
with performance index j = txT(O)(K + K)x(O) = 6.95. This value as com-
pared with J* = 3.1353 indicates that (4.4.69) is a nonbeneficial perturba-
tion. The performance suboptimality index p = i.22 and an upper bound
p 0.674.
The AMREs were solved by Newton's iterative formulation (Kleinman,
1968), while the Lyapunov-type equations were solved by an infinite series
method given by Smith (1971). For a survey on the solutions of both
equations, see Jamshidi (1980). Now consider a second example.
Example 4.4.3. Tllis example deals with a two-reach segment of
pollution problem considered in Example 4.3.2. Consider
l
-1.32 0 0 0 j l 0.1
x= -6.
2
x+
o 0.9 -0.32 -1.2 0
a flver
where each reach of the river is represented by two state variables, BOD
(biochemical oxygen demand) and DO (dissolved oxygen). The remaining
matrices were chosen as Q = diag( 1,2, 1,2) and R = 1
2

I
" i '
lIicrarc\}i Gil C(1ntrnl of Largc-Scale Systcms
SOII I IION: t Inlier dCC\ lup\cd c\lllditi(lIlS, the systeIll can he cOIl .,idcred as
(\\'t) (lll e reach subsyQc\1ls:
. [-.1 .. '\2
x, = , - . O .. ll.
(4.4.73 )
\\ ith V, - ..ti :lg( 1. 2 ) and I. For illter:lction 1l1<ltrix Ci '-7 0, the il1<.kpcll -
den t 111 :lt Ii X R in:a ti equa t iOlls lead to the rollowing solutions:
1\ _,_ /\ C7 1 ().40JX --- 0.10)(1 ,1 (4 .4.74)
1 l l - O.IOS() o)trn
:111\ 1 ,,kn:l1traii /C(i cOlltrollers
IIi' ( I )
,,;(1)
() 11 -\ I ( I ) 1
-- O.OI056\ 2(t)
with .I" -- tl .5tl(, 5 rc.)r\,(O) - ( 1
(4.4.7:1 )
0.) )', i 7= 1, 2. The interaction matrix ror
.. .. - ...--.--.-.------------ - --------- -------
,1.14 linn' 1'C'p(1n:<cs [pr Ihc slrllclmal pl'IllII'bati(l1l in Example 4.4.3 show-
in?, n:wl qllil1l1l!ll and appnl"\illlalc "a" "e" s"lulions: (a) states, (b) outputs, and
((' ) C(l 1> I rtlls.
11
I I
.. L. _____ _ L . ________._1 _ ___ .. __.L _______ ._ .. 1., ______ _ L _ __ ___ _ L . ______ 1 _____ J _ __ J
0:; I I.) :'. 'i.l .I.'i 4 4.5
TIM!' Isec)
(a)
t
o
14
1.3
12 --
.5 -
.4
..1 --
. 1
o
-. 1 _ ....J,__ --'-_
o .5
U
O
\
I
\
. 1 -
If\
.(1 2:' .
\
\
\
\
\
\
\
"
___ 1' 1
145
. ..j.I__ --'-__ .....l... _ ___ L _ _ ___ LI __ -'
1.5 2 2.5 3.5 4 4.5
(sec )
Ih)
\
\
\
\

" ----
'-.. ------
'-- --
............ _-----
- .05 L......J. __ ......J __ -----'L... __ L.. __ ___ __ _' ___ ......J.__ _.l
o .5 1.5 2.5
TIME (sec)
(e)
3.5 4 4.5
I tl ,
(; =
()
()
0.9
()
ll irra rr hil: a l l"lJl lmlni' 1.argc Sc: 1i r
o
o
o
0.9
()
o
()
o
1.11
()
(l
() -
(4.4 .76)
II I,i( h IIl av he ll!.' lI tr:. lI . h(' ndici:lI , or ll ollhcll d ici:lI . III order t(l rind out
\I hi "h ,': 1."'.' it we need 10 lI SC Theorem 4,2 to factor (, as in (4.4.]0). If wc
I :,ke ( ; ddincd hy (4.4.7(, ) ;1Ild I( = Bl ock-di ag(K
I
thCll the cnrrc-
,\ 111 :1 lri :\ hccoll1 cs
()
0
() ()

() 0 () ()
(4.4 .n)
2.] (1.3 () ()
() .. \ 1. 1
f)
0
Iy hi l' h j " 11 (11 :1 SkCI\ svtlllll elri c l11:ltri x. il11pl ying lhal the inter:1cli o11 nla tri x.
(4.4.7(,) i, !lll i ncutral. In ll nkr to fill d out whether (, in (4.4. 76) is beneficial
Ill' 1](1 ll h ' lldicial. The(1 rCl ll 4.3 and Equali on (4.4.4 1) are used 10 fin d ,)' :
:ll1 d Ih':11 hv virtue of (4.'1.42).
f) . l m
.- 0
() I X I
- o.on
0.5 I
.- () .2
().I g I
- o.on
() -- (J . 17
()
(I .023X
(UR
.- o. on x
()
- OJ lJ 7
- (J .042 0. 142
O.2]R - () .059
- (J . I22 (J.(}07
0.30X 0. 028
-- 0. 147 0. 142
1.07 - 0.059
- 0. 122 () A I I
-- 0.077 O. 30X
(l .067 '1
-- lUX
O.ll ] 7
o
-- 0.055
0.3 17
- 0.03
0
(J055 1
0.3 17
.. - 0.1 36
() .1\]3
(4.4. 7X )
(4.4.79)
; 111<1 11 )1' \: tllI e of 11.7 19J7 fm\(O) c (I 0. 5
(l .5 )' , whi ch indi -
(' :lt e" Ih:lI the (l ri ginal int erac ti on l11:1tri :\ is nonhell eri ci:ti . The
(f.'l lrt et'l llr) controll er is givell by
fI ' (I) -
n I ' f 'y = .... I (l ,I
- 3. 55
1.77
-- -- - -
-- 15.2.
1.32
0.93 I
- 4.(,:1
1
._ .. __ 1
- 4 I
3.45 1
- 12.4 3. 25 I
n.R_. _ -::..0 :_2 __ ( ...
4.2
- 24. 2 tl .6]
(4.4.XO)
\\ ith ; - () .7 19.l7, the perf(l rlllall Ce slihoptillwlil y index f> = O.2.l)(i (J . and
"",,,, , 1, "'111< 1 " r Il <. I n . The above nerfortll ance index IV" S also checked hv
ll icrarchi cal Con trol of Di serete-Timc Systcms 147
the ori gin,,1 sys tem's cost functi on by solving a fourth-order J\ MRE, whi ch
provided the overall optimal system performancc index as J " = 0.7132,
indicat ing that the proposed decentrali zed-global (corrcctor) control pcrfor-
Illance index is within less than 1 % of the overall centrali zed optimum cost.
To check the ori ginal system's optimal control solution versus the com-
bined decentrali zed global solution, the fourth-order syst em (4.4. 72) was
solved llsing
and
U,, = II * + lI
c
wi th x (O) = (1 , 0.5, 1,0.5)T and output matri x
c = [6
0.25
a
()
1
(4.4. 81)
(4 .4.82)
() ]
0.25
(4.4. 83)
The rcsulting x ;(t), i = I, 2.3,4; yJ(t ), for j = 1, 2 appl ying exact optimal
control (4.4. XI) and approximate oplimal cont rol (4.4.82) responses for
o t 5.0 and a step size 61 = 0.1 were found to be very cl ose. The slates
and outputs of the fourth-order system utili zing two controls are undis-
tingui shahly close. Figure 4.14 indi cat es that the structural pertu rba tion
closed-l oop cont rol considered here is a good approximal ion. Further
commcnts on this suboptimality are given in Sccl ion 6.5.
4.5 Hierarchical Control of Discrete-Time Systems
Thus far, the multil evel hi erarchi cal control has been used for linear
stati onary continuous- time systems. In practi ce, however, ma ny systems are
ncither linear nor continuous-t ime. In fact, many pract ical engineering
systems, such as traffi c control, wa ter rcsources, and manufacturing
processes. are both di screte and delayed in naturc. Duc to the import ance of
such appli calions, many researchers have made signi ficant contri butions in
appl ying or extending hi erarchi cal control to both l1 0ndelay di screte-time
and time-del ay di screte-t ime systcms. In this secti on the extensions of
hi erarchi cal control to such sys tems in both linear and nonlinear for ms arc
considered.
4.5.1 Three- Level Coordinat ion for Discretc -Ti me !:>)'sfcli/s
Thi s secli on deals with a three-l evel goal-coord inat ion st rategy suitable for
di screte-time sys tems and their time-delay modifi cat ions. Such a stra tc!!.v
was first proposed by Tamura (1974, 1975 ) and trea tec! further by
(
148 Hierarchical Control of Large-Scale Sys tems
Consider a large-scale linear discrete-time system
x(k+l) = Ax(k)+Bu(k) , x(o)=x
o
(4.5.1)
where the usual definitions hold for x , u, A, and B. The optimal control
problem is to find a sequence of discrete-time control vectors u( k) , k =
0, 1,2, ... ,K -I which minimizes a quadratic cost function
I I K - \
J = "2 xT(K)Q(K )x(K) +"2 L {xT(k )Q(k )x(k) + uT(k) R (k )u(k)}
k=O
(4.5.2)
while (4.5.1) holds. Following the decomposition procedure discussed in
Section 4.3, this problem can be reformulated by minimizing
N{l lK- l
J= "2
x
;"(K)Q;(K)x;(K)+"2
+ z;(k)S;(k)z;(k)+ U;(k)R;(k)U;(k)]} (4.5.3)
while subsystem dynamic state equations
x;(k + 1) = A;x;(k) + B;u;(k) + C;z;(k), x;(o) = X;o ,
i=I,2, . . . , N, k=0, 1, 2, . . . , K-I
and interaction relations
N
(4.5.4)
z;(k)= L Gijxj (k) , k=O, I, . .. , K - I, i=I,2, .. . ,N (4.5 .5)
j- I
are satisfied.
Following the decomposition of the Lagrangian and the duality of
two-level strategy formulated by (4.3.21 )- ( 4.3.22), let us define a dual
function
q ( ex) = Min L ( x, u, z, ex)
x, u.z
where
N
L(x,u, z , ex)= L L;( x pu; , z; , ex;)
; = 1
(4.5.6)

Hierarchical Control of Discrete-Time Systems 149
Through this decomposition, as in the continuous-time case, one can pro-
ceed to apply the two-level iterative procedure to improve on the values of
the Lagrangian multipliers ex; using the gradient of the dual function q( ex) as
in (4.3.25):
N
\7
a
q(ex)la,=ai = z;(k)- L G;jxj(k) = e;(k) (4.5.8)
j = I
for i=I,2, ... ,N; k=0, 1, 2, ... ,K-1. Although one may proceed to use
(4.5.8) and a few iterations of conjugate gradient or steepest descent to
obtain a feasible and optimum solution, our objective here is to present a
three-level modification of the problem due to Tamura (1974, 1975).
The essential point in this modification is to recognize the fact that the
"first-level" solutions of a two-level structure can be obtained by utilizing
the concepts of duality and decomposition instead of solving N independent
problems of minimizing L
j
defined by (4.5.7) and subject to (4.5.4)-(4.5.5).
At this level, the sub-Lagrangian for every subsystem is further decomposed
by the discrete-time index k, thereby reducing a "functional" optimization
problem at the first level of a two-level structure to one of a "parametric"
optimization at the first level of a three-level structure. This decomposition
was mentioned in Section 4.1 under" time" or "functional" division of the
control task. It can also be considered a "temporal" decomposition versus a
"spatial" one in Section 4.3.
In order to determine the optimal strategy of this three-level structure, let
us define a dual problem for minimizing L;() in (4.5.7) subject to (4.5.4) as
follows :
p ( f3;) = Min L; ( x;, U;, Z;, (X i , f3; ) }
Xj, Uj , Z;
(4.5.9)
where
+ zT{k )Sj(k )z;(k)+ uj(k )R;(k) u;(k)
+ f3r(k)( A;x;(k) + B;u;(k) + C;z;(k) - x; (k + I))]
N
+ exiT(k)z;(k)- L exf(k)Gjix; (k) (4.5.10)
j = I
and f3; is the i th subsystem Lagrange multiplier vector corresponding to
dynamic equality constraint of (4.5.4), and all the other variables and
parameters are defined earlier. The last constraint to be determined is the
initial condition, also given by (4.5.4). In other words, the dual problem to
minimizing J in (4.5.3) subject to (4.5.4)-(4.5.5) is maximizing the dual
function p(f3;) defined by (4.5.9)-(4.5.10) and subject to (4.5.4) at a given
() - ()* /, - 1 '") M Thp or<:>rlipnt "f n( R \ tn th P: r.nnti1111()lI s-fimp.
150 Hierarchical Control of Large-Scale Systems
case, the error is satisfying equality constraint (4.5.4), i.e. ,
'ilp,p (f3n = - x;(k + I) + A;x;(k) + B;u;(k)+ C;z,( k) (4.5 .11)
for k=O,I, ... ,K-I, and i=I,2, ... ,N, with x;(k) and u;Ck) being the
state and control vectors obtained from minimizing L; in (4.5.7) subject to
(4.5.4) for a given f3; = f3r Now let us define the Hamiltonian H;() of the
i th subsystem:
H; (x; (k), u;(k), z;(k), k) = iX;(k )Q;(k )x;(k)+ iU;(k) R;u;(k)
1 N
+ 2z;(k)S;z; (k)+ajT(k)z;(k)- L aY(k)Gj;x;(k)
j= I
(4.5.12)
for k = 0, 1,2, ... ,K -I, and i = 1,2, ... ,N. Note that without loss of general-
ity R . and S. matrices are assumed to be constant. By regrouping the last
I I
term f3r(K - l)x;(K) inside the bracket in (4.5.10) with
ix;(K)Q;(K)x;(K) and adding the term f3;( -1)x;(O) to the sum inside
the bracket with f3;( -1) defined to be zero, the functionp(f3;) in (4.5.9) can
be rewritten in terms of H;(), i.e.,
K - I
+ L {H;(x;(k), u;(k) , z;(k), k)- f3;*T(k -I)x;(k)} (4.5.13)
k=O
The minimization problem at the first level is divided into three portions:
for k = 0, k = 1,2, ... ,K -I, and k = K. For k = 0, the problem is defined as
MinimizeH;(x;(O), u;(O), z,(O),O) (4.5.14)
uj(O) . z, (O)
subject to
x;(O) = x;o (4.5.15)
In view of H;(x;(k), u;(k), z;(k), k) in (4.5.12) for k = 0, the necessary
conditions for the minimization problem (4.5.14)-(4.5.15) are
or
'ilu'(O)H;() = R;u;(O)+ BF/3;*(O) = 0 (4.5.16)
u;(O) = - Rj IBrf3;*(O)
z;(O) = - S;- I( Crf3;*(O) + al(O))
(4.5 . 17)
(4.5 . 18)
(4.5.19)
In a similar fashion, for the second portion k = 1, 2, ... , K - I, by virtue of
(4.5. 13), the minimization problem is
Mi"irni7P (H(x,(k),u,(k),z,(k),k) - f3;*T(k-l)x;(k)} (4.5 .20)
Hierarchical Control of Discrete-Time Systems 151
which leads to the following relations (Singh, 1980; Tamura, 1974):
x;(k) = - Qj l(k){ A;f3;*(k)+f3;*(k -1)+ [ajT(k)Gj; r} (4.5 .21)
u;(k) = - RjIB!fJ;*(k) (4.5.22)
and
z;(k) = - S;- I( Crf3;*(k) + aj( k)) (4.5.23)
The final portion of the first-level minimization is defined by
Minimize {ixT(K )Q;(K )x;(K) - f3;*T(K - J)x; (K)} (4.5.24)
which results in
(4.5.25)
Therefore, by virtue of this three-level hierarchical structure, the large-scale
optimal control problem defined by (4.5.3)- (4.5.5) can be solved by the
following algorithm.
Algorithm 4.3. Three-Level Coordination of Discrete-Time Systems
Step 1: At the first level, for given Lagrangian mUltiplier ai(k), f3;*(k)
sequences, (4.5.18)-(4.5.19), (4.5.21)- (4.5.22), and (4.5.25) can
be used to find x;(k), u;(k), and z;(k) for k = 0, 1,2, ... , K and
i=I,2, ... ,N.
Step 2: At the second level, x;(k), u;(k), and z;( k) of the first level can
be used along with the gradient of p C f3;) in (4.5.11) to improve
on the value of f3;, i.e.,
f3;*'+ I (k) = f3t' (k) + o/J' (k) ,
k=O,I, ... ,K, i=I,2, ... ,N (4.5.26)
where /' (k) is a function of gradient of p C f3n, and again a simple
gradient or conjugate gradient method can be used. Steps I and
2 are repeated alternatively until the optimal f3;*(k), i = 1, 2, ... ,N,
and k = 0, 1,2, ... ,K, are obtained.
Step 3: At level 3, the optimal f3;*(k) values can be used to improve on
the values of aiCk) using the gradient of q(a),
ai' + '(k) = ai'(k)-t- ;g{(k),
k = O,I , ... , K, i=I,2, ... ,N (4.5.27)
where g{Ck) is a function or given by (4.5.8).
f:t
j"(
152
Hierarchical Control of Discrete-Time Systems
153
Q
Once the above three steps are complete, the interaction errors defined by
Z>-l b>-l (4.5.8) and (4.5.11) would become zero and the optimal control of the

,-.

origi nal large-scale system is obtained (Singh, 1980). Figure 4.15 shows a u>

0::;>

......
schematic of the three-level modified coordination principle proposed by tIl >-l u...>-l
Q>-l

Tamura (1974). The above structure and algorithm is ill ustrated by the
...... >
r- followi ng example .
0-
b>-l

"'< oj
Example 4.5.1. Consider a third-order system,
....
;:J


[ x, ( k + I) H -
0
I 005: 005 f (k) 1 [ 025: 0 1 [ u, ( k) 1
b
>-.
.D
x 2(k+ l ) - 0 -0.2 1 0 x
2
(k) + 0 10 - ___

0
'"0
;3(k+l) -0.2--::"'0.1-1--=-3-- ;;(k) 0 - -1-0 u2(k) II
..,
</)
"'<
0
0..
0
(4.5 .28)
....
0..
<:
..,
wi th a cost function
....
N
;:l
0::;
t)
2 {I 9 1
0 .5
J = "2 x;(lO)Q;(lO)X;(lO) + "2 [x;(k) Qi(k )x
i
(k)
b
II
</)
...:t:
"'<
s:1
+ z;(k )S;(k )zi(k) + ui(k) Ri(k) [l i(k)] }

Z 0
2S
'LJ
(4.5.29)
oj
"".
s:1
-
0::;

:a
0

....
0
0
where
Qi(k) = 21", Ri(k) = 0.5InJ, SiCk) = I
k
, n
l
= 2, n
2
= 1, /11
1
= /11
2
=
0
U <.> , , ,
0
OJ
1, kl = 2, k2 = 1. It is desired to find an optimum control strategy through
II >
the three-level coordination Algorithm 4.3.
"'<

oJ
..,
SOLUTION: The algorithm was simulated on an HP-9845 computer using
....
.;3
BASIC and an initial condition x(O) = (2 -3 4)T. The results were generally ..,
,-,
.;3
very satisfactory, especially with respect to the interaction error between
"'<

"-<
levels 2 and 3. Typical resulting error and states are shown in Figures 4. 16 0
II
and 4.17. The interaction error between levels I and 2 reduced from 1,000 to
C;;
"'<
149 in some 50 iterations and the
convergence was not as fast as the
S
..,
second-third levels interaction errors . A
<.>
</)
-<t::
- 0
I/)
4.5.2 Discrete -Time Systems with Delays N
- >-l
-
II -.i
...:t:
In this section one of the more powerful algorithms for multilevel opti miza- ......
;:!
"'<
..,
b
-
...
tion of large linear discrete-time systems with delays in both state and
...:t:
N

0.. >-l
control is presented. The problem was first introduced by Tamura (1974, til
...:t:
0::;
1975) and has been further applied by many others (Fallaside and Perry,
0
1975; Jamshidi and Heggen, 1980; Singh, 1980).
0..
::8
Consider the following large-scale discrete-time system with delays
b
x(k + 1) = Aox(k)+ A1x(k - 1)+ A2X(k - 2)+ .. .
+A
s
x(k-s)+B
o
u(k)+B
1
u(k- l )+ ... +Bsu(k-s)
(4.5.30)
154
\60
140
...,
I
N
120
....!
w
>
w
!::!, 100
t:t::
0
t:t::
t:t:: 80
w
z
0
f:::
60
u
<t:
t:t::
W
I-
40
~
20
0
I 2 3 4 5 6 7 8 9 10 II 12
ITERATIONS
Figure 4.16 Typical interaction errors of Example 4.5.1.
Figure 4.17 Typical state trajectories for the third-order system of Example 4.5.1.
4
"'"
~
k
C/l
IJ-l
I-
<t: 0
I-
C/l
I
I
- I
I
I
I
-2 I
I
I
-3
I 2 3 4 G
DISCRETE TIME k
7
--- x2 (k)
- . - x )(k)
8 9 10
Hierarchical Control of Discrete-Time Systems 155
where A; and B;, k = 0, 1, 2, . . . ,s, arc n X /1 and /1 X m matrices, respectively,
and x and u are /1- and m-dimensional state and control vectors. The system
(4.5.30) has 2s delayed terms; hence it requires 2s discrete-time initial
functions, assumed to be zero without loss of generality:
x(k)=O, u(k)=O, - s ~ k < O , x(O) = xo (4.5.31)
Physical interpretation of initial functions in (4.5.31) for system (4.5.30) is
that the system is operating at its steady-state and receives a disturbance at
k = 0 and derives it to a known value xo' The system cost function is
assumed to be quadratic:
1 K - I 1
J = 2:xT(K)Q(K)x(K)+ L 2:{x
T
(k )Q(k )x(k)+ uT(k )R(k) u(k)}
k = O
(4.5.32)
where Q(k) and R(k) are both assumed to be positive-definite. The optimal
control problem is to find a sequence of control vectors u(O), u( 1), ... ,
u(K -1) such that (4.5.32) is minimized while (4.5.30)- (4.5.31) and a set of
inequality constraints
X min ~ X ( k ) ~ X max
U min ~ U ( k ) ~ U max
(4.5.33)
(4.5.34)
are satisfied. It goes without saying that a solution to the problem
(4.5.30)- (4.5.34) in usual "centralized" methods by the application of the
maximum principle, as it is demonstrated in Section 6.4, results in a TPEY
(two-point boundary value) problem which involves both delay and advance
terms, making the attainment of an optimum solution very difficult indeed,
if not impossible. In the literature, there are several approximation tech-
niques to deal with continuous-time systems with delay which will be
discussed in Chapter 6. Here the objective is the application of hierarchical
control via the interaction balance principle.
Following the formulation of discrete-time maximum principle (Dorato
and Levis, 1971), let us define the Hamiltonian:
H(x(k), u(k), A(k), k) = 1{x
T
(k )Q(k ) x(k)+ uT(k )R(k) u(k)}
of
i = 0
(4.5.35)
where k = 0, 1, ... ,K -1, A(k) is a vector of Lagrange multipliers at k, and
A (K), A (K + 1), '" are defined as zero vectors. In a manner similar to
previous discussions regarding Equation (4.5.13) for a given vector A = A* ,
156 Hierarchical Control of Large-Scale Systems
the Lagrangian can be defined as
L(x, u, >-.*, k) = t xT(K)Q(K)x(K)- >-.*T(K -1)x(K)
K - l
+ L {H( x(k), u(k), >-. *(k) , k)- >-.*(k - 1)x(k
k = O
(4.5.36)
Thus the optimization problem is to minImiZe (4.5.36) subject to
(4.5.33)- (4.5.34). As before, this problem can be altered to that of maximiz-
ing the minimum of L() with respect to >-.. The power behind the" time-
delay algorithm" of Tamura (1974) is the decomposition of this problem
into (K + I) independent minimization problems for a given >-. *, as in the
three-level coordination formulation di scussed in the last section, which
reduces a "Junctional" optimization problem to a "parametric" one.
4.S.2.a Problem k = 0
By virtue of (4.5.36), definition (4.5.35), and constraints (4.5.33)- (4.5.34),
the optimization problem for k = 0 is
MinH(x(O) , u(O), >-'(0 = uT(O)R(O) u(O)}
u(O)
s
+ L >-.*T(i)(Aix(O)+Biu(O) (4 .5.37)
i = l
subject to
(4.5.38)
Now if R(O) is assumed to be a diagonal matrix, the necessary conditions
for (4.5.37)- (4.5.38) lead to a set of m independent relations, each of which
has an explicit solution given by setting JH(')/ Ju(O) = 0, i.e.,
(4.5.39)
where the" saturation" function Sat u(' ) is
(4.5 .40)
and the indexj represents thejth element of control tl
j
, j=1,2, .. . ,m.
(/
Hierarchical Control of Discrete-Time Systems
4.S.2.b Problem k = 1, 2, .. . ,K-l
The intermediate problem is defined by
157
Min H( x (k), u (k), >-. *(k), k) - >-. *T( k - I)x(k) (4.5 AI)
x (k ) , lI(k )
subject to
xmin x(k) x
max (4 .5042)
and
umin u(k) u
max
(4.5.43)
<?nce again, assuming that R(k) and Q(k) are diagonal, the partial deriva-
tives JH()/ Jxi(k) and JH(-)/Juj(k) for i=I , 2, ... , n andj=1,2, ... , 111
lead .to a. set of n + m independent one-parameter equations whose general
solutIOn IS
x'Ck) Sa'. ( - Q- 'Ck)[ - )..'Ck - 1) + ,to Ar)..' Ck+;)])
u*(k) = satu{ - BT>-. *(k + i)]} (4.5044)
where the I th element of the saturation function Sat x( v) is
{
X max , { if v{>x
max
, {
Sat x(v{)= v{ if xmin,{ v{xmax.{
xmin, { if v{ < x
min
,{
4.S.2.c Problem k = K
This problem is
{ xT(K)Q(K )x(K) - A*T(K -I)X(K)} x (K)
subject to
x . ,:::: x(K) ,:::: X
mln max
whose solution is similarly given by
x(K) = Sat x{Q- l(K)A*(K -I)}
(4.5.45)
(4.5.46)
(4.5.47)
(4.5.48)
The above so-called" time-delay algorithm" can be summarizcd as follows:
Algorithm 4.4. Time-Delay Algorithm
Step 1: At level one, solve K + 1 analytic problems defined by
(4.5.39)-(4.5.44) and (4.5.48) for a fi xed set of Lagrange
multipliers A(k) = A*(k), k = 0, 1, ... ,K-1.
158
Hierarchical Control of Large-Scale Systems
Step 2: At level two, the value of "A*(k) is improved through a
gradient-type iteration
where dr(k) is a function of the error er(k), i.e.,
dr(k) = f(er(k))
(4.5.49)
= [A;x(k - i)+ B;x(k - i)] - x(k + I)} (4.5 .50)
which follows from our previous discussions, i.e., Equation
(4.5.30).
The following example illustrates the time-delay algorithm.
Example 4.5.2. Consider a simple second-order system

+[1.
5
O][U\(k)]+[O 0.25][U\(k-I)] (4.5.51)
u
2
(k) I I u2 (k -I)
with cost function
I I 4
J = "2 XT(S)Q(S)x(S)+"2 L {xT(k )Q(k )x(k)+ uT(k )R(k) u(k))
k =O
(4.S.S2)
constraints
(4.S.S3)
and Q(S) = diag(l , 2), Q(k) = 1
2
, and R(k) = diag(l,O.S).
SOLUTION: The problem was solved for an error tolerance of 0.00 I, a step
size of 0.1 for the conjugate gradient iteration, and >"(0) = (0.1 O.ll. The
algorithm converged in 49 iterations, as shown in Figure 4.18. Several other
initial x(O) and >"(0) were tried and the convergence was achieved in a
similar fashion.
4.5.3 Interaction Prediction Approach
Consider the following linear discrete-time system in its state-space form,
x(k+I)=Ax(k)+Bu(k)+d(k), x(O)=xo (4.5 .S4)
Hierarchical Control of Discrete-Time Systems 159
.9
.8
.7
p::
0
p::
.6
p::

z
0 .5
f::
u
<t:
.4 p::

f-<
is
.3
.2
.1
0
0 4 81216202428323640444850
ITERATIONS
Figure 4.18 Interaction error vs iterations for the time-delay algorithm in Example
4.5.2.
where x(k), u(k), and d(k) are 11 X I, III X I, and II X I state, control, and
disturbance vectors, respectively. Let the system be decomposed into N
subsystems
x;(k + I) = A;x;(k) + B;u;(k) + C;z;( k) + d; (k), xi(O) = XiII
(4.S.SS)
for i = 1, ... ,N and X;, u;, and d; are n; X I, m; X I, and l1i X I state, control ,
and disturbance vectors of the i th subsystem, respectively. The vector z, (k),
to be defined shortly, represents the interactions between the i th subsystem
and the remaining (N -1) subsystems. The matrices A;, Bi' and C; are,
respectively, n;Xlli' I1;Xm;, and n;Xr;. The integer r; represents the
number of incoming interactions to the ith subsystem. The optimal control
problem can be stated as follows:
(4.S.S6)
subject to (4.S.SS),
(4.S .S7)
160
and
Hierarchical Control of Large-Scale Systems
N
z;(k)= L {D;ju;(k)+G;jxj (k) }
j = 1
(4.5.58)
where D;j and G;j are the appropriate matrices between controlllj(k), state
x,(k ), and the state x;(k + I), respectively.
Two solutions of the above optimization problem and its modified forms
have already been presented. One is based on the time-delay algorithm due
to Tamura (1975), which was discussed in the previous section. The other
was di scussed in Section 4.5.1 without the bounds (4.5.57) and utilizing the
three-l evel discrete-time application of goal coordination. However, instead
of utili zing those methods, the continuous-time interaction prediction
method of Section 4.4. 1 is extended to discrete-time, which has not received
much attention in literature. Here again, we will ignore the bounds (4.5.57)
for the time being. Moreover. the objective function (4.5.56) is assumed to
be quadratic:
N - 1
J; = {xi(k )Q;x;(k)+ u{'(k )R;u;(k)}
k = O
(4.5.59)
where weighting matrices Q; and R; follow the usual regulatory conditions.
Consider the Lagrangian of the problem (4.5.55), (4.5.58)-(4.5.59):
N ,'II (Nr- 1 { 1
L = L; = 2. xJ'(k )Q;x;(k) + 1
u
;(k )R;u; (k) + )z;(k)
N
.... L

+ p;(k + 1)[ - x; (k + I) + A;x; (k) + B;lI; (k) + C;z;( k) + d; (k)] } )
(4.5.60)
Here it is assumed that the Lagrangian Lis additively separable for Z; and
A; trajectories. This would imply that [or any given z; and A;, there are N
independent maximization problems, each with a sub-Lagrangian L
j
What
is left is to find a mechanism for updating Zj and A
j
A necessary condition
[or this is
which result in
N
zj(k)- (Dj j ll j(k)+GjjXj (k)) =0
j = 1
(4.5.61)
(4.5.62)
(4.5 .63)
Hierarchical Control of Discrete-Time Systems
161
The coordination at the second level for this interaction prediction scheme
would be
l
-C
T
P (k + 1) j 1
[
(
)]
1+ 1 I I
A; k N
zj(k) = {Djjllj(t() + Gjjxj(k))
(4.5.64 )
for k = 1,2, ... ,N
j
, and I is the iteration number. Here again it is noted thaI
the computational effort at the second level IS very small compared WIth
that of the gradient, Newton, or conjugate gradient techniques.
For a known set of augmented interaction vectors [A *T( k) I Z*T( k) J, the
dh subsystem Hamiltonian is
H
j
(-) = 1X;(k) Qjx;(k) + 1U;(k) R;uj(k)
N
+ >/i ( k ) z; ( k ) - ( k ) ( D
J
j U j ( k ) + GFA) k ) )
j=1
+ pi( k + 1) [Ajx j (k) + Bju; (k) + CjZj (Ie)]
(4.5.65)
Then the necessary conditions for optimality would lead to
p j ( k ) = () Hj (- ) / () x j ( k ) = Q j x j ( k ) + A rp J k + I)
N T
- , pj(Nj )=O (4.5.66)
) = 1
and
N T
o = c7 H, ( . ) / a II j ( k ) = R j U j ( k ) + B;r Pi (k + I) - L ( k) D,; ) ( 4.5 .67 )
)= 1
or
(4.5.68)
where
Let us assume that the costate p/k) and state xj(k) are related by a linea r
vector equation
(4.5.69)
Now eliminating p(k) in (4.5.68), solving for x; (k + I) , and substituting it
in (4.5.66) while (4.5.69) and equating the coefficients of x/k)
and zeroeth-order terms would lead to the following Riccati and adJo1l1t
162
Hierarchical Control of Large-Scale Sys tems
K, ( k ) = Q, + A ;'i( ( k + I) Ei - I ( k ) A, ' K, ( N
j
) = 0 ( 4. 5.70)
Xi (k) = ;/: [/ - Ki {k + I) Ei - l(k)r;]Xi(k + 1) - lIi(k) , g, (N/ ) = 0
wiIere
(4.5.7 1)
E, (k. ) = I + 1'; K, (k + I )
r; = B,R, IB,T
h,( k) = ;/{K,(k + I)b,(k) + /,(k.)
h, (k) = E,- I (k) [R j Ie, (k) + e,z, (k) + d, (k)]
(4.5.72)
N
.t; ( k. ) = L (k) GjJ'
j = 1
Recalling the bounds (4.5.57), the expressions for 11 , (k) i 11 (4.5.68) a nd
.\, ( k + I) in (4.5.55) a ft er eliminating {J,(k + I) via (4.5.69), the followi ng
expressions arc sugges ted for conlrol and stale vect ors:
where
a Illi
{
J.I' ( k)
1I,( k) = R,- IGi (k)
u, (k)
LI, ( Ie ) <!:I, (J.: )
!:Ii ( k ) LI i ( k ) u, ( k )
u,(ku,(k)
(4.5.73)
G,(k) = B,"[K,(k + I) x,(k + 1) + g,( k + I)] - e,(k) (4 .5.74)

+ I) .\ ,(k + I) <.::..,,(/.; + I)
. L, ( k) X I (k ) + M, (k ) g, (k + I) - b (k ) .. . x (Ie - I)
x, (k + I) = , - ,

x, (k + I) x,(/.; + I) > .l: ,(k + I)
(4. 5.75)
where
L,(k) = L,I(k)A" M,(k) =E,
(4.5.76)
The di screte-t ime interaction prediction approach is summarized by the
following algorithm.
Algorifhm 4.5. Interaction Prediction Approach for Discrete-Time Sys tems
,\IC{J I : At the second level set I = I, assumc initial values for
z, (k) = zt (k) and AI( k) = A;( k), anu pass them down to first
level, i = 1, .. 'ON and k = 1, .. . ,N/.
Hierarchical Control of Discrete-time Systems
Step 2: At the first level solve N ma trix Riccati equations (4.5.70) and
s tore.
Step 3: Solve N adj oint equations (4.5.7 1) and store for k = I , ... ,N/.
Step 4: Using the stored values of K,(k), K,(k), (4.5.73)- (4.5.76), find
and store u,(k) and x,(k), i = I , ... ,N and Ie = I , . .. , N/.
163
Step 5: Check for the convergence of (4.5. 62) - (4.5.63) i.c. , whether their
left-hand sides are within E = 10-
6
o f zero. If not, use (4.5.64) to
updat e A,(k) and z,( k) , increment I = I + I, and go to Step 3.
Step 6: Stop.
An application of this algorithm is given in Example 4.5.3 .
4.5.4 Structural Perturbation Approach
In thi s section the optimal hierarchical control scheme via s tructural per-
turbation of Section 4.4.2 for continuous-time systems is extended for
discrete-time sys tems. For the sake of discussion, the di screte quantiti es are
represent ed by index i in a subscripted form and the subsys tems by index)
in a superscripted form; e.g., xl represents the)th subsys tem's state at ith
interval. The present development, in part, foll ows the work of Sundareshan
( 1976).
Consider a large-scale di screte-time sys tem which may consist or M
subsystems:
(4.5.77)
for ) = 1,2, ... , M and i = 1,2, ... , N. In the above relati on, xl and tI ! arc 11 ,-
and m-dimensional state and control vectors of the )th subsystem at i th
interva'1. The G
jk
is the II j X II k interconnection matrix between .the )th and
k th subsys tems. The relati ons of Ai' B
i
, and G
i k
matnces are gIven below:
AI I G
I 2
I
__ i _ _ l _ _ _
G
2 1
I A2 I G
n
I
_ _ L __ L __ L
. I
- - - -
I AM
(4.5.78)
The optimal control problem would be to find cont rol vectors Llf, j =
1,2, ... , M, sllch thaI (4.5.77)- (4.5.78) are sati sfied while minimi zing a
164
Hierarchical Control of Large-Scale Systems
quadratic cost function
N
Jl( x{" lin = L (xrQ;x( + Rj u; _ I) (4.5.79)
i = 1
Qj and. R; are 11; X 11 ; and m; X Ill ; positive-semidefinite and positive-
defll1lte The optimal control problem, in its large-
scale composIte IS to find an m-dimensional control u
T
= (U
IT
, ... , U
MT
)
WhICh would satisfy the overall state equation
X; + 1 = (A + C)x; + Bu; (4.5.80)
A = I ' A 2 , .. ,A,If ): B = Block-diag(B I' B
2
, ... , B
M
), and
[0,k J.J . k - 1, 2, . . . ,M, willie the cost
M
J( x", ul) ) = L Jl(x /" (4.5.81)
j = I
is minimized. The solution of this prohlem requires the solution of a
large-order Riccati equation (Dorato and Levis, 1971), and a single central
controller would become necessary which may require too much, and often
unnecessary or unavailable, information from all the states. In order to
alleviate these problems, Sundareshan (1976) has proposed a hierarchical
control which consists of a local and a global component, i.e.,
u. =u'+u
g
1 1 1
(4.5.82)
where the local control is given by
, _ { IT 2T MT}
U; - Ii; ,U; , ... ,U; (4.5.83)
and each compon.ent u{ is the optimal control of the jth decoupled subsys-
tem problem deflI1ed by (4.5.77) --(4.5.78) and (4.5.79) which is obtained
through a Riccati formulation as in (4.5.65)- (4.5.72). The jth subsystem
local control is given by
u
j
= - pi
x
j
1 1 1 (4.5.84)
where and K/ls the positive-definite
solutIOn of the discrete matrix Riccati equation:
Kj= Q +ATKj (I+SKj )- I
A
(
; j .; ; + 1 j / + 1 ;' KN=O 4.5.85)
= B;Rj IB! and all other matrices are defined by (4.5.77)- (4.5.79).
1 he global component II f is given by
(4.5.86)
where r is an 111 X 11 gain matrix yet to be determined. Using the local and
global controls defined by (4.5.84) and (4.5.86), the closed-loop composite
system becomes
(4.5.87)
Hierarchical Control of Discrete-Time Systems
165
where P; = lllock-diag(P;', p/, .. . , p;M) and the tcrm (C - B r) is termcd as
the effective interconnection matrix C
e
. The matrix r should be chosen such
that the norm IIC - Bfll is minimized. The solution to this minimization
problem is well known:
(4.5 .88)
where B+ is the generalized inverse, i.e., B+ = (l3
T
B) - IBT. The solution
(4.5.88) is subject to the rank condition, rank(B) = rank(BC). Thus, the
global control component gain matrix is obtained [rom
(4.5.89)
The above choice would in effect neutrali ze the interconnections, i.e.,
IICe l1 = O. The application of this hierarchical control to the large-scale
interconnected discrete-time system would, in general, cause a deterioration
of performance whose sUboptimality degree is p if the following inequality
holds (Sundareshan, 1976) :
(4.5.90)
where
V=LTQ L+P/TRPJ, H=(I+p)K+Q
J JJJ 1 JI 1 ,
(4.5.91)
The terms AIII(D) and AM(D) are, respectively, the minimum and maximum
eigenvalues of matrix D. This control scheme is summarized by an algo-
rithm.
Algorithm 4.6. Structural Perturbation Approach [or Discrete-Time Systems
Step 1: Input all matrices {Aj' B
j
, Qj' R
j
, C
jk
}, initial states etc.,
j=I, .. . ,M.
Step 2: For each subsystemj, solve (4.5.85) for Riccati matrix K( ,
i = I, ... ,N - 1, and store.
Step 3: Evaluate the local control components by solving (4.5.84).
Step 4: Solve (4.5.89) and use (4.5.86) for the global control component
and form the overall control (4.5.82).
Step 5: Using the control u and Equation (4.5.80), find state x;,
i=I ,2, ... ,N.
In sequel, Algorithms 4.5 (interaction prediction) and 4.6 (structural
perturbation) are applied to a three-region energy resources system.
166
REGION
3
REGION
2
Hierarchical Control of Large-Scale Systems
Figure 4.19 A three-region energy resources system.
4.5.3. COI?sider a three-region energy resources sys tem shown in
Figure 4.19. For thiS system, let the foll owing discrete-time model hold:
r IJ
I
: () : () j
Xi + I = l qo?J x, + 1/,
Hierarchical Control of Discrete-Time Systems
167
where

J
m
0.5
0.75 0.5

0.5 0.25

0.5 0
(4.5.92)
0 0.5
0.2 0.25 0.25 0

0.5

0.25
0.3 0.2
H
rO
0


0 0
10 0.5
0.5 0
G
21
= l
0
1.5 0 0
0
The quadratic cost function has weighting matrices QI = diag(J 10 1),
Q2 = 51
4
, Q
3
= diag(1O 110). and R;=I,j=L2,3. lt is desired to apply
Algorithms 4.5 and 4.6 to find a hierarchical allocation policy for tbe
energy-resources system of (4.5.92).
SOLUTION: For the hierarchical control policies, three matrix Riccati equa-
tions of the form in (4.5.70) or (4.5.85) and three adjoint vector equations of
the form in (4.5.71) were solved usi ng an HP-9845 computer and BASIC.
Based on the sol utions of these and vector equat ions. three local control
functions were obtained. Using the interation prediction method, an accep-
table (within 10-
4
) convergence was reached wi th :line iterations. Using the
structural perturbation method, Equation (4.5.89) was used to ohtain the
global cont rol component and added to the vector of local controls to fi nd
the overall control (4.5.82). The resulting states ami control function via the
hierarchical control Algorithms 4.5 and 4.6 along with the optimum central-
ized trajectories which were obtained by solving a tenth-order discrete-time
Riccati equation are shown in Figures 4.20 and 4.21, respectively.
The state and control responses for the interaction prediction (Algorithm
4.5) were closer to the optimum than those for the structural perturbation
(Algorithm 4.6). However, the third subsystem's trajectories were closer than
the other two. A reason for this may be the fact tilat the third subsystem is
completely decoupled.
4.5.5 Costate PredictioJl Approach
In this section a computationally effective approach for hierarchical control
of nonlinear discrete-time systems due to Mahmoud et al. (1977), Hassan
and Singh (1976, 1977) is presented. The scheme is appl ied to nonlinear
1611
4 .
3
2 -
.<
LW
-- .1
t;:; - 4 _
-. S
7 -
III
()
0
- I
___ "l
- 3
- 4
-o S
-- -C)
-'<
-- 7
-R
" LW --- 9
f-
III <!;
f-
- 1 I (/)
- \:'
-- \. 1
- 14
- I S
1(,
I 7
- IX
- 19
- 20
-
-
..
_.
-
-
-
-
..
_.
-
--- CTNTRALIZED TRAJECTORY
---- NEAR- OPTIMUM lll ERARCl1 l CAL TRA.1FCTORY (Algorith11l4.6)
!
/
-'-' - NEAR-OPTIMUM IIIERARCIIICAL TRAJFCTORY (Al gorithm 4. 5 )
i
/
i /
. /
1/
1/
.-'-_--.JI ___ L - _ ___ '--_. __ J __ __. ___._L ____ ._ I __ --'-- I _.__ L ___ .J
S 10 15 20 25 30 35 40 45 50
PERIOD. k
(a)

--- OPTIMUM C'I'NTRALlZI'IJ TRAJFCTORY
- - -- NI ' AR-OPT1MUM IIlI'RARC'IIiCAL TRAJHTORY (Al gorilhlll 4 .(1) 0-
- . -.- NEAR- OPTIMUM IIIERARClIlCAL TRAJECTORY (AlgorilhIll 4.5) '",
0
L---l __ __ __ _ _ __
10 15 20 25 30 35 40 45 50
PERIOD. k
(b)
Hierarchical Control of Discrete-Time Systems
169
.95
.85
.75
.65
.55
.45
.35
..:,-:
.25
0
'"
. I 5
LW
f-
.05
<!;
f-
en -.05
- . 15
- .25
-. 35
- -.45
- .55
-- .65
- .75
___ OPTlMUM CENTRALIZED TRAJECTORY
___ NEAR-OPTIMUM HIERARCIIlCAL TRAJECTORY (Algorithm 4.6)
_._._ NEAR-OPTlMUM HIERARCHICAL TRAJECTORY (Algorithm 4.5)
L-_ _ __ _ _ .L I __ L __ __
35 40 45 50
0 10 15 20 25 30
PERIOD, I;
(c)
Figure 4.20 Optimum centralized and near-optimum hierarehical trajeetories for
Example 4.5.3: (a) state x2(t), (b) state x
1
(t), (e) state XlO(t) .
systems, and its linear extension is rather simple (see Problem 4.8). The
hierarchical control of nonlinear continuous-time systems is considered in
Chapter 6 (Section 6.3.1.) under the context of near-optimum design of
large-scale systems.
Consider a nonlinear discrete-time system described by
x(k+1)=/(x,u,(k+1,k)), x(O) = x"
with a quadratic cost function
K - I
L (xT(k)Qx(k)+uT(k)Ru(k))
1; = 0
(4.5.93)
(4.5.94)
The procedure begins by rewriting the nonlinear state equation (4.5.93) as
x (k + 1) = A (x, u, (k + 1, k) )x( k) + B (:x, fl, (Ie + 1, k)) u (k)
+ c(x, u, (k + 1, k)) (4.5.95)
where A(), B() are block diagonal matrices and
c ( x, u, (k + 1 , k )) = C I ( x, u, ( k + 1 , k ) ) x ( k ) + C
2
( x, u, (k + 1, k ) ) 11 ( k )
(4.5.96)
It is notecl that the reformulation (4.5.95) - (4.5.%) of' the system (4.5.93) is
/ 170
.,1.
-5
-10
-15
-20
-25
3:
-30
:;
o-l
0 -35
p::
b
-40 Z
0
U
-45
-50
-55
-60
-65
-70
0
0
- 2.5
- 5
-7.5
- 10
- 12.5
- 15
:;: - 17.5
N
-20
" o-l
-22.5
0
p::
- 25 b
Z
0
U
-27.5
-30
-32.5
-35
-37.5
- 40
- 42.5
- 45
--=--:::- .--.
....... -.
.......... """'- --- .----
............... ........ ........ --...... '-...
........ --......
'- "
'-" ."'"'-
'- "'"'-
'-.. "
" "-
"" '",
" '"
"" '"
" \
"-
'\ \
\ \
OPTIMUM CENTRALIZED TRAJECTORY \ '\
___ NEAR-OPTIMUM HI ERARCHICAL TRAJ ECTORY (Algorithm 4.6) \ .
_. _ . _ NEAR-OPTIMUM HIERARCHICAL TRAJECTORY (Algorithm 4. 5) "
0
\0 15 20 25
PERIOD, k
(a)
30
___ OPTIMUM CENTRALIZED TRAJECTORY
35 40 45
_ _ __ NEAR-OPTIMUM HI ERARCII ICAL TRAJ ECTORY (Algorithm 4.6)
_._. _ NEAR-OPTIMUM HIERARCHICAL TRAJECTORY (Algorithm 4.5)
5 10 15 20 25
PERIOD, k
(b)
30 35 40 45
50
50
P.
Hierarchical Control of Discrete-Time Systems 171
0
-. 1
-.2
-.3
:;:
- .4
~
" .
o-l
-.5 0
p::
b
Z
0 -.6
u
- .7
-.8
- .9
I
0
_ _ OPTIMUM CENTRALIZED TRAJECTORY
___ NEAR-OPTIMUM HIERARCHICAL TRAJECTORY (Algorithm 4.6)
._. _ NEAR-OPTIMUM HIERARCHICAL TRAJ ECTORY (Algorithm 4.5)
10 IS 20 25 30 35 40 45 50
PERIOD, k
(c)
Figure 4.21 Optimal centralized and near-optimum hierarchical trajectories for
Example 4.5.3: (a) control Ul (t), (b) control U2(t) , (c) control u
3
(t).
always possible. Moreover, for N blocks in A and B, it is assumed that
matrices Q and R also have N blocks. The basic reason for the reformula-
tion (4.5.95)- (4.5.96) is to provide" predicted" state and control vectors x*
and u* to fix the arguments in the nonlinear coefficient matrices A(), B(),
C 1 (.), and C
2
(-). Therefore, the problem (4.5.93)- (4.5.96) can be rewrit-
ten as
_ I K - I
min J =2 L (xT(k)Qx(k)+uT(k)Ru(k))
k ~ O
(4.5 .97)
subject to
x (k + I) = A (x* , u*, (k + I, k)) x (k) + B (x* , u* , (k + I , k)) u (k)
+ c(x*,u* ,(k +l , k)) (4.5.98)
x*(k) = x(k), u*(k) = u(k) (4.5.99)
The modified problem can be solved by defining a Hamiltonian:
H( ) = 1XT(k )Qx(k)+ 1u
T
(k )Ru(k)
+ pT(k + l){A(x*, u*, (k + 1, k )x(k)
+ B (x* , u* , (k + I, k) u (k) + c (x*, u* , (k + 1, k))
+ exT ( k ) ( x ( k ) - x * ( k ) ) + f3 T ( k ) ( u ( k ) - u * ( k ) ) (4.5 . 100)
172
Hierarchical Control of Large-Scale Systems
In view of the assumptions made on matrices A, E, C
I
, C
2
, Q, and R , it is
clear that the Hamiltonian H() in (4.5. 100) is additively separable for given
x* and 11*. i.e.,
N N
H = L H, = L
, = 1 , = 1
+p/(k + I)[A,(. )x,(k)+ BJ )u, (k)] + d,)}
+cx[(k)[x,(k)-x
i
(k)]+f3,'(k)[u,(k)-U
i
(k)]} (4.S. IOI)
The necessary conditions for optimality are given by
0 = oHjou,
x,(k + I) = oH,/op,(k + I)
p,(k) = aHjox,(k)
O= r7 IJ/aH, 0 = () J/ / iJ!J
0 = rJll/ox*(k), 0 = olJ/ou*(k)
Rela tion (4.S.102) yields an expression for u,( k),
(4.S.102)
(4.S.103)
(4.S.104)
(4 .S .IOS)
(4.S . 106)
u,(k) =- R,:'
I
{B,'(x* , u*,(k+l,kp,(k + I) + f3, (k)} (4.S. 107)
and subs tituting 1I,(k) in Equati on (4.S.103) yields a new expression for
x,(k + I). i.e.,
.\Jk + I) = A,(x*, u*,(k + I, kx,(k) - B,( x*, u*,(k + I, k
R I I{ B,'( x*, 11*. (k + I, k pJ'( k + I) + f3, (k )} + c, (x*, u*, (k + I, k
(4.S . IOX)
with x, (O) = x'o' The condition (4.S.104) gives the costate equation
P, (k ) = Q,x, (k) + Ai( x*, 11*, (k + I, k p, (k + I) + H, (k) , P, (K) = 0
(4.S.109)
and the necessary conditions (4.S.IOS) lead to the equality constraints
(4.S.99), i.e.,
x*(k)=x(k). u*(k) = u(k)
(4.S.110)
The first of the two conditions in (4.S.106) leads to an expression for cy(k),
I.e ..
H ( k ) = {F\T ( X * , u * , x, (k + 1 , k ) ) + ( x * , II * , u, (k + I , Ie ) )
+ D\T( x*, u*, (k + I, k } p (k + I) (4.S .111)
Hierarchical Control of Discrete-Time Systems 173
where
) = 0 Z (- ) /0 x* 0 {A ( x* , u* , (k + I , k ) ) x ( k ) / 0 x*
(4 .S.112)
D
x
(') = oc(x*, u*,(k + I , k/iJx*
Finally, in order to obtain an expression for f3(k), the second condition in
(4.S.106) yields
f3 (k) = {F,;r( x*, u*, x, (k + I, k + G;;( x* . u* , u, (k + I, k
+ D,;(x*, u*,(k + I , k}p(k + I) (4.S.113)
where F,,('), G,,('), and D,,(') are derivatives of the expressions in the
brackets of (4.S.112) with respect to u*. The following algorithm sum-
marizes the costate prediction method.
A 4.7. Costate Prediction Approach
Stcp 1: Set iteration index / = I and guess vectors 1'1, x*', u*'. and f31 .
Step 2: At the first-level, substitutcP'(k), x*'(k), u*'( k) , and (3'(k),
k = O, ... ,K - I , in (4.S. 107)- (4.S.108) to obtain u;(I.;,) and x;(I.;)
for i = 1.2, ... ,N. Simi larly, use (4.S.111) to find H'(k).
St ep 3: At the second-level, use xi(k), !liCk), and (4.S. 109)- (4.S.113) to
update coordination vector, q' = (pi, x*', U*' , f3') , i.e.,
P:+ I(k) = Q,xi(k) + AJ'(x*, u*, (k + 1, k pi + 1(1-: + I) + a;(k.)
X*f.lI(k) = x'(k)
11 *' 1 l(k) = 1I '( k) (4.S .114)
f3 ' + I (k) = { F.,T( . ) 1" /I". x') + G,;( . )1(.\' ,. ,,". 1/) + D,;(- ) /(.\" ', I'-')}
xp(k+I)
Step 4: lfql +l(k)=ql(k)fork=O,I, ... ,K-I,stopandu' + I(k)isthe
optimal control. Otherwise go to Step 2.
Before this algorithm is illustrated by a numerical example, a few com-
ments on the costate-prediction method are due. First, since the costate
vector P'(k) is a component of the coordination '/ector ql(k). the first-level
problem (Step 2) involves simple substitution and docs not require the
solution of a TPBV problem or even a Riccati equation. Moreover, the
second-level problem (Step 3) is also ra ther trivi al, requiring the substitution
of x, U, and f3 vectors which have been obtained from the previolls iteration.
Therefore, unlike the other multilevel formulations, such as the goal coordi-
174 Hierarchical Control of Large-Scale Systems
nation and interaction prediction approaches, both the first- and second-level
problems are mere substitution problems. The only question remaining is
the nature of and/or conditions for the convergence of costate prediction.
Hassan and Singh (1981) and Singh (1980) have proved theorems by which
an open interval of time is obtained to guarantee convergence of the
algorithm for the continuolls-time case (see Problem 4.12). Simi larl y for the
discrete-time case, Hassan and Singh (1976) in an unpublished report have
given some conditions for the convergence. They have shown that Algo-
rithm 4.7 converges uniformly over an interval (0, K - I) if the following
conditions hold: (i) g, y , Z, and C are bounded functions of k ; and (ii) z, y,
and c in (4.5.112) are differentiable with respect to x* and u* for each
o k K - I and their derivatives are also bounded functions of Ie (Singh
and Titli, 1978). Further discussion on costate coordination is given in
Section 4.6.
The following example deals with an open-loop power system consi sting
of a synchronous machine connected to an infinite bus through a trans-
fonner and a transmission line. This system, in continuous-time form, is
treated in detail in Section 6.3 and has been treated by many authors
(Mukhopadhyay and Malik, 1973; Jamshidi, 1975; Hassan and Singh,
1977). For the sake of the present discussion, a discrete-time formulation of
the original continuous-time model (lyer and Cory, 1971) considered by
Singh and Titli (1978) is used.
Example 4.5.4. Consider a sixth-order nonlinear di screte-time model of the
open-loop synchronous machine system:
xl(k + I) = x , (k)+0 .05x
2
(k)
x
2
(k + I) = (1-0.05c, )x
2
(k) - 0.05c
2
x, (k)sin x , (k)
- 0.025c
3
sin 2x I (k) + (0.05/ M )X5 (k)
x,(k + I) = (I -0.05(4) x3 (1e)+0.05x6(k)+0.05 cs cosxl(k) (4.5 . 115)
x
4
(k + I) = (1 - 0.05K3)x4(k)+0.05K2x 2(k) +0.05Klul(k)
xs(k + I) = (I -0.05Ks)xs(k)+0.05K4x4(k)
x(,(k + 1) = (1 - 0.05K
7
)xc, (k)+O.05Kc,u
2
(k)
where ( C
I
' (' 2' c1, c
4
, c
s
) = (2.1656, 13.997, - 55.565,1.03, 4.049) , (KI' K
2
, ... ,
K 7) = (9.4429, 1.0198, 5,2.0408,2.0408, 1.5,0.5) and M = 1. Apply the costate
prediction approach of Algorithm 4.7 to satisfy (4.5.115) starting with an
initial state X7(0) = (0.71050.04.20.80.80.5) while minimizing a quadratic
cost fUll ction
1<)
.I = t { QII(xl(k) -X/ I) 2+QD( X3 (k) - xn )2
k - ()
+ RII(1I1(k) - u/I)2 + R
n
(u
2
(k) - 1I
12
)2} (4 .5. 116)
where QII = QJ] = 0.2 and RII = Rn = 1.
Hierarchical Control of Di screte-Time Systems 175
SOLUTION: In order to apply Algorithm 4.7, system (4.5. 115) must be
reformulated in the form of Equation (4.5.98), i.e.,
x,(k + I) = x ,(k)+0.05x! (k)
x
2
(k + 1) = (1 - 0.05c , )X2 (k) -0.05c
2
x)( k) sin x7(k)
- 0.025c
J
sin 2x7( k) + (0.05/ M )X5 (k)
x
3
(x + I) = (4.5 . 117)
x
4
(k + 1) =
x s(k + 1) = (1 - 0.05Ks)xs(k)+0 .05K4 x! (k)
x
6
(k + 1) = (1-0 .05K
7
)x
6
(k)+0.05K
6
u
2
(k)
Additional equality constraints are
x7(k) = xJk), i = 1, ... ,6 (4 .5.1 IS)
This problem was simulat ed on an HP-9845 computer using BASI C. The
necessary conditions of optimality through the Hamiltonian formulatJ on
leads to the following control, costate, and f3(k ) vector equations:
uJk) = u
/ I
-0.05K,P4(k + I)
1I
2
(k) = 1I
12
-0.05K
6
P6(k + 1)
P I ( k ) = 0 .2 ( x I (k ) - xII) + PI ( k + 1) + 0: I ( k )
P2 (k) = (I - 0.05c
l
) P2 (k + 1) + 0:
2
(Ie)
P3(k) = 0.2(x
3
(k) -x/
3
)+ P3(k + 1)(1 - 0.05c
4
)+ 0: 3(k)
P4 (k ) = (I - 0 .05K 3) P4 (k + 1) + 0:
4
(k)
Ps(k) = (I - 0. 05Ks) Ps{k + 1)+ lX s (k)
Pc, (Ie) = (1 - 0 .05K 7) P6 (k + I) + lX6 ( k )
withp;(K) = O, i = I, . .. , 6, and
(4.5 . 119)
(4 .5. 120)
lXl(k) = - 0.05c
2
P2(k + l)xr(k)cosx7(k)-0.05c
3
P2(k + 1)
. cos2x'i(/..: ) -0.05c
s
PJ (k + 1 )sin x7Ud
lX
2
(k) = 0.05(PI(k + 1) - K
2
P4(k + I))
lX3(k) = -0.05c
2
P2(k + l)sinxf (k)
lX
4
(k) = 0.05K
4
Ps(k + 1) (4.5.121)
lX s(k) = (0.05/M)P2(k + I)
lX6(k) = 0.05PJ(k + I)
(4. 5. 121)
In order to compare the results of thi s algori tbm to those reported earlier
(Hassan and Singh, 1977), the same values of parameters and initial values
were chosen. Here p(k) = x*(k) = 0 was chm-:en, and the algorithm con-
verged in 86 second-level iterations. Figure 4.22 shows the optimum
176
u
I 1 .
4
Perind . k
2.0
10
0.0 -
.' .0 .. . ....... 1 _ _ .J _ ... _ I . ____ J
I) c.1 " 5
Period. I,
4 ' .-
3.4
1 ' - .. __ l __ . L ___ l __ J
. " ' Il I 3 4 5
Peri od .
D.R -
0.4
. O. C ----L---'-__ J '----.JIL---'
() I 2 .J 4
Per iod. k
0.0
- O. I '::-0--'-- ---'-2- ---.l---.J
4
---l
Peri od .
0. 5
0.4
0.3
0. 2
0. 1 -
D.D --- - J._ ... __ L __ L __ L __ J
o I 2 3 4 5
Pcri od.k
o
- 0.0004
- 0.0012

o I 2 3 4 5
Peri od .
0.003
112 0.001
- O. 00 10 L. ---'---..L
2
---'3---'4L....-.-J
5
Period, k
Figure 4.22 Optimum timl' responses for the opcn-Ioop power system of Example
4.5.4.
Discussion and Conclusions J77
responses for states and controls, which turned out to be identical to those
reported earlier.
4.6 Discussion and Conclusions
In this section, the open- and closed-loop hierarchical control of continu-
ous-time systems and hierarchical control of discrete-time systems will be
discussed and compared. Some aspects, such as computer time, storage,
information transfer requirements, and potential practical applicability. will
be discussed and an attempt is made to point out advantages and disad-
vantages of each scheme.
The first method considered in this chapter was goal coordination. based
on the interaction balance principle. The computational effort in the
hierarchical control of a large-scale system reduces to that of a set of
lower-order subsystems and a coordination procedure. The computational
requirements of this method at the first level are normally much less than
those of the original large-scale composite system. However, the overall
computations depend heavily on the convergence characteristics of the
second-level linear search iterations, e.g., gradient, Newton's, conjugate
gradient, etc., methods. Since between each two successive second-level
iterations, N decoupled first-level problems must be solved, the slower the
second-level problem's convergence is, the more times the N first-level
problems must be solved. In practice, one may use multiprocessors [or the
first level in an attempt to save computational time for the N subsystems
computations. Although this seems to be a good proposition and has been
suggested by others (Singh, 1980), not many in the published literature
support it (Sandell et a!., 1978). An exception to this is due to Ti tli et a!.
(1978), who have used multiprocessors in the hierarchical can trol of a
distributed parameter and a traffic control system. A major improvement in
computations can be made when the first-level problems are both smaller
and simpler. This latter observation is illustrated for coupled nonlinear
systems with and without delays in Chapter 6. From the computer storage
point of view, no significant limitations exist for the method. The scheme
can handle constraints o[ inequality type on both the state and control, and,
furthermore, the coordination at the second level has been shown (Varaiya.
1969) to converge to the optimum solution. The main disadvantage of goal
coordination stems from the numerical convergence at the second level and
its adverse effects on the first-level calculations mentioned before. Another
shortcoming is the inadequacy of methods to find a ncar-optimum control
in an attempt to avoid too many second-level iterations. A near-optimum
control without convergence at the second level would be an unfeasible
solution. Yet another difficulty is the need to add a quadratic term z
7
Sz of
the interaction vector z in the cost function to avoid singular solutions. This
178:
Hierarchical Control of Large-Scale Systems
problem, as demonstrated in Section 4.3.3, can, however, be avoided in two
different ways.
Another hierarchical control technique discussed was the interaction
prediction method. Based on its development in Section 4.3.2, it is clear that
the second-level problem is much simpler here than in the goal coordination
method. Furthermore, no possibility for a singular solution exists and the
experience of this author and others (Singh and' Hassan,
1976) ven.fy that the convergence at the second level is reasonably fast.
the interaction prediction method compares very favor-
a?ly wIth the goal coordination approach. First from the storage point of
VIew, the two methods are very similar. Both can be simulated in such a way
so as to save on sto.rage requirements by using a common block of memory
for both levels. This was done for almost all illustrative examples of this
chap.tee. . From. the computational viewpoint, for a number of reported
appitcatlOns (SIngh and Titli 1978, Singh 1980), including the 12th-order
system of Example 4.3. I, the interaction prediction method took on the
order of one-quarter of the computer time of goal coordination. It should be
mentioned, however, that the extent of computational benefits resulting
from the interaction prediction and goal coordination as compared to
the ongInal large-scale composite system depends on the format of the
decomposition process. For example, if the 12th-order system of Example
4.3 .. 1 had been decomposed into two sixth-order systems, the computational
savIngs woul? not been that appreciable. Another important point
concerns the InfOrmatIOn structure between subsystems. An argument which
can go against both the goal coordination and interaction prediction meth-
ods is the . need for subsystems' full state information in a feedback
configuration. Still another argument against the hierarchical control meth-
ods has been that they are insensitive with respect to modeling errors and
component failure (Sandell et aL, 1978). Part of the latter difficulty can be
through hierarchical control strategies for structural perturba-
tIOns developed by Siljak and Sundareshan (l976a, b).
The feedback control based on interaction prediction discussed in Section
4.4.1 has the advantages of being able to handle large interconnected
systems regulator problem with a complete decentralized structure while the
feedback gains remain independent of the initial conditions. The main
disadvantage of this scheme is the extensive amount of off-line calculations
for the finite-time case, as suggested in Section 4.4.1.
The feedback control based on structural perturbation presented in
Section 4.4.2 is perhaps one of the first attempts to come up with a
c!osed-loop control ' law for hierarchical systems. The method is relatively
SImpler than both the goal coordination and interaction prediction methods
in that no second-level iterations or even simple updatings are involved.
Moreover, no large-scale AMREs must be solved, but rather only a
Discussion and Conclusions 11 ':1
Lyapunov-type equation, i.e., (4.4.41). Although the solution of a Lyapunov
equation is much easier than a Riccati equation, when the system order is
very large (n > 200), finding an efficient method with reasonable storage as
well as computer time is questionable (Jamshidi, 1980). It has been argued
by Sundareshan (1977) that all calculations are done off-line in contrast to
the goal coordination which involves extensive on-line iterations on the
first-level subsystems calculations. In this author's opinion, the best situa-
tion would be a balance between on-line and off-line calculations. The most
interesting outcome of feedback control based on structural perturbation is
that it takes possible beneficial effects of interactions into account. The
establishment of a class of interconnection perturbations, which provide a
foreseen effect on the overall system cost function, is a very useful flexibility
for the coordinator. In spite of these advantages, it becomes clear that when
the interconnection matrix Gis nonbeneficial, the near-optimality index p in
(4.4.45) may become very large. Under such conditions, it would .be desira-
ble to have an additional global controller, such as those of Siljak and
Sundareshan (1976a), which would neutralize the effects of the interactions.
In all, the feedback control based on structural perturbation is an effective
step in the right direction for closed-loop hierarchical control, although the
total state information structure still remains the same.
In Section 4.5, the hierarchical control of discrete-time systems was
considered. Five approaches were presented. In Section 4.5.1 the continu-
ous-time version of goal coordination approach was extended to linear
discrete-time systems. The computational behavior of this scheme is very
similar to continuous-time goal coordination, which is heavily dependent
upon the second-level problem'S iterations. Moreover, both methods require
the quadratic term in the cost function to avoid singularities. The only
advantage of discrete-time over continuous-time goal coordination (Section
4.3.1) is that its first-level problem is very simple, as is evident from
(4.5.21 )-(4.5.23).
The time-delay algorithm discussed in Section 4.5.2 is perhaps the most
appropriate goal coordination method for practical problems. The applica-
tion of the algorithm to traffic control problems (Tamura, 1974, 1975) is
excellent evidence of this. This algorithm as well as goal coordination have
been used for the optimal management of water resources systems by many
authors (Fallaside and Perry, 1975; Haimes, 1977; Jamshidi and Heggen,
1980). Perhaps the most striking benefit from the time-delay algorithm of
Tamura (1974) is the fact that it avoids the solution of a TPBV problem,
which involves both delay (in state vectors) and advance (in costate vectors)
terms. This type of TPBV problem is a common difficulty with optimal
control of time-delay systems by the maximum principle, which is discussed
in Chapter 6, where a near-optimum solution is proposed. Further
advantages of the method are the convenient handling of inequality con-
l80 Hierarchical Control of Large-Scale Systems
straints and avoiding the possibility of singular solutions, which is not
always possible in goal coordination procedure.
In Section 4.5.3 the continuous-time version of the interaction prediction
approach was extended to discrete-time systems for the first time in the
author's best recollection. The first-level problem constitutes the solution of
the discrete-time Riccati equation, which can prove to be rather time-
consuming in a real-time situation. This scheme, tested numerically in
Example 4.5.3, seems less effective than the costate prediction method of
Section 4.5.5. The reasons for this are the convenient solutions of the costate
prediction's first- and second-level problems. More specifically, for the
system of Example 4.5.4, the first-level problem consists of straight substitu-
tions on the right sides of 14 equations (six for states x, six for Lagrange
multipliers (3, and two for controls u), while the second-level problem
involves six substitutions for the costate p(k). The computational effort per
iteration for this approach is a small fraction of that required for the other
methods, such as discrete-time goal coordination (Section 4.5.1) and interac-
tion prediction (Section 4.5.3). The costate prediction scheme can be easily
extended to the linear case (see Problem 4.8) or continuous-time case
(Problem 4.12). Jamshidi and Merryman (1982) have extended the costate-
prediction method to take on continuous-time and discrete-time nonlinear
systems with delays in state and control.
The other scheme considered in Section 4.5 was the structural perturba-
tion approach of Section 4.5.4, which is computationally more attractive
than goal coordination or interaction prediction due to its convenient
scheme for obtaining a global (corrector) component for control vectors.
Also, the solution of a discrete-time Riccati equation is sought only once for
the N subsystems. In other words, this scheme, as in its continuous-time
version (Section 4.3.1), does not involve any iterations between first- or
second-level hierarchies. The main difficulty with almost all of these
hierarchical control schemes is that their convergence is not yet a settled
issue. The only exception is perhaps continuous-time goal coordination
(Varaiya, 1969). More discussions on hierarchical methods are in Section
6.7.
Problems
C4.1. Consider a two-subsystem problem,
: . .
Problems
181
C4.2.
C4.3.
Use the two-level goal coordination Algorithm 4.1 to find an optimum
control which minimizes
Q=diag(I,2,2,1), R=/
2
, and !:l.t=O.l. Use your favorite computer
guage and an integration routine such as Runge-Kutta to the aSSOCI-
ated differential matrix Riccati equation and the state equatIOn.
Algorithm 4.3 represents a goal coordination proccdure for the hicrarchical
control of a large-scale discrete-time system. Consider the system
0.Q2 0
-0.2 0
o -I
o 0
with a cost function
4 4
J=.!. L L [xT(k)Q(k)x(k) +uT(k)R(k)u(k)]
2
i
= 1 k=O
where Q(k) = 14 and R(k) = 1
2
. Find the optimal hierarchical control of
this system by decomposing it into two second-order subsystems.
For a second-order discrete-time delay system
where
XI (k + 1) = 2xI (k)+ XI (k -1)+ 111 (k )+0.5112 (k - I)
x
2
(k + 1) = - 2xI (k) - x
2
(k)+ X2 (k - I) + ul (k -I)
-u2(k) - u2(k-l)


x (k ) + u T (k ) (
0
6
5
182
. Hierarchical Control of Large-Scale Systems
Problems (continued)
with initial conditions
x(k)=u(k)=O, xT(O) = [1 I].
use the time-delay Algorithm 4.4 to find the optimal sequence u*( k),
k = 0, I, ... ,6. It is possible to solve this problem analytically; however, a
computer is more desirable for higher-order systems.
C4.4. Consider an interconnected system
x = r h - F -- --= q'j x + r H j u
0. 1 - 0.2 1 0 - 2 0 01 I
0.4 0. 1 I - 0.5 0 -4 0 I I
with cost function
2
J= .L [xT(2)Q;X;(2)+ l\xT(t)Q;x;(t)+ u;(t)R;u(t) dt]
I - I 0
with Qi = diag(2, I, I, I, I), R; = 1
2
, and initial conditions XT(O) =
[ I 0 0.5 - I 0], 6.t = 0.1. Use the interaction prediction Algorithm
4.2 to find the optimal control u*( I) for the above problem.
4.5. Prove that the control law (4.4.53) is a stabilizing controller for the
large-scale system (4.4.22). [Hilll: F= K + k is a positive-definite solution
of AMRE (4.4.47) for the perturbed system (4.4.46).]
4.6. Repeat Example 4.4.2 for the following system:
and a cost function
4.7. For the system
XI=XI+U
I
, xl(O)=1
x
z
=2x
z
+u
z
, x
2
(0)=1
J
-.! ('Xl (2 z + z 2 Z )
- 2 J
o
X I x 2 + u
l
+ U
z
dt
X I = X I + ax z + u I ' X I (0) = I
xz=xZ+bxl+u
Z
' xz(O)=O
Problems 183
and a quadratic cost function with weighting matrices Q = R = 1
2
, find
regions in the (b - a )-plane where the interconnected syslem can be
beneficial, neutral, and nonbeneficial.
4.8. Extend the costate prediction approach of Section 4.5.5 to a composite
interconnected linear discrete-time system
4.9.
C4.10.
x(k + I) = A(k)x(k)+ B(k)u(k)+C(k)z(k)
where A and B are block-diagonal and Cis block-antidiagonal and z(k) is
the interconnection vector.
Use the costate prediction Algorithm 4.7 to find the optimum control (or
( )
[
0.9 0.2 .. ] ( ) [ O. I
x k+1 = 0.1 O.I-O.lxl(k) x k + 0
wi th x (0) = ( 10 5) T and a cost function
1 20
J='2 L

For a system with the matrices
r
-5
A =
. 0
- I
0.2
-2
o
- I
o
0.5
0.2
-I
o
- 0.5
0. 1
o
o
- 0.5
o
r1' B= ;1
-1 0 0
with Q = diag(l, 1,2,2,2) and R = 1
2
, find a hierarchical control law based
on the structural perturbation method of Section 4.4.2. What is the value of
the cost function deviation p? Use initial state x (O) = [ 1 I I 1 If
4.11. Usc Algorithm 4.6 on the structural perturbation approach for discrete-time
systems to find the optimum control for
with x(O) = (1 I )T and quadratic cost matrices Q = 1
2
, R = 0.51
2
,
~
..
~ I' . . : . ~ .
j' . ~ . I'
I
I
oj
,;1
:' '1
i
I ,
'1.
184 Hierarchical Control of Large-Scale Systems
Problems (continued)
4.12. Extend the discrete-time system's costate-prediction method of Section
4.5.5 to the continuous-time system
x(t) = f(x(t), u(t), t)
J = i x T ( If) Qx ( t f) + i fol, [ X T ( t) Qx ( t ) + u T ( t ) Ru ( t )] dt
References
Arafeh, S., and Sage, A P. 1974a. Multi-level discrete time system identification in
large scale systems. Int . J. Syst . Sci. 5:753-791.
Arafeh, S., and Sage, A P. 1974b. Hierarchical system identification of states and
parameters in interconnected power system. Int. J. Syst. Sci. 5:817-846.
Bauman, E. J. 1968. Multi-level optimization techniques with application to trajec-
tory decomposition, in C. T. Leondes, ed., Advances ill Control Systems, pp.
160-222.
Beck, M. B. 1974. The application of control and systems theory to problems of
river pollution. Ph.D. thesis, University of Cambridge, Cambridge, England.
Benveniste, A; Bernhard, P.; and Cohen, A 1976. On the decomposition of
stochastic control problems. IRIA Research Report No. 187, France.
Cheneveaux, B. 1972. Contribution a I'optimisation hierarchisee des systemes dy-
namiques. Doctor Engineer Thesis, No.4. Nantes, France.
Cohen, G.: Benveniste, A; and Bernhard, P. 1972. Commande hierarchisee avec
coordination en ligne d'un systeme dynamique. Renue Francaise d'Automatique,
Informatique et Recherche Operationnelle. J4:77-1O 1.
Cohen, G.; Benveniste, A; and Bernhard, P. 1974. Coordination algorithms for
optimal control problems Part I. Report No. A/57, Centre d'Automatique, Ecole
des Mines, Paris.
Cohen, G., and Jolland, G. 1976. Coordination methods by the prediction principle
in large dynamic constrained optimization problems. Proc. IFAC Symposium on
Large-Scale Systems Theory and Applications, Udine, Italy.
Davison, E. J., and Maki, M. C. 1973. The numerical solution of the matrix Riccati
differential equation. IEEE Trans. Aut. Cont. AC-18:71-73.
Dorato, P., and Levis, A H. 1971. Optimal linear regulators: The discrete-time case.
IEEE Trans. Aut. Cont. AC-16:613-620.
Fallaside, F., and Perry, P. 1975. Hierarchical optimization of a water supply
network. Proc. lEE 122:202-208.
Geoffrion, A M. 1970. Elements of large-scale mathematical programming: Part I,
Concepts, Part II, Synthesis of Algorithms and Bibliography. Management Sci.
16.
Geoffrion, A M. 1971a. Duality in non-linear programming. SIAM Review 13 : 1-37.
Geoffrion, A M. 1971 b. Large-scale linear and nonlinear programming, in D. A
Wismer, ed., Optimization Methods for Large Scale Systems. McGraw-Hill, New
York.
References
185
Haimes, Y. Y. 1977. Hierarchical Analyses of Water Resources Systems. McGraw-Hill,
New York.
Hassan, M. F., and Singh, M. G. 1976. A two-level costate prediction algorithm for
nonlinear systems. Cambridge University Engineering Dcpt. Report No.
CUEDF-CAMS/TR( 124).
Hassan, M. F., and Singh, M. G. 1977. A two-level costate prediction algorithm for
nonlinear systems. IFAC J. Automatica 13:629-634.
Hassan, M. F., and Singh, M. G. 1981. Hierarchical successive approximation
algorithms for non-linear systems. Part II. Algorithms based on costate coordina-
tion. J. Large Scale Systems. 2:81-95.
Hewlett-Packard Company, 1979. Interpolation: Chebyschev polynomial. System
9845 Numerical Analysis Library Part No. 9282-0563, pp. 247-256.
lyer, S. N., and Cory, B. J. 1971. Optimization of turbogcnerator transient perfor-
mance by differential dynamic programming. IEEE Trans. Power. Appal'. SySI.
90:2149-2157.
Jamshidi, M. 1975. Optimal control of nonlinear power systems by an imbedding
method. IFAC J. AU/olllatica. 11:633-636.
Jamshidi, M. 1980. An overview on the solutions of the algebraic matrix Riccati
equation and related problems. J. Large Scale Systems I: 167-192.
J anlshidi, M., and Heggen, R. 1. 1980. A multilevel stochastic managemen tmodel
for optimal conjunctive use of ground and surface water. Proc. I FA C SymposiulII
on Water and Related Land Resources Systems. (C\eveland, OH).
Jamshidi, M., and Merryman, 1. E. 1982. On the hierarchical optimization of
retarded systems via costate prediction. Proc. American COllII'. Conf., pp. 899-904
(Arlington, VA).
Kleinman, D. L. 1968. On the iterative technique for Riccati equation computations.
IEEE Trans. Aut. Cant. AC-13:114-115.
Mahmoud, M. S. 1977. Multilivel systems control and applications. IEEE Trans.
Syst . Man. and Cybern. SMC-7:125-143.
Mahmoud, M. 1979. Dynamic feedback methodology for interconnected control
systems. Int. J. Contr. 29:881-898.
Mahmoud, M.; Vogt, W.; and Mickle, M. 1977. Multi-level control and optimiza-
tion using generalized gradients technique. lilt. J. Contr. 25:525- 543.
March, J. G., and Simon, H. A 1958. Organizations. Wiley, New York.
Mesarovic, M. D.; Macko, D.; and Takahara, Y. 1969. Two coordination principles
and their applications in large-scale systems control. Proc. IFAC COllgr. Warsaw,
Poland.
Mesarovic, M. D; Macko, D.; and Takahara, Y. 1970. TheOlY of Hierarchical
Multilevel Systems. Academic Press, New York.
Mukhopadhyay, B. K., and Malik, O. P. 1973. Solution of nonlinear optimization
problem in power systems. Int. J. Colllr. 17:1041-1058.
Newhouse, A 1962. Chebyschev curve fit. C0Il1111 . ACM 5:281.
Pearson, J. D. 1971. Dynamic decomposition techniques, in Wismer, D. A, cd.,
Optimization Methods for Large-Scale Systems. McGraw-Hill, New York.
Sage, A P. 1968. Optimum Systems COlllrol . Prentice Hall, Englewood Cliffs, NJ.
186 Hierarchical Control of Large-Scale Systems
Sandell, N. R., Jr.; Varaiya, P. ; Athans, M.; and Safonov, M. G. 1978. Survey of
decentralized control methods for large scale systems. IEEE Trails. Aut. COI/t.
AC-23: \08-128. (special issue on large scale systems).
Schoeffler, J. D., and Lasdon, L. S. 1966. Decentralized plant control. ISA Trans.
5:175-183.
Schoeffler, J. D. 1971. Static multilevel systems, in D. A Wismer, ed., Optimization
Methods for Large-Scale Systems. McGraw-Hill, New York.
Siljak, D. D. 1978. Large-Scale Dynamic Systems. Elsevier North Holland, New
York.
Siljak, D. D., and Sundareshan, M. K. 1974. On hierarchical optimal control of
large-scale systems. Proc. 8th Asilomar Con/. Circuits Systems Computers, pp.
495-502.
Siljak, D. D., and Sundareshan, M. K. 1976a. A multi-level optimization of large
scale dynamic systems. IEEE TrailS. Aut. Cont. AC-21 :79-84.
Siljak, D. D., and Sundareshan, M. K. 1976b. Large-scale systems: Optimality vs.
reliability. Internal Report, University of California, Berkeley, Department of
Electrical Engineering.
Singh, M. G. 1975. River pollution control. lilt . J. Syst . Sci. 6:9- 21.
Singh, M. G. 1980. DYllamical Hierarchical COlltrol, rev. ed. North Holland,
Amsterdam, The Netherlands.
Singh, M. G., and Hassan, M. 1976. A comparison of two hierarchical optimisation
methods. lilt. J. Syst. Sci. 7:603-611.
Singh, M. G., and Titli, A 1978. Systems : Decomposition, Optimizatioll and Control.
Pergamon Press, Oxford, England.
Singh, M. G.; Drew, A W.; and Coales, J. F. 1975. Comparisons of practical
hierarchical control methods for interconnected dynamical systems. I FA C J.
Automatica II :331- 350.
Singh, M. G.; Titli, A; and Galy, J. 1976. A method for improving the efficiency of
the goal coordination method for large dynamical systems with state variable
coupling. Computers alld Electrical Engineering 2:339-346.
Smith, P. G. 1971. Numerical solution of the matrix equation AX + XA
T
+ B = O.
IEEE Trans. Aut. Cont . AC-16:278- 279.
Smith, N. H., and Sage, A P. 1973. An introduction to hierarchical systems theory.
Computers alld Electrical Engineerillg 1:55-71.
Sundareshan, M. K. 1976. Large-scale discrete systems: A two-level optimization
scheme. Int. 1. Syst . Sci. 7:901-909.
Sundareshan, M. K. 1977. Generation of multilevel control and estimation schemes
for large-scale systems: A perturbation approach. IEEE Trans. Syst.Man . Cyber.
SMC-7:144-152.
Takahara, Y. 1965. A multi-level structure for a class of dynamical optimization
problems. M.S. thesis, Case Western Reserve University, Cleveland, OH.
Tamura, H. 1974. A discrete dynamic model with distributed transport delays and
its hierarchical optimization to preserve stream quality. IEEE Trans. Sysl. Man.
Cyber. SMC-4:424- 429.
Tamura, H. 1975. Decentralized optimization for distributed-lag models of di scretc
systems. IFAC J. Automatica II :593-602.
References 10 1
Titli, A 1972. Contribution a l'etude des structures de hierarchisees en
vue de l'optimization des processes complexes, These d' Etat. No. 495, Toulouse,
France.
Ti tli , A ; Singh, M. G.; and Hassan, M. F. 1978. Hierarchical optimization of
dynamical systems using multiprocessors. Computers alld Electrical Engineering
5:3- 14.
Varaiya, P. 1969. A decomposition technique for non-linear programming. Internal
Report, University of California, Berkeley, Department of Electrical Engineering.

You might also like