zar97/09/03

Numerical Methods For Engineers

page 3-31

3.3 LU Decomposition
Recalling that in the Gauss elimination method we have forward elimination process
which involves considerable computations. The task is especially more cumbersome
for large systems of LAEs. Thus, the improved method is recommended.
The coefficient matrix A can be decomposed (or factored) into the product of two
triangular matrices, L and U, a lower and upper triangular matrix, respectively.
A = LU

(3.24)

Using a set of four LAEs as illustration, eq.(3.24) can be represented into two
forms,
a. Dolittle decomposition:
a11 a12
a
a 22
21
a31 a 32

a41 a 42

a13 a14
a23 a 24

a33 a 34

a43 a 44

1
l
21
l31

l41

0
1

0
0

a32

l42

l43

l11 0
l
l
21 22
l31 a 32

l41 l42

0
0

0 u11
0 0

0 0

1 0

u12
u22

u13
u23

u33

0 1 u12
0 0 1

0 0 0

l44 0 0

u13
u23

u14
u24

u34

u44

(3.25a)

u14
u24

u34

(3.25b)

b. Crout decomposition:
a11 a12
a
a 22
21
a31 a 32

a41 a 42

a13 a14
a23 a 24

a33 a 34

a43 a 44

l33
l43

1
0

Both forms yield a comparable degree of accuracy and efficiency. It is just a matter
of ones preference. For this class, we will only concentrate on the latter.

zar97/09/03

Numerical Methods For Engineers

page 3-32

Once we have L and U matrices, we can then determine the solution of the LAEs
through the following manner
Ax = b
LUx = b
and defining
y = Ux

(3.26a)

Ly = b

(3.26b)

Eqs.(3.26b) is first solved for y, then substitutes it into eq.(3.26a) to solve for the
unknown x. Note, they can easily be solved since they are both in the triangular
form.
To derive the L and U matrices, lets now perform matrix multiplication of eq.(3.25b),

which yields
l11 = a11
or in general,

l21 = a21
li1 = ai1

l31 = a31
for i = 1,2,...,n

l41 = a41
(3.27)

Next, we have
l11 = a11

l11u12 = a12

l11u13 = a13

l11u14 = a14

The first result is already found, but the remaining relationships can be solved for
u12, u13, and u14. These terms can be expressed as

u1 j

a1 j
l11

for

j 2 ,3,..., n

(3.28)

zar97/09/03

Numerical Methods For Engineers

page 3-33

Similar operations are done to obtain the remaining columns of L and rows of U.
The steps to implement the entire procedures are expressed in the following
concise formulas.
li1 ai1
a1 j
u1 j
l11
For

for

i 1,2 ,..., n

(3.27)

for

j 2 ,3,..., n

(3.28)

j 2 ,3,..., n 1
j 1

lij aij

lik ukj

for

i j , j 1,..., n

(3.29)

for

k j 1, j 2 ,..., n

(3.30)

k 1
j 1

a jk
u jk

l ji uik
i 1

l jj

and
n 1

lnn ann

lnk ukn

(3.31)

k 1

Remark:
1. The method is just as fast, but more accurate than the Gauss elimination method.
Thus, it becomes the basis for the decomposition of positive definite matices which
are frequently encountered in modern engineering and statistics.
2. The pivoting is done immediately after computing each column of L.

zar97/09/03

Numerical Methods For Engineers

page 3-34

Example 3.10: LAEs using LU decomposition.

Problem Statement: Solve the following set of linear algebraic equations using
Crout LU decomposition method.
5x1 + 2x2 + 3x3 + x4 = 17
x1 - x2 + 2x3 + 5x4 = -3
2x1 + 3x2 + 5x3 - 2x4 = 4
8x1 + 5x2 - x3 - 2x4 = 50
Solution:
Using eq.(3.25b),

5 2 3 1
1 1 2 5

2 3 5 2

8 5 1 2

l11 0
l
l22
21

l31 a32

l41 l42

0
0
l33
l43

0 1 u12
0 0 1

0 0 0

l44 0 0

u13 u14
u23 u24

1 u34

0
1

Eq.(3.27) implies that the first column of L is

l11 = a11 = 5
l21 = a21 = 1
l31 = a31 = 2
l41 = a41 = 8
Then eq.(3.28) gives the first row of U as
u12

a12 2

l11 5

u13

a13 3

l11 5

Eq.(3.29) gives the second column of L as

2
7
5
5
2
11
- l31u12 = 3 - 2( ) =
5
5
2
9
- l41u12 = 5 - 8( ) =
5
5

l22 = a22 - l21u12 = -1- 1( ) =

l32 = a32
l42 = a42

u14

a14 1

l11 5

zar97/09/03

Numerical Methods For Engineers

page 3-35

Eq.(3.30) gives the second row of U as

3
a23 l21u13 2 1( 5 )
u23

1
7
l22
5

u24

1
a24 l21u14 5 1( 5 ) 24

7
l22
7
5

j=
I=
lij aij

lik ukj

l33 a33 ( l31u13 l32 u23 ) 5 [(2)( 35 ) ( 11

)( 1)] 6
5

k 1

l43 a43 ( l41u13 l42 u23 ) 1 [(8)( 35 ) ( 95 )(1)] 4

a jk
jk

l ji uik
i 1

l jj

u34

1
11
a34 ( l31u14 l32 u24 ) 2 [(2)( 5 ) ( 5 )(24)] 6

l33
6
7

j=
I=
lnn ann

lnk ukn

l44 a44 ( l41u14 l42 u24 l43u34 )

k 1

2 [(8)( 15 ) ( 95 )( 724 ) ( 4)( 76 )] 6

zar97/09/03

Numerical Methods For Engineers

page 3-36

Worksheet 3.2: LAEs using LU decomposition method.

Problem Statement: Solve the following set of linear algebraic equations using
Crout Decomposition with partial pivoting .
2x1 - 6x2 - 2x3 = -24
3x1 - 2x2 + x3 = 0
3x1 - 3x2 + 6x3 = 5
Solution:

Answer:

zar97/09/03

Numerical Methods For Engineers

page 3-37

Banded systems
In many engineering problems, linear systems with a band structure arise. This
structure can be seen, for example, in computations of spline function, solution of
boundary-value problems of differential equatuions using finite diffrence method,
etc.
In such systems, the band matrices have the nonzero elements on the major
diagonal and on the diagonals just above and below the main diagonal.
In general, a square matrix A = [ aij ] is a band matrix if there is a positive integer
k substantially smaller than the order n of the matrix such that aij = 0 if i j k .
For example, in the case of tridiagonal matrix, k = 2 which is the width of the band
matrix.
A band matrix saves storage requirements i.e. only 3n-2 as compared to nxn = n2
elements to be stored.
The number of additions and multiplications for the band matrix is 3n-3 and for
divisions is 2n-1, which for large n, it is nothing compared to that n3/3 for
conventional Gauss elemination method.

zar97/09/03

Numerical Methods For Engineers

page 3-38

Worksheet 3.3: Banded LAEs using LU decomposition method.

Problem Statement: Solve the following set of linear algebraic equations using
Crout Decomposition with partial pivoting .
x1 + 2x2
= 2
x1 + 3x2 + 2x3
= 7
x2 + 4x3 + 2x4 = 15
4x3 + x4 = 11
Solution:

zar97/09/03

Numerical Methods For Engineers

page 3-39

Choleskys Method ( or the method of sqaure roots)

If the coefficient matrix A is real and symmetric, the LU decomposition can be
modified so that the two factors are the transpose of each other. That is
A = L TL

or

UTU

For the kth row, it can be shown that

i 1

a ki
lki

lij lkj
j 1

lii
k 1

lkk akk

lkj2
j 1

for

i 1,2,..., k 1