You are on page 1of 23

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1 Matrices and determinants


1.1

Matrices

Definition: An mn matrix is a rectangular array of numbers


(m rows and n columns) enclosed in brackets. The numbers
are called the elements of the matrix.
Examples:
(i) A 2 3 matrix has 2 rows and 3 columns:

1 2 3
A=
5 6 7
(ii) Heres a 3 3 square matrix:

1 2 3
A=5 6 7
8 9 10
(iii) Column vectors are matrices with only one column:

1
b=5
8
(iv) Row vectors are matrices which only have one row:
b = (1 2 3) .
Unless specifically stated otherwise, we will assume that vectors are column vectors.

1.1

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

A general real matrix, A Rmn with m n elements is of


the form

a11 a12 a13 . . . a1n


a

21 a22 a23 . . . a2n

A = a31 a32 a33 . . . a2n


(1)
..

.
.
.
.
..
..
. . ..
.
am1 am2 am3 . . . amn
We refer to the elements via double indices as follows
(i) The first index represents the row.
(ii) The second index represents the column.
Example
a32 is the element in row 3, column 2 of the matrix A.
Notation/Conventions:
Use lowercase boldface (or underlined) letters for vectors
a b c

(or

a,

b,

c)

Use uppercase boldface (or underlined) letters for matrices


A

B C

(or

A,

B,

C)

Refer to the respective elements by lowercase letters with the


appropriate number of indices e.g.
bi
aij

is a vector element
is a matrix element

1.2

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.2 Special matrices


The unit matrix, I, is a square matrix whose only non-zero
elements are on the diagonal and are equal to one, e.g.

1 0 0 ... 0 0
0 1 0 ... 0 0

1 0 0

0
0
1
.
.
.
0
0

I = 0 1 0 , I = .. .. .. . . .. ..
. . . . . .

0 0 1
0 0 0 ... 1 0
0 0 0 ... 0 1
All elements of the zero matrix, 0, are equal to zero, e.g.

0 0 0 ... 0 0
0 0 0 ... 0 0

0 0 0

0
0
0
.
.
.
0
0

0 = 0 0 0 , 0 = .. .. .. . . .. ..
. . . . . .

0 0 0
0 0 0 ... 0 0
0 0 0 ... 0 0
A diagonal matrix only has non-zero elements on the main
diagonal. These non-zero elements can have any value, e.g.

d11 0 . . .
0
0

0
0
d11 0 0
0 d22 . . .

.
.
.
.
.
..
..
..
..
D = 0 d22 0 , D = ..

0 0 d33
0 0 . . . dn1,n1 0
0 0 ...
0
dnn
are square diagonal matrices.

1.3

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3 Matrix algebra


1.3.1

Matrix equality

Two matrices are equal if they have the same size and if their
corresponding elements are identical , i.e.
A=B
if and only if
aij = bij

for i = 1, ..., m; j = 1, ..., n

1.3.2 Matrix addition


Two matrices can only be added if they have the same size.
The result is another matrix of the same size.
We add matrices by adding their corresponding elements, i.e.
A=B+C
is obtained (element-wise) via
aij = bij + cij
Example

A=

A+B=

1 2 3
5 6 7

for i = 1, ..., m; j = 1, ..., n

B=

1 + 10 2 + 1 3 + 23
5 + 5 6 + 16 7 + 3

1.4

10 1 23
5 16 3

11 3 26
10 22 10

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3.3 Multiplication of a matrix by a scalar


A matrix is multiplied by a scalar (= a number) by multiplying
each element of the matrix by that scalar.
The result is a matrix of the same size.
Hence
A=B
is given (element-wise) by
aij = bij

for i = 1, ..., m; j = 1, ..., n

Example

1 2 3
A=
5 6 7

31 32 33
3 6 9
3A=
=
35 36 37
15 18 21
1.3.4 Matrix Vector multiplication
Before defining the product of a matrix and a vector, let us
recall the notion of the dot product (or, scalar product) of two
vectors.
Let a Rn and b Rn be two column vectors with n real
elements each:


a1
b1
a
b
2
2


a = a3 ,
b = b3 .
..
..
.
.
an
bn
1.5

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

Then the dot product (or, scalar product) of a and b is defined


as
a b = a1b1 + a2b2 + a3b3 + + anbn.
Note that the row vector corresponding to the (column) vector
a can be defined by the vector transpose operation:

T
a1
a
2

T
a = a3 = a1 a2 a3 . . . an .
..
.
an
Then the product of the row vector aT and the column vector
b can be defined as follows:
def

aT b = a b = a1b1 + a2b2 + a3b3 + + anbn

(2)

Example

2
5
5

a = 1 , b =
2 ; a b = 2 1 3 2
3
4
4

5
2
= 1 2 = 10 2 12 = 4.
4
3

1.6

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

Now we can define a matrix vector multiplication.


The product of an m n matrix A and a column vector x of
length n produces a column vector b of length m:
A |{z}
x = |{z}
b .
|{z}
mn n1

m1

This implies that the number of columns of the matrix must


be equal to the number of rows in the column vector!
The elements of the vector b are calculated by taking
dot products of the rows of the matrix with the column vector:
if rows of A Rmn are vectors aT1 , aT2 , . . ., aTm Rn then

aT1
aT2

A = ..
.
aTm

aT1
aT2

A x = .. = b
.
aTm x

where aTi x are dot products defined in equation (2).


Element wise this is
n
X
aij xj = bi

for i = 1, ..., m.

j=1

1.7

(3)

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

Example

A=

1 2 3
5 6 7

3
x=2
4

then A consists of the row vectors


aT1 = ( 1 2 3 )
aT2 = ( 5 6 7 )
and the matrix vector product

( 1 2 3 ) 2

1 2 3
4
.
Ax =
2 =
5 6 7
3

4
( 5 6 7 ) 2
4
The vector products are simply the dot products of the respective row aTi with the column vector x

13+22+34
Ax =
53+62+74
and hence

b=

3 + 4 + 12
15 + 12 + 28

1.8

19
55

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3.5 Matrix Vector multiplication using Falks scheme


For practical computations use Falks scheme and evaluate
Ax = b
in tabular form as follows:

x
A b
Example

A=

1 2 3
5 6 7

3
x=2
4

Evaluate Ax as follows:
3
2
4
1 2 3
5 6 7
Now place the elements of the resulting vector at the intersection of the column vector and the rows of the matrix:

1 2 3
5 6 7

3
2
4

13+22+34
19
=
53+62+74
55

1.9

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3.6 Matrix Matrix multiplication


The product of an m n matrix A and an n p matrix B
produces an m p matrix C, i.e.
A |{z}
B = |{z}
C .
|{z}
mn np

mp

This implies that the number of columns of the first matrix


must be equal to the number of rows in the second matrix.
Here are some examples:

A=

1 2 3
5 6 7

1 2 3
B=5 6 7
8 9 10

23

33

1 2 3 2
C=5 6 7 2
8 9 10 7

5
D=
8
3

34

9
7

42

We can form
AB the result is a 2 3 matrix
AC the result is a 2 4 matrix
CD the result is a 3 2 matrix
It is not possible to form
BA

or
1.10

AD

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

The product matrix of two matrices is obtained by taking dot


products of the rows of the left matrix with the columns of the
right matrix.
If the rows of A Rmn are the vectors aT1 , aT2 , . . ., aTm Rn
and the columns of B Rnp are b1, b2, . . ., bp Rn. Let

T
a1
T

a2
A = .. and B = b1 b2 . . . bp
.
aTm
then the matrix matrix product is

aT1
aT2

b1
b1

aT1
aT2

bp

bp

AB = ..
Rmp
.
.
.
.
aTm b1 aTm b2 . . . aTm bp

Element-wise this is
n
X
aij bjk = cik

b2 . . .
b2 . . .
...
...

aT1
aT2

for i = 1, ..., m; k = 1, ..., p.

j=1

1.11

(4)

2E1: Linear Algebra | Lecture Notes

Example

A=

1 Matrices and matrix algebra

1 2
3 4

B=

4 3 2
2 1 1

The respective row and column matrices are





aT1 = (1 2)
4
3
2
b1 =
, b2 =
, b3 =
,
T
2
1
1
a2 = (3 4)
the matrix-matrix product is then

4
3
(1
2)
(1
2)

2
1

AB =

4
3
(3 4)
(3 4)
2
1

(1 2)

2
1


2
(3 4)
1

which can be evaluated by a series of dot products. That is

14+22 13+21 12+21

AB =
34+42 33+41 32+41
Giving the final result

C=

20

13

10

1.12

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3.7 Matrix Matrix multiplication using Falks scheme


For practical computation we again use Falks scheme and
evaluate AB = C in tabular form as follows:

B
A C
Example

A=

1 2
3 4

B=

4 3 2
2 1 1

Evaluate AB as follows
4
3
2
2
1
1
1 2 (1 4 + 2 2) (1 3 + 2 1) (1 2 + 2 1)
3 4 (3 4 + 4 2) (3 3 + 4 1) (3 2 + 4 1)
Place the elements at the intersection of the rows of the left
matrix and the columns of the right matrix
The result
4
3
2
2
1
1
1 2 (1 4 + 2 2) (1 3 + 2 1) (1 2 + 2 1)
3 4 (3 4 + 4 2) (3 3 + 4 1) (3 2 + 4 1)
therefore gives

AB =

8 5 4
20 13 10
1.13

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.3.8 Differences from multiplication with numbers


(i) Matrix multiplication is not commutative
AB 6= BA
Example

1 2
2 2
12+23 12+21
=
4 1
3 1
42+13 42+11

8 4
=
11 9

2 2
3 1

1 2
4 1

21+24 22+21
=
31+14 32+11

10 6
=
7 7

We must be careful how we multiply out!


(ii) AB = 0 does not imply A = 0, B = 0 or BA = 0
Example

1 1
1 1
0 0
=
2 2
1 1
0 0

1 1
1 1

1 1
2 2

1.14

1 1
1 1

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

(iii) AC = AD does not necessarily imply C = D


Example

1 1
2 2
1 1
2 2

2 1
2 2
3 0
1 3

4 3
8 6
4 3
8 6

(iv) BUT other properties are similar to numbers


A(B + C) = AB + AC

distributive law

A(BC) = (AB)C

associative law

1.15

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.4 Transpose of a matrix


The transpose of a matrix is obtained by interchanging its
rows and columns
aTij = aji

for i = 1, ..., m; j = 1, ..., n

The transpose is denoted by a superscript T and the general


matrix given in equation (1) becomes

a11 a21 a31 . . . am1


a a a ... a
12 22 32
m2

T
A = a13 a23 a33 . . . am3
..
...
... . . . ...
.

a1n a2n a3n . . . amn


Example

A=

1 2 3
5 6 7

1 5
AT = 2 6
3 7

If A = AT then A is a symmetric matrix, e.g.

3 2 1
A= 2 7 0
1 0 8
The matrix transpose also satisfies the following rules:
i) (AT )T = A for any matrix A;
ii) (A + B)T = AT + BT and (AB)T = BT AT ,
provided that matrices A and B have compatible dimensions.
1.16

2E1: Linear Algebra | Lecture Notes

?<

1 Matrices and matrix algebra

Proof of the identity (AB)T = BT AT

If the rows of A Rmn are the vectors aT1 , aT2 , . . ., aTm Rn


and the columns of B Rnp are b1, b2, . . ., bp Rn.
Taking the transpose of equation (4)

T
(AB) =

aT1
aT1

...

b1 aT2
b2 aT2

b1 . . .
b2 . . .
...
...

aTm b1
aTm bp
...

aT1 bp aT2 bp . . . aTm bp


For column vectors ai and bj vector multiplication aTi bj is
defined as the dot product between ai and bj . The dot product
is commutative so
aTi bj = ai bj = bj aj = bTj ai
which implies that

bT1
bT2

a1 bT1
a1 bT2

am

am

T
(AB) = ..
.
..
.
bTp a1 bTp a2 . . . bTp am

since

bT1
bT2

= B T AT

B = ..
.
bTp
T

a2 . . .
a2 . . .
...
...

bT1
bT2

and AT =

a1 a2 . . . am .

>?

Here and further in these notes, the material between the


markers ? < and > ? is for advanced reading.
1.17

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.5 Determinant of a matrix


The determinant of a 2 2 matrix

a11 a12
A=
a21 a22
is written det A or |A| or

a11 a12

a21 a22 = a11a22 a12a21


Example
A=

1 2
4 7

det A = 7 8 = 15.

The determinant of a 3 3 matrix is written as

a11 a12 a13

|A| = a21 a22 a23


a31 a32 a33

a21 a23
a21 a22
a22 a23

a12
+ a13
= a11

a31 a32
a32 a33
a31 a33
= a11(a22a33 a32a23) a12(a21a33 a31a23)
+ a13(a21a32 a31a22)
That is the 3 3 determinant is defined in terms of determinants of 2 2 sub-matrices of A. These are called the minors
of A.

1.18

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

Example

a11 a12 a13

a21 a23

m12 = a21 a22 a23 =


a31 a33
a31 a32 a33
is obtained by suppressing the elements in row 1 and column
2 of matrix A.
Cofactors
The cofactor cij is defined as the coefficient of aij in the determinant A. If is given by the formula
cij = (1)i+j mij
where the minor is the determinant of order (n 1) (n 1)
formed by deleting the column and row containing aij .
Examples

a11 a12 a13

c11 = (1)1+1m11 = +1 a21 a22 a23 = a22a33 a32a23


a31 a32 a33

a11 a12 a13

2+3

c23 = (1) m23 = 1 a21 a22 a23 = a11a32 + a31a12


a31 a32 a33

1.19

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

General determinant
The value of an n n determinant equals the sum of the products of the elements in any row (or column) and their cofactors, i.e.
n
X
|A| =
aij cij , for i = 1, . . . , n 1, or n
j=1

or
|A| =

n
X

aij cij ,

for

j = 1, . . . , n 1, or n

i=1

Example
For a 3 3 matrix
det A = a11c11 + a12c12 + a13c13

(1st row)

or
det A = a12c12 + a22c22 + a32c32

(2nd column)

Points to note:
the determinant det A is equal to zero if
(i) rows or columns of A are multiples of each other,
(ii) rows or columns are linear combinations of each other,
(iii) entire rows or columns are zero;
if det A = 0 the matrix A is called a singular matrix;
for any square matrices A and B there holds
det A = det(AT ),

det(AB) = det(A) det(B).

for the unit matrix I one has det I = 1.


1.20

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

1.6 The matrix inverse


The inverse a1 of a scalar (=a number) a is defined by
a a1 = 1.
For square matrices we use a similar definition: the inverse
A1 of a n n matrix A fulfils the relation
AA1 = I
where I is the n n unit matrix defined earlier.
Note: if A1 exists then
det(A) det(A1) = det(AA1) = det I = 1.
Hence, det(A1) = (det A)1.
Example
The inverse of

A=

is given by

3 2
7 5

B = A1 =

5 2
7 3

since
5
2
7
3
AB =
3 2 (3 5 2 7) (3 2 + 2 3)
7 5 (7 5 5 7) (7 2 + 5 3)
which gives

AB =

1 0
0 1

as required.
1.21

=I

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

The matrix inverse can be computed as follows


1. Find the determinant det A
2. Find the cofactors of all elements in A and form a new
matrix C of cofactors, where each element is replaced by
its cofactor.
3. The inverse of A is now given as
A

CT
=
det A

Note: the inverse A1 exists if (and only if) det A 6= 0.


Example Find the inverse of

1 1 2
A = 3 1 2 .
3 2 1

1 2
3 2
3 1
(1)

det A = 1
3 1 + 2 3 2
2 1
= 1 3 + 1 (3) + 2 3
=6
Since the determinant is nonzero an inverse exists.

1.22

2E1: Linear Algebra | Lecture Notes

1 Matrices and matrix algebra

Calculate the matrix of minors


3 2
1
2

2 1 3 1

1 2 1 2

M=
2 1 3 1

1 2 1 2

1 2 3 2

3 3 3
= 5 7 1
4 8 2

3 1

3 2

1 1

3 2

1 1

3 1

Modify the signs according to whether i + j is even or odd to


calculate the matrix of cofactors

3 3 3
C = 5 7 1 .
4 8 2
It follows that

A1

3 5 4
1
1
= CT = 3 7 8 .
6
6
3 1 2

To check that we have made no mistake we can compute

1 0 0
1 1 2
3 5 4
1
A1A = 3 7 8 3 1 2 = 0 1 0 .
6
0 0 1
3 2 1
3 1 2
This way of computing the inverse is only useful for hand
calculations in the cases of 2 2 or 3 3 matrices.

1.23

You might also like