EMTH211 Tutorial 9 Solutions

Textbook exercises
7.3, #7,8,13,14,18,27,44
#7 To find the least square solution to

1 1  
0
1 2 c0 = 1
c1
1 3
5
Using MATLAB ([1, 1; 1, 2; 1, 3]\[0; 1; 5]) we get c0 = 3 and c1 = 2.5. So the line
y = 3 + 2.5x best fits these three points.
The residual error is

0
1 1  
0.5
c
r = 1 1 2 0 = 1
c1
5
1 3
0.5

So krk = 1.5.
#8 To find the least square solution to

1 1  
5
1 2 c0 = 3
c1
1 3
2
Using MATLAB ([1, 1; 1, 2; 1, 3]\[5; 3; 2]) we get c0 = 6 13 and c1 = 1.5. So the line
y = 6 13 1.5x best fits these three points. The residual error is

1
5
1 1  
6
c
r = 3 1 2 0 = 31
c1
1
2
1 3
6
So krk 0.4082.
#13,14 In both cases we are trying to find the least square solution to the system of the form

1 x1  
y1
.. .. c0
..
= .
. .
c1
1 xn
yn
With the help of MATLAB (simply using A\y): we get c0 = 0.2, and c1 = 1.4 for #13,
and c0 = 12.7, and c1 = 2.5 for #14. Notice that we have a positive slope for the first
one and a negative one for the second one, which is exactly, what we would expect,
since the measurements for #13 go up, and the ones for #14 go down.
#18 As in the previous exercise, but now were solving

1 x1 x21
y1
c0
.. ..
.. c = ..
. .
.
. 1
2
c2
1 xn xn
yn
1

Using MATLAB ([ones(5, 1) x x. x]\y), we get c0 = 12.4, c1 = 1.8, and c2 = 4. So

the parabola
y = 4x2 + 1.8x 12.4 = 4(x 1.55)(x 2)
best fits these measurements.
#27 We have seen that in this the solution is given by solving the equation
c = R1 QT b ,
instead of solving the slightly more complicated normal equation. So in this case the
least square solution c is

 
1
 2
 
 
1 1
1 5
3
2 2 1 1 15
3
3 =
c=
=
0 1 3 1 2 2
9 6
3 2
1
#44

1. P is symmetric:
P T = (A(AAT )1 AT )T = (AT )T (AAT )T AT = A(AAT )1 AT ,
since AAT is symmetric and therefore also its inverse is; that is (AAT )T =
(AAT )1
2. P is idempotent:
P P = (A(AAT )1 AT )(A(AAT )1 AT ) = A(AAT )1 (AT A)(AAT )1 AT
= A(AAT )1 AT = P

Supplementary exercise
1. Compute the pseudo-inverse for A, where
(a)

(b)

 
1
A=
2

1 3
A = 1 1
0 2

(c)

1 1
A=
0 1
2. Orthogonally diagonalise

3
0
A=
0
1
2

0
2
0
0

0
0
2
0

1
0

0
3

1.

(a) To compute the pseudo-inverse

AT A = [5]
So
(AT A)1 AT =


1
1 2
5

(b) To compute the pseudo-inverse



1 3
1 1 0
2
2
T
1 1 =
A A=
3 1 2
2 14
0 2


So

 1 3 T




1
1 2 4 1
1 8 16 4
14 2
T
1 T

1 1 =
(A A) A =
=
4
1
28 4 2 2
24 4 4
6 1 1
0 2


(c) To compute the pseudo-inverse



1 0
A A=
1 1
T

So


 

1 1
1 1
=
0 1
1 2


 

2 1 1 0
1 1
(A A) A =
=
1 1
1 1
0 1
T

Notice that this is simply the inverse; which, as weve seen in the lectures is always the case if A is square (and the columns of A are invertible, which was a
requirement to actually compute the pseudo-inverse).
2. First, we compute the eigenvalues:
det(AI) = (3)2 (2)2 (2)2 = (2)2 ((3)2 1) = (2)2 (4)(2)
Hence the eigenvalues are 2, 2, 2, 4. The corresponding eigenvector for 4 is

1
0

E4 = span
0
1
The general solution for (A 2I)x = 0 is

s
r
,
t
s
for arbitrary s, t, r. So

1
0
0
0 1 0

E2 = span
0 , 0 , 1
1
0
0
3

These are already orthogonal! So there is nothing to do, apart from scaling them, to get orthonormal vectors. Let

1
2

0
Q=
0

1
2

then A = QDQT , where

4
0
D=
0
0

1
2

0
0
12
0
2
0
0

0
1
0
0

0
0
,
1
0

0 0
0 0
,
2 0
2