AGEC 317 Dr.

Capps
Fall 2008
Derivation of Ordinary Least Squares Estimators
u x
y
i i
i
+ +
1 0
where
y
i
is a dependent variable, xi is an independent right-hand side
(RHS) variable,
i
u
is the error term (unobservable),

1 0
and
are coefficients. The
ordinary least squares procedure minimizes the error sum of squares (SSE). The
minimization problem is given as follows:
b) - (1 0 ) ( 2
a) - (1 0 ) 1 ( 2 . . .
) , (
1
^
1
^
0 ^
1
1
^
1
^
0 ^
0
2
1
^
1
^
0
1
2
^
1
^
0

,
_

,
_

,
_

i
n
i
i i
n
i
i i
n
i
i i
n
i
i
x x y
SSE
x y
SSE
C O F
x y u SSE Minimize

Step 1: Derive the OLS estimate of )

(
0 0


0
1
^
1
^
0

,
_

n
i
i i
x y
divide equation (1-a) by -2

0
1 1
^
1
1
^
0



n
i
i
n
i
n
i
i
x y
re-arrange



n
i
i
n
i
i
x n y
1 1
^
1
^
0
0
divide both sides by n.

0
_ ^
1
^
0
_
x y
re-arrange

_ ^
1
_ ^
0
x y
solve the OLS estimate
^
0

Step 2: Derive the OLS estimate of )

( ,
1 1



+
n
i
n
i
i i
x n y
1 1
^
1
^
0

re-arrange equation (1-a) (2)


+
n
i
n
i
i
n
i
i i i
x x y x
1 1
2
^
1
1
^
0

re-arrange equation (1-b) (3)
Now multiply equation (2) by the sum of
i
x
and equation (3) by n. Subsequently,



1
]
1

+
n
i
n
i
i
n
i
i i
n
i
i
x x n y x
1
2
1
^
1
1
^
0
1
(4)



+
n
i
i
n
i
i i
n
i
i
x n x n y x n
1
2
^
1
1
^
0
1
(5)
Subtract equation (4) from equation (5).
2
1
^
1
1 1
2
^
1
1 1
1
]
1

n
i
i
n
i
n
i
i i
n
i
i i
n
i
i
x x n y x y x n (6)
1
AGEC 317 Dr. Capps
Fall 2008
Solving equation (6) yields the OLS estimate of : )

(
1 1

2
1 1
2
1 1 1
^
1
1
]
1

n
i
i
n
i
i
n
i
i
n
i
i i
n
i
i
x x n
y x y x n

(7)
Step 3:
) (
) , (
) (
) )( (
1
2
_
1
_ _
2
1 1
2
1 1 1
^
1
x Var
y x Cov
x x n
y y x x n
x x n
y x y x n
n
i
i
n
i
i i
n
i
i
n
i
i
n
i
i
n
i
i i
n
i
i

1
]
1

The numerator of equation (7) can be re-written as follows:

_ _
2
1
_
1
_
1 1
_ _
) ( ) ( ) ( ) )( ( y x n x y n y x n y x n y y x x n
n
i
i
n
i
i i
n
i
i
n
i
i i





+
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i i
n
i
i
x y x y x y y x n
1 1 1 1 1 1 1
) (



n
i
i
n
i
i i
n
i
i
x y y x n
1 1 1
) (
(8)
The denominator of equation (7) can be re-written as follows:
(9)
2
) ( 2 ) (
2
1 1
2
2
1
2
1 1
2
1
2
_
2
_
1
2
1
2
_

,
_

,
_

+
,
_

+






n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
x x n
x x x n
x n x x n x n x x n
2