Econometrics

An Introduction to Asymptotic Theory

Yuyi LI
University of Manchester

2012

Yuyi LI (University of Manchester)

An Introduction to Asymptotic Theory

2012

1 / 12

Finite Sample Results - Revision

Multiple linear regression model
1

Linear in parameters y = X + u Random Sample {(yi , xi1 , xi2 , . . . , xik ) : i = 1, 2, . . . , n} = independently and identically distributed (iid) data No perfect collinearity: X has full (column) rank (1 + k ). Zero conditional mean: E [u|X] = 0 Homoskedasticity: Var (u|X) = 2 In

3 4 5

1-4 OLS Unbiased 1-5 OLS BLUE 6 Normality: u N (0, 2 In )

1-6 OLS: N (, 2 (X X)1 ) Hypothesis Testing (Inference)

[Notation remains consistent with previous notes, whenever possible.] Asymptotic Theory Reading: Wooldridge Chapter 5, Appendix C
Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 2 / 12

Asymptotic Theory

Convergence in Probability and LLNs

Convergence in Probability
Consider the deterministic sequence an = n1 , n where lim an = 1

As n , an converges to 1 or the limit of an is 1 Let Zn denote a random variable indexed by the sample size n. Denition (Convergence in Probability) Random variable Zn converges (in probability) to a constant c if
n

lim Pr (|Zn c | > ) = 0

for any > 0

and it is written as Zn c
Yuyi LI (University of Manchester)

as n ,

or

plim Zn = c
n
2012 3 / 12

An Introduction to Asymptotic Theory

Asymptotic Theory

Convergence in Probability and LLNs

Consistency
Let be an estimator of the population parameter
Denition (A Consistent Estimator)

Estimator is a consistent estimator of if and only if as n or

p

plim =
n

Note, sample size plays a vital part in consistency

Unbiased estimator of : E [] = - Sample size plays no part

Consistency and Unbiasedness are very Different concepts! Consistency v.s. Unbiasedness (not correct but helpful)
Consistency: get it right when the sample size is large enough Unbiasedness: get it right on average

Yuyi LI (University of Manchester)

An Introduction to Asymptotic Theory

2012

4 / 12

Asymptotic Theory

Convergence in Probability and LLNs

Law of Large Numbers (LLN)

Theorem (LLN for IID Data) Let Z1 , Z2 , . . . , Zn be iid random vectors with mean vector < . Then

Zn

1 n

Zi
i =1

as n .

Sample averages of iid data converge to the means Example (LLN) Consider random regressors matrix X (iid) with the i-th row xi = [1, xi1 , xi2 , , xik ]. If E [xi xi ] = Qxx < , positive denite (p.d.), then XX 1 n n
Yuyi LI (University of Manchester)

xi xi Qxx
i =1

An Introduction to Asymptotic Theory

2012

5 / 12

Asymptotic Theory

Convergence in Distribution and CLT

Convergence in Distribution
Random variable Zn Let the Cumulative Distribution Function (CDF) of Zn be Fn (z ). Let the CDF of another random variable Z be F (z ) Denition (Convergence in Distribution) Random variable Zn converges in distribution to Z is
n

lim |Fn (z ) F (z )| = 0

at all continuous points of F (x )

and it is written as Zn Z
d

as n

F (z ) is known as the limit distribution of Zn If F (z ) is the standard normal CDF, then Zn is said to have an asymptotic standard normal distribution.
Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 6 / 12

Asymptotic Theory

Convergence in Distribution and CLT

Central Limit Theorem (CLT)

Theorem (Lindberg-Levy CLT) Let {Z1 , Z2 , . . . , Zn } be a random sample from a multivariate distribution with mean vector < and p.d. variance-covariance matrix . Then

1 n

(Zi )
i =1

d n Zn N (0, )

Weighted sample averages converge to a multivariate normal Example (CLT) Consider the IID random vector xi ui . Let E [xi ui ] = 0 and Var (xi ui ) = E [ui2 xi xi ] = 2 Qxx < . Then Xu n

1 n

xi ui N (0, 2 Qxx )
i =1
2012 7 / 12

Yuyi LI (University of Manchester)

An Introduction to Asymptotic Theory

Asymptotic Theory

Asymptotic Results for OLS Estimators

Assumptions
Assumptions 1 A1. y = X + u, X = [x1 , x2 , , xn ] is n (1 + k ) 2 A2. Random sampling (yi , xi : i = 1, 2, . . . , n) are iid 3 A3. rank (X) = (1 + k ) < n 4 A4. E [xi ui ] = 0, E [xi xi ] = Qxx < and p.d. 5 A5. E [ui2 xi xi ] = 2 Qxx < and p.d. and var (ui |xi ) = 2 i Comments A1 (Linear in parameters) yi = xi + ui , i = 1, 2, . . . , n A2 (ui , xi ui ) are IID

A1+A3 OLS: = (X X)1 X y = + (X X)1 X u A1+A4 OLS Estimators: Consistent A5 Homoskedastic ui A1+A5 OLS Estimators: Asymptotically Normal
Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 8 / 12

Asymptotic Theory

Asymptotic Results for OLS Estimators

LLNs and CLT

As n LLN 1 XX 1 n n LLN 2 X uu X 1 n n LLN 3 Xu 1 n n CLT 1 Xu n 1 n
n n p

xi xi E [xi xi ] = Qxx
i =1

ui2 xi xi E [ui2 xi xi ] = 2 Qxx

i =1 n

xi ui E [xi ui ] = 0
i =1 n

xi ui N (0, 2 Qxx )
i =1
2012 9 / 12

Yuyi LI (University of Manchester)

An Introduction to Asymptotic Theory

Asymptotic Theory

Asymptotic Results for OLS Estimators

Consistency of OLS Estimators

Theorem (OLS Consistency) Let Assumptions A1-A4 hold. Then as n , the OLS estimators

p , i.e. are consistent estimators of the population parameters .

Proof.

Note = (X X)1 X y = + (X X)1 X u, or = (X X)1 X u. Then =

p

XX n

1 Xu = n n 0 = 0,

i =1

1 1 xi xi n

i =1

xi ui

(Qxx )

as n
p

using LLNs 1 and 3 and Slutskys theorem. Thus, , as n .

Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 10 / 12

Asymptotic Theory

Asymptotic Results for OLS Estimators

Asymptotic Normality of OLS Estimators

Theorem (OLS Asymptotic Normality) Let Assumptions A1-A5 hold. Then as n , the OLS estimators

i.e.

n N 0, 2 Q1 , xx

n( ) has a normal limit distribution.

Proof.

Note, = (X X)1 X u. By LLNs 1-3, CLT 1 and Continuous Mapping Theorem, n =

d

XX n

Xu

1 = n

i =1

1 1 xi xi

i =1

xi ui

(Qxx )1 N 0, 2 Qxx = N 0, 2 Q1 , as n xx
Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 11 / 12

Asymptotic Theory

Remarks
Lecture notes L2-L4 provide nite-sample results
Assume error terms are normally distributed Fixed or predetermined regressors OLS estimators have exact normal distribution Exact inference (hypothesis testing) - t, F tests

Lecture notes L5 provide asymptotic (large-sample) results

Normality of the error terms is Not assumed Regressors can be random/stochastic Asymptotic normality can be achieved using CLT (and LLNs) OLS estimators have normal limit distribution Asymptotic (large-sample) inference - asymptotically valid t, F tests

Asymptotic theory: what happens when n ? as an approximation LLNs and CLTs are fundamental to asymptotic analysis Consistency and asymptotic normality are 2 typical asymptotic results Asymptotic analysis is essential to advanced econometric theory
Yuyi LI (University of Manchester) An Introduction to Asymptotic Theory 2012 12 / 12