Lecture Notes

On Binary Choice Models:

Logit and Probit
Thomas B. Fomby
Department of Economic
SMU
March, 2010

Maximum Likelihood Estimation of Logit and Probit Models

1 with probabilit y Pi
yi
0 with probabilit y 1 - Pi
Consequently, if N observations are available, then the likelihood function is
N

L Pi i 1 Pi

1 y i

(1)

i 1

The logit or probit model arises when Pi is specified to be given by the logistic or
normal cumulative distribution function evaluated at X i . Let F X i denote either
of theses cumulative distribution functions. Then, the likelihood function of both
models is
N

y
1 y
L F X i i 1 F X i i .

(2)

i 1

Then, the log-likelihood function is

N

ln L l yi ln F X i 1 yi ln 1 F X i .

(3)

i 1

Now, the first order conditions arising from equation (3) are nonlinear and non-analytic.
Therefore, we have to obtain the ML estimates using numerical optimization methods,
eg, the Newton-Raphson method.
This method (which will be explained further later) implies the following recursion.

2l
l
n 1 n

~n ~n
~

(4)

~
In equation (4), n is the n-th round estimate and the Hessian and score vectors are
evaluated at this estimate.
From our previous ML theorem, we know that

2l 1

N ML N 0, N E

asy

(5)

~
where ML represents the last iteration of the Newton-Raphson procedure. For finite
~
samples, the asymptotic distribution of ML can be approximated by
1

2l

.
N ,

ML

For the logit model, Pi F X i where

F t

1
1 et

(6)

is the logistic cdf and the logistic pdf is

F t f t

et

(7)

1 e

t 2

Also, note that

1 F t

et
F t
1 et

f t
1 F t
F t

(8-1)
(8-2)

f t f t F t 1 e t

Using these results it can be shown for the logit model,

(8-3)

N
N
l
1
1
yi
X i 1 yi
Xi
i 1 1 exp X i
1 exp X i
i 1
N

yi F X i 1 yi F X i X i

(9)

i 1

The Hessian can be shown to be

N
2l
exp X i

Xi Xi
2

i 1 1 exp X i
N

f X i X i X i

(10)

i 1

Note that this X i X i matrix is p.d. for all .

1

2l
l
~
~
So, iterate n 1 n
until n 1 n .

~n ~n
~

For the probit model, Pi F X i where

1
1
exp t 2
2
2
is the probit pdf and the probit cdf is
f t

F t f v dv
t

Also, note that

f t tf t
F t 1 F t

(11)

(12)

(13-1)
(13-2)

Then, the score vector for the probit model is

N
f X i
l
f X i
yi
1 yi
X i
i 1 F X i
1 F X i

(14)

The probit Hessian is then

N
f X i X iF X i
2l
f X i X i 1 F X i
f X i yi
1 yi
X i X i
2

1 F X i 2
F X i
i 1

Estimation of Marginal Effects in the Logit and Probit Models

The analysis of marginal effects requires that we examine
Pi
f X i j , i 1,2,, N , j 1,2,, K .
X ij

Pi
X ij

~ ~
f X i ML ML, j , i 1,2,, N , j 1,2,, K
X X

Talk about applications of logit and probit : credit scoring, target marketing, bond
Rating.

Go over example of German Credit.xls on class website.