Moment Generating Function

Moments and Moment-Generating Functions

Moment
The r th moment about the origin of the random variable X is
given by



= and
2
= E[X
2
]-{E[X]}
2
=


2
The purpose of moment-generating function is
To determine moments of distributions
To establish distributions of functions of random variables
= = ' ) (
r
r
X E

x
r
x f x ) (
}

dx x f x
r
) (
if X is discrete.
if X is continuous.
1

'

'
2

Moments and Moment-Generating Functions
Moment generating Function
The moment-generating function (MGF) of the random
variable X is given by E(e
tX
) and denoted by M
x
(t). Hence



How to get moments from MGF
Let X be random variable with MGF M
x
(t). Then

= = ) ( ) (
tx
X
e E t M

x
tx
x f e ) (
}

dx x f e
tx
) (
0
) (
=
=
'
t
r
x
r
r
dt
t M d

(a) If X is a discrete r.v., the

.
(b) If X is a continuous r.v., then

.

) x ( p e ) t ( M
x
x t
X

=
dx ) x ( f e ) t ( M
x t
X

=
Example (Poisson distribution) X~ Poisson ()
- MGF of X:
Moments and Moment-Generating Functions
) 1 (
0 0
!
) (
!
] [ ) (

= = = =

t
e
x
x t
x
x
tx tX
X
e
x
e
e
x
e
e e E t M

= = = =
=

=
0
) 1 (
0
'
1
) (
] [
t
e t
t
X
t
e e
dt
t dM
X E
) 1 ( ) 1 (
) (
] [
0
0
2
2
2 '
2


+ = + = = =
=
+
=
t
e t t
t
X
t
e e
dt
t M d
X E
= + = =
2 2 2
) 1 ( ] [ ] [ ] [ X E X E X Var
Moments and Moment-Generating Functions
Related Theorems about MGF
For two random variables X, Y , if = for all value t, then
X and Y have the same probability density.



X
1
,X
2
,X
3
, . . . X
n
are independent random variables and
Y = X
1
+X
2
+X
3
+ + X
n
, then
Example (Sum of two independent Poisson rvs)
X
1
~Poisson(
1
) X
2
~ Poisson(
2
) and they are independent.
Let Y= X
1
+ X
2.
Distribution of Y ?

) (t M
x
) (t M
y
) ( ) ( t M e t M
X
at
a X
=
+
) ( ) ( bt M t M
X bX
=
) ( ) ( ) ( ) (
2 1
t M t M t M t M
n
X X X Y
=
MGF uniquely determines
the distribution.
) ( ) ( bt M e t M
X
at
a bX
=
+
) 1 )( ( ) 1 ( ) 1 (
2 1 2 1
2 1
) ( ) ( ) (
+
= = =
t t t
e e e
X X Y
e e e t M t M t M


Moments and Moment-Generating Functions
Linear Combination of Normal r.v.s
If X
1
,X
2
,X
3
, . . . X
n
are independent r.v.s having normal
dists with means
1
,
2
,
3
, . . .
n
and variances
respectively, then

r.v. has normal distribution
with mean
and variance

2 2
3
2
2
2
1
, , , ,
n
o o o o
2 2 2
2
2
2
2
1
2
1
2
n n Y
a a a o o o o + + + =
n n Y
a a a + + + =
2 2 1 1
n n
X a X a X a Y + + + =
2 2 1 1
.
2 /
2 2
) (
t t
X
i i
i
e t M
o +
=
> Note
Binomial Distribution
Example: Let X be a discrete r.v.
with pmf
.


n ,..., 2 , 1 , 0 x , ) p 1 ( p ) x ( p
x n x n
x
= =

= =

=
x n x n
x
n
0 x
tx
X
) p 1 ( p e ) t ( M , Then
( )
n t x n x t n
x
n
x
p pe p p e ) 1 ( ) 1 ( ) (
0
+ = =

=

Exponential Distribution
Let X be a r.v. with density
fX(x) = e- x , x>0,
) t (
dx e
dx e e ) t ( M
t) - x( -
0
x -
0
x t
X

= =
= =

MGF of Sums of Independent R.V.s

Let Xi, I=1, 2, , n be independent r.v.s with moment-ge
nerating functions (mgf) ,
then
.
=
=

=
n
1 i
X
X
) t ( M ) t ( M
i
n
1 i
i