You are on page 1of 33

Random Variates

ICS 2307
Modelling and Simulation

Random Variates
There are several techniques for generating random
variates
These include:
Inverse Method
Convolution Method
Acceptance-rejection method
The inverse method is particularly suited for analytically
tractable probability density functions,
The remaining methods deal with more complex cases
such as the normal
2

The Inverse Method


Method applicable to cases where the CDF can be
inversed analytically
Assume that we wish to generate stochastic
variates from a probability density function pdf
f(x).
Let F(x) be the cumulative density function
F(x) is defined in the region [0,1]
First we generate a random variable r which we
set to be equal to F(x) i.e. F(x) =r
The quantity x is obtained by inverting F i.e x=F
1
(r)
The inverse function is applied
on both sides of
3

The Inverse Method


Example
Assume that we wish to generate random variates with
the pdf f(x)=2x, 0 x 1
We first generate an expression for the CDF.
x

F ( x) 2tdt

F ( x) [t ]

2 x
0

F ( x) x

Now if we know that F(x)=r, then


x2=r

x r
4

The Inverse Method


r is obtained by generating a random variable between 0
and 1 i.e. sampling the CDF
Remember that the CDF for any pdf takes values
uniformly distributed in the interval 0 to 1
This is illustrated in the figures below

Fig 1: Sampling A continuous


5

The Inverse Method

Fig 2: Sampling a discrete


distribution

The Inverse Method


1) Sampling from a uniform distribution

f ( x)

1
,a x b
ba
0, otherwise

We get the CDF expression


X

1
F ( x)
dt
ba
a

t x
F ( x) [
]a
ba
xa
F ( x)
ba
7

The Inverse Method


Get the inverse of the CDF expression

xa
r F ( x)
ba

x r (b a ) a

The Inverse Method


2) Sampling from an exponential distribution

f ( x) e

,0 x

We get the CDF expression

f (t ) e

,t 0

F (t ) e x dx
0

F (t ) [e x ]t0
9

The Inverse Method

F (t ) 1 e

Get the inverse of the CDF expression

r F (t ) 1 e t
r 1 e t

1 r e t
ln(1 r ) ln(e t )
ln(1 r ) t
ln(1 r )
t

Because r is a complement of 1-r we have

log(r )

10

Convolution Method
3) Sampling from an Erlang distribution
The m-Erlang random variable is the statistical sum
(convolutions) of m independent and identically
distributed exponential random variables
If y represents the m-Erlang random variable, ten
y=y1+y2++ym
where yi, i=1,2,,m are independent and identically
distributed exponential random variables whose
probability density function is defined as

11

Convolution Method
f ( yi ) e yi

yi>0, i=1,2,,m

The ith exponential sample is given by


1
yi ln(ri ), i 1..m

The m-Erlang sample is computed as

y
y

ln(r1 ) ln(r2 ) ... ln(rm )


ln(r1 r2 ... rm )

12

Convolution Method
4) The Normal Distribution
A random variable x with pdf

is said to have a normal distribution with mean and


standard deviation
The expectation and variance are and 2 respectively
If =0 and =1, then the variable has a standard normal
distribution
13

Convolution Method
A random variable x follows a normal distribution with
mean and standard deviation then the random
variable z
z

follows a standard normal distribution


In order to generate variates from the normal distribution
with parameters and , we use the central limit theorem

14

Convolution Method
The central limit theorem states that
if x1, x2, xn are n independent random variates
with parameters mean and std then, and we
n
have y defined
as

y xi
1

y approaches a normal distribution as n becomes


large
E(y)=n
Var(y)=n2

15

Convolution Method
The procedure for generating random variables from the
normal distribution will this require generation of n values
of the uniform distribution
The random variables associated with these values have
mean=1/2 and variance =1/12
Therefore when the variable got from a summation of n of
these variables will have a normal distribution with
=n/2 and 2=n/12
i.e y has the normal distribution given below
n
n
f ( y) N ( ,
)
2 12
16

Convolution Method
y can be mapped into the standard normal distribution to
produce a new variable z using
z

Thus z is given by
z

yn/2
n / 12

Now if we consider the normal distribution we want to


produce with parameters and , we have
y n/2 x u

n
12
(( y n / 2) )
x
u
n / 12

17

Convolution Method
If we take n=12, this simplifies to

x ( y 6) u
In summary, this means that
1) We generate y by generating and summing 12 random
variables (r1r12), y=r1++r12
2) We subtract 6 from y,
3) Multiply the difference in two with the standard deviation
for the distribution we want to sample
4) Add the mean to the product of 3 to get our random
variate x
18

Convolution Method
This method is slow because of the need to generate 12
random values
A more efficient procedure calls for using the
transformation
z1 ( 2 ln(r1 ) ) cos(2r2 ), and
z 2 ( 2 ln(r1 ) ) sin( 2r2 ),

This method is Box-Mueller procedure (Taha,2003)


Two values in the with the standard normal distribution
are generated using two random variables
These can be mapped into the distribution of interest with
mean N(,)
19

Convolution Method
5) Poisson Distribution
If the distribution of the time between the occurrence of
successive events is exponential, then the distribution of
the number of events per unit time is Poisson
This relationships is used to sample the Poisson
distribution
If the Poisson distribution has a mean value of events
per unit time,
The time between events is exponential with mean=1/
time units
This means that a Poisson sample,n, will occur during t
time units iff
20

Convolution Method
The time between events is exponential with mean=1/
time units
This means that a Poisson sample,n, will occur during t
time units iff
Period until event n occurs t<period until event
n+1 occurs
this is translated to
t1+t2++tn t < t1+t2++tn+1 , n>0
0 t<t1 , n=0
This means that we sample the exponential samples do
we need so that the condition that the left side of the
inequality is less than or equal
to t and the right side
21

Convolution Method
The following discussion shows how this is implemented
Using the explanation for the m-Erlang random variable,
We can get a sample consisting of n identically distributed
exponential random variables using the formula
1
1
ln(r1 r2 ... rn ) t ln(r1 r2 ... rn 1 ), n 0

0t

ln(r1 ), n 0

Which reduces to
r1 r2 rne-t> r1 r2 rn+1, n>0
1 e-t>r1, n=0
22

Convolution Method
In summary, this means that
is the number of events per unit time,
And we want a sample of how many events will occur in a
time t
First we obtain the value of e-t
Case 1
If we generate the first random value r1 and it is less than
e-t, then we have a sample of 0, I.e. our Poisson random
variable n takes the value 0.
Note that if e-t is greater than 1, we will never get a
sample of 0 coz the relevant inequality will never be
satisfied
23

Convolution Method
Case 2
i.

We generate a series of random variables ri until the


product of the the random variables we have generated is
less than e-t, these variable will be r1rm+1

ii. We then check if the product of r1rm is greater or equal


to e-t, if so, then out Poisson random variable n takes the
value m.
iii. If ii is not satisfied, we have to discard the m values and
start generating new ri values

24

Acceptance Rejection Method


6) Sampling the binomial distribution
p is the probability of success and q is the probability of
failure
The CDF for one trial is as shown below

Fig 3: CDF of a binomial


distribution
25

Acceptance Rejection Method


If we assume 0 corresponds to a success and 1 to a
failure (from the diagram)
To generate a random sample, k assuming that we have
n trials
i.

We set k to 0

ii. If we generate a random number ri (between 0 and 1) and


it is less or equal to p, set k=k+1, else k doesnt change
iii. Repeat from ii until you have generated n random values
The value of k will hold the value of the binomial variate
This is an example of acceptance rejection method
26

Acceptance Rejection Method


The acceptance-rejection method is designed for
complex pdfs that are hard to deal with analytically
The general idea is to replace this complex pdf f(x) with a
more analytically manageable proxy pdf h(x)
Samples from h(x) can then be used to sample the
original pdf
Procedure
Define the majorizing g(x) such that it dominates f(x) in
its entire range
g(x)f(x), -<x<-

27

Acceptance Rejection Method


Define the proxy pdf, h(x) by normalizing g(x) using the
formula
h( x )

g ( x)

, x

g ( y)dy

The steps of the acceptance Obtain a sample x=x1 from


h(x) using the inverse or convolution method
1) Obtain a 0 to 1 random number r,
f (x )
r

g ( x1 )
2) If
, accept x1as a sample from f(x) else
reject x1 and repeat from 1

28

Acceptance Rejection Method


Example
Use the acceptance-rejection method to generate at one
value from the following distribution
f(x)=6x(1-x), 0x 1
Assume the following random values are generated
.0589
.6733
.4799
.9486
.6319
29

Sampling From an Empirical Distribution


Quite often, an empirical probability distribution may not
be approximated well by one of the well known
theoretical distributions
In such a case, we have to generate variates from the
empirical distribution
Sampling from a discrete probability distribution
Let x be a discrete random variable taking the values
x1,x2,xn, with probabilities p1, p2,,pn
Let P(xxi)=Pi, i=0, 1,,n (Cumulative Probability)
Let r be a random variable generated from the uniform
distribution [0,1]
30

Sampling From an Empirical Distribution


Then x=xi when Pi-1<rPi

Fig 4: Sampling a discrete


empirical distribution. From the
figure, x=x2
31

Sampling From an Empirical Distribution


Sampling from a continuous probability distribution
Sometimes we need to group the data into classes
(especially when the samples take continuous values)
Each sample will have the lower class boundary and the
upper class boundary
We will use the upper class boundary of each class to
plot the CDF,
Let us assume that the upper class boundary values are
x1, x2, xn where xi-1 is the lower class boundary for the
class with the upper boundary xi
Lets also assume that the p(xxi)=Pi
32

Sampling From an Empirical Distribution


If we generate a random value r in the interval [0,1]
Then the associated value x is obtained using the
following formula

r Pi 1
x xi 1 ( xi xi 1 )
Pi Pi 1
Where r is greater than Pi-1 but less or equal to Pi

33

You might also like