Lecture 4

Simulation of Communication Systems
Adrish Banerjee
Department of Electrical Engineering
Indian Institute of Technology Kanpur
Kanpur, Uttar Pradesh
India
Jan. 3rd, 2014
Lecture #4: Monte Carlo simulation and

random number generation
Adrish Banerjee
Jan. 3, 2014
References:
Chapter 7 of the textbook, Simulation of Communication Systems
Adrish Banerjee
Department of Electrical Engineering Indian Institute of Technology Kanpur Kanpur, Uttar Pradesh India

Adrish Banerjee
Jan. 3, 2014
References:
D. E. Knuth, The art of computer programming, vol. 2 seminumerical
algorithms, Addison Wesley Publishing Company, 1981.
Adrish Banerjee

Adrish Banerjee
Jan. 3, 2014
References:
W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical
Recipes in C: The art of scientific computing, Cambridge University Press, 1992.
Adrish Banerjee

Adrish Banerjee
Jan. 3, 2014
References:
P. Bratley, B. L. Fox, and L. E. Schrage A Guide to Simulation, Springer-Verlag.
Adrish Banerjee

Adrish Banerjee
Jan. 3, 2014
References:
P. Bratley, B. L. Fox, and L. E. Schrage A Guide to Simulation, Springer-Verlag.
L. Devroye, Non-Uniform Random variate Generation, Springer-Verlag.
Adrish Banerjee
Outline of the lecture
Introduction
Adrish Banerjee
Introduction
Uniform random number generation
Adrish Banerjee
Introduction
Linear Congruential Generators
Adrish Banerjee
Introduction
Random variate generation
Adrish Banerjee
Introduction

Inverse transform method
Adrish Banerjee
Introduction

Acceptance/Rejection method
Adrish Banerjee
Introduction

Gaussian random number generation
Adrish Banerjee
Introduction


Box Muller method
Adrish Banerjee
Introduction


Box Muller method
Correlated Gaussian random number generation
Adrish Banerjee
Introduction


Box Muller method

Time domain approach
Adrish Banerjee
Introduction


Box Muller method

Time domain approach
Frequency domain approach
Adrish Banerjee
Introduction
Monte Carlo methods are stochastic techniquesmeaning they are
based on the use of random numbers and probability statistics to
investigate problems.
Adrish Banerjee
Introduction
Assuming that the evolution of the physical system can be described
by probability density functions (pdfs), then the Monte Carlo
simulation can proceed by sampling from these pdfs, which
necessitates a fast and effective way to generate random numbers
uniformly distributed on the interval [0,1].
Adrish Banerjee
Introduction
Assuming that the evolution of the physical system can be described
by probability density functions (pdfs), then the Monte Carlo
simulation can proceed by sampling from these pdfs, which
necessitates a fast and effective way to generate random numbers
uniformly distributed on the interval [0,1].
The name Monte Carlo was coined by Nick Metropolis during the
Manhattan Project of World War II, because of the similarity of
statistical simulation to games of chance, and because the capital of
Monaco was a center for gambling and similar pursuits.
Adrish Banerjee
Introduction
Major components of a Monte Carlo algorithm
Adrish Banerjee
Introduction

Probability distribution functions
Adrish Banerjee
Introduction

Random number generator
Adrish Banerjee
Introduction

Tallying the output
Adrish Banerjee
Introduction

Tallying the output
Error estimation
Adrish Banerjee
Introduction

Tallying the output
Error estimation
Variance reduction techniques
Adrish Banerjee
Introduction

Tallying the output
Error estimation
Variance reduction techniques
Parallelization and vectorization
Adrish Banerjee
Introduction
Steps for Monte Carlo simulation for estimating bit error rate in a
digital communication system.
N(t)
A(t)
Adrish Banerjee
Model of a
Communication System
Y(t)
Introduction
Steps for Monte Carlo simulation for estimating bit error rate in a
digital communication system.
N(t)
A(t)
Model of a
Communication System
Y(t)
Generate sample values of the input bit sequence

A(k), k = 1, 2, , N,, and the noise samples N(j), j = 1, 2, , mN
(m samples per bit).
Adrish Banerjee
Introduction
Process the samples through the models of the functional blocks,

and generate an output sequence, Y(k).
Adrish Banerjee
Introduction
Process the samples through the models of the functional blocks,

and generate an output sequence, Y(k).
Estimate E (g (Y (k))) as
N
X
e = 1
g (Y (k))
P
N
k=1
where
g (Y (k)) =
Adrish Banerjee
1
0
if Y(k) 6= A(k)
if Y(k) = A(k)
Desired features of random number generators
Adrish Banerjee

Repeatability
Adrish Banerjee

Repeatability
Same initial value must produce same sequence.
Adrish Banerjee

Repeatability
Randomness
Adrish Banerjee

Repeatability
Randomness
Numbers that pass statistical tests for randomness.
Adrish Banerjee

Repeatability
Randomness
Long period
Adrish Banerjee

Repeatability
Randomness
Long period
Much larger than the number of random numbers to be generated.
Adrish Banerjee

Repeatability
Randomness
Long period
Unbiased
Adrish Banerjee

Repeatability
Randomness
Long period
Unbiased
Numbers that are generated uniformly in the desired interval.
Adrish Banerjee
Adrish Banerjee

Insensitive to seeds
Adrish Banerjee

Period or randomness should not depend on the seed values.
Adrish Banerjee

Portability
Adrish Banerjee

Portability
Give same results on different computers.
Adrish Banerjee

Portability
Efficient
Adrish Banerjee

Portability
Efficient
Be fast and not use too much memory.
Adrish Banerjee

Linear Congruential Rule: LCG(M, a, c, X(0))
X (k) = [aX (k 1) + c]
where
Adrish Banerjee
M
a
c
X (0)
is
is
is
is
the
the
the
the
mod M
modulus, M > 0
multiplier, 0 < a < M
increment, usually = 1 or 0
seed, 0 < X (0) < M

X (k) = [aX (k 1) + c]
where
M
a
c
X (0)
is
is
is
is
the
the
the
the
mod M
modulus, M > 0
seed, 0 < X (0) < M
The above operation produces a random sequence of integer values

in the range 0 X (k) M 1.
Adrish Banerjee

X (k) = [aX (k 1) + c]
where
M
a
c
X (0)
is
is
is
is
the
the
the
the
mod M
modulus, M > 0
seed, 0 < X (0) < M
The above operation produces a random sequence of integer values

in the range 0 X (k) M 1.
Uniform random numbers can be generated in the interval [0, 1] by
executing the following floating point operation
U(k) = X (k)/M
Adrish Banerjee
Linear Congruential Rule: Examples
Adrish Banerjee

rand() function in ANSI C:
Adrish Banerjee

LCG (231 , 1103515245, 12345, 12345)
Adrish Banerjee

LCG (231 , 1103515245, 12345, 12345)
drand48() function in ANSI C:
Adrish Banerjee

LCG (231 , 1103515245, 12345, 12345)

LCG (248 , 25214903917, 0, 1)
Adrish Banerjee

LCG (231 , 1103515245, 12345, 12345)

LCG (248 , 25214903917, 0, 1)
rand function in Maple:
Adrish Banerjee

LCG (231 , 1103515245, 12345, 12345)

LCG (248 , 25214903917, 0, 1)
rand function in Maple:

LCG (1012 , 427419669081, 0, 1)
Adrish Banerjee
Properties of LCG(a, c, M, X(0))
Adrish Banerjee

M = 2m , c > 0
Adrish Banerjee

M = 2m , c > 0
The full period of M = 2m is obtained if and only if
a 1( mod 4), and c is odd.
Adrish Banerjee

M = 2m , c > 0
M = 2m , c = 0
Adrish Banerjee

M = 2m , c > 0
M = 2m , c = 0
The maximum period of M = 2m2 is obtained if and only if
a 3 ( mod 8) or a 5( mod 8), and the initial seed X(0) is odd.
Adrish Banerjee

M = 2m , c > 0
M = 2m , c = 0
M=p (prime), c=0 or c 6= 0
Adrish Banerjee

M = 2m , c > 0
M = 2m , c = 0
M=p (prime), c=0 or c 6= 0

The maximum period of M = p 1 is obtained if and only if a is a
primitive element modulo p.
Adrish Banerjee
Variation of Linear Congruential Rule:
Adrish Banerjee
Variation of Linear Congruential Rule:

Random numbers with period p m 1
X (k) = [a1 X (k 1) + + am X (k m)]
mod p
where (a1 , a2 , , am ) are the coefficients of a primitive polynomials

f (x) = x m a1 x m1 am
over the Galois field GF(p).
Adrish Banerjee
Wichman Hill Algorithm:
Adrish Banerjee

This algorithm is based on the approach, that if we sum two periodic
sequences with periods N1 and N2 , the resulting sequence will have a
period
N = lcm(N1 , N2 )
Adrish Banerjee

This algorithm is based on the approach, that if we sum two periodic
sequences with periods N1 and N2 , the resulting sequence will have a
period
N = lcm(N1 , N2 )
If N1 and N2 are relatively prime with respect to each other, then
N = N1 N2
Adrish Banerjee

Wichman Hill algorithm combines the outputs of three random
number generators according to
X (n) =
Y (n) =
171X (n 1) mod 30269

172X (n 1) mod 30307
Z (n)
170X (n 1) mod 30323
and
U(n) =
Adrish Banerjee
X (n)
Y (n)
Z (n)
+
+
30269 30307 30323
mod 1
Inverse transform method:
Adrish Banerjee
Inverse transform method:

Let U be a uniform [0,1] random variable. For any continuous
cumulative distribution function, FX (x), the random variable X is
defined by
X = F 1 (U)
has the pdf fX (x)
Adrish Banerjee
Inverse transform method: Proof
Adrish Banerjee
Inverse transform method: Proof

Let FX denote the cdf of X = F 1 (U). Then
FX (x)
Adrish Banerjee
P{X x}
P{F 1 (U) x}
P{F (F 1 (U)) F (x)}
P{U F (x)}
F (x)

Inverse transform method: Example (Exponential random variable)
Adrish Banerjee

Inverse transform method: Example (Exponential random variable)
By applying a transformation to a uniform variable U[0, 1], we can a
random variable X with an exponential pdf fX (x) as follows.
fX (x) = e x
and
FX (x) = 1 e x
Hence
U = FX (x) yields U = 1 e x
or
X
Adrish Banerjee
(1/) ln(1 U)
(1/) ln(U)

Inverse transform method (Analytical):
Adrish Banerjee

defined by
X = F 1 (U)
has the pdf fX (x)
Adrish Banerjee

defined by
X = F 1 (U)
has the pdf fX (x)
Inverse transform method (Empirical):
Adrish Banerjee

defined by
X = F 1 (U)
has the pdf fX (x)

Let Fi =
Adrish Banerjee
Pi
j=1
pj , i = 0, 1, , N, with F0 = 0.

defined by
X = F 1 (U)
has the pdf fX (x)

P
Let Fi = ij=1 pj , i = 0, 1, , N, with F0 = 0.
Find the smallest value of i that satisfies
Fi1 < U < Fi , i = 1, 2, , N
Adrish Banerjee

defined by
X = F 1 (U)
has the pdf fX (x)

P
Let Fi = ij=1 pj , i = 0, 1, , N, with F0 = 0.
Find the smallest value of i that satisfies
Fi1 < U < Fi , i = 1, 2, , N
Output the interpolated value of X in the interval [xi1 , xi ].
Adrish Banerjee
Acceptance/Rejection method:
Adrish Banerjee
This method requires that the pdf f be bounded and nonzero only on
some finite interval [a, b].
Adrish Banerjee
Let
c = max{f (x)|a x b}
The method can be summarized as:
Adrish Banerjee
Let
1. Generate X uniform on (a,b).
Adrish Banerjee
Let
2. Generate Y uniform on (0,c).
Adrish Banerjee
Let
3. If Y f(X), then output X; otherwise return to step 1.
Adrish Banerjee
Let
3. If Y f(X), then output X; otherwise return to step 1.
This method is inefficient when X is rejected in step 3 with

significant probability.
Adrish Banerjee

Acceptance/Rejection method: Example Generation of beta(2,4)
Adrish Banerjee

Generation of random variable having probability distribution
function,
f (x) = 20x(1 x)3 , 0 < x < 1
Adrish Banerjee

function,
f (x) = 20x(1 x)3 , 0 < x < 1
Since the beta random variable is concentrated in the interval (0,1),
lets consider
g (x) = 1, 0 < x < 1
Adrish Banerjee

function,
f (x) = 20x(1 x)3 , 0 < x < 1
Since the beta random variable is concentrated in the interval (0,1),
lets consider
g (x) = 1, 0 < x < 1
To determine the smallest constant c such that f (x)/g (x) c, we
use calculus to determine the maximum value of f(x)/g(x).
Differentiating with respect to x, we get
f (x)
d
= 20[(1 x)3 3x(1 x)2 ]
dx g (x)
Adrish Banerjee
Setting the differentiation to 0, we get the maximum value at x =

1/4, and
3
3
135
1
f (x)
=
20
c
g (x)
4
4
64
Adrish Banerjee

1/4, and
3
3
135
1
f (x)
=
20
c
g (x)
4
4
64
Hence,
Adrish Banerjee
256
f (x)
=
x(1 x)3
cg (x)
27

1/4, and
3
3
135
1
f (x)
=
20
c
g (x)
4
4
64
Hence,
256
f (x)
=
x(1 x)3
cg (x)
27
The procedure can be summarized.
Adrish Banerjee

1/4, and
3
3
135
1
f (x)
=
20
c
g (x)
4
4
64
Hence,
256
f (x)
=
x(1 x)3
cg (x)
27

Step 1: Generate random numbers U1 and U2 .
Adrish Banerjee

1/4, and
3
3
135
1
f (x)
=
20
c
g (x)
4
4
64
Hence,
256
f (x)
=
x(1 x)3
cg (x)
27

Step 1: Generate random numbers U1 and U2 .
U1 (1 U1 )3 , stop and set X = U1 . Otherwise, return to
Step 2: If U2 256
27
Step 1.
Adrish Banerjee
Modified Acceptance/Rejection method:
Adrish Banerjee

Let g(x) be a probability density function such that there exists a
constant c with f (x) cg (x) for all x.
Steps:
Adrish Banerjee

Steps:
1. To generate X according to the pdf g (x).
Adrish Banerjee

Steps:
2. Generate Y uniform on [0, cg(X)].
Adrish Banerjee

Steps:
2. Generate Y uniform on [0, cg(X)].
3. If Y < f (x), then output X; otherwise return to Step 1.
Adrish Banerjee

Acceptance/Rejection method: Proof
P{X x|Y f (X )}
P{X x and Y f (X )}
P{Y f (X )}
F (x)/c
= F (x)
1/c
=
=
P[X x and Y f (X )]
=
=
P[X x and Y f (X )|X = z]dG (z)
f (z)
g (z)dz = F (x)/c
cg (z)
Z x
where the correctness of the bracketed term follows from the fact
that Y is (conditionally) uniform on [0, cg(z)]; likewise
P[Y f (X )] = 1/c
Adrish Banerjee

No, empirical form
pdf in closed form
Quantize pdf
compute empirical cdf and
use empirical inverse transform
Yes, closed form
Yes
No
cdf in closed form
Yes
Inverse cdf
in closed form
Inverse
Transform
(Analytical)
Adrish Banerjee
No
Inverse
Transform
(Empirical)
Finite
Support
Infinite
Support
Acceptance/
Rejection Method
Modified
A/R Method
Box-Muller method:
Adrish Banerjee
Box-Muller method:
If U1 and U2 are two independent variables uniformly distributed in
the unit interval, then
X = [2 ln(U1 )]1/2 cos(2U2 )
and
Y = [2 ln(U1 )]1/2 sin(2U2 )
are independent Gaussian random variables with mean zero and

variance 1.
Adrish Banerjee

Box-Muller method: Proof
Adrish Banerjee

The joint pdf of two independent Gaussian random variables is given
by
1
exp[(x 2 + y 2 )/2]
f (x, y ) =
2
Adrish Banerjee

by
1
exp[(x 2 + y 2 )/2]
f (x, y ) =
2
Writing the pdf in polar form
fp (r , ) =
Adrish Banerjee
r
exp[r 2 /2]
2

by
1
exp[(x 2 + y 2 )/2]
f (x, y ) =
2
fp (r , ) =
r
exp[r 2 /2]
2
Absence of from the joint density function implies is uniformly

distributed over [0, 2], generated with 2U2
Adrish Banerjee

by
1
exp[(x 2 + y 2 )/2]
f (x, y ) =
2
fp (r , ) =
r
exp[r 2 /2]
2
Absence of from the joint density function implies is uniformly

distributed over [0, 2], generated with 2U2
Integrating over all possible values of , we get the marginal pdf of R,
fp (r ) = r exp[r 2 /2]
Adrish Banerjee

Thus, the cdf is
Fp (r ) =
t. exp[t 2 /2]dt = 1 exp[r 2 /2]
Adrish Banerjee

Thus, the cdf is
Fp (r ) =
t. exp[t 2 /2]dt = 1 exp[r 2 /2]
Setting
U1 = 1 exp[R 2 /2]
one gets
R=
2 log(1 U1 )
which has the same distribution as

p
R = 2 log(U1 )
Adrish Banerjee

Thus, the cdf is
Fp (r ) =
t. exp[t 2 /2]dt = 1 exp[r 2 /2]
Setting
U1 = 1 exp[R 2 /2]
one gets
R=
2 log(1 U1 )
which has the same distribution as

p
R = 2 log(U1 )
Projecting the vector (R, ) onto the axes, we get

p
X = R cos() = 2 log(U1 ) cos(2U2 )
and
Y = R sin() =
Adrish Banerjee
2 log(U1 ) sin(2U2 )
Polar method:
Adrish Banerjee
Polar method:
Step 1: Generate uniform random numbers, U1 and U2 .
Adrish Banerjee
Polar method:
Step 2: Set V1 = 2U1 1, V2 = 2U2 1, S = V12 + V22 .
Adrish Banerjee
Polar method:
Step 2: Set V1 = 2U1 1, V2 = 2U2 1, S = V12 + V22 .
Step 3: If S > 1, return to Step 1.
Adrish Banerjee
Polar method:
Step
Step
Step
Step
1:
2:
3:
4:
Generate uniform random numbers, U1 and U2 .

Set V1 = 2U1 1, V2 = 2U2 1, S = V12 + V22 .
If S > 1, return to Step 1.
Return the independent standard normals
r
r
2 log S
2 log S
X =
V1 Y =
V2
S
S
Adrish Banerjee
Correlated Gaussian random sequence generation

Time domain approach based on correlation function: ARMA model
Adrish Banerjee

A correlated Gaussian sequence Y(n) can be generated from an
uncorrelated Gaussian sequence X(n) using an autoregressive moving
average [ARMA (p,q)] model of the form
Y (n) =
p
X
i=1
Adrish Banerjee
pi Y (n i) +
q
X
qj X (n j) + X (n)
j=1

Y (n) =
p
X
i=1
pi Y (n i) +
q
X
qj X (n j) + X (n)
j=1
There exists many approaches in the literature for calculating the

parameters pi and qj (see e.g. [1])
Adrish Banerjee

Y (n) =
p
X
i=1
pi Y (n i) +
q
X
qj X (n j) + X (n)
j=1
There exists many approaches in the literature for calculating the

parameters pi and qj (see e.g. [1])
1 . Kashyap, R. L., Optimal choice of AR and MA parts in
autoregressive moving average models. IEEE Trans. Pattern Analysis
and Machine Intelligence, Vol. PAMI-4 (1982), No. 2, 99-104.
Adrish Banerjee
Frequency domain approach based on the power spectral density :

Spectral Factorization
Adrish Banerjee

Output Gaussian process power spectral density is given by:
SYY (f ) = SXX (f )|H(f )|2
where SXX (f ) = X2 and H(f) is the filter transfer function.
Adrish Banerjee

SYY (f ) = SXX (f )|H(f )|2
If power spectral density, SYY (f ) is represented as a ratio of
polynomials in f 2 , SYY (f ) and hence |H(f )|2 can be factored into a
product of the form H(s)H(s), where s = j2f .
Adrish Banerjee

SYY (f ) = SXX (f )|H(f )|2
If power spectral density, SYY (f ) is represented as a ratio of
polynomials in f 2 , SYY (f ) and hence |H(f )|2 can be factored into a
product of the form H(s)H(s), where s = j2f .
Design a filter with transfer function H(s) to transform an
uncorrelated Gaussian sequence into a correlated sequence.
Adrish Banerjee

Lecture 4

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 4

Uploaded by

Copyright:

Available Formats

Simulation of Communication Systems

Jan. 3rd, 2014

Lecture #4: Monte Carlo simulation and

Lecture #4: Monte Carlo simulation and

Lecture #4: Monte Carlo simulation and

Lecture #4: Monte Carlo simulation and

Lecture #4: Monte Carlo simulation and

Outline of the lecture

Outline of the lecture

Outline of the lecture

Outline of the lecture

Random variate generation

Outline of the lecture

Random variate generation

Outline of the lecture

Random variate generation

Outline of the lecture

Random variate generation

Gaussian random number generation

Outline of the lecture

Random variate generation

Gaussian random number generation

Outline of the lecture

Random variate generation

Gaussian random number generation

Correlated Gaussian random number generation

Outline of the lecture

Random variate generation

Gaussian random number generation

Correlated Gaussian random number generation

Outline of the lecture

Random variate generation

Gaussian random number generation

Correlated Gaussian random number generation

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Major components of a Monte Carlo algorithm

Generate sample values of the input bit sequence

Process the samples through the models of the functional blocks,

Process the samples through the models of the functional blocks,

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators

Uniform random number generation

Desired features of random number generators