STAT/MATH 395 A - PROBABILITY II UW

Winter Quarter 2017 Nhmy Lim

Continuous Random Variables

Notation.

The indicator function of a set S is a real-valued function defined by :

;
1 if x S
1S (x) =
0 if x S

Suppose that f : D R is a real-valued function whose domain is an

arbitrary set D. The support of f , written supp(f ), is the set of points in
D where f is nonzero

supp(f ) = {x D | f (x) = 0}.

1 Probability Density Function and Cumulative

Distribution Function
Definition 1.1 (Probability density function). A rrv is said to be (absolutely)
continuous if there exists a real-valued function fX such that, for any subset
B R:
P(X B) = fX (x) dx (1)
B

Then fX is called the probability density function (pdf) of the random vari-
able X.
In particular, for any real numbers a and b, with a < b, letting B = [a, b],
we obtain from Equation (1) that :
b
P(a X b) = fX (x) dx (2)
a

Property 1.1. If X is a continuous rrv, then

For all a R,
P(X = a) = 0 (3)
In other words, the probability that a continuous random variable takes on
any fixed value is zero.

1
 For any real numbers a and b, with a < b

P(a X b) = P(a X < b) = P(a < X b) = P(a < X < b) (4)

The above equation states that including or not the bounds of an interval
does not modify the probability of a continuous rrv.
Proof. Let us first prove Equation (3) :
a
P(X = a) = P(X [a, a]) = fX (x) dx = 0
a

To prove Equation (4), we simply notice that

b
P(a X b) = P(a X < b) = P(a < X b) = P(a < X < b) = fX (x) dx
a

Theorem 1.1. A probability density function completely determines the distri-

bution of a continuous real-valued random variable.
Remark : This theorem means that two continuous real-valued random vari-
ables X and Y that have exactly the same probability density functions follow
the same distribution. We say that they are identically distributed.

Definition 1.2 (Cumulative distribution function). Let ( , A, P) be a proba-

bility space. The (cumulative) distribution function (cdf) of a real-valued
random variable X is the function FX given by

FX (x) = P(X x), for all x R (5)

Property 1.2. Let FX be the cdf of a random variable X. Following are some
properties of FX :
FX is increasing : x y FX (x) FX (y)
limx FX (x) = 1 and limx FX (x) = 0
FX is cdlg :
FX is right continuous : limxx0 FX (x) = FX (x0 ) , for x0 R
FX has left limits : limxx0 FX (x) exists, for x0 R
Property 1.3 (Cumulative distribution function of a continuous rrv). Let X
be a continuous rrv with pdf fX . Then the cumulative distribution function FX
of X is given by : x
FX (x) = fX (t) dt (6)

2
Proof. We have the following :

FX (x) = P(X x)
= P(X (, x))
x
= fX (t) dt

Theorem 1.2. A cumulative distribution function completely determines the

distribution of a continuous real-valued random variable.
Lemma 1.3. Let X be a continuous rrv with pdf fX and cumulative distribution
function FX . Then

FX (x) = fX (x), if fX is continuous at x;
For any real numbers a and b with a < b,

P(a X b) = FX (b) FX (a) (7)

Proof. This is a direct application of the Fundamental Theorem of Calculus.

Proposition 1.1. Let X be a continuous rrv on probability space ( , A, P) with

pdf fX . Then, we have :
fX is nonnegative on R :

fX (x) 0, for all x R; (8)

fX is integrable on R and

fX (x) dx = 1 (9)

Proof. Proof of (8) : Property 1.2 states that the cumulative distribution
function FX is increasing on R. Therefore FX
(x) 0. According to
Lemma 1.3, FX (x) = fX (x) if fX is continuous at x. This completes the

proof.
Proof of (9) :

fX (x) dx = P(X (, )) = P(X R) = P( ) = 1

In other words, the event that X takes on some value (X R) is the sure
event .

3
Definition 1.3. A real-valued function f is said to be a valid pdf if the following
holds :
f is nonnegative on R :

f (x) 0, for all x R; (10)

f is integrable on R and

f (x) dx = 1 (11)

This means that if f is a valid pdf, then there exists some continuous rrv X
that has f as its pdf

Example. A continuous rrv X is said to follow a uniform distribution on

[0, 1/2] if its pdf is :
;
c if 0 x 1/2
fX (x) = = c1[0,1/2] (x)
0 otherwise

Questions.
(1) Determine c such that fX satisfies the properties of a pdf.
(2) Give the cdf of X.

Proof. (1) Since fX is a pdf, fX (x) should be nonnegative for all x R. This
is the case for x (, 0) and x (1/2, ) where fX (x) equals zero. On
the interval [0, 1/2], fX (x) = c. This implies that c should be nonnegative
as well.
Let us now focus on the second condition,
0 1/2
fX (x) dx = 1 fX (x) dx + fX (x) dx + fX (x) dx = 1
0 1/2
0 1/2
0 dx + c dx + 0 dx = 1
0 1/2
-1/2
-
cx-- =1
0
1
c =1
2
c=2

And we check that indeed c = 2 is nonnegative.

(2) The cumulative distribution function FX of X is piecewise like its pdf :

4
 If x < 0, then fX (t) = 0 for all t (, x].
x x
FX (x) = fX (t) dt = 0 dt = 0

If 0 x 1/2, then fX (t) = 2 for all t [0, x].

x
FX (x) = fX (t) dt

0 x
= fX (t) dt + fX (t) dt
0
x
= FX (0) + 2 dt
0
-x
-
= 0 + 2 t--
0
= 2x

If x > 1/2, then fX (t) = 0 for all t [1/2, x].

x
FX (x) = fX (t) dt

1/2 x
= fX (t) dt + fX (t) dt
1/2

= FX (1/2) + 0
1
=2
2
=1
In a nutshell, FX is given by :
Y
] 0 if x < 0
FX (x) = 2x if 0 x 1/2
[
1 if x > 1/2

Example. Let X be the duration of a telephone call in minutes and suppose

X has pdf :
fX (x) = c ex/10 1[0,) (x)
Questions.
(1) Which value(s) of c make(s) fX a valid pdf?
Answer. c = 1/10.
(2) Find the probability that the call lasts less than 5 minutes.
Answer. P(X < 5) = 1 e1/2 0.393.

5
2 Expectation and Variance
Definition
s 2.1 (Expected Value). Let X be sa continuous rrv with pdf fX . If
(x) dx is absolutely convergent, i.e. |x|fX (x) dx < 1 , then, the


xf X
mathematical expectation (or expected value or mean) of X exists, is
denoted by E[X] and is defined as follows :

E[X] = xfX (x) dx (12)

Definition 2.2 (Expected Value of a Function of a Random Variable). Let X be

a continuous rrv with pdf fX . Let g : R R be a piecewise continuous function.
If random variable g(X) is integrable. Then, the mathematical expectation of
g(X) exists, is denoted by E[g(X)] and is defined as follows :

E[g(X)] = g(x)fX (x) dx (13)

Example. Compute the expectation of a continuous rrv X following a uniform

distribution on [0, 1/2]. As seen earlier, its pdf is given by:
;
2 if 0 x 1/2
fX (x) = = 2 1[0,1/2] (x)
0 otherwise

Proof. The expectation of X can be computed as follows :

E[X] = xfX (x) dx

0 1/2
= xfX (x) dx + xfX (x) dx + xfX (x) dx
0 1/2
1/2
=0+ x 2 dx + 0
0
-1/2
-
= x2 --
0
1
=
4

Property 2.1. Let X be a continuous rrv with pdf fX .

for all c R,
E[c] = c (14)
1 We then say that X is integrable.

6
 If g : R R is a nonnegative piecewise continuous function and g(X) is
integrable. Then, we have :

E[g(X)] 0 (15)

If g1 : R R and g2 : R R are piecewise continuous functions and

g1 (X) and g2 (X) are integrable such that g1 g2 . Then, we have :

E[g1 (X)] E[g2 (X)] (16)

Proof. Proof of Equation (14) : Here we consider the function of the

random variable X defined by : g(X) = c. We then get :

E[c] = cfX (x) dx


=c fX (x) dx


=1 since fX is a pdf

=c

Proof of Equation (15) : This comes from the nonnegativity of the integral
for nonnegative functions.
Proof of Equation (16) : This is a direct application of Equation (15)
applied to function g2 g1 .

Property 2.2 (Linearity of Expectation). Let X be a continuous rrv with pdf

fX . If c1 , c2 R and g1 : R R and g2 : R R are piecewise continuous
functions and g1 (X) and g2 (X) are integrable. Then, we have :

E[c1 g1 (X) + c2 g2 (X)] = c1 E[g1 (X)] + c2 E[g2 (X)] (17)

Note that Equation (17) can be extended to an arbitrary number of piecewise

continuous functions.
Proof.

E[c1 g1 (X) + c2 g2 (X)] = (c1 g1 (x) + c2 g2 (x))fX (x) dx


= c1 g1 (x)fX (x) dx + c2 g2 (x)fX (x) dx

= c1 E[g1 (X)] + c2 E[g2 (X)]

7
Definition 2.3 (VarianceStandard Deviation). Let X be a real-valued random
variable. When E[X 2 ] exists2 , the variance of X is defined as follows :

Var(X) = E[(X E[X])2 ] (18)

2
Var(X) is sometimes denoted X . The positive square root of the variance is
called the standard deviation of X, and is denoted X . That is:

X = Var(X) (19)

Property 2.3. The variance of a real-valued random variable X satisfies the

following properties :

Var(X) 0
If a, b R are two constants, then Var(aX + b) = a2 Var(X)
Proof. This property is true for any kind of random variables (discrete or con-
tinuous). See proof of Property 4.1 given in the lecture notes of the chapter
about discrete rrvs.

Theorem 2.1 (Knig-Huygens formula). Let X be a real-valued random vari-

able. When E[X 2 ] exists, the variance of X is also given by :

Var(X) = E[X 2 ] (E[X])2 (20)

Proof. This property is true for any kind of random variables (discrete or con-
tinuous). See proof of Theorem 4.1 given in the lecture notes of the chapter
about discrete rrvs.

In the case of continuous random variables, Equation (20) becomes :

3 42
Var(X) = x2 fX (x) dx xfX (x) dx

Example. Compute the variance of a continuous rrv X following a uniform

distribution on [0, 1/2]. As seen earlier, its pdf is given by:

fX (x) = 2 1[0,1/2] (x)

2 We then say that X is square integrable.

8
Proof. We computed previously the expectation of X that is E[X] = 1/4. Com-
puting the variance of X thus boils down to calculating E[X 2 ] :

2
E[X ] = x2 fX (x) dx

0 1/2
= x2 fX (x) dx + x2 fX (x) dx + x2 fX (x) dx
0 1/2
1/2
=0+ x2 2 dx + 0
0
-1/2
1 3 --
=2 x -
3 0
1
=
12
Therefore,
3 42
1 1 1
Var(X) = =
12 4 48

3 Common Continuous Distributions

3.1 Uniform Distribution
Definition 3.1. A continuous rrv is said to follow a uniform distribution
U(a, b) on a segment [a, b], with a < b, if its pdf is
1
fX (x) = 1[a,b] (x) (21)
ba
Proof. Let us prove that the pdf of a uniform distribution is actually a valid
pdf :
Is fX (x) 0 for x R?
If x < a or x > b, then fX (x) = 0 0.
If x [a, b], then fX (x) = 1/(b a) > 0 since a < b.
s
Let us show that fX (x) dx = 1 :
a b
fX (x) dx = fX (x) dx + fX (x) dx + fX (x) dx
a b

1b
=0+ dx + 0
a b a
-b
1 -
= x--
ba a
=1

9
Property 3.1 (Mean and Variance for a Uniform Distribution). If X follows
a uniform distribution U(a, b), then
its expected value is given by :
a+b
E[X] = (22)
2

its variance is given by :

(b a)2
Var(X) = (23)
12

Proof. Expectation :

E[X] = xfX (x) dx


1 b
= x dx
ba a
-b
1 1 2 --
= x -
ba 2 a
1 b2 a2
=
2 ba
a+b
=
2

Variance : Let us first compute E[X 2 ]

E[X 2 ] = x2 fX (x) dx


1 b
= x2 dx
ba a
-b
1 1 3 --
= x -
ba 3 a
1 b3 a3
=
3 ba
b2 + ab + a2
=
3
Then using the shortcut formula (20), we find :

b2 + ab + a2 (a + b)2 (b a)2
Var(X) = =
3 4 12

10
Motivation. Most computer programming languages include functions or li-
brary routines that provide random number generators. They are often designed
to provide a random byte or word, or a floating point number uniformly dis-
tributed between 0 and 1.
The quality i.e. randomness of such library functions varies widely from
completely predictable output, to cryptographically secure.
There are a couple of methods to generate a random number based on a
probability density function. These methods involve transforming a uniform
random number in some way. Because of this, these methods work equally well
in generating both pseudo-random and true random numbers.

3.2 Gaussian Distribution

Example. A restaurant wants to advertise a new burger they call The Quarter-
kilogram. Of course, none of their burgers will exactly weigh exactly 250 grams.
However, you may expect that most of the burgers weights will fall in some
small interval centered around 250 grams.

Definition 3.2. A continuous random variable is said to follow a normal (or

Gaussian) distribution N (, 2 ) with parameters, mean and variance 2 if
its pdf fX is given by:
I 3 42 J
1 1 x
fX (x) = exp , for x R (24)
2 2

We also say that X is normally distributed or X is a normal (or Gaussian) rrv

with parameters and 2 .
Property 3.2 (Mean and Variance for a Normal Distribution). If X follows a
normal distribution N (, 2 ), then
its expected value is given by :

E[X] = (25)

its variance is given by :

Var(X) = 2 (26)

Remark. The normal distribution is one of the most (even perhaps the most)
important distributions in Probability and Statistics. It allows to model many
natural, physical and social phenomenons. We will see later in this course how
all the distributions are somehow related to the normal distribution.

Property 3.3. If X follows a normal distribution N (, 2 ). Then its pdf fX

has the following properties:

11
(1) fX is symmetric about the mean :
fX ( x) = fX ( + x), for x R (27)

(2) fX is maximized at x = .
(3) The limit of fX (x), as x approches or , is 0:
lim fX (x) = lim fX (x) = 0 (28)
x x

Definition 3.3 (Standard Normal Distribution). We say that a continuous rrv

X follows a standard normal distribution if X follows a normal distribution
N (0, 1) with mean 0 and variance 1. We also say that X is a standard normal
random variable. In particular, the cdf of a standard normal random variable is
denoted , that is :
x
1 t2
(x) = e 2 dt, for x R (29)
2
Property 3.4. The cdf of a standard normal random variable satisfies the
following property:
(z) = 1 (z), for z R (30)
Property 3.5. Let a R, a = 0 and b R. If X follows a normal distribution
N (, 2 ), then random variable aX + b follows a normal distribution N (a +
b, a2 2 ).
Proof. Let us assume that a > 0. Consider random variable Y = aX +b. We will
prove that FY , the cdf of Y is the cdf of a normal distribution N (a + b, a2 2 ).
For y R, we have that :
FY (y) = P(Y y)
= P(aX + b y)
3 4
yb
=P X
a
3 4
yb
= FX
a
yb I 3 42 J
a 1 1 x
= exp dx
2 2
Now, consider the change of variable : t = ax + b dt = a dx. Hence, we have
:
Y A B2 Z
y ] ^ dt
1 1 tb
a
FY (y) = exp
2 [ 2 \ a
y I 3 42 J
1 1 t (a + b)
= exp dt
a 2 2 a

12
We now recognize the cdf of a normal distribution N (a + b, a2 2 ), which com-
pletes the proof.
The proof in the case where a is negative is left as an exercise.
Corollary 3.1. If X follows a normal distribution N (, 2 ), then random vari-
able Z defined by:
X
Z=

is a standard normal random variable.
Finding Normal probabilities. If X N (, 2 ). The above property leads
us to the following strategy for finding probabilities P(a X b) :
(1) Transform X, a, and b, by:
X
Z=

(2) Use the standard normal N (0, 1) Table to find the desired probability.

Example. Let X be the weight of a so-called quarter-kilogram burger. As-

sume X follows a normal distribution with mean 250 grams and standard devi-
ation 15 grams.
(a) What is the probability that a randomly selected burger has a weight below
240 grams?

(b) What is the probability that a randomly selected burger has a weight above
270 grams?
(c) What is the probability that a randomly selected burger has a weight be-
tween 230 and 265 grams?

3.3 Quantiles
Previously, we learned how to use the standard normal curve N (0, 1) to find
probabilities concerning a normal random variable X. Now, what would we do if
we wanted to find some range of values for X in order to reach some probability?

Definition 3.4 (Quantile). Let X be a rrv with cumulative distribution function

FX and [0, 1]. A quantile of order for the distribution of X, denoted q ,
is defined as follows :

q = inf {x R |FX (x) } (31)

Remarks :
The quantile of order 1/2 is called the median of the distribution
If FX is a bijective function, then q = FX
1
()

13
Property 3.6. For (0, 1), the quantile function of a standard normal
random variable is given by :
1
() = 1
(1 ) (32)

Finding the Quantiles of a Normal Distribution. In order to find the

value of a normal random variable X N (, 2 ):
(1) Find in the Table the z = (x )/ value associated with the desired
probability.
(2) Use the transformation x = + z.

Example. Suppose X, the grade on a midterm exam, is normally distributed

with mean 70 and standard deviation 10.
(a) The instructor wants to give 15% of the class an A. What cutoff should the
instructor use to determine who gets an A?
(b) The instructor now wants to give the next 10% of the class an A-. For which
range of grades should the instructor assign an A-?

3.4 Exponential Distribution

Motivating example. Let Y be the discrete rrv equal to the number of people
joining the line to visit the Eiffel Tower in an interval of one hour. If , the mean
number of people arriving in an interval of one hour, is 800, we are interested
in the continuous random variable X, the waiting time until the first visitor
arrives.
Definition 3.5. A continuous random variable is said to follow an exponential
distribution E() with > 0 if its pdf fX is given by:
fX (x) = ex 1[0,) (x) (33)
Property 3.7 (Mean and Variance for an Exponential Distribution). If X
follows an exponential distribution E(), then
its expected value is given by :
1
E[X] = (34)

its variance is given by :
1
Var(X) = (35)
2

Example. Suppose that people join the line to visit the Eiffel Tower according
to an approximate Poisson process at a mean rate of 800 visitors per hour. What
is the probability that nobody joins the line in the next 30 seconds?

14