Introduction to Probability

Experiments
Counting Rules
Combinations
Permutations
Assigning Probabilities

These are
processes that
generate welldefined
outcomes

Experiments

Experiment

Experimental Outcomes

Toss a coin

Head, tail

Select a part for

inspection

Defective, nondefective

Conduct a sales call

Purchase, no purchase

Roll a die

1, 2, 3, 4, 5, 6

Play a football game

Win, lose, tie

Sample Space
The sample space for an experiment
is the set of all experimental outcomes
S Head, Tail

For a coin toss:

Selecting a part
for inspection:
S 1, 2, 3, 4, 5, 6

Rolling a die:

S Defective, Nondefective

Event
Any subset of the sample space is
called an event.
S 1, 2, 3, 4, 5, 6

Rolling a die:

Events:
{1}
// the outcome is 1 (elementary event)
{ 1, 3, 5 } // the outcome is an odd number
{ 4, 5, 6 } // the outcome is at least 4.

Probability is a
numerical measure of
the likelihood of an
event occurring

0.5

Probability:

The occurrence of the event is just

as likely as it is unlikely

1.0

Basic Requirements for

Assigning Probabilities
Let Ei denote the ith experimental
outcome (elementary event) and P(Ei) is
its probability of occurring. Then:

0 P( Ei ) 1 for all i

The sum of the probabilities for all

experimental outcomes must be must
equal 1. For n experimental outcomes:

P ( E1 ) P ( E2 ) . . . P ( En ) 1

Principle of Indifference
We assign equal probability to
elementary events if we have
no reason to expect one over
the other.

1
P ( Ei )
n

For a coin toss:

This method of
assigning probabilities
is indicated if each
experimental outcome
is equally likely

S Head, Tail

P(Head) = P(Tail) = 1/2

Rolling a die:

S 1, 2, 3, 4, 5, 6

P(1) = P(2) = = P(6) = 1/6

Selecting a part
for inspection:

S Defective, Nondefective
P(Defective) = ?

Relative Frequency
Method
This method is indicated when the data are available
to estimate the proportion of the time the
experimental outcome will occur if the experiment is
repeated a large number of times.
What if experimental
outcomes are NOT equally
likely. Then the Principle of
Indifference is out. We must
assign probabilities on the
basis of experimentation or
historical data.

Selecting a part
for inspection:

S Defective, Nondefective
N parts: n1 defective and n2 nondefective
P(Defective) = n1/N, P(Nondefective) = n2/N

Counting Experimental Outcomes

To assign probabilities, we must
first count experimental
outcomes. We have 3 useful
counting rules for multiple-step
experiments. For example, what
is the number of possible
outcomes if we roll the die 4
times?

1. Counting rule for multi-step

experiments
2. Counting rule for combinations
3. Counting rule for permutations

Example: Lucas Tool Rental

Relative Frequency Method

Ace Rental would like to assign probabilities to the number

of carpet cleaners it rents each day. Office records show the
following frequencies of daily rentals for the last 40 days.

Number of
Number
Cleaners Rented of Days
0
4
1
6
2
18
3
10
4
2

Relative Frequency Method

Each probability assignment is given by dividing
the frequency (number of days) by the total frequency
(total number of days).

Number of
Number
Cleaners Rented of Days Probability
.10
4
0
.15
6
1
4/40
.45
18
2
.25
10
3
.05
2
4
1.00
40

Subjective Method

When economic conditions and a companys

circumstances change rapidly it might be
inappropriate to assign probabilities based solely on
historical data.
We can use any data available as well as our
experience and intuition, but ultimately a probabilit
value should express our degree of belief that the
experimental outcome will occur.

The best probability estimates often are obtained by

combining the estimates from the classical or relati
frequency approach with the subjective estimate.

Counting Rule for Multi-Step

Experiments
If an experiment can be described as a
sequence of k steps with n1 possible outcomes
on the first step, n2 possible outcomes on the
second step, then the total number of
experimental outcomes is given by:

(n1 )(n2 ) . . . (nk )

Example: Bradley Investments

Bradley has invested in two stocks, Markley Oil
and Collins Mining. Bradley has determined that
the possible outcomes of these investments
three months from now are as follows.

Investment Gain or Loss

in 3 Months (in $000)
Markley Oil Collins Mining
10
8
5
2
0
20

A Counting Rule for

Multiple-Step Experiments
Bradley Investments can be viewed as a
two-step experiment. It involves two stocks, each
with a set of experimental outcomes.
Markley Oil:
n1 = 4
Collins Mining:
n2 = 2
Total Number of
Experimental Outcomes:
n1n2 = (4)(2) = 8

Tree Diagram
Markley Oil Collins Mining
(Stage 2)
(Stage 1)
Gain 8
Gain 10

Gain 8

Gain 5 Lose 2

Lose 2

Gain 8

Even
Lose 20

Gain 8
Lose 2

Lose 2

Experimental
Outcomes
(10, 8)

Gain $18,000
$18,00

(10, -2)

Gain

(5, 8)

$8,000
$8,00

Gain $13,000

(5, -2)

Gain

$3,000
$3,00

(0, 8)

Gain

$8,00

(0, -2)

Lose

$2,00

(-20, 8)

Lose

$12,00

(-20, -2)

Lose

$22,00

This
rule allows us to
Counting Rule
for
count the number of
Combinations
experimental outcomes
when we select n objects
from a (usually larger)
set of N objects.

The number of N objects taken n

at a time is
N
N!
C
n n!( N n)!
N
n

where

And by definition

N ! N ( N 1)( N 2) . . .(2)(1)
n! n(n 1)(n 2) . . .(2)(1)

0! 1

Example: Quality Control

An inspector randomly selects 2 of 5
parts for inspection. In a group of 5
parts, how many combinations of 2
parts can be selected?
5
5!
(5)(4)(3)(2)(1) 120

10
2 2!(5 2)! (2)(1)(3)(2)(1) 12
5
2

Let the parts de designated A, B, C, D, E. Thus we could

select:

AB AC AD AE BC BD BE CD CE and DE

Iowa Lottery
Iowa randomly selects 6 integers from a group of 47 to
determine the weekly winner. What are your odds of winning if
you purchased one ticket?
47
47!
(47)(46)(45)(44)(43)(42)

C647

10,737,573
(6)(5)(4)(3)(2)(1)
6 6!(47 6)!

Counting Rule for

Sometimes the order of
Permutations
selection matters. This
rule allows us to count the
number of experimental
outcomes when n objects
are to be selected from a
set of N objects and the
order of selection matters.

N
N!
P n!
n ( N n)!
N
n

Example: Quality Control Again

An inspector randomly selects 2 of 5
parts for inspection. In a group of 5
parts, how many permutations of 2
parts can be selected?

5!
5! (5)(4)(3)(2)(1) 120
P

20
(5 2)! 3! (3)(2)(1)
6
5
2

Again let the parts be designated A, B, C, D, E. Thus we could

select:

AB BA AC CA AD DA AE EA BC CB BD DB BE
EB CD DC CE EC DE and ED

Some Basic Relationships of

Probability
There are some basic probability relationships that
can be used to compute the probability of an event
without knowledge of all the sample point
probabilities.

Complement
Complement of
of an
an Event
Event
Union
Union of
of Two
Two Events
Events
Intersection
Intersection of
of Two
Two Events
Events
Mutually
Mutually Exclusive
Exclusive Events
Events

Complement of an Event

The
The complement
complement of
of event
event AA is
is defined
defined to
to be
be the
the even
eve
consisting
consisting of
of all
all sample
sample points
points that
that are
are not
not in
in A.
A.
c
The
The complement
complement of
of AA is
is denoted
denoted by
by AAc..

Event A

Venn
Diagra
m

Ac

Sample
Space S

Union of Two Events

The
The union
union of
of events
events AA and
and BB is
is the
the event
event containing
containing
all
all sample
sample points
points that
that are
are in
in AA or
or BB or
or both.
both.
The
The union
union of
of events
events AA and
and BB is
is denoted
denoted by
by AA
BB

Event A

Event B

Sample
Space S

Union of Two Events

Event M = Markley Oil Profitable
Event C = Collins Mining Profitable
M C = Markley Oil Profitable
or Collins Mining Profitable

M C = {(10, 8), (10, 2), (5, 8), (5, 2), (0, 8), (2
P(M C) = P(10, 8) + P(10, 2) + P(5, 8) + P(5, 2)
+ P(0, 8) + P(20, 8)
= .20 + .08 + .16 + .26 + .10 + .02
= .82

Intersection of Two Events

The
The intersection
intersection of
of events
events AA and
and BB is
is the
the set
set of
of all
all
sample
sample points
points that
that are
are in
in both
both AA and
and BB..

The
The intersection
intersection of
of events
events AA and
and BB is
is denoted
denoted by
by AA

Event A

Event B

Intersection of A and B

Sample
Space S

Intersection of Two Events

Event M = Markley Oil Profitable
Event C = Collins Mining Profitable
M C = Markley Oil Profitable
and Collins Mining Profitable
M C = {(10, 8), (5, 8)}
P(M C) = P(10, 8) + P(5, 8)
= .20 + .16
= .36

Addition Law

The
The addition
addition law
law provides
provides aa way
way to
to compute
compute the
the
probability
probability of
of event
event A,
A, or
or B,
B, or
or both
both AA and
and BB occurrin
occurrin
The
The law
law is
is written
written as:
as:
P(A B) = P(A) + P(B) P(A B

Addition Law
Event M = Markley Oil Profitable
Event C = Collins Mining Profitable
M C = Markley Oil Profitable
or Collins Mining Profitable
We know: P(M) = .70, P(C) = .48, P(M C) = .36
Thus: P(M C) = P(M) + P(C) P(M C)
= .70 + .48 .36
= .82
(This result is the same as that obtained earlier
using the definition of the probability of an event.)

Mutually Exclusive Events

Two
Two events
events are
are said
said to
to be
be mutually
mutually exclusive
exclusive ifif the
the
events
events have
have no
no sample
sample points
points in
in common.
common.

Two
Two events
events are
are mutually
mutually exclusive
exclusive if,
if, when
when one
one even
even
occurs,
occurs, the
the other
other cannot
cannot occur.
occur.

Event A

Event B

Sample
Space S

Mutually Exclusive Events

IfIf events
events AA and
and BB are
are mutually
mutually exclusive,
exclusive, PP((AA
BB =
=
The
The addition
addition law
law for
for mutually
mutually exclusive
exclusive events
events is:
is:
P(A B) = P(A) + P(B)

theres no need to
include P(A
B

Conditional Probability

The
The probability
probability of
of an
an event
event given
given that
that another
another event
event
has
has occurred
occurred is
is called
called aa conditional
conditional probability
probability..
The
The conditional
conditional probability
probability of
of AA given
given BB is
is denoted
denoted
by
by PP((AA||BB).).
AA conditional
conditional probability
probability is
is computed
computed as
as follows
follows ::

P( A B)
P( A| B)
P(B)

Conditional Probability
Event M = Markley Oil Profitable
Event C = Collins Mining Profitable

P(C| M ) = Collins Mining Profitable

given Markley Oil Profitable
We know: P(M C) = .36, P(M) = .70

P(C M ) .36

.5143
Thus:P(C| M )
P( M )
.70

Multiplication Law

The
The multiplication
multiplication law
law provides
provides aa way
way to
to compute
compute th
th
probability
probability of
of the
the intersection
intersection of
of two
two events.
events.
The
The law
law is
is written
written as:
as:
P(A B) = P(B)P(A|B)

Multiplication Law
Event M = Markley Oil Profitable
Event C = Collins Mining Profitable
M C = Markley Oil Profitable
and Collins Mining Profitable
We know: P(M) = .70, P(C|M) = .5143
Thus: P(M C) = P(M)P(M|C)
= (.70)(.5143)
= .36
(This result is the same as that obtained earlier
using the definition of the probability of an event.)

Independent Events
IfIf the
the probability
probability of
of event
event AA is
is not
not changed
changed by
by the
the
existence
existence of
of event
event BB,, we
we would
would say
say that
that events
events AA
and
and BB are
are independent
independent..
Two
Two events
events AA and
and BB are
are independent
independent if:
if:
P(A|B) = P(A)

or

P(B|A) = P(B)

Multiplication Law
for Independent Events

The
The multiplication
multiplication law
law also
also can
can be
be used
used as
as aa test
test to
to ss
ifif two
two events
events are
are independent.
independent.
The
The law
law is
is written
written as:
as:
P(A B) = P(A)P(B)

Multiplication Law
for Independent Events
Event M = Markley Oil Profitable
Event C = Collins Mining Profitable
Are events M and C independent?
DoesP(M C) = P(M)P(C) ?
We know: P(M C) = .36, P(M) = .70, P(C) = .48
But: P(M)P(C) = (.70)(.48) = .34, not .36
Hence: M and C are not independent.

Terminology
Eventsmayormaynotbemutuallyexclusive.
IfEandFaremutuallyexclusiveevents,then
P(EUF)=P(E)+P(F)
IfEandFarenotmutuallyexclusive,then
P(EUF)=P(E)+P(F)P(EnF).
Allelementaryeventsaremutuallyexclusive.

Thebirthofasonora
daughteraremutually
exclusiveevents.
The event that the outcome
of rolling a die is even and
the event that the outcome of
rolling a die is at least four
are not mutually exclusive.

Simpleprobabilities
IfAandBaremutuallyexclusiveevents,
thentheprobabilityofeitherAorBtooccur
istheunion
P(A B) P(A) P(B)
Example:Theprobabilityofahatbeingredis,theprobabilityof
thehatbeinggreenis,andtheprobabilityofthehatbeingblackis
.Then,theprobabilityofahatbeingredORblackis.

Simpleprobabilities

IfAandBareindependentevents,thenthe
probabilitythatbothAandBoccuristhe
intersection

P(A B) P(A) P(B)

Simpleprobabilities
Example: The probability that a US president is bearded is
~14%, the probability that a US president died in office is
~19%. If the two events are independent, the probability that
a president both had a beard and died in office is ~3%. In
reality, 2 bearded presidents died in office. (A close enough
result.)
Harrison, Taylor, Lincoln*, Garfield*, McKinley*, Harding, Roosevelt, Kennedy* (*assassinated)

Conditionalprobabilities
WhatistheprobabilityofeventAtooccur
giventhateventBdidoccur.Theconditional
probabilityofAgivenBis

P(A B)
P(A | B)
P(A)
Example: The probability that a US president dies in office
if he is bearded 0.03/0.14 = 22%. Thus, out of 6 bearded
presidents, 22% are expected to die in office. In reality, 2
died. (Again, a close enough result.)

ProbabilityDistribution
Theprobabilitydistribution
referstothefrequencywithwhich
allpossibleoutcomesoccur.There
arenumeroustypesofprobability
distribution.

TheUniformDistribution
Avariableissaidtobeuniformlydistributedifthe
probabilityofallpossibleoutcomesareequaltoone
another.Thus,theprobabilityP(i),whereiisoneofn
possibleoutcomes,is

1
P(i)
n

TheBinomialDistribution
Aprocessthathasonlytwopossibleoutcomesiscalleda
binomialprocess.Instatistics,thetwooutcomesare
frequentlydenotedassuccessandfailure.The
probabilitiesofasuccessorafailurearedenotedbypand
q,respectively.Notethatp+q=1.Thebinomial
distributiongivestheprobabilityofexactlyksuccessesin
ntrials

n k
n

k
P(k) p 1 p
k

TheBinomialDistribution
Themeanandvarianceofabinomiallydistributedvariable
aregivenby

np

V npq

ThePoissondistribution

Poisson dApril

Simon
Simon Denis
Denis Poisson
Poisson
1781-1840
1781-1840

ThePoissondistribution
Whentheprobabilityofsuccessisverysmall,e.g.,the
probabilityofamutation,thenpkand(1p)nkbecometoo
smalltocalculateexactlybythebinomialdistribution.In
suchcases,thePoissondistributionbecomesuseful.Let
betheexpectednumberofsuccessesinaprocess
consistingofntrials,i.e.,=np.Theprobabilityof
observingksuccessesis

e
P(k)
k!

ThemeanandvarianceofaPoissondistributed
variablearegivenby=andV=,respectively.

Normal Distribution

Gamma Distribution