Multinomial Distribution

Statistical Learning

Stevanato Alberto - Statistical Learning - 2011/2012

Preface
The multinomial distribution is a discrete
distribution of probability, coming from a
generalization of the Binomial Distribution.

Fig. 1 Binomial Distribution Formula

Stevanato Alberto - Statistical Learning - 2011/2012

Preface
As we know, the Binomial Distribution describes
a Bernoulli process for a general variable X, with
a range of value for the single test on X that
consists in 0 and 1, where 1 is success and 0 is
failure.
The Multinomial, instead, counts the number of
times that a test gives a certain value. For
example, the number of times that the roll of a
dice gives every single face (1,2,3,4,5,6).
Stevanato Alberto - Statistical Learning - 2011/2012

The MD Formula
Like the BD, the MD has a formula to calculate
its value, that is the following:

Fig. 2 Multinomial Distribution Formula

Lets see what each number stands for.

Stevanato Alberto - Statistical Learning - 2011/2012

The MD Formula
As for the BD, we have a numerosity for our
sample that is the number n. Every ni , (i:1..s)
defines the value of each result for xi. For the
dice example, every ni stands for the number of
times that it gives the face i.
Then, pi stands for the probability of each test to
give the result xi.
Finally, the Multinomial Coefficient counts all
the possible combination of n1 results x1, n2
results x2, etc.
Stevanato Alberto - Statistical Learning - 2011/2012

Properties of the MD
The main properties for the MD concern on the
value of pi and ni :

n = ni, the sum ofevery possible
combination has to give the numerosity of
the sample;

pi = 1, the sum of the probability has to
give 1 (as for BD).
Stevanato Alberto - Statistical Learning - 2011/2012

Expected Value and Variance

The expected value for the MD is the following:
E[M] = n*(p1, , ps)
And the variance:
Var[M] = n*pi*(1 - pi)
This is true for the following property:
If S = (S1,...,Ss) is an aleatory vector following the
Multinomial Distribution of parameters ((p1,...,ps),n),
every Si can be treated as a Binomial Variable
Stevanato Alberto - Statistical Learning - 2011/2012

Expected value and Variance

For the expected value we have:

and for the variance (jumping preliminar steps):

Si,i = n*Var(Si) = n*pi*(1 - pi)
because we know that the Binomial Distribution
has mean like n*p and variance n*p*(1-p).
Stevanato Alberto - Statistical Learning - 2011/2012

A simple example: roll of a dice

Taking the dice example explained at the
beginning, we can say that the number of results
described by each face follows the multinomial
distribution of parameters ((1/6, 1/6, 1/6, 1/6,
1/6, 1/6), n).
If we say that the face 6 comes out 1 times, 5
comes out 2 times, 4 comes out 4 times, 3 comes
out 0 times, 2 comes out 2 times and 1 comes out
3 times, we have n = 12.
Stevanato Alberto - Statistical Learning - 2011/2012

10

A simple example: roll of a dice

So, we can calculate the probability of this
experiment as:
P(3, 2, 0, 4, 2, 1) =
=

So, in this particular case, we have a very small

probability.
Stevanato Alberto - Statistical Learning - 2011/2012

11

Summarizing
When we have a distribution that seems like a
Binomial Distribution, but we have different
results for each entity involved in the test, we
can use the Multinomial Distribution. Those
cases are: roll of a dice, extraction of colored
spheres from an urn, etc.

Stevanato Alberto - Statistical Learning - 2011/2012