2017-13

Multivariate Probability Theory: Determination of Probability Density

Functions

Hugo Hernandez
ForsChem Research, 050030 Medellin, Colombia
hugo.hernandez@forschem.org

doi: 10.13140/RG.2.2.28214.60481

Abstract

Many random variables can be described as arbitrary functions of one or more independent
random variables with arbitrary probability distributions. The resulting probability density
function of any dependent continuous random variable can be mathematically described in
terms of the probability density functions of the independent variables, as long as the function
can be expressed explicitly in terms of at least one independent variable. Such theory of
random variable transformation has been very well established but it is not widely known.
Thus, the sole purpose of the present report is to provide an illustrative reference by means of
several examples, of the calculation of probability density functions for nonlinear and
multivariate random functions. One of the most relevant examples shows that the norm of a
three-dimensional vector of identical zero-mean normal random variables is a Maxwell-
Boltzmann distribution, whereas its direction corresponds to a vector of uniform distributions.
In another relevant example, the validity of the Central Limit Theorem is illustrated for the
average of several independent exponential random variables.

Keywords

Exponential distribution, Non-linear transformation, Normal distribution, Maxwell-Boltzmann,

Multivariate probability theory, Probability density function, Random functions, Random
variables, Uniform distribution

1. Introduction

Random variables are ubiquitous in Nature. For example, all variables that are measured
experimentally present fluctuation and therefore, they are random. Also, the lack of
information about a certain process or system leads to uncertainty and randomness.[1]

18/11/2017 ForsChem Research Reports 2017-13 (1 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

The behavior of random variables is described by probability distributions. A probability

distribution indicates the relative frequency of occurrence of a certain result or realization of
the random variable, which is observed after a very large number of measurements or
experiments. In particular, for continuous random variables, such probability distribution is fully
described by the probability density function. Thus, all the properties of a continuous random
variable can be obtained from the knowledge of its probability density function.

Certain random variables may depend on the behavior of one or more independent random
variables, and such dependency can be described by means of mathematical functions. The
purpose of the present report is to illustrate, by means of various examples, the determination
of the probability density function of a dependent random variable which is an arbitrary
function of one or more independent random variables, given that the probability density
functions of the independent variables are known, and that the random function can be solved
explicitly for at least one independent random variable.

In Section 2, the general expression for the calculation of the probability density function of a
random variable which is an arbitrary non-linear function of a single random variable is
presented and illustrated. An additional degree of difficulty is found in Section 3, when the
probability density function of a random variable depending on two independent random
variables is considered. Finally, the more general case of functions of several independent
random variables is explained in Section 4, where some relevant examples of application are
also discussed. In one example, the probability distribution functions of the norm and the
direction vector of any three-dimensional vector of identical zero-mean normal random
variables are obtained, indicating that the norm behaves as a Maxwell-Boltzmann distribution,
whereas the direction is a uniform random vector. In another example, the probability density
function for the average value of several independent exponential random variables is derived,
showing that for a large number of variables, the normal distribution is obtained, in agreement
with the Central Limit Theorem.

2. Nonlinear Functions of a Single Random Variable

Let us first consider a general non-linear transformation of random variable with any
arbitrary probability density distribution , where denotes any particular realization of .
Its probability density function can be formally defined as:

(2.1)

where is the cumulative probability function of the random variable , and

18/11/2017 ForsChem Research Reports 2017-13 (2 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(2.2)

Because the non-linear transformation of a random variable is also a random variable, it is then
possible to define the random variable as:

(2.3)

This random variable can also be described by its own probability density function , such
that:

(2.4)

Now, the probability of taking an arbitrary value , is:

(2.5)

where the function is the inverse function of , such that:

(2.6)

Since the inverse function may lead to multiple values if it turns out to be a multivalued
function, then Eq. (2.5) should be more generally expressed as:

(2.7)

where denotes the -th value, and is the total number of different possible values of
.

Combining Eq. (2.1), (2.2), (2.4) and (2.7) results in:

( )

(2.8)

18/11/2017 ForsChem Research Reports 2017-13 (3 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Therefore, the probability density function of the nonlinear transformation of is:

( )| | ( )| |

(2.9)

The absolute value of the derivative is required because the probability density function is non-
negative by definition.

Eq. (2.9) is usually known as the change of variables theorem or the variable transformation
theorem.[2]

The applicability of this theorem will be demonstrated with two examples.

Let us first consider a uniform random variable with the following probability density
function:

(2.10)

Let us also consider in this example that ; and make the following transformation:

(2.11)

Thus, the inverse function will be:

(2.12)

which is a one-to-one function, and therefore there is only one single value for each value .

The resulting probability density function of will then be (from Eq. 2.9):

( )| | ( )| | {

(2.13)

18/11/2017 ForsChem Research Reports 2017-13 (4 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Figure 1 shows the probability density functions of random variables and , considering
and .

Figure 1. Probability density function of uniform random variable (Eq. 2.10) defined in the
range from 2 to 10 (left) and the corresponding probability density function (Eq. 2.13) of its
reciprocal random variable (right).

As the second example, let us consider the normal random variable with mean 0 and
standard deviation 1, with the following probability density function:

(2.14)

Now, let us consider the transformation:

| |
(2.15)

The inverse function will be:

(2.16)

which clearly results in two possible values of for each value . Thus, the probability density
function of will be (from Eq. 2.9):

18/11/2017 ForsChem Research Reports 2017-13 (5 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

( )| ( )| ( )| ( )|
( ) ( )

| | | |

(2.17)

Figure 2 shows the probability density functions of random variables (Eq. 2.14) and (Eq.
2.17).

Figure 2. Probability density function of the standard normal random variable given by Eq.
(2.14) (blue line) and the corresponding probability density function (Eq. 2.17) of its absolute
value (green line).

3. Functions of two Random Variables

Let us now consider the case when a random variable is a function of two independent
random variables and , with arbitrary probability distributions given by and
. The function is:

(3.1)

Since and are independent, then the probability of taking any arbitrary value is:

18/11/2017 ForsChem Research Reports 2017-13 (6 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(3.2)

where the function given in Eq. (3.1) is solved explicitly for (with possible solutions) as:

(3.3)

and for (with possible solutions) as:

(3.4)

Thus, as long as one of the independent variables can be expressed explicitly as a function of
and the other independent variable, then it will be possible to obtain the probability density
function of :

( )| |

( )| |

(3.5)

Again, let us consider some examples. First, assume the following function:

(3.6)

where and are independent uniform random variables defined between 0 and 1, and
therefore:

{
(3.7)

{
(3.8)

Solving Eq. (3.6) explicitly in terms of :

18/11/2017 ForsChem Research Reports 2017-13 (7 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(3.9)

which corresponds to a one-to-one relationship between and (for a constant value of ),

and with partial derivative:

(3.10)

Thus, using Eq. (3.5), the probability density function of is:

( )| |

(3.11)

Now, from Eq. (3.7):

{
(3.12)

So there are two conditions on for having a non-zero probability density function:
and . Since both conditions are necessary, then it is possible to use
the following integrated condition:

(3.13)

which also implies that:

(3.14)

And therefore:

18/11/2017 ForsChem Research Reports 2017-13 (8 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(3.15)

Figure 3 shows the probability density function obtained, according to Eq. (3.15).

Figure 3. Probability density function of the sum of two independent uniform random variables
(both defined in the range [0,1]), , calculated from Eq. (3.15).

In another example, let us consider two independent random variables and , both
described by the standard normal distribution:

(3.16)

(3.17)

Let us also consider their norm as a random function:

18/11/2017 ForsChem Research Reports 2017-13 (9 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(3.18)

Expressing Eq. (3.18) explicitly in terms of yields:

(3.19)

with partial derivative:

| |

(3.20)

Thus, from Eq. (3.5):

( ) ( )

( ) ( )

( )

(3.21)

Since the integral only exists in the real plane if , then:

( )|

[ ] [ ( )]
(3.22)

Figure 4 shows the probability density function obtained in Eq. (3.22).

18/11/2017 ForsChem Research Reports 2017-13 (10 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Figure 4. Probability density function of the positive square root of the sum of squares of two
independent standard normal random variables, , calculated from Eq. (3.22).

4. Functions of several Random Variables

The results obtained in the previous sections can be extended to a more general random
multivariate function of independent random variables with arbitrary probability
distributions:

(4.1)

If such function can be expressed explicitly in terms of any of the independent variables (e.g.
) as:

(4.2)

with possible solutions, then, the probability density function of will be:

( ( ) )

( )| |

(4.3)

18/11/2017 ForsChem Research Reports 2017-13 (11 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Let us for consider again the norm as a random function:

(4.4)

where are independent standard normal random variables, and the realizations of
are non-negative.

Solving for yields:

(4.5)

with

| |

(4.6)

And from Eq. (4.3):

( ) ( )

( )

(4.7)

The probability density function obtained corresponds to the Maxwell-Boltzmann

distribution.[3] Again, such probability density function in three dimensions obtained is
presented in Figure 5, compared to the equivalent functions for one and two dimensions.

18/11/2017 ForsChem Research Reports 2017-13 (12 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Figure 5. Probability density function of the positive square root of the sum of squares of three
independent standard normal random variables, , calculated from Eq. (4.7)
(blue line), compared to the probability density functions for one (green line) and two
dimensions (red line).

Please notice that the average value of the 3-dimensional function is:

[ | ]

[ | ]

(4.8)

Thus, if we define the random function as:

(4.9)

then, the standard Maxwell-Boltzmann distribution is obtained.[4] Please notice that the
expected value of the standard Maxwell-Boltzmann random variable is .

Now, given that , then from Eq. (2.9):

18/11/2017 ForsChem Research Reports 2017-13 (13 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

( )

( )| | ( )

(4.10)

which is the probability density function of the standard Maxwell-Boltzmann distribution.

On the other hand, let us consider the following direction random function for the same three
independent standard normal random variables in the:

(4.11)

which can also be expressed as:

(4.12)

This is a one-to-one function valid only for realizations of in the range , and with
partial derivative:

| | ( )

(4.13)

Thus, from Eq. (4.3):

( )

( )

( )

( )

(4.14)

Eq. (4.14) can be transformed using the following substitutions:

18/11/2017 ForsChem Research Reports 2017-13 (14 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

(4.15)

(4.16)
resulting in:

( )

(4.17)

Solving the inner integral yields:

( ( ) ( ))

( )

( ( ) ( ) )

(4.18)

where is the modified Bessel function of the second kind.

Solving the remaining integrals:

[ ]

(4.19)

Eq. (4.19) corresponds to the probability density function of a uniform distribution in the range
[ ]. The expected value and the variance for this distribution are, respectively:

(4.20)

Integration performed with the help of WolframAlpha

18/11/2017 ForsChem Research Reports 2017-13 (15 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

( ) |

(4.21)

As an interpretation of these results, let us consider the vector random variable , defined as:

[ ] [ ]

(4.22)

where are zero-mean normal random variables with standard deviation . The norm of this
random vector will be:

(4.23)

From the definitions given in Eq. (4.4) and (4.9):

(4.24)

where corresponds to the average value of the magnitude of .

Now, let us define the direction vector of as:

[ ]

[ ]

(4.25)

This means that the direction vector of a 3-dimensional vector of identical normal random
variables with mean zero is a vector composed of uniform random variables, each one
described by the probability density function given in Eq. (4.19).

The expected value of is:

18/11/2017 ForsChem Research Reports 2017-13 (16 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

( ) [ ] [ ]

(4.26)
and its variance is:
( ) ( ) ( ) ( )
(4.27)
Combining Eq. (4.24) and (4.25) yields:

(4.28)

Therefore, the vector of zero-mean normal random variables can be expressed in terms of the
standard Maxwell-Boltzmann variable and the uniform unit vector .

As a final example, let us now consider the average of a large number of independent
exponential random variables with a probability distribution given by:

(4.29)
and where .

Such average function is:

(4.30)

Now, solving for , results in a one-to-one function:

(4.31)
with partial derivative:

| |

(4.32)

Thus, from Eq. (4.3) and given that can only take positive values:

18/11/2017 ForsChem Research Reports 2017-13 (17 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

( )
( )

( )

(4.33)
Thus, Eq. (4.33) can be solved recursively yielding:

( )

( )

( )

(4.34)

Figure 6 shows the behavior of the probability density function of the average value as the
number of independent exponential random variables increases.

For large values of , it is possible to use Stirlings approximation:[5]

(4.35)
Thus, Eq. (4.34) becomes:

(4.36)

18/11/2017 ForsChem Research Reports 2017-13 (18 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Figure 6. Evolution of the probability density function of the average function for
independent exponential random variables calculated from Eq. (4.34) for different values of .

Now, let us consider the following approximation for based on the first two terms of a
series expansion:

(4.37)

and replacing in Eq. (4.36) results in:

( )
( )

( )

(4.38)

which corresponds to the probability density function of a normal distribution with mean and
variance , which is in agreement with the Central Limit Theorem.[6]

18/11/2017 ForsChem Research Reports 2017-13 (19 / 20)

www.forschem.org
 Multivariate Probability Theory:
Determination of Probability Density Functions
Hugo Hernandez
ForsChem Research
hugo.hernandez@forschem.org

Acknowledgments

This research did not receive any specific grant from funding agencies in the public,
commercial, or not-for-profit sectors.

References

[1] Hernandez, H. (2017). Ergodic-Stochastic Transformations. ForsChem Research Reports

2017-12. doi: 10.13140/RG.2.2.20325.70881

[2] Casella, G., & Berger, R. L. (2002). Statistical Inference. 2nd Ed. Pacific Grove: Duxbury.

[3] Hernandez, H. (2017). On the generalized validity of the Maxwell-Boltzmann distribution and
the zeroth Law of Thermodynamics. ForsChem Research Reports 2017-4. doi:
10.13140/RG.2.2.26937.16480.

[4] Hernandez, H. (2017). Standard Maxwell-Boltzmann distribution: Definition and properties.

ForsChem Research Reports 2017-2. doi: 10.13140/RG.2.2.29888.74244.

[5] Marsaglia, G., & Marsaglia, J. C. (1990). A new derivation of Stirling's approximation to n!.
American Mathematical Monthly, 826-829.

[6] Canavos, G. C. (1984). Applied probability and statistical methods. Englewood Cliffs:
Prentice Hall.

18/11/2017 ForsChem Research Reports 2017-13 (20 / 20)

www.forschem.org