Professional Documents
Culture Documents
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 2
Discrete Random Variables
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 3
Continuous Random Variable
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 4
Joint Probability
Consider two random variables x and y
If we observe multiple paired instances, then some
combinations of outcomes are more likely than
others
This is captured in the joint probability distribution
Written as Pr(x,y)
Can read Pr(x,y) as probability of x and y
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 5
Joint Probability
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 6
Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 7
Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 8
Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 9
Marginalization
We can recover probability distribution of any variable in a joint distribution
by integrating (or summing) over the other variables
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 10
Conditional Probability
Conditional probability of x given that y=y1 is relative
propensity of variable x to take different outcomes given that
y is fixed to be equal to y1.
Written as Pr(x|y=y1)
11
Computer vision: models, learning and inference. 2011 Simon J.D. Prince
Conditional Probability
Conditional probability can be extracted from joint probability
Extract appropriate slice and normalize
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 12
Conditional Probability
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 13
Conditional Probability
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 14
Bayes Rule
From before:
Combining:
Re-arranging:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 15
Bayes Rule Terminology
Likelihood propensity for Prior what we know
observing a certain value of about y before seeing x
x given a certain value of y
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 17
Independence
If two variables x and y are independent then variable x tells
us nothing about variable y (and vice-versa)
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 18
Independence
When variables are independent, the joint factorizes into a
product of the marginals:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 19
Expectation
Expectation tell us the expected or average value of some
function f [x] taking into account the distribution of x
Definition:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 20
Expectation
Expectation tell us the expected or average value of some
function f [x] taking into account the distribution of x
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 21
Expectation: Common Cases
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 22
Expectation: Rules
Rule 1:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 23
Expectation: Rules
Rule 2:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 24
Expectation: Rules
Rule 3:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 25
Expectation: Rules
Rule 4:
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 26
Conclusions
Rules of probability are compact and simple
Computer vision: models, learning and inference. 2011 Simon J.D. Prince 27