Coin Flip

The Problem
A coin has two sides: heads and tails. When you flip it, what are the chances it will come up heads? What are the chances it will come up tails?

The Analysis
Assuming the coin is not tainted, it has an equal chance of coming up on either side. Therefore, it should come up heads one of every two times or 50%. It should also come up tails one of every two times or 50%.

Playing the Game

An image of a coin appears near the top left corner of the window. To flip the coin, click on the image. To flip it again, click on it again. The actual and expected results are shown in the bottom half of the window. You can choose to see just results for the current session or to see historical results (since about June 1, 2003). The results are updated after each flip. To automatically flip the coin multiple times, enter a number in the field to the right of the coin and then click the Auto Flip button. The program automatically performs the number of flips you specified and updates the displayed results. If you enter an especially high number, you might experience some delay as the flips are processed.

What is Probability?
'Probability is the bane of the age,' said Moreland, now warming up. 'Every Tom, Dick, and Harry thinks he knows what is probable. The fact is most people have not the smallest idea what is going on round them. Their conclusions about life are based on utterly irrelevant - and usually inaccurate premises.' The very name calculus of probabilities is a paradox. Probability opposed to certainty is what we do not know, and how can we calculate what we do not know?

H. Poincar Science and Hypothesis Cosimo Classics, 2007, Chapter XI

Anthony Powell Casanova's Chinese Restaurant 2nd Movement in A Dance to the Music of Time University of Chicago Press, 1995

Intuitively, the mathematical theory of probability deals with patterns that occur in random events. For the theory of probability the nature of randomness is inessential. (Note for the record, that according to the 18th century French mathematician Marquis de Laplace randomness is a perceived phenomenon explained by human ignorance, while the late 20th century mathematics came with a realization that chaos may emerge as the result of deterministic processes.) An experiment is a process - natural or set up deliberately - that has an observable outcome. In the deliberate setting, the word experiment and trial are synonymous. An experiment has a random outcome if the result of the experiment can't be predicted with absolute certainty. An event is a collection of possible outcomes of an experiment. An event is said to occur as a result of an experiment if it contains the actual outcome of that experiment. Individual outcomes comprising an event are said to be favorable to that event. Events are assigned a measure of certainty which is called probability (of an event.) Quite often the word experiment describes an experimental setup, while the word trial applies to actually executing the experiment and obtaining an outcome. A formal theory of probability has been developed in the 1930s by the Russian mathematician A. N. Kolmogorov. The starting point is the sample (or probability) space - a set of all possible outcomes. Let's call it . For the set , a probability is a real-valued function P defined on the subsets of :
(*) P: 2 *0, 1+

Thus we require that the function be non-negative and that its values never exceed 1. The subsets of for which P is defined are called events. 1-element events are said to be elementary. The function is required to be defined on the empty subset and the whole set :
(1) P() = 0, P() = 1.

This says in particular that both and are events. The event that never happens is impossible and has probability 0. The event has probability 1 and is certain or necessary. If is a finite set then usually the notions of an impossible event and an event with probability 0 coincide, although it may not be so. If is infinite then the two notions practically never coincide. A similar dichotomy exists for the notions of a certain event and that with probability 1. Examples will be given shortly. The probability function (or, as it most commonly called, measure) is required to satisfy additional conditions. It must be additive: for two disjoint events A and B, i.e. whenever A B = ,
(2) P(AB) = P(A) + P(B),

which is a consequence of a seemingly more general rule: for any two events A and B, their union AB and intersection AB are events and
(2') P(AB) = P(A) + P(B) - P(A B).

Note, however, that (2') can be derived from (2). Indeed, assuming that all the sets involved are events, events A - B and A B are disjoint as are B - A and A B. In fact, all three events A - B, B - A, and A B are disjoint and the union of the three is exactly AB. We have,
P(A) + P(B) = (P(A - B) + P(A B)) + (P(B - A) + P(A B)) = (P(A - B) + P(A B) + (P(B - A)) + P(A B) = P(AB) + P(A B)

which is exactly (2'). For an infinite space , we require -additivity: for mutually disjoint sets Ai, i = 1, 2, ...,, their union is an event and
(3) P(i1Ai) = i1P(Ai).

In general, the collection of events is assumed to be a -algebra, which means that the complements of events are events and so are the countable unions and intersections. Some properties of P follow immediately from the definition. For example, denote the complement of an event A, A = - A. It is also called a complementary event. Then, naturally, A and A are disjoint, A A = , and from (2),
1 = P() = P(A) + P(A).

In other words,
(4) P(A) = 1 - P(A).

Also from (2) and (*), if B = AC for disjoint A and C, then

P(B) = P(AC) = P(A) + P(C) P(A).

In other words, if A is a subset of B, A

(5) P(B) P(A).

B, then

Probability is a monotone function - the fact that jibes with our intuition that a larger event, i.e. an event with a greater number of favorable outcomes, is more likely to occur than a smaller event. Disjoint events do not share favorable outcomes and, for this reason, are often called incompatible or, even more assertively, mutually exclusive. In addition to a sample space, an experiment may be profitably associated with what is known as a random variable. X is a random variable if its values are not determined with certainty but come from a sample space defined by a random experiment. If x is a possible outcome of an experiment, we often write P(x) for the probability P({x}) of the elementary event {x}. In terms of random variable X, the same quantity is described as P(X = x), the probability that the random variable X takes the value of x.

References 1. R. B. Ash, Basic Probability Theory, Dover, 2008 2. A. N. Kolmogorov, The Theory of Probability, in Mathematics: Its Content, Methods and Meaning, Dover, 1999 3. P. S. de Laplace, Concerning Probability, in The World of Mathematics, Dover, 2003 4. P. S. de Laplace, A Philosophical Essay on Probability, in God Created the Integers: The Mathematical Breakthroughs That Changed History , Running Press, 2007

Event (probability theory)

In probability theory, an event is a set of outcomes (a subset of the sample space) to which a probability is assigned. Typically, when the sample space is finite, any subset of the sample space is an event (i.e. all elements of the power set of the sample space are defined as events). However, this approach does not work well in cases where the sample space is infinite, most notably when the outcome is a real number. So, when defining a probability space it is possible, and often necessary, to exclude certain subsets of the sample space from being events (see Events in probability spaces, below).

A simple example
If we assemble a deck of 52 playing cards with no jokers, and draw a single card from the deck, then the sample space is a 52-element set, as each individual card is a possible outcome. An event, however, is any subset of the sample space, including any single-element set (an elementary event, of which there are 52, representing the 52 possible cards drawn from the deck), the empty set (an impossible event, defined to have probability zero) and the sample space itself (the entire set of 52 cards), which is defined to have probability one. Other events are proper subsets of the sample space that contain multiple elements. So, for example, potential events include:

A Venn diagram of an event. B is the sample space and A is an event. By the ratio of their areas, the probability of A is approximately 0.4.

"Red and black at the same time without being a joker" (0 elements), "The 5 of Hearts" (1 element), "A King" (4 elements), "A Face card" (12 elements), "A Spade" (13 elements), "A Face card or a red suit" (32 elements),

"A card" (52 elements).

Since all events are sets, they are usually written as sets (e.g. {1, 2, 3}), and represented graphically using Venn diagrams. Venn diagrams are particularly useful for representing events because the probability of the event can be identified with the ratio of the area of the event and the area of the sample space. (Indeed, each of the axioms of probability, and the definition of conditional probability can be represented in this fashion.)

[edit] Events in probability spaces

Defining all subsets of the sample space as events works well when there are only finitely many outcomes, but gives rise to problems when the sample space is infinite. For many standard probability distributions, such as the normal distribution the sample space is the set of real numbers or some subset of the real numbers. Attempts to define probabilities for all subsets of the real numbers run into difficulties when one considers 'badly-behaved' sets, such as those that are nonmeasurable. Hence, it is necessary to restrict attention to a more limited family of subsets. For the standard tools of probability theory, such as joint and conditional probabilities, to work, it is necessary to use a -algebra, that is, a family closed under countable unions and intersections. The most natural choice is the Borel measurable set derived from unions and intersections of intervals. However, the larger class of Lebesgue measurable sets proves more useful in practice. In the general measure-theoretic description of probability spaces, an event may be defined as an element of a selected -algebra of subsets of the sample space. Under this definition, any subset of the sample space that is not an element of the -algebra is not an event, and does not have a probability. With a reasonable specification of the probability space, however, all events of interest are elements of the -algebra.

[edit] A note on notation

Even though events are subsets of some sample space , they are often written as propositional formulas involving random variables. For example, if X is a real-valued random variable defined on the sample space , the event

can be written more conveniently as, simply,

This is especially common in formulas for a probability, such as

Probability

Three Different Concepts of Probability The classical interpretation of probability is a theoretical probability based on the physics of the experiment, but does not require the experiment to be performed. For example, we know that the probability of a balanced coin turning up heads is equal to 0.5 without ever performing trials of the experiment. Under the classical interpretation, the probability of an event is defined as the ratio of the number of outcomes favorable to the event divided by the total number of possible outcomes. Sometimes a situation may be too complex to understand the physical nature of it well enough to calculate probabilities. However, by running a large number of trials and observing the outcomes, we can estimate the probability. This is the empirical probability based on long-run relative frequencies and is defined as the ratio of the number of observed outcomes favorable to the event divided by the total number of observed outcomes. The larger the number of trials, the more accurate the estimate of probability. If the system can be modeled by computer, then simulations can be performed in place of physical trials. A manager frequently faces situations in which neither classical nor empirical probabilities are useful. For example, in a one-shot situation such as the launch of a unique product, the probability of success can neither be calculated nor estimated from repeated trials. However, the manager may make an educated guess of the probability. This subjective probability can be thought of as a person's degree of confidence that the event will occur. In absence of better information upon which to rely, subjective probability may be used to make logically consistent decisions, but the quality of those decisions depends on the accuracy of the subjective estimate.

Outcomes and Events An event is a subset of all of the possible outcomes of an experiment. For example, if an experiment consists of flipping a coin two times, the possible outcomes are:

heads, heads heads, tails tails, heads tails, tails

One can define the showing of heads at least one time to be an event, and this event would consist of three of the four possible outcomes.

Given that the probability of each outcome is known, the probability of an event can be determined by summing the probabilities of the individual outcomes associated with the event. A composite event is an event defined by the union or intersection of two events. The union of two events is expressed by the "or" function. For example, the probability that either Event A or Event B (or both) will occur is expressed by P(A or B). The intersection of two events is the probability that both events will occur and is expressed by the "and" function. For exampe, the probability that both Event A and Event B will occur is expressed by P(A and B).

Law of Addition Consider the following Venn diagram in which each of the 25 dots represents an outcome and each of the two circles represents an event.

In the above diagram, Event A is considered to have occurred if an experiment's outcome, represented by one of the dots, falls within the bounds of the left circle. Similarly, Event B is considered to have occurred if an experiment's outcome falls within the bounds of the right circle. If the outcome falls within the overlapping region of the two circles, then both Event A and Event B are considered to have occurred. There are 5 outcomes that fall in the definition of Event A and 6 outcomes that fall in the definition of Event B. Assuming that each outcome represented by a dot occurs with equal probability, the probability if Event A is 5/25 or 1/5, and the probability of Event B is 6/25. The probability of Event A or Event B would be the total number of outcomes in the orange area divided by the total number of possible outcomes. The probability of Event A or Event B then is 9/25. Note that this result is not simply the sum of the probabilities of each event, which would be equal to 11/25. Since there are two outcomes in the overlapping area, these outcomes are counted twice if we simply sum the

probabilities of the two events. To prevent this double counting of the outcomes common to both events, we need to subtract the probability of those two outcomes so that they are counted only once. The result is the law of addition, which states that the probability of Event A or Event B (or both) occurring is given by: P(A or B) = P(A) + P(B) - P(A and B) This addition rule is useful for determining the probability that at least one event will occur. Note that for mutually exclusive events there is no overlap of the two events so: P(A and B) = 0 and the law of addition reduces to: P(A or B) = P(A) + P(B)

Conditional Probability Sometimes it is useful to know the probability that an event will occur given that another event occurred. Given two possible events, if we know that one event occurred we can apply this information in calculating the other event's probability. Consider the Venn diagram of the previous section with the two overlapping circles. If we know that Event B occurred, then the effective sample space is reduced to those outcomes associated with Event B, and the Venn diagram can be simplified as shown:

The probability that Event A also has occurred is the probability of Events A and B relative to the probability of Event B. Assuming equal probability outcomes, given two outcomes in the overlapping area and six outcomes in B, the probability that Event A occurred would be 2/6. More generally, P(A and B) P(A given B) = ---------------------------P(B)

Law of Multiplication The probability of both events occurring can be calculated by rearranging the terms in the expression of conditional probability. Solving for P(A and B), we get: P(A and B) = P(A given B) x P(B) For independent events, the probability of Event A is not affected by the occurance of Event B, so P(A given B) = P(A), and P(A and B) = P(A) x P(B)