Skip to main content

Probability Axioms

In the stochastic truth table below, you can change the probabilities assigned to each row. Recall that the sum of the numbers assigned to each row must be 1 and each number must be greater than or equal to 0. The probabilities of the formulas are updated when the stochastic truth table changes. When you hover over a formula with your mouse, the rows of the truth table where that formula is true is highlighted. As you change the probabilities assigned to each row, answer the following questions:

  • What is the minimum and maximum value of a probability assigned to a formula?
  • What is the relationship between Pr(¬P∨(P∧Q))Pr(\neg P\vee (P\wedge Q)) and the sum Pr(¬P)+Pr(P∧Q)Pr(\neg P) + Pr(P\wedge Q)?
  • What is the relationship between Pr(P),Pr(Q)Pr(P), Pr(Q) and Pr(P∨Q)Pr(P \vee Q)?
  • What is the relationship between Pr(P)Pr(P) and Pr(¬P)Pr(\neg P) (and between Pr(Q)Pr(Q) and Pr(¬Q)Pr(\neg Q))?
PPQQ
T\mathsf{T}T\mathsf{T}
T\mathsf{T}F\mathsf{F}
F\mathsf{F}T\mathsf{T}
F\mathsf{F}F\mathsf{F}
Pr(P)Pr(P)  = \ =\ 0.5000Pr(Q)Pr(Q)  = \ =\ 0.5000
Pr(¬P)Pr(\neg P)  = \ =\ 0.5000Pr(¬Q)Pr(\neg Q)  = \ =\ 0.5000
Pr(P∧Q)Pr(P \wedge Q)  = \ =\ 0.2500
Pr(¬P∨(P∧Q))Pr(\neg P \vee (P\wedge Q))  = \ =\ 0.7500
Pr(P∨Q)Pr(P \vee Q)  = \ =\ 0.7500

Returning to the first two questions asked above:

  • What is the minimum and maximum value of a probability assigned to a formula? In any stochastic truth table, the probabilities assigned to a formula must be greater than or equal to 0 and less than or equal to 1. This means that in any stochastic truth table, for any formula XX, if XX is a tautology (so, XX is true in every row), then Pr(X)=1Pr(X)=1.

  • What is the relationship between Pr(¬P∨(P∧Q))Pr(\neg P\vee (P\wedge Q)) and the sum Pr(¬P)+Pr(P∧Q)Pr(\neg P) + Pr(P\wedge Q)? Note that ¬P\neg P and P∧QP\wedge Q are mutually exclusive: There is no row of the truth table that makes both formulas true. That is, ¬P\neg P is true in rows 3 and 4, while P∧QP\wedge Q is true in row 1. Since the disjunction of these two formulas ¬P∨(P∧Q)\neg P\vee (P\wedge Q) is true in rows 1, 3, and 4, in any stochastic truth table, Pr(¬P∨(P∧Q))=Pr(¬P)+Pr(P∧Q)Pr(\neg P\vee (P\wedge Q)) = Pr(\neg P) + Pr(P\wedge Q).

These two observations motivate the following three core principles of probability, which are known as the Kolmogorov Axioms (for a further discussion, see Section 1 of Interpretations of Probability, Stanford Encyclopedia of Philosophy by Alan Hajek).

Kolmogorov Axioms

Given any stochastic truth table, the following is true:

  1. For all XX, Pr(X)≥0Pr(X)\ge 0.
  2. For all XX, if XX is a tautology, then Pr(X)=1Pr(X) = 1.
  3. For all XX and YY, if XX and YY are mutually exclusive, then Pr(X∨Y)=Pr(X)+Pr(Y)Pr(X\vee Y) = Pr(X)+ Pr(Y).

Since XX and ¬X\neg X are contradictory and hence mutually exclusive, the Kolmogorov axiom 3 implies that

Pr(X∨¬X)=Pr(X)+Pr(¬X).Pr(X\vee\neg X)=Pr(X) + Pr(\neg X).

Furthermore, since X∨¬XX\vee\neg X is a tautology, Kolmogorov axiom 3 implies that Pr(X∨¬X)=1Pr(X\vee\neg X)=1. Putting these two equations together, we have 1=Pr(X)+Pr(¬X)1=Pr(X) + Pr(\neg X). This justifies the following:

Complement Law

Given any stochastic truth table, for all formulas XX,

Pr(¬X)=1−Pr(X).Pr(\neg X)=1-Pr(X).

Another important property is that equivalent formulas are always assigned the same probability. If XX and YY a tautologically equivalent (that is, in every row of the truth table XX and YY have the same truth vale), then the probabilities assigned to XX must be the same as the probability assigned to YY. For instance, since XX is equivalent to (X∧Y)∨(X∧¬Y)(X\wedge Y)\vee (X\wedge\neg Y), they must have the same probability. These two observations are gathered below:

Observation

Given any stochastic truth table, for all formulas XX and YY,

  1. if XX and YY are tautologically equivalent (i.e., X↔YX\leftrightarrow Y is a tautology), then Pr(X)=Pr(Y)Pr(X)=Pr(Y); and
  2. Pr(X)=Pr((X∧Y)∨(X∧¬Y))Pr(X)=Pr((X\wedge Y)\vee (X\wedge\neg Y)).

Using the observations above, we can derive the following further properties of probability.

Observation

Given any stochastic truth table, the following are true:

  1. For all formulas XX, if XX is a contradiction, then Pr(X)=0Pr(X)=0.
  2. For all formulas XX and YY, Pr(X)=Pr(X∧Y)+Pr(X∧¬Y)Pr(X)=Pr(X\wedge Y) + Pr(X\wedge\neg Y).
  3. For all formulas XX and YY,
    if X→YX\rightarrow Y is a tautology, then Pr(X)≤Pr(Y)Pr(X)\le Pr(Y).
  4. For all formulas XX and YY,
    Pr(X∨Y)=Pr(X)+Pr(Y)−Pr(X∧Y)Pr(X\vee Y)= Pr(X) + Pr(Y) - Pr(X\wedge Y)

Principles of Conditional Probability#

As in the example above, in the stochastic truth table below, you can change the probabilities assigned to each row. The conditional probabilities of the formulas are updated when the stochastic truth table changes. As you change the probabilities assigned to each row, answer the following questions:

  • What is the relationship between Pr(P∧Q)Pr(P\wedge Q), Pr(Q)Pr(Q) and Pr(P∣Q)Pr(P\mid Q)?
  • Can you find a stochastic truth table such that Pr(P)≠Pr(Q)Pr(P∣Q)+Pr(¬Q)Pr(P∣¬Q)Pr(P)\ne Pr(Q) Pr(P\mid Q) + Pr(\neg Q) Pr(P\mid \neg Q)?
PPQQ
T\mathsf{T}T\mathsf{T}
T\mathsf{T}F\mathsf{F}
F\mathsf{F}T\mathsf{T}
F\mathsf{F}F\mathsf{F}
Pr(P)Pr(P)  = \ =\ 0.5000Pr(Q)Pr(Q)  = \ =\ 0.5000
Pr(¬P)Pr(\neg P)  = \ =\ 0.5000Pr(¬Q)Pr(\neg Q)  = \ =\ 0.5000
Pr(P∣Q)Pr(P \mid Q) = \ =\ Pr(P∧Q)Pr(P\wedge Q) // Pr(Q)Pr(Q)  = \ =\ 0.2500 / 0.5000 = \ =\ 0.5000
Pr(P∣¬Q)Pr(P \mid \neg Q)  = \ =\ Pr(P∧¬Q)Pr(P\wedge \neg Q) // Pr(¬Q)Pr(\neg Q)  = \ =\ 0.2500 / 0.5000 = \ =\ 0.5000
Pr(Q)Pr(P∣Q)+Pr(¬Q)Pr(P∣¬Q)Pr(Q)Pr(P\mid Q) + Pr(\neg Q)Pr(P \mid \neg Q)  = \ =\
0.5000 ×\times 0.5000 ++ 0.5000 ×\times 0.5000  = \ =\ 0.5000

The following observation follows from the definition of conditional probability:

Observation

Given any stochastic truth table, the following are true:

  1. For all formulas XX and YY,
    Pr(X∧Y)=Pr(Y)×Pr(X∣Y)Pr(X\wedge Y)=Pr(Y)\times Pr(X\mid Y).
  2. For all formulas XX, YY and ZZ,
    Pr(X∧Y∣Z)=Pr(Y∣Z)×Pr(X∣Y∧Z)Pr(X\wedge Y\mid Z)=Pr(Y\mid Z)\times Pr(X\mid Y\wedge Z).

The next observation follows from the following two facts:

  1. Pr(X)=Pr(X∧Y)+Pr(X∧¬Y)Pr(X) = Pr(X\wedge Y) + Pr(X\wedge\neg Y); and
  2. Pr(Y)Pr(X∣Y)+Pr(¬Y)Pr(X∣¬Y)=Pr(Y) Pr(X\mid Y) + Pr(\neg Y) Pr(X\mid \neg Y) =
    Pr(Y)Pr(X∧Y)Pr(Y)+Pr(¬Y)Pr(X∧¬Y)Pr(¬Y)=Pr(Y)\frac{Pr(X\wedge Y)}{Pr(Y)} + Pr(\neg Y)\frac{Pr(X\wedge\neg Y)}{Pr(\neg Y)} =
    Pr(X∧Y)+Pr(X∧¬Y)Pr(X\wedge Y)+Pr(X\wedge\neg Y)
Law of Total Probability

Given any stochastic truth table, for all formulas XX and YY,
Pr(X)=Pr(Y)Pr(X∣Y)+Pr(¬Y)Pr(X∣¬Y)Pr(X) = Pr(Y) Pr(X\mid Y) + Pr(\neg Y) Pr(X\mid \neg Y).

Practice Questions#

  1. True or False: For any formulas XX and YY, for any stochastic truth table, Pr(X ∣ Y)=Pr(Y ∣ X)Pr(X\ |\ Y) = Pr(Y\ |\ X)

  2. True or False: For any formulas XX and YY, in any stochastic truth table, if Pr(X↔Y)=1Pr(X\leftrightarrow Y)=1, then Pr(X)=Pr(Y)Pr(X)=Pr(Y).

  3. True or False: For any formulas XX and YY, in any stochastic truth table, if X→YX\rightarrow Y is a tautology, then Pr(Y∣X)=1Pr(Y\mid X)=1.

  1. Suppose that Pr(P∧B)=0.75Pr(P\wedge B)=0.75, Pr(P∧¬Q)=0.1Pr(P\wedge \neg Q)= 0.1, find Pr(P)Pr(P).
  1. Suppose that Pr(P)=0.35Pr(P)=0.35, Pr(Q∣P)=0.7Pr(Q\mid P)=0.7, Pr(Q∣¬P)=0.1Pr(Q\mid\neg P)= 0.1, find Pr(Q)Pr(Q).
  1. Suppose that Pr(Q)=0.85Pr(Q)=0.85, Pr(P∣Q)=0.2Pr(P\mid Q)=0.2, Pr(P∣¬Q)=0.3Pr(P\mid\neg Q)= 0.3, find Pr(P)Pr(P).