Probability vs Statistics

Probability Terminologies

[PSDR] Definition 2.1.

Terminology for statistical experiments:

• An experiment is a process that produces an observation.
• An outcome is a possible observation.
• The set of all possible outcomes is called the sample space.
• An event is a subset of the sample space.
• A trial is a single running of an experiment.

Set Theory Basics

[PSDR] Definition 2.5

Let \(A\) and \(B\) be events in a sample space \(S\).

1. \(A \cap B\) is the set of outcomes that are in both \(A\) and \(B\).
2. \(A \cup B\) is the set of outcomes that are in either \(A\) or \(B\) (or both).
3. \(A − B\) is the set of outcomes that are in \(A\) and not in \(B\).
4. The complement of \(A\) is \(\bar{A} = S - A\). So, \(\overline{A}\) is the set of all outcomes that are not in \(A\).
5. The symbol \(\emptyset\) is the empty set, the set with no outcomes.
6. \(A\) and \(B\) are disjoint if \(A \cap B = \emptyset\).
7. \(A\) is a subset of \(B\), written \(A \subset B\), if every element of \(A\) is also an element of \(B\).

Probability Axioms Part 1

[PSDR] Definition 2.7

Let \(S\) be a sample space. A valid probability satisfies the following probability axioms:

1. Probabilities are non-negative real numbers. That is, for all events \(E\), \(P(E) > 0\).
2. The probability of the sample space is \(1\), \(P(S) = 1\).
3. Probabilities are countably additive: If \(A_1, A_2, \cdots\) are pairwise disjoint, then \[P\left( \bigcup_{n=1}^{\infty} \right) = \sum_{n=1}^{\infty} P(A_n)\]

Probability Axioms Part 2

[PSDR] Theorem 2.1

Let \(A\) and \(B\) be events in the sample space \(S\).

1. \(P(\emptyset) = 0\).
2. If \(A\) and \(B\) are disjoint, then P(A B) = P(A) + P(B).
3. If \(A \subset B\), then \(P(A) \le P(B)\).
4. \(0 \le P(A) \le 1\)
5. \(P(A) = 1 - P(\overline{A})\)
6. \(P(A - B) = P(A) - P(A \cap B)\)
7. \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\)

Independence

[PSDR] Definition 2.20

Two events are said to be independent if knowledge that one event occurs does not give any probabilistic information as to whether the other event occurs. Formally, we say that \(A\) and \(B\) are independent if \(P(A \cap B) = P(A)P(B)\).

Events \(A\) and \(B\) are said to be dependent if they are not independent.

Mini-Activity

Mini-Assignment: Probability Theory Basics Part 1

Back to Tentative Topics Schedule.