• Introduce conditional probability
• Develop an understanding of dependent events
• Know how to compute conditional probabilities
• Introduce Bayes’ theorem and how it relates to classical statistics

Understanding Associations:

• Conditional probability shows how one variable changes given another.
• Forms the conceptual foundation of linear regression.

Basis for Statistical Inference:

• Helps compute probabilities of outcomes conditional on observed data.
• Supports hypothesis testing, confidence intervals, and predictive modeling.

Improves Decision-Making:

• Guides predictions by updating beliefs with observed information.
• Applied in health, biology, and economics to make informed, data-driven decisions.

Clarifies Intuition About Data

• Prevents misinterpretation of correlations and dependencies.
• Essential for correctly interpreting regression coefficients and p-values.

Characteristics:

• The population remains unchanged after each selection.
• Each item can be chosen more than once.
• Items are independently chosen with the same probability.

Characteristics:

• The population changes after each selection.
• Each item can be chosen only once.
• Items are dependently chosen with changing probability.

With Replacement:

• After drawing a card, you put it back into the deck.
• The deck resets to \(52\) cards every time.
• Each draw is independent.

Example:

• Probability of drawing an Ace on each draw: \[P(\text{Ace}) = \frac{4}{52} \text{ or } \frac{1}{13}\] because there are four Aces in the deck out of 52 cards.
• Same probability at every draw.

Without Replacement:

• After drawing a card, you do not put it back.
• The number of cards from the deck decreases each time you draw.
• Each draw is dependent, except for the first draw.

Example:

• 1st draw: \(P(\text{Ace}_1) = \frac{4}{52}\)
• 2nd draw: \(P(\text{Ace}_2 \text{ given } \text{Ace}_1) = \frac{3}{51}\)

Multiplication Rule:

• What is the probability that the 1st draw is an Ace and the 2nd draw is another Ace?

\[ \begin{aligned} P(\text{Ace}_1 \text{ and Ace}_2) & = P(\text{Ace}_1) P(\text{Ace}_2 \text{ given } \text{Ace}_1) \\ & = \left( \frac{4}{52} \right) \left( \frac{3}{51} \right) \\ & = \frac{1}{221} \\ P(\text{Ace}_1 \text{ and Ace}_2) & \approx 0.0045 \end{aligned} \]

Why Conditional Probability?

• Sometimes probabilities change when we have new or extra information.
• That’s true in life where change is always happening.

What is Conditional Probability?

• The probability of an event \(A\), given that event \(B\) has already occurred: \[P(A | B)\] which read as “Probability of \(A\) given \(B\)”.

Definition:

• Given events \(A\) and \(B\) in the sample space \(S\), \[P(A | B) = \frac{P(A \cap B)}{P(B)}\] where:

  • \(P(A \cap B)\) is the probability that both \(A\) and \(B\) occur and
  • \(P(B) > 0\) is the probability that \(B\) occurs (event \(B\) must be possible).

Independence: Given events \(A\) and \(B\) with non-zero probability in the sample space \(S\), the following are equivalent:

• \(A\) and \(B\) are independent
• \(P(A \cap B) = P(A)P(B)\)
• \(P(A|B) = P(A)\)
• \(P(B|A) = P(B)\)

Dependence: Given events \(A\) and \(B\), then by definition:

• \(P(A \cap B) = P(B)P(A|B)\) if \(A\) is dependent on \(B\) or
• \(P(B \cap A) = P(A)P(B|A)\) if \(B\) is dependent on \(A\).

Not Always Symmetric:

\[P(A|B) \ne P(B|A)\]

Set-up:

• Suppose there are two jars \(J_1\) and \(J_2\) continaing colored balls.

• Assumptions:

  • Each jar is equally likely to get chosen.
  • Balls are identical for each color.
  • Each ball is equally likely to get drawn.

• Jar contents as frequencies:

Based on the given information:

• Jar probabilities:

  • \(\displaystyle P(J_1) = \frac{1}{2}\)
  • \(\displaystyle P(J_2) = \frac{2}{2}\)

• Jar contents probabilities:

• Jar contents as column proportions (relative frequencies):

Scenario:

• Randomly choose a Jar.
• Then, randomly draw one ball.
• The goal is to determine the probability of a desired outcome of the drawn ball.

Probability tree:

Scenario:

• Randomly choose a Jar.
• Then, randomly draw one ball.
• Given the drawn ball, the goal is to determine the probability on what jar it came from.

Probability tree in reverse:

Scenario:

• Randomly choose a Jar.
• Then, randomly draw two balls.
• The goal is to determine the probability of a desired outcome of the drawn balls.

Probability tree:

Scenario:

• Randomly choose a Jar.
• Then, randomly draw two balls.
• Given the drawn balls, the goal is to determine the probability on what jar they came from.

Probability tree in reverse:

Scenario:

A person takes a test for a rare disease. How do we know the probability they actually have the disease given a positive test result?

• Let \(D\) be the event that the person actually have the disease.
• Let \(+\) be the event that the person tested positive of the disease.

Information given:

• Disease prevalence: \(P(D) = 0.01\)
• True positive rate of the test: \(P(+|D) = 0.99\)
• False positive rate of the test: \(P(+|D^c) = 0.05\)

Bayes’ Rule:

\[ \begin{aligned} P(D|+) & = \frac{P(+|D)P(D)}{P(+)} \\ & = \frac{P(+|D)P(D)}{P(+|D)P(D) + P(+|D^c)P(D^c)} \\ & = \frac{(0.99)(0.01)}{(0.99)(0.01) + (0.05)(1-0.01)} \\ & = \frac{0.0099}{0.0594} \\ P(D|+) & \approx 0.167 \end{aligned} \]

Interpretation:

• Even with a highly accurate test, the probability of actually having the disease given a positive result can be low if the disease is rare.

Hypothesis statements and data:

Consider hypothesis statements as events and data as an observed evidence.

• Let \(H\) be a hypothesis.
• Let \(E\) be a an observed evidence.

According to Bayes’ Rule,

\[ \begin{aligned} P(H|E) & = \frac{P(E|H)P(H)}{P(E)} \\ \text{or} & \\ \text{posterior} & = \frac{\text{likelihood} \times \text{prior}}{\text{marginal}}. \end{aligned} \]

Terms explained:

• Posterior: The probability of your hypothesis given the observed evidence. Your updated degree of belief.
• likelihood: The probability of the observed evidence given the hypothesis is true. This describes how well the evidence fits the hypothesis.
• Prior: The probability of your hypothesis before observing the evidence. Your initial hunch or the baseline probability.
• Marginal: The probability of the observed evidence under all possible hypotheses describes how common or rare the observation actually is.

Frequentist Perspective:

• Probability is interpreted as the long-run relative frequency of events.
• Population parameters are fixed but unknown.
• Data is random and it comes from repeated sampling.

In inference:

• It assumes the null hypothesis \(H_0\) to be true.
• Then, it computes the p-value, which is interpreted as the probability of observing data as extreme as what we saw, given \(H_0\) is true.
• The decision is either reject or fail to reject \(H_0\) according to some subjective threshold.

Bayesian Perspective:

• Probability is interpreted as the degree of belief updated with evidence.
• Population parameters are random variables.
• It combines initial belief, observed evidence, and updated belief.

In inference:

• It assumes a prior belief or an initial belief about a hypothesis.
• Then, it calculates the likelihood on how well the observation supports that initial hypothesis and determining how common or rare is the observation overall.
• It makes decisions based on the updated belief after having the observed evidence.