MTH-361A | Spring 2025 | University of Portland
February 12, 2025
Random Variables
A random variable (r.v.) is a numerical outcome of a random experiment. It assigns a number to each possible outcome in a sample space.
In other words, a random variable is a function that maps the sample space into real numbers.
Types:
A Bernoulli r.v. represents a single experiment with two possible outcomes: “success” (\(X=1\)) with probability \(p\) and failure (\(X=0\)) with probability \(1-p\). We denote this r.v. and its PMF as \[ \begin{aligned} \text{R.V. } & \longrightarrow X \sim \text{Bern}(p) \\ \text{PMF } & \longrightarrow P(X = x) = p^x (1-p)^{1-x}, \ \ x \in \{0,1\} \end{aligned}. \]
We have shown that the expected value of \(X\) is \(\text{E}(X) = p\) and the variance of \(X\) is \(\text{Var}(X) = p(1-p)\) using the definition of expected value and variance respectively.
The sampling with replacement is a sampling method where each selected item is returned to the population before the next selection.
Key Characteristics:
Example:
The sampling without replacement is a sampling method where each selected item is not returned to the population before the next selection.
Key Characteristics:
Example:
A sequence of random variables \(X_1, X_2, \cdots, X_n\) is independent and identically distributed (i.i.d) if:
Examples:
Why is i.i.d. Important?
A sequence of multiple Bernoulli trials consists i.i.d. Bernoulli random variables where each follows a Bernoulli distribution with “success” probability \(p\).
\(\star\) Key Assumption: Trials are independent (i.e., one outcome does not affect the next).
A geometric R.V. is a discrete random variable that represents the number of Bernoulli trials until the first “success” where each trial is independent, with a constant “success” probability \(p\): \[X \sim \text{Geom}(p)\]
Sample Space:
\[ \begin{aligned} 1 & \longrightarrow 0 \text{ fail until success} \\ 0,1 & \longrightarrow 1 \text{ fail until success} \\ 0,0,1 & \longrightarrow 2 \text{ fail until success} \\ & \vdots \\ 0,0,0,\cdots,1 & \longrightarrow k \text{ fail until success} \\ \end{aligned} \]
Probabilities:
\[ \begin{aligned} 1 & \longrightarrow (1-p)^0 p \\ 0,1 & \longrightarrow (1-p)^1 p \\ 0,0,1 & \longrightarrow (1-p)^2 p \\ & \vdots \\ 0,0,0,\cdots,1 & \longrightarrow (1-p)^k p \\ \end{aligned} \]
\(\star\) Key Idea: The geometric random variable counts the number of “failures” before a “success” and can also be viewed as counting the number of trials including the first “success.”
The geometric r.v. \(X \sim \text{Geom}(p)\) has infinite possible outcomes (or infinite sized sample space) where \(p\) is the “success” probability.
The PMF of the geometric r.v. can be written in two ways:
\(\star\) Key Idea: The geometric random variable models a situation where samples are taken with replacement, and the number of failures until the first success is counted.
What is the probability of “success” on the 6th trial with \(p=0.50\)? \[P(X=5) = (1-0.50)^5 (0.50) \approx 0.016\] because there are 5 “failures” before the 6th trial.
What is the probability that the first “success” occurs before the 6th trial, given \(p=0.50\)? \[ \begin{aligned} P(X \le 5) & = \sum_{i=0}^5 P(X = i) \\ & = \sum_{i=0}^5 (1-0.50)^{i} (0.50) \\ P(X \le 5) & \approx 0.984 \\ \end{aligned} \] because we need to count five or fewer “failures” before a “success” occurs.
A binomial R.V. is a discrete random variable representing the number of “success” in \(n\) independent Bernoulli trials, each with “success” probability \(p\): \[X \sim \text{Binom}(n,p)\]
Sample Space:
Suppose \(n = 3\).
\[ \begin{aligned} 0,0,0 & \longrightarrow 3 \text{ fail and } 0 \text{ success} \\ 0,0,1 & \longrightarrow 2 \text{ fail and } 1 \text{ success} \\ 0,1,0 & \longrightarrow 2 \text{ fail and } 1 \text{ success} \\ 0,1,1 & \longrightarrow 1 \text{ fail and } 2 \text{ success} \\ 1,0,0 & \longrightarrow 2 \text{ fail and } 1 \text{ success} \\ 1,0,1 & \longrightarrow 1 \text{ fail and } 2 \text{ success} \\ 1,1,0 & \longrightarrow 1 \text{ fail and } 2 \text{ success} \\ 1,1,1 & \longrightarrow 0 \text{ fail and } 3 \text{ success} \\ \end{aligned} \]
Probabilities:
Suppose \(n = 3\). \[ \begin{aligned} 0,0,0 & \longrightarrow (1-p)^3 p^0 \\ 0,0,1 & \longrightarrow (1-p)^2 p^1 \\ 0,1,0 & \longrightarrow (1-p)^2 p^1 \\ 1,0,0 & \longrightarrow (1-p)^2 p^1 \\ 0,1,1 & \longrightarrow (1-p)^1 p^2 \\ 1,0,1 & \longrightarrow (1-p)^1 p^2 \\ 1,1,0 & \longrightarrow (1-p)^1 p^2 \\ 1,1,1 & \longrightarrow (1-p)^0 p^3 \end{aligned} \]
\(\star\) Key Idea: The binomial random variable counts the number of “successes” in \(n\) independent Bernoulli trials, where each trial has a “success” probability \(p\).
Permutations
An arrangement of objects in a specific order. For \(n\) objects, we pick \(k\) objects to permute with number of permutations given by \[_n P_k = \frac{n!}{(n-k)!}.\]
Combinations
A selection of objects where order does not matter. For \(n\) objects, we pick \(k\) objects to combine with number of combinations given by \[_n C_k = \binom{n}{k} = \frac{n!}{k!(n-k)!}.\]
The binomial coefficient, denoted as \(\binom{n}{k}\) represents the number of ways to choose \(k\) objects from a set of \(n\) objects without regard to order. It is given by the formula: \[\binom{n}{k} = \frac{n!}{k!(n-k)!}\]
Expanding binomial expressions using the Binomial Theorem: \[(x+y)^n = \sum_{k=0}^n \binom{n}{k} y^k x^{n-k}\]
If we let \(x=1-p\) and \(y=p\) (Bernoulli PMF), then \[(1-p+p)^n = \sum_{k=0}^n \binom{n}{k} p^k (1-p)^{n-k} = 1.\]
\(\star\) Key Idea: Since \(p\) is the “success” probability and the Binomial Theorem reduces to \(1\), then this satisfies the probability axioms.
The binomial r.v. \(X \sim \text{Binom}(n,p)\) has finite possible outcomes with PMF given by \[P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}, \ k = 0,1,2,3, \cdots, n\] where \(p\) is the “success” probability. The term \(\binom{n}{k} = \frac{n!}{k! (n-k)!}\) is the binomial coefficient.
\(\star\) Key Idea: The binomial random variable models a situation where samples are taken with replacement, and the number of successes is counted within a finite number of trials.
What is the probability of getting 4 “success” in 10 trials with \(p=0.50\)? \[P(X=4) =\binom{10}{4} (0.50)^4 (1-0.50)^{n-k} \approx 0.205\]
What is the probability of getting at most 4 “success” in 10 trials with \(p=0.50\)? \[ \begin{aligned} P(X \le 4) & = \sum_{i=0}^4 P(X = i) \\ & = \sum_{i=0}^4 \binom{10}{i} (0.50)^i (1-0.50)^{10-i} \\ P(X \le 4) & \approx 0.377 \\ \end{aligned} \] because we need to count four or fewer “success” in 10 trials.
.pdf file.