Visualizing the Geometric Distribution (1/2)

Geometric Probability Mass Function (PMF)

\(\star\) The geometric r.v. has infinitely sized sample space. The plot of the geometric PMF only shows the first \(11\) possible outcomes.

Geometric R.V. and PMF:

Let \(p\) be the “success” probability.

\(\displaystyle X \sim \text{Geom}\left(p\right)\)
\(\displaystyle P(X=k) = \left(1-p\right)^k p \quad \text{for } k = 0,1,2, \cdots\)

Relevant R Functions:

dgeom \(\leftarrow\) PMF
rgeom \(\leftarrow\) random sampling simulation

Visualizing the Geometric Distribution (2/2)

Geometric Cumulative Distribution Function (CDF)

\(\star\) The CDF, in general, computes the cumulative probability of a given PMF.

Geometric PMF and CDF:

Let \(p\) be the “success” probability.

\(\displaystyle P(X=k) = \left(1-p\right)^k p \quad \text{for } k = 0,1,2, \cdots\)
\(\displaystyle P(X \le k) = \sum_{i=0}^k P(X=i) \quad \text{for } k = 0,1,2, \cdots\)

Relevant R Functions:

pgeom \(\leftarrow\) CDF
qgeom \(\leftarrow\) Inverse CDF

Geometric Probabilities (1/3)

Geometric Distribution

\(\star\) The dgeom() function computes the probability \(P(X = k)\), meaning it computes the probability at exactly \(X=k\) number of “failures” before a “success” using the Geometric PMF.

Example:

What is the probability of “success” on the 4th trial with \(p = \frac{1}{2}\)? \[ \begin{aligned} P(X=3) & = \left(1-\frac{1}{2}\right)^3 \left(\frac{1}{2}\right) \\ P(X=3) & \approx 0.063 \end{aligned} \]

Using R:

dgeom(3,1/2)

## [1] 0.0625

Geometric Probabilities (2/3)

Geometric Distribution

\(\star\) The pgeom() function computes the probability \(P(X \le k)\), meaning it computes the sum of all probabilities from \(X=0\) to \(X=k\) number of failures” before a “success” using the Geometric PMF.

Example:

What is the probability that the first “success” occurs before the 4th trial, given \(p = \frac{1}{2}\)? \[ \begin{aligned} P(X \le 3) & = \sum_{i=0}^3 P(X = i) \\ & = \sum_{i=0}^3 \left(1-\frac{1}{2}\right)^{i} \left(\frac{1}{2}\right) \\ P(X \le 3) & \approx 0.938 \\ \end{aligned} \]

Using R:

pgeom(3,1/2)

## [1] 0.9375

Geometric Probabilities (3/3)

Geometric Distribution

\(\star\) Since \(k\) is the number of “failures” before a “success” occurs, the probability of that “success” occurs at least on the fifth trial is \(P(X \ge 4)\). Then, we use the complement rule.

Example:

What is the probability that “success” occurs at least on the fifth trial, given \(p = \frac{1}{2}\)? \[ \begin{aligned} P(X \ge 4) & = 1 - P(X \le 3) \\ & = 1 - \sum_{i=0}^3 P(X = i) \\ & = 1 - \sum_{i=0}^3 \left(1-\frac{1}{2}\right)^{i} \left(\frac{1}{2}\right) \\ & \approx 1 - 0.938 \\ P(X \ge 4) & \approx 0.063 \\ \end{aligned} \]

Using R:

1-pgeom(3,1/2)

## [1] 0.0625

Geometric R.V. Expected Value

Geometric Distribution with Expected Value:

\(\star\) The law of large numbers still can be used to interpret the expected value, which in this case, you would expect to have \(1\) “failure” before a “success” occurs in the long run.

Geometric R.V.:

Let \(p=\frac{1}{2}\) be the success probability.

\(\displaystyle X \sim \text{Geom}\left(\frac{1}{2}\right)\)
\(\displaystyle P(X=k) = \left(1-\frac{1}{2}\right)^k \left(\frac{1}{2}\right) \quad \text{for } k = 0,1,2, \cdots\)
\(\displaystyle \text{E}(X) = 1\)

In general, \[ \begin{aligned} \text{E}(X) & = \sum_{k=0}^{\infty} k P(X=k) \\ & = \sum_{k=0}^{\infty} k \left(1-p\right)^k p \\ \text{E}(X) & = \frac{1-p}{p} \end{aligned}. \]

Geometric R.V. Variance

Recall that the formula for the variance is \[\text{Var}(X) = \text{E}\left(X^2 \right) - \left( \text{E}(X) \right)^2,\] where:

\(\displaystyle \text{E}(X) = \frac{1-p}{p} \leftarrow \text{ expected value}\)
\(\displaystyle \text{E}\left(X^2\right) = \frac{(1-p)(2-p)}{p^2} \leftarrow \text{ 2nd raw moment}\)

In general, \[ \begin{aligned} \text{Var}(X) & = \text{E}\left(X^2 \right) - \left( \text{E}(X) \right)^2 \\ & = \frac{(1-p)(2-p)}{p^2} - \left( \frac{1-p}{p} \right)^2 \\ \text{Var}(X) & = \frac{1-p}{p^2} \end{aligned} \]

\(\star\) The Geometric r.v. is at its most uncertain if \(p \to 0\), meaning that if the probability of “success” is very low, then expected number of “failures” increases.

Simulating the Geometric Distribution

Random Sampling from the Geometric Distribution:

A simulation of \(100\) random samples using the Geometric PMF with \(p = \frac{1}{2}\).

Sample Mean vs the Expected Value:

The sample mean of \(0.81\) is not exactly equal to the expected value of \(1\) due to sampling variability.

If we increase the number of samples, the sample mean will get closer to the expectation due to the law of large numbers.

Using R:

set.seed(42) # set seed for reproducibility
samples <- rgeom(100,1/2) # generate random samples
mean(samples) # compute sample mean

## [1] 0.81

Visualizing the Binomial Distribution (1/2)

Binomial Probability Mass Function (PMF)

\(\star\) In comparison with the Geometric PMF, the Binomial PMF has a finitely size sample space, which is \(n\), the number of trials.

Binomial R.V. and PMF

Let \(p=\frac{1}{2}\) be the “success” probability and \(n=10\) the number of trials.

\(\displaystyle X \sim \text{Binom}(p)\)
\(\displaystyle P(X=k) = \binom{n}{k} p^k (1-p)^{n-k} \quad \text{for } k = 0,1,2, \cdots, n\)

Relevant R Functions:

dbinom \(\leftarrow\) PMF
rbinom \(\leftarrow\) random sampling simulation

\(\star\) If you set \(n=1\), the Binomial PMF reduces to the Bernoulli PMF.

Visualizing the Binomial Distribution (2/2)

Binomial Cumulative Distribution Function (CDF)

\(\star\) Since the Binomial r.v. has finitely sized sample space, then \(P(X \le n) = 1\).

Binomial PMF and CDF:

Let \(p=\frac{1}{2}\) be the “success” probability and \(n=10\) the number of trials.

\(\displaystyle P(X=k) = \binom{n}{k} p^k (1-p)^{n-k} \quad \text{for } k = 0,1,2, \cdots, n\)
\(\displaystyle P(X \le k) = \sum_{i=0}^k P(X=i) \quad \text{for } k = 0,1,2, \cdots, n\)

Relevant R Functions:

pbinom \(\leftarrow\) CDF
qbinom \(\leftarrow\) Inverse CDF

Binomial Probabilities (1/3)

Binomial Distribution

Example:

What is the probability of getting 4 “success” in 10 trials with \(p=\frac{1}{2}\)? \[ \begin{aligned} P(X=4) & = \binom{10}{4} \left(\frac{1}{2}\right)^4 \left(1-\frac{1}{2}\right)^{10-4} \\ & = 210 \left(\frac{1}{2}\right)^4 \left(1-\frac{1}{2}\right)^{6} \\ P(X=4) & \approx 0.205 \end{aligned} \]

Using R:

dbinom(4,10,1/2)

## [1] 0.2050781

\(\star\) The dbinom() function computes the probability \(P(X = k)\), meaning it computes the probability at exactly \(X=k\) using the Binomial PMF.

Binomial Probabilities (2/3)

Binomial Distribution

\(\star\) The pbinom() function computes the probability \(P(X \le k)\), meaning it computes the sum of all probabilities from \(X=0\) to \(X=k\) using the Binomial PMF.

Example:

What is the probability of getting at most 4 “success” in 10 trials with \(p=\frac{1}{2}\)? \[ \begin{aligned} P(X \le 4) & = \sum_{i=0}^4 P(X = i) \\ & = \sum_{i=0}^4 \binom{10}{k} \left(\frac{1}{2}\right)^i \left(1-\frac{1}{2}\right)^{10-i} \\ P(X \le 4) & \approx 0.377 \\ \end{aligned} \]

Using R:

pbinom(4,10,1/2)

## [1] 0.3769531

Binomial Probabilities (3/3)

Binomial Distribution

\(\star\) Since we want the probability of 4 or more “success” in 10 trials, we need \(P(X \ge 4)\). Then, we use the complement rule.

Example:

What is the probability of getting at least 4 “success” in 10 trials with \(p=\frac{1}{2}\)? \[ \begin{aligned} P(X \ge 4) & = 1 - P(X \le 3) \\ & = 1 - \sum_{i=0}^3 P(X = i) \\ & = 1 - \sum_{i=0}^3 \binom{10}{i} \left(\frac{1}{2}\right)^i \left(1-\frac{1}{2}\right)^{10-i} \\ & \approx 1 - 0.172 \\ P(X \ge 4) & \approx 0.828 \\ \end{aligned} \]

Using R:

1-pbinom(3,10,1/2)

## [1] 0.828125

Binomial R.V. Expected Value

Binomial Distribution with Expected Value:

\(\star\) The law of large numbers still can be used to interpret the expected value, which in this case, you would expect to \(5\) “successes” out of \(10\) trials in the long run.

Binomial R.V.:

Let \(p=\frac{1}{2}\) be the success probability and \(n=10\) the number of trials.

\(\displaystyle X \sim \text{Binom}\left(\frac{1}{2}\right)\)
\(\displaystyle P(X=k) = \binom{10}{k} \left(\frac{1}{2}\right)^k \left(1-\frac{1}{2}\right)^{10-k} \quad \text{for } k = 0,1,2, \cdots, 10\)
\(\displaystyle \text{E}(X) = 5\)

In general, \[ \begin{aligned} \text{E}(X) & = \sum_{k=0}^{n} k P(X=k) \\ & = \sum_{k=0}^{n} k \binom{n}{k} p^k \left(1-p\right)^{n-k} \\ \text{E}(X) & = np \end{aligned}. \]

Binomial R.V. Variance

Recall that the formula for the variance is \[\text{Var}(X) = \text{E}\left(X^2 \right) - \left( \text{E}(X) \right)^2,\] where:

\(\displaystyle \text{E}(X) = np \leftarrow \text{ expected value}\)
\(\displaystyle \text{E}\left(X^2\right) = (np)^2 + np(1-p) \leftarrow \text{ 2nd raw moment}\)

In general, \[ \begin{aligned} \text{Var}(X) & = \text{E}\left(X^2 \right) - \left( \text{E}(X) \right)^2 \\ & = (np)^2 + np(1-p) - (np)^2 \\ \text{Var}(X) & = np(1-p) \end{aligned} \]

\(\star\) The variance of the Binomial r.v. is similar to the variance of the Bernoulli r.v., where the r.v. is at its most uncertain is when \(p = \frac{1}{2}\).

Simulating the Binomial Distribution

Random Sampling from the Geometric Distribution:

A simulation of \(100\) random samples using the Binomial PMF with \(p = \frac{1}{2}\) and \(n = 10\).

Sample Mean vs the Expected Value

The sample mean of \(5.063\) is not exactly equal to the expected value of \(5\) due to sampling variability.

If we increase the number of samples, the sample mean will get closer to the expectation due to the law of large numbers.

Using R:

set.seed(42) # set seed for reproducibility
samples <- rbinom(100,10,1/2) # generate random samples
mean(samples) # compute sample mean

## [1] 5.07

Summary of Discrete R.V.s and their Properties

	Bernoulli	Geometric	Binomial
Description	“success” or “failure” outcome in 1 independent trial	Number of “failure” independent trials before a “success”	Number of “success” in \(n\) independent trials
Parameters	\(p \leftarrow\) probability of “success”	\(p \leftarrow\) probability of “success”	\(n \leftarrow\) number of trials \(p \leftarrow\) probability of “success”
\(\displaystyle X\)	\(\displaystyle \text{Bern}(p)\)	\(\displaystyle \text{Geom}(p)\)	\(\displaystyle \text{Binom}(n,p)\)
\(\displaystyle \text{E}(X)\)	\(\displaystyle p\)	\(\displaystyle \frac{1-p}{p}\)	\(\displaystyle np\)
\(\displaystyle \text{Var}(X)\)	\(\displaystyle p(1-p)\)	\(\displaystyle \frac{1-p}{p^2}\)	\(\displaystyle np(1-p)\)

Summary of Discrete R.V.s and their PMFs

	Bernoulli	Geometric	Binomial
\(\displaystyle P(X = k)\) PMF	\(\displaystyle p^k (1-p)^{1-k}\) \(\text{for } k = 0,1\)	\(\displaystyle (1-p)^k p\) \(\text{for } k=0,1,2,\cdots\)	\(\displaystyle \binom{n}{k} p^k (1-p)^{n-k}\) \(\text{for } k = 0,1,2, \cdots, n\)
R PMF	`dbinom`	`dgeom`	`dbinom`
R CDF	`pbinom`	`pgeom`	`pbinom`
R Inverse CDF	`qbinom`	`qgeom`	`qbinom`
R Simulations	`rbinom`	`rgeom`	`rbinom`

\(\star\) The Binomial r.v. reduces to the Bernoulli r.v. if \(n=1\).

Geometric and Binomial Distributions

Applied Statistics

Objectives

Visualizing the Geometric Distribution (1/2)

Visualizing the Geometric Distribution (2/2)

Geometric Probabilities (1/3)

Geometric Probabilities (2/3)

Geometric Probabilities (3/3)

Geometric R.V. Expected Value

Geometric R.V. Variance

Simulating the Geometric Distribution

Visualizing the Binomial Distribution (1/2)

Visualizing the Binomial Distribution (2/2)

Binomial Probabilities (1/3)

Binomial Probabilities (2/3)

Binomial Probabilities (3/3)

Binomial R.V. Expected Value

Binomial R.V. Variance

Simulating the Binomial Distribution

Summary of Discrete R.V.s and their Properties

Summary of Discrete R.V.s and their PMFs