Previously…

The Law of Large Numbers

It states that as the number of trials in a random experiment increases, the sample mean approaches the expected value.

Example Bernoulli Trials Simulation

Let \(p=0.60\) be the “success” probability of a Bernoulli r.v. \(X\), where \(\text{E}(X) = p\).

Visualizing the Exponential Distribution

Exponential Distribution

Exponential R.V.

Let \(\lambda=\frac{1}{15}\) be the rate of “success”.

\[ \begin{aligned} \text{R.V. } & \longrightarrow X \sim \text{Exp}\left(\frac{1}{15}\right) \\ \text{PDF } & \longrightarrow f(x) = \frac{1}{15} e^{- \frac{1}{15} x} \\ \text{for } & x \in [0,\infty) \end{aligned} \]

Exponential Probabilities

Exponential Distribution

Example:

What is the probability that “success” happens on 15 unit length or less, given \(\lambda=\frac{1}{15}\)? \[ \begin{aligned} P(X \le 15) & = \int_0^{15} f(x) \ dx \\ & = \int_0^{15} \frac{1}{15} e^{-\frac{1}{15} x} \ dx \\ P(X \le 15) & \approx 0.632 \end{aligned} \]

Using R:

lambda <- 1/15
pexp(15,lambda)

## [1] 0.6321206

\(\star\) Note that the pexp() function computes the probability \(P(X \le x)\), meaning it computes the sum of all probabilities from \(X=0\) to \(X=x\) using the Exponential PDF. The dexp() function computes the density, not probability because \(P(X = x)=0\) at any \(x\).

Exponential Expected Value

Exponential Distribution with Expected Value

Exponential R.V.

Let \(\lambda=\frac{1}{15}\) be the rate of “success”.

In general, the expected value of the exponential r.v. is given by \[\text{E}(X) = \frac{1}{\lambda},\] which is the reciprocal of the “success” rate.

Simulating the Exponential Distribution

Random Sampling from the Exponential Distribution

Sample Mean vs the Expected Value

The sample mean of \(0.88\) is not exactly equal to the expected value of \(1\) due to sampling variability. As we increase the number of samples, the sample mean gets closer to the expectation.

Geometric Random Sampling using R

N <- 100 # number of simulations
mean <- 15
lambda <- 1/15 # set rate
rexp(N,lambda)

Binomial R.V. (Revisited)

Binomial Distribution

Binomial R.V.

Let \(p=0.50\) be the success probability and \(n=10\) the number of trials.

\[ \begin{aligned} \text{R.V. } & \longrightarrow X \sim \text{Binom}(n,p) \\ \text{PMF } & \longrightarrow P(X=k) = \binom{n}{k} p^k (1-p)^{n-k} \\ \text{for } & k = 0,1,2,3, \cdots, n \\ \text{expected value} & \longrightarrow \text{E}(X) \approx 5 \end{aligned} \]

Flipping \(\mathbf{n}\) Coins: Possible Outcomes

Suppose we conduct an experiment of flipping \(n\) fair coins in a sequence, where \(n\) is an integer. The sample space \(S\) contains all possible sequences of \(H\) and \(T\). Number of possible outcomes is \(|S| = 2^n\).

Visualizing the possible outcomes using Pascal’s triangle

\(\star\) Key Idea: Pascal’s Triangle helps us visualize the total possible sequences of “success” (\(H\)) outcomes given \(n\) independent trials.

Flipping \(\mathbf{n}\) Coins: Counting the Number of \(H\) outcomes

Let \(X\) be the r.v. that counts the number of \(H\) outcomes in \(n\) trials.

Pascal’s triangle helps us count

\(\dagger\) Can you determine the ways \(H\) can occur in \(4\) trials using Pascal’s triangle?

Flipping \(\mathbf{n}\) Coins: Probability of Observing \(H\) outcomes in \(n\) Trials

Compute the probability of observing a certain number of “success” (\(H\)) outcomes in \(n\) trials.

\(\dagger\) Can you determine the probabilities of observing \(H\) outcomes in \(4\) trials?

Flipping \(\mathbf{n}\) Coins: The Expected Number of \(H\) outcomes in \(n\) Trials

How many \(H\) outcomes do we expect to have in \(n\) independent Bernoulli trials with “success” probability \(p\)?

Example

Each fair coin flip is an independent Bernoulli trial with:
- \(P(H) = \frac{1}{2}\)
- \(P(T) = 1 - \frac{1}{2}\)
Suppose that \(n=2\).
- We expect to have \(2 \times \frac{1}{2} = 1\) \(H\) outcome in \(2\) trials.

In general:

The expected value of “success” outcomes is \(\text{E}(X) = n \times p\).
The binomial coefficient tells you how many ways \(k\) “success” outcomes can occur in \(n\) trials, corresponding to the \(k\)-th column in the \(n\)-th row of Pascal’s Triangle.

\(\star\) Key Idea: Over many repetitions, the long-run average number of “success” outcomes is \(n \times p\), reflecting the frequentist interpretation of probability.

Normal Approximation to the Binomial

\(\star\) Key Idea: The Binomial distribution is approximately the normal distribution given large enough samples because of the Law of Large Numbers.

The Normal R.V.

A normal r.v. is a type of continuous r.v. whose probability distribution follows the normal distribution, also known as the Gaussian distribution. The normal distribution is characterized by two parameters, \(\mu\) as the mean and \(\sigma^2\) as the variance: \[X \sim \text{N}(\mu,\sigma^2)\]

Sample Space:

\(x \in (-\infty,\infty)\) because the normal r.v. can take any value from the entire real number line and it is a continuous random variable.

Parameters

\(\mu\) is the mean (center) or the mode of the distribution.
\(\sigma^2\) measures the spread of the distribution.

The Normal R.V.: PDF

The normal r.v. \(X \sim \text{N}(\mu,\sigma^2)\) has infinite possible outcomes (or infinite sized sample space) where \(\mu\) is the mean and \(\sigma^2\) is the variance with PDF given as \[f(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}, \ -\infty < x < \infty\]

\(\star\) Key Idea: The normal r.v. often approximates the distribution of many types of data, especially when there are large numbers of independent factors contributing to the outcome.

Normal Probabilities

Normal Distribution

Example:

What is \(P(X \le 13)\) for \(X \sim \text{N}(10,2.24)\)? \[ \begin{aligned} P(X \le 13) & = \int_0^{13} f(x) \ dx \\ & = \int_0^{13} \frac{1}{\sqrt{2 \pi (2.24)^2}} e^{-\frac{(x-10)^2}{2(2.24)^2}} \ dx \\ P(X \le 13) & \approx 0.9098 \end{aligned} \]

Using R:

mu <- 10
sd <- 2.24
pnorm(13,mu,sd)

## [1] 0.9097612

\(\star\) Note that the pnorm() function computes the probability \(P(X \le x)\), meaning it computes the area under \(f(x)\) from \(X=0\) to \(X=x\) using the Normal PDF. The dnorm() function computes the density, not probability because \(P(X = x)=0\) at any \(x\).

Normal Expected Value

Normal Distribution with Expected Value

Normal R.V.

Let \(\mu=10\) and \(s=2.24\) be the mean and standard deviation respectively.

\[ \begin{aligned} \text{R.V. } & \longrightarrow X \sim \text{N}\left(10,2.24^2\right) \\ \text{PDF } & \longrightarrow f(x) = \frac{1}{\sqrt{2 \pi (2.24)^2}} e^{-\frac{(x-10)^2}{2(2.24)^2}} \\ \text{for } & x \in (-\infty,\infty) \\ \text{expected value} & \longrightarrow \text{E}(X) = 10 \end{aligned} \]

In general, the expected value of the normal r.v. is given by \[\text{E}(X) = \mu,\] which is the center of the normal distribution.

Simulating the Normal Distribution

Random Sampling from the Exponential Distribution

Sample Mean vs the Expected Value

The sample mean of \(10.07\) is not exactly equal to the expected value of \(10\) due to sampling variability. As we increase the number of samples, the sample mean gets closer to the expectation.

Normal Random Sampling using R

N <- 100 # number of simulations
mu <- 10
sd <- 2.24
rnorm(N,mu,sd)

Normal Approximation to the Binomial (Revisited)

When Can we use it?

The binomial distribution \(X \sim \text{Binom}(n,p)\) can be approximated by a normal distribution when:

\(np\) (expected “success”) and \(np(1-p)\) (expected “fail”) are sufficiently large

Approximation Formula

If \(X \sim \text{Binom}(n,p)\), then for large \(n\), \[X \approx \text{N}(\mu,\sigma^2)\] where \(\mu = np\) and \(\sigma^2 = np(1-p)\).

\(\star\) The normal approximation simplifies binomial probability calculations for large \(n\).

Why Use Normal Approximation for Binomial?

Computational convenience: Calculating binomial probabilities for large \(n\) can be computationally intense because you have to evaluate a large number of terms. The normal approximation simplifies this by allowing you to use the normal distribution.
Simplifies calculations: When \(n\) is large, the binomial distribution can become cumbersome to work with directly. The normal approximation provides a simpler alternative that still gives reasonably accurate results.

Exponential vs Normal Distribution Summary

R.V. \(X\)	Exponential	Normal
Description	Unit length until a “success” event happens	Approximation to the Binomial with sufficiently large number of \(n\) independent trials
Sampling	With replacement	With replacement
Parameters	\(\lambda \longrightarrow\) rate of “success”	\(\mu \longrightarrow\) mean \(\sigma^2 \longrightarrow\) variance
PMF	\(f(x) = f(x) = \lambda e^{- \lambda x}\) \(x \in [0,\infty)\)	\(f(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}\) \(x \in (-\infty,\infty)\)
Expected Value \(\text{E}(X)\)	\(\frac{1}{\lambda}\)	\(\mu\)
\(P(X = x)\)	`dexp(rate)`	`dnorm(mean,sd)`
\(P(X \le x)\)	`pexp(rate)`	`pnorm(mean,sd)`
\(N\) Simulations	`rexp(N,rate)`	`rnorm(N,mean,sd)`

Activity: The Galton Board and the Normal Distribution

Watch this video: https://youtu.be/UCmPmkHqHXk?si=LT1apiUwe0meA8PS.
Get together with another student.
Discuss the video by answering the following question.
- How does the structure of Pascal’s triangle relate to the binomial distribution?
- Why does the shape of the bead distribution on a Galton board become approximately normal as the number of rows increases?
Make sure to write your name in the attendance sheet.

References

Diez, D. M., Barr, C. D., & Çetinkaya-Rundel, M. (2012). OpenIntro statistics (4th ed.). OpenIntro. https://www.openintro.org/book/os/

Speegle, Darrin and Clair, Bryan. (2021). Probability, statistics, and data: A fresh approach using r. Chapman; Hall/CRC. https://probstatsdata.com/

Appendix: The Binomial Coefficient

Pascal’s Triangle and Combinations This formula calculates the number of ways to choose \(k\) elements from a set of \(n\). Each number in Pascal’s Triangle corresponds to a combination. Also known as the binomial coefficient.

Normal and Exponential Distributions

Applied Statistics

Objectives