Objectives

Previously… (1/3)

The guiding principle of statistics is statistical thinking.

Statistical Thinking in the Data Science Life Cycle

Previously… (2/3)

Types of Inference

Previously… (3/3)

Types of Decision Errors

Case Study I

Two scientists want to know if a certain drug is effective against high blood pressure.

1. The first scientist wants to give the drug to 1000 people with high blood pressure and see how many of them experience lower blood pressure levels.
2. The second scientist wants to give the drug to 500 people with high blood pressure, and not give the drug to another 500 people with high blood pressure, and see how many in both groups experience lower blood pressure levels.

Which is the better way to test this drug?

Case Study I: Results from the GSS

The GSS (General Social Survey) asks the same question, below is the distribution of responses from the 2010 survey:

Case Study I: Parameter and Point Estimate

We would like to estimate the proportion of all Americans who have good intuition about experimental design, i.e. would answer “500 get the drug 500 don’t”? What are the parameter of interest and the point estimate?

• Parameter of interest: proportion of all Americans who have good intuition about experimental design. \[p \longrightarrow \text{a population proportion}\]

• Point estimate: proportion of sampled Americans who have good intuition about experimental design. \[\hat{p} \longrightarrow \text{a sample proportion}\]

Case Study I: Inference on a Proportion

What percent of all Americans have good intuition about experimental design, i.e. would answer “500 get the drug 500 don’t”?

We can answer this research question using a confidence interval, which we know is always of the form \[\text{point estimate} \pm z^{\star} \times \text{SE}.\]

Standard error (SE) of a sample proportion \[SE_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}.\]

Case Study I: Central Limit Theorem for Proportions

Sample proportions will be nearly normally distributed with mean equal to the population mean, \(p\), and standard error equal to \(\sqrt{\frac{p(1-p)}{n}}\).

This is true only under certain conditions:

• independent observations, at least 10 successes and 10 failures

Note:

• If \(p\) is unknown (most cases), we use \(\hat{p}\) in the calculation of the standard error.

Case Study I: Infering the True Proportion

The GSS found that 571 out of 670 (85%) of Americans answered the question on experimental design correctly. Estimate (using a 95% confidence interval) the proportion of all Americans who have good intuition about experimental design?

Given: \(n = 670\), \(\hat{p} = 0.85\). First check conditions.

• Independence: The sample is random, and \(670\) which is less than \(10%\) of all Americans, therefore we can assume that one respondent’s response is independent of another.
• Success-failure: 571 people answered correctly (successes) and 99 answered incorrectly (failures), both are greater than 10.

Case Study I: Confidence Interval

We are given that \(n = 670\), \(\hat{p} = 0.85\), we also just learned that the standard error of the sample proportion is \[SE_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}.\]

Which of the below is the correct calculation of the 95% confidence interval?

1. \(0.85 \pm 1.65 \times \sqrt{\frac{0.85 \times 0.15}{670}}\)
2. \(0.85 \pm 1.96 \times \sqrt{\frac{0.85 \times 0.15}{670}}\)
3. \(0.85 \pm 1.96 \times \frac{0.85 \times 0.15}{\sqrt{670}}\)
4. \(571 \pm 1.96 \times \sqrt{\frac{571 \times 99}{670}}\)

Case Study I: Choosing a sample size

Previously, for \(n=670\) the margin of error is \(1.96 \times \sqrt{\frac{0.85 \times 0.15}{670}} \approx 0.027\).

How many people should you sample in order to cut the margin of error of a 95% confidence interval down to 0.01?

\[z^{\star} \times SE_{\hat{p}}\]

\[ \begin{aligned} 1.96 \times \sqrt{\frac{0.85 \times 0.15}{n}} & \le 0.01 \\ 1.96^2 \times \frac{0.85 \times 0.15}{n} & \le 0.01^2 \\ n & \ge \frac{1.96^2 \times 0.85 \times 0.15}{0.01^2} \\ n & \ge 4898.04 \end{aligned} \]

Parameter Estimation for One Proportion

Conditions

• Independent samples
• Success-failure outcomes is \(np \ge 10\) and \(n(1-p) \ge 10\).
• We can assume Normal approximation of the Binomial PMF.

Confidence Intervals for One Proportion

\[\hat{p} \pm z^{\star} SE_{\hat{p}}\]

• \(\hat{p}\) is the sample proportion
• \(z^{\star}\) is the critical z-score for a given confidence level
• \(SE_{\hat{p}} = \sqrt{\frac{\hat{p}\left(1-\hat{p}\right)}{n}}\) is the standard error

Parameter Estimation for Two Proportions

Conditions

• Independent samples
• Success-failure outcomes is \(n_Ap_A \ge 10\), \(n_A(1-p_A) \ge 10\), \(n_Bp_B \ge 10\), and \(n_B(1-p_B) \ge 10\)
• We can assume Normal approximation of the Binomial PMF.

Confidence Intervals for Two Proportions

\[\hat{p}_B - \hat{p}_A \pm z^{\star} SE_{\hat{p}_B - \hat{p}_A}\]

• \(\hat{p}_A\) is the sample proportion for group \(A\)
• \(\hat{p}_B\) is the sample proportion for group \(B\)
• \(z^{\star}\) is the critical z-score for a given confidence level
• \(SE_{\hat{p}_B - \hat{p}_A} = \sqrt{ \frac{\hat{p}_A \left( 1-\hat{p}_A \right)}{n_A} + \frac{\hat{p}_B \left( 1-\hat{p}_B \right)}{n_B} }\) is the standard error

Case Study II

Customer Satisfaction at a Coffee Shop

Background: A local coffee shop prides itself on high customer satisfaction. The shop’s management claims that at least 85% of its customers are satisfied with their service. A market research firm is hired to assess this claim by conducting a survey.

Data Collection: The research firm randomly surveys 200 customers who visited Brew Haven in the past month. Among them, 160 reported being satisfied with the service.

Objective: Use hypothesis testing for one proportion to determine whether the claim that at least 85% of customers are satisfied is supported by the survey data.

Case Study II: Define Hypotheses

Let \(p\) represent the true proportion of satisfied customers.

• Null Hypothesis \(H_0\): \(p = 0.85\) (The satisfaction rate is equal to 85%.)

• Alternative Hypothesis \(H_A\): \(p < 0.85\) (The satisfaction rate is less than 85%.)

• Choose Significance Level: A significance level of \(\alpha = 0.05\) is chosen.

Case Study II: Compute the Test Statistic

The sample proportion is \[\hat{p} = \frac{160}{200} = 0.80.\]

Test statistic formula for one proportion: \[z = \frac{\hat{p} - p_0}{SE_{p}}\]

• \(p_0 = 0.85\) (claimed proportion or the null value)
• \(SE_{p} = \sqrt{\frac{p_0(1-p_0)}{n}}\) (standard error)
• \(n = 200\) (sample size)

Substituting values: \[z = \frac{0.80 - 0.85}{\sqrt{\frac{0.85(1-0.85)}{200}}} \approx -1.9803\]

Case Study II: Determine the P-Value

Determine the probability associated with the computed test statistic.

pe <- 0.80 # point estimate
p_0 <- 0.85 # null value
n <- 200 # sample size
SE_p <- sqrt((p_0*(1-p_0))/(n)) # standard error
Z <- (pe-p_0)/SE_p # test statistic

# p-value
pnorm(Z,0,1) 

## [1] 0.02383519

Case Study II: Make a Conclusion

We compare the p-value to our chosen significance level of \(\alpha = 0.05\).

Conclusion:

• If \(\text{p-value} < \alpha\), reject the null hypothesis and conclude that the satisfaction rate is significantly less than 85%.

• If \(\text{p-value} \ge \alpha\), do not reject the null hypothesis; there is not enough evidence to claim that the satisfaction rate is lower than 85%.

Case Study II: What does the Significance Level Mean?

Remember that we defined \(\alpha = 0.05\) arbitrarily before we conducted the hypothesis test.

The significance value \(\alpha\) is related to the confidence level of the confidence interval of the point estimate, which is \(1-\alpha\).

Case Study II: One Proportion Confidence Interval

If we set a significance level \(\alpha = 0.05\), then the confidence level for the sample proportion is \(1-\alpha = 1 - 0.05 = 0.95\).

That is a 95% confidence interval of the sample proportion \(\hat{p} = 0.80\).

• The critical z-value of a 0.95 confidence level is \(z^{\star} = 1.96\).
• Standard error of the null value is \(SE_{p} = \sqrt{\frac{0.85(1-0.85)}{200}} \approx 0.0252\).
• 95% confidence interval is \(0.80 \pm 1.96 \times 0.0252\) or \((0.7506,0.8494)\).

Hypothesis Testing for One Proportion (1/2)

Let \(p\) be the population parameter and \(p_0\) be the null value.

State Hypotheses

• Null Hypothesis \(H_0\): The population proportion remains unchanged. \[p = p_0\]
• Alternative Hypothesis \(H_A\): The population proportion has changed. \[p \ne p_0\]

Set Significance Value \(\alpha\)

Common values are \(\alpha = 0.10, 0.05, 0.01\). Note that \(\alpha\) is the Type I error rate.

Hypothesis Testing for One Proportion (2/2)

Compute the Test Statistic

\[z = \frac{\hat{p}-p_0}{SE_p}\]

• \(\hat{p}\) is the point estimate
• \(SE_p = \sqrt{\frac{p_0\left( 1-p_0 \right)}{n}}\) is the standard error of the null sampling distribution

Determine the P-Value

• You need to find \(P(Z \le z)\) for one-sided test or \(2 \times P(Z \le z)\) for a two sided test from \(Z \sim N(0,1)\), the standard normal distribution

Make a Decision and conclusion

• Reject \(H_0\) if the \(\text{p-value} < \alpha\): There is enough evidence to support \(H_A\).
• Fail to reject the \(H_0\) if the \(\text{p-value} \ge \alpha\): There is not enough evidence to support \(H_A\).

Hypothesis Testing for Two Proportions (1/2)

Let \(p_A\) and \(p_B\) be the population parameters for groups \(A\) and \(B\) respectively and \(p_0\) be the null value.

State Hypotheses

• Null Hypothesis \(H_0\): The difference in population proportion remains unchanged. \[p_B - p_A = p_0\]
• Alternative Hypothesis \(H_A\): The difference in population proportion has changed. \[p_B - p_A \ne p_0\]

Set Significance Value \(\alpha\)

Common values are \(\alpha = 0.10, 0.05, 0.01\). Note that \(\alpha\) is the Type I error rate.

Hypothesis Testing for Two Proportions (2/2)

Compute the Test Statistic

\[z = \frac{\left(\hat{p}_B-\hat{p}_A\right)-p_0}{SE_{p_B - p_A}}\]

• \(\hat{p}_A\) is the point estimate for group \(A\)
• \(\hat{p}_B\) is the point estimate for group \(B\)
• \(SE_{p_B - p_A} = \sqrt{\hat{p}_{pool}\left( 1-\hat{p}_{pool} \right)\left( \frac{1}{n_A} + \frac{1}{n_B} \right)}\) is the standard error
• \(\hat{p}_{pool} = \frac{n_A \hat{p}_A + n_B \hat{p}_B}{n_A + n_B}\)

Determine the P-Value

• You need to find \(P(Z \le z)\) for one-sided test or \(2 \times P(Z \le z)\) for a two sided test from \(Z \sim N(0,1)\), the standard normal distribution

Make a Decision and conclusion

• Reject \(H_0\) if the \(\text{p-value} < \alpha\): There is enough evidence to support \(H_A\).
• Fail to reject the \(H_0\) if the \(\text{p-value} \ge \alpha\): There is not enough evidence to support \(H_A\).

Activity: Rock-Paper-Scissors Hypothesis Testing

1. Make sure you have a copy of the W 3/19 Worksheet. This will be handed out physically and it is also digitally available on Moodle.
2. Work on your worksheet by yourself for 10 minutes. Please read the instructions carefully. Ask questions if anything need clarifications.
3. Get together with another student.
4. Discuss your results.
5. Submit your worksheet on Moodle as a .pdf file.

RPS Wins Data

Data collection was done on 3/19/2025

Number of Wins

RPS Proportion of Wins Data

Data collection was done on 3/19/2025

Proportions

Each group played 15 rounds for blind and 15 rounds for non-blind.

References