Objectives

Previously…

Linear Regression

\[ y = \beta_0 + \beta_1 x + \epsilon \]

• \(y\) is the outcome, \(x\) is the predictor, \(\beta_0\) is the intercept, and \(\beta_1\) is the slope. The notation \(\epsilon\) is the model’s error (residuals).
• We estimate the slope \(b_1\) and intercept \(b_0\) of the least squares regression line by minimizing the sum of squared residuals.
• The slope is given by \(b_1 = \frac{s_y}{s_x} r\) , and the intercept is \(b_0 = \bar{y} - b_1 \bar{x}\) , where \(r\) is the correlation coefficient, \(s_x\) and \(s_y\) are the standard deviations of \(x\) and \(y\), and \(\bar{x}\), \(\bar{y}\) are their respective means.

Case Study I

Consider data births gathered originally from the US Department of Health and Human Services. The births14 data can be found in the openintro R package.

Case Study I: The Linear Model

We want to predict the baby weight based on number of weeks. The population linear model is \[y_{weight} = \beta_0 + \beta_1 x_{weeks} + e\]

The relevant hypotheses for the linear model setting can be written in terms of the population slope parameter.

Here the population refers to a larger population of births in the US.

• \(H_0: \beta_1= 0\), there is no linear relationship between weight and weeks.
• \(H_A: \beta_1 \ne 0\), there is some linear relationship between weight and weeks.

Let’s set the significance value to be \(\alpha = 0.01\).

Technical Conditions

• Linearity. The scatterplot of the explanatory and response must be nearly linear.
• Independent Observations. The samples must be independent.
• Normally Distributed Residuals. The errors must show a nearly normal distribution.
• Constant or equal variability. The error must exhibit homoscedasticity.

Case Study 1: Residual Analysis (1/2)

• The residuals appear approximately normal, supporting the normality assumption.
• The residual plot shows roughly “constant” variance, suggesting homoskedasticity.
• Slight clustering near the center may require further investigation but otherwise the residuals are independent of the model (no pattern).

Case Study 1: Residual Analysis (2/2)

Case Study 1: Least Squares Approximation

The least squares regression model uses the data to find a sample linear fit: \[\hat{y}_{weight} = -3.5980 + 0.2792 \times x_{weeks}.\]

R code:

lm(weight ~ weeks, data = births14)

where the data is stored as a data frame named births14 and weight and weeks are two numerical variables in the data frame.

Hypothesis Testing - Randomization Method (1/2)

Two different permutations of the weight variable with slightly different least squares regression lines.

Hypothesis Testing - Randomization Method (2/2)

Histogram of slopes given different permutations of the weight variable. The vertical red line is at the observed value of the slope, 0.28.

Hypothesis Testing - Theoretical Method

• Computing the test statistic. \[t = \frac{b_1 - \text{null value}}{SE} = \frac{0.2792 - 0}{0.0135} = 20.69\]
• Degrees of Freedom: \(df = n - k - 1\), where \(n\) is the sample size and \(k\) is the number of predictors. So, in this case, \(df = 1000 - 2 = 998\).

Using R to compute the p-value

2*(1-pt(20.69,998))

## [1] 0

• We multiply the p-value since it’s a two sided test, but it’s still 0.
• Since the p-value is less than \(\alpha = 0.01\), then we reject the null hypothesis, meaning we have enough evidence to support that the true slope (relationship) between weeks and weight is non-zero.
• More specifically, since the test statistic is positive, then we can futher conclude the the relationship is strongly positive.

Confidence Interval (1/2)

99% Confidence interval for the slope

\[b_1 \pm t_{df}^* \text{SE}_{b_1}\]

• Critical t-star for a 0.99 confidence level: \(t_{df}^* = t_{998}^* = 2.5808\). Note that we use \(1-\alpha\) as the confidence level, where \(alpha = 0.01\).

Using R to compute the critical t-star

qt((1-0.99)/2,998)

## [1] -2.580765

\[ \begin{aligned} 0.2792 & \pm 2.5808 \times 0.0135 \end{aligned} \] \[(0.2444,0.3140)\]

We are 99% confident that the true slope is in between 0.2444 and 0.3140. Note that the null value of 0 (no slope) is not within the interval.

Confidence Interval (2/2)

Activity: Conduct a Hypothesis Test for Linear Regression

1. Log-in to Posit Cloud and open the R Studio assignment F 4/11 - Conduct a Hypothesis Test for Linear Regression.
2. Make sure you are in the current working directory. Rename the .Rmd file by replacing [name] with your name using the format [First name][Last initial]. Then, open the .Rmd file.
3. Change the author in the YAML header.
4. Read the provided instructions.
5. Answer all exercise problems on the designated sections.

References