Construct and interpret regression models

Detailed Explanation: Construct and interpret regression models

A linear regression model describes the relationship between a quantitative explanatory variable $x$ and a quantitative response variable $y$ using

where:

• $\hat y$ is the predicted value of $y$,
• $b$ is the slope,
• $a$ is the $y$-intercept.

Worked example

A teacher records the number of hours five students studied and their test scores.

1. Examine the scatterplot

A scatterplot would show an approximately straight, upward pattern. This suggests using a linear regression model: students who study more hours tend to have higher scores.

2. Find the regression equation

For these data, the means are

The least-squares slope is

The intercept is

Therefore, the regression model is

The least-squares line is chosen because it makes the sum of the squared residuals as small as possible.

3. Interpret the slope and intercept

The slope is $4.1$. In context:

For each additional hour of studying, the predicted test score increases by about $4.1$ points.

The intercept is $47.7$. This means:

A student who studies $0$ hours is predicted to score about $47.7$ points.

The intercept may not be especially meaningful if $0$ hours was not represented in the data, but it is needed to define the line.

4. Make a prediction

To predict the score for a student who studies $4.5$ hours, substitute $x=4.5$:

The predicted score is about $66.2$ points. Since $4.5$ hours is between the observed values of $1$ and $5$, this is an interpolation, which is generally more reliable than predicting outside the observed range.

The predicted score is not necessarily the student’s actual score. It is the value given by the model.

5. Use residuals to judge individual predictions

A residual is

or observed value minus predicted value.

For the student who studied $2$ hours,

The actual score was $55$, so the residual is

The model overpredicted this student’s score by $0.9$ points.

For all five students, the residuals are approximately

Residuals close to $0$ indicate predictions close to the observed values. A residual plot with no clear pattern supports using a linear model. A curved pattern or a pattern that spreads out would suggest that a line may not be appropriate.

6. Interpret $R^2$

For this model,

This means that about $98.9\%$ of the variation in test scores is explained by the linear relationship with study time in this data set. The remaining variation is due to other factors and random differences.

A high $R^2$ does not prove that studying causes higher scores. These data show an association, but other variables—such as prior knowledge, attendance, or motivation—could also affect scores.

Finally, avoid extrapolation: predicting a score for someone who studies $20$ hours would be outside the observed range and may be unreliable.