Module 7 Simple Linear Regression

Statistics for Managers using
Microsoft Excel
6th Edition
Module 7
Simple Linear Regression
Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall
13-1
Learning Objectives
In this chapter, you learn:
How to use regression analysis to predict the value of
a dependent variable based on an independent
variable
The meaning of the regression coefficients b0 and b1
To make inferences about the slope and correlation
coefficient
To predict individual values
13-2
Correlation vs. Regression

DCOVA
A scatter plot can be used to show the

relationship between two variables
Correlation analysis is used to measure the

strength of the association (linear relationship)
between two variables
Correlation is only concerned with strength of the

relationship
No causal effect is implied with correlation
13-3
Introduction to
Regression Analysis
DCOVA
Regression analysis is used to:
Predict the value of a dependent variable based on

the value of at least one independent variable
Explain the impact of changes in an independent

variable on the dependent variable
Dependent variable: the variable we wish to

predict or explain
Independent variable: the variable used to predict
or explain the dependent
variable
13-4

Model
DCOVA
Only one independent variable, X
Relationship between X and Y is

described by a linear function
Changes in Y are assumed to be related

to changes in X
13-5
Types of Relationships
DCOVA
Linear relationships
Y
Curvilinear relationships
Y
X
Y
X
Y
X
X
13-6
DCOVA
(continued)
Strong relationships
Y
Weak relationships
Y
X
Y
X
Y
X
X
13-7
DCOVA
(continued)
No relationship
Y
X
Y
X
13-8

Model
DCOVA
Population
Y intercept
Dependent
Variable
Population
Slope
Coefficient
Independent
Variable
Random
Error
term
Yi 0 1Xi i
Linear component
Random Error
component
13-9

DCOVA
Model
(continued)
Yi 0 1Xi i
Observed Value
of Y for Xi
Predicted Value
of Y for Xi
Slope = 1
Random Error
for this Xi value
Intercept = 0
Xi
X
13-10

Equation (Prediction Line) DCOVA
The simple linear regression equation provides an
estimate of the population regression line
Estimated
(or predicted)
Y value for
observation i
Estimate of
the regression
Estimate of the
regression slope
intercept
Yi b0 b1Xi
Value of X for
observation i
13-11
The Least Squares Method

DCOVA
b0 and b1 are obtained by finding the values of

that minimize the sum of the squared
differences between Y and Y :
2
2
min (Yi Yi ) min (Yi (b 0 b1X i ))
13-12
Finding the Least Squares

Equation
DCOVA
b1
xy nxy
x
nx
b0
x y
xy
n
2
2
x n
y
x
b
n
13-13
Interpretation of the
Slope and the Intercept
DCOVA
b0 is the estimated average value of Y

when the value of X is zero
b1 is the estimated change in the

average value of Y as a result of a
one-unit increase in X
13-14

Example
DCOVA
A real estate agent wishes to examine the

relationship between the selling price of a home
and its size (measured in square feet)
A random sample of 10 houses is selected

Dependent variable (Y) = house price in $1000s
Independent variable (X) = square feet
13-15

Example: Data
House Price in $1000s
(Y)
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
DCOVA
13-16

Example: Scatter Plot
House price model: Scatter Plot
DCOVA
13-17
Simple Linear Regression Example:

Using Excel Data Analysis Function
DCOVA
2. Choose Data Analysis
1. Choose Data
3. Choose Regression
13-18

Using Excel Data Analysis Function
(continued)
Enter Ys and Xs and desired options
DCOVA
13-19

Excel Output
DCOVA
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
house price 98.24833 0.10977 (square feet)
41.33032
Observations
ANOVA
The regression equation is:
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
13-20

Graphical Representation
DCOVA
House price model: Scatter Plot and Prediction Line
Slope
= 0.10977
Intercept
= 98.248

13-21

Example: Interpretation of bo DCOVA
b0 is the estimated average value of Y when the

value of X is zero (if X = 0 is in the range of
observed X values)
Because a house cannot have a square footage

of 0, b0 has no practical application
13-22

Example: Interpreting b1
DCOVA
b1 estimates the change in the average

value of Y as a result of a one-unit
increase in X
Here, b1 = 0.10977 tells us that the mean value of a

house increases by .10977($1000) = $109.77, on
average, for each additional one square foot of size
13-23

Example: Making Predictions
Predict the price for a house
with 2000 square feet:
DCOVA
house price 98.25 0.1098 (sq.ft.)

98.25 0.1098(200 0)
317.85
The predicted price for a house with 2000
square feet is 317.85($1,000s) = $317,850
13-24

Example: Making Predictions
DCOVA
When using a regression model for prediction,

only predict within the relevant range of data
Relevant range for
interpolation
Do not try to
extrapolate
beyond the range
of observed Xs
13-25
Measures of Variation
DCOVA
Total variation is made up of two parts:
SST
SSR
Total Sum of
Squares
Regression Sum
of Squares
SST ( Yi Y )2
SSR ( Yi Y )2
SSE
Error Sum of
Squares
SSE ( Yi Yi )2
where:
= Mean value of the dependent variable
Yi = Observed value of the dependent variable
Yi = Predicted value of Y for the given X value

i
13-26
(continued)
DCOVA
SST = total sum of squares
Measures the variation of the Yi values around their

mean Y
SSR = regression sum of squares (Explained Variation)
(Total Variation)
Variation attributable to the relationship between X

and Y
SSE = error sum of squares (Unexplained Variation)
Variation in Y attributable to factors other than X
13-27
(continued)
DCOVA
Y
Yi
SSE = (Yi - Yi )2
SST = (Yi - Y)2

_
SSR = (Yi - Y)2
Xi
_
Y
X
13-28
Coefficient of Determination, r2
DCOVA
The coefficient of determination is the portion

of the total variation in the dependent variable
that is explained by variation in the
independent variable
The coefficient of determination is also called
r-squared and is denoted as r2
SSR regression sum of squares
r
SST
total sum of squares
2
note:
0 r 1
13-29
Examples of Approximate
r2 Values
DCOVA
Y
r2 = 1
r2 = 1
100% of the variation in Y is

explained by variation in X
r =1
2
Perfect linear relationship

between X and Y:
13-30
r2 Values
DCOVA
Y
0 < r2 < 1
Weaker linear relationships

between X and Y:
Some but not all of the
variation in Y is explained
by variation in X
X
13-31
r2 Values
DCOVA
r2 = 0
No linear relationship
between X and Y:
r2 = 0
The value of Y does not

depend on X. (None of the
variation in Y is explained
by variation in X)
13-32

Coefficient of Determination, r2 in Excel
DCOVA
SSR 18934.9348
r
0.58082
SST 32600.5000
2
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
58.08% of the variation in

house prices is explained by
variation in square feet
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
13-33
Standard Error of Estimate

DCOVA
The standard deviation of the variation of

observations around the regression line is
estimated by
n
SSE
S YX
n2
i 1
(Yi Yi ) 2
n2
Where
SSE = error sum of squares
n = sample size
13-34

Standard Error of Estimate in Excel
DCOVA
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
S YX 41.33032
10
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
13-35
Comparing Standard Errors

SYX is a measure of the variation of observed
Y values from the regression line
Y
DCOVA
small SYX
large SYX
The magnitude of SYX should always be judged relative to the

size of the Y values in the sample data
i.e., SYX = $41.33K is moderately small relative to house prices in
the $200K - $400K range
13-36
Assumptions of Regression
L.I.N.E
DCOVA
Linearity
The relationship between X and Y is linear
Independence of Errors
Error values are statistically independent
Normality of Error
Error values are normally distributed for any given
value of X
Equal Variance (also called homoscedasticity)
The probability distribution of the errors has constant
variance
13-37
Inferences About the Slope

DCOVA
The standard error of the regression slope

coefficient (b1) is estimated by
S YX
Sb1
SSX
S YX
(X X)
where:
Sb1
= Estimate of the standard error of the slope
S YX
SSE
= Standard error of the estimate

n2
13-38
Inferences About the Slope:

t Test
DCOVA
t test for a population slope
Null and alternative hypotheses
Is there a linear relationship between X and Y?

H 0 : 1 = 0
H 1 : 1 0
(no linear relationship)

(linear relationship does exist)
Test statistic
t STAT
b1 1
Sb
d.f. n 2
where:
b1 = regression slope
coefficient
1 = hypothesized slope
Sb1 = standard
error of the slope
13-39

t Test Example
DCOVA
House Price
in $1000s
(y)
Square Feet
(x)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Estimated Regression Equation:

house price 98.25 0.1098 (sq.ft.)
The slope of this model is 0.1098

Is there a relationship between the
square footage of the house and its
sales price?
13-40

t Test Example
DCOVA
H0: 1 = 0
H1: 1 0
From Excel output:

Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
b1
Sb1
t STAT
b1 1
Sb
0.10977 0
3.32938
0.03297
13-41

t Test Example
DCOVA
Test Statistic: tSTAT = 3.329
H0: 1 = 0
H1: 1 0
d.f. = 10- 2 = 8
/2=.025
Reject H0
/2=.025
Do not reject H0
-t/2
-2.3060
Reject H0
t/2
2.3060
3.329
Decision: Reject H0
There is sufficient evidence
that square footage affects
house price
13-42

t Test Example
DCOVA
H0: 1 = 0
H1: 1 0
From Excel output:
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
Decision: Reject H0, since p-value <
p-value
There is sufficient evidence that

square footage affects house price.
13-43
F Test for Significance

DCOVA
F Test statistic:
where
MSR
FSTAT
MSE
MSR
SSR
k
MSE
SSE
n k 1
where FSTAT follows an F distribution with k numerator and (n k - 1)

denominator degrees of freedom
(k = the number of independent variables in the regression model)
13-44
F-Test for Significance

Excel Output
DCOVA
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
ANOVA
MSR 18934.9348
FSTAT
11.0848
MSE 1708.1957
10
df
With 1 and 8 degrees

of freedom
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
p-value for
the F-Test
Significance F
0.01039
13-45
F Test for Significance

(continued)
DCOVA
Test Statistic:
H 0 : 1 = 0
H 1 : 1 0
= .05
df1= 1
FSTAT
df2 = 8
Decision:
Reject H0 at = 0.05
Critical
Value:
F = 5.32
Conclusion:
= .05
Do not
reject H0
Reject H0
MSR
11.08
MSE
There is sufficient evidence that

house size affects selling price
F.05 = 5.32
13-46
Pitfalls of Regression Analysis
Lacking an awareness of the assumptions

underlying least-squares regression
Not knowing how to evaluate the assumptions
Not knowing the alternatives to least-squares
regression if a particular assumption is violated
Using a regression model without knowledge of
the subject matter
Extrapolating outside the relevant range
13-47
Strategies for Avoiding

the Pitfalls of Regression
Start with a scatter plot of X vs. Y to observe

possible relationship
Perform residual analysis to check the
assumptions
Plot the residuals vs. X to check for violations of

assumptions such as homoscedasticity
Use a histogram, stem-and-leaf display, boxplot,
or normal probability plot of the residuals to
uncover possible non-normality
13-48
Strategies for Avoiding

the Pitfalls of Regression
(continued)
If there is violation of any assumption, use

alternative methods or models
If there is no evidence of assumption violation,
then test for the significance of the regression
coefficients and construct confidence intervals
and prediction intervals
Avoid making predictions or forecasts outside
the relevant range
13-49
Chapter Summary
Introduced types of regression models

Reviewed assumptions of regression and
correlation
Discussed determining the simple linear
regression equation
Described measures of variation
Described inference about the slope
Discussed possible pitfalls in regression and
recommended strategies to avoid them
13-50
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording, or otherwise, without the prior written permission of the publisher.
Printed in the United States of America.
13-51

Module 7 Simple Linear Regression

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Module 7 Simple Linear Regression

Uploaded by

Copyright:

Available Formats

Statistics for Managers using

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Correlation vs. Regression

A scatter plot can be used to show the

Correlation analysis is used to measure the

Correlation is only concerned with strength of the

No causal effect is implied with correlation

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Regression analysis is used to:

Predict the value of a dependent variable based on

Explain the impact of changes in an independent

Dependent variable: the variable we wish to

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

Only one independent variable, X

Relationship between X and Y is

Changes in Y are assumed to be related

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

Simple Linear Regression

The Least Squares Method

b0 and b1 are obtained by finding the values of

min (Yi Yi ) min (Yi (b 0 b1X i ))

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Finding the Least Squares

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

b0 is the estimated average value of Y

b1 is the estimated change in the

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

A real estate agent wishes to examine the

A random sample of 10 houses is selected

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression Example:

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression Example:

Enter Ys and Xs and desired options

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression Example:

house price 98.24833 0.10977 (square feet)

The regression equation is:

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression Example:

House price model: Scatter Plot and Prediction Line

house price 98.24833 0.10977 (square feet)

Simple Linear Regression

b0 is the estimated average value of Y when the

Because a house cannot have a square footage

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

house price 98.24833 0.10977 (square feet)

b1 estimates the change in the average

Here, b1 = 0.10977 tells us that the mean value of a

Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall

Simple Linear Regression

house price 98.25 0.1098 (sq.ft.)

Simple Linear Regression

When using a regression model for prediction,