Professional Documents
Culture Documents
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
11/25/2018 http://numericalmethods.eng.usf.edu 1
Regresi Linier
http://numericalmethods.eng.usf.edu
What is Regression?
What is regression? Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn)
best fit y f (x ) to the data. The best fit is generally based on
minimizing the sum of the square of the residuals, Sr.
y f (x )
Sum of the square of the residuals
n
Sr ( yi f ( xi )) 2 ( x1, y1)
i 1
Figure. Basic model for regression
3 http://numericalmethods.eng.usf.edu
Linear Regression-Criterion#1
Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn ) best fit y a0 a1 x
to the data.
y
xi , yi
i yi a0 a1 xi xn , y n
x2 , y2
x3 , y3
x1 , y1 i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
n
Does minimizing
i 1
i work as a criterion, where i yi (a 0 a1 xi )
4 http://numericalmethods.eng.usf.edu
Example for Criterion#1
Example: Given the data points (2,4), (3,6), (2,6) and (3,8), best fit
the data to a straight line using Criterion#1
8
x y
2.0 4.0 6
y
3.0 6.0 4
2.0 6.0 2
3.0 8.0 0
0 1 2 3 4
x
5 http://numericalmethods.eng.usf.edu
Linear Regression-Criteria#1
Using y=4x-4 as the regression curve
x y ypredicted ε = y - ypredicted 8
y
3.0 6.0 8.0 -2.0 4
2.0 6.0 4.0 2.0
2
3.0 8.0 8.0 0.0
0
4
0 1 2 3 4
i 0
i 1 x
6 http://numericalmethods.eng.usf.edu
Linear Regression-Criteria#1
Using y=6 as a regression curve
y
4
2.0 6.0 6.0 0.0
2
3.0 8.0 6.0 2.0
4 0
i 1
i
0 0 1 2 3 4
x
7 http://numericalmethods.eng.usf.edu
Linear Regression – Criterion #1
i 1
i 0 for both regression models of y=4x-4 and y=6.
8 http://numericalmethods.eng.usf.edu
Linear Regression-Criterion#2
n
Will minimizing
i 1
i work any better?
y
xi , yi
i yi a0 a1 xi xn , y n
x2 , y2
x3 , y3
x1 , y1 i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
9 http://numericalmethods.eng.usf.edu
Linear Regression-Criteria 2
Using y=4x-4 as the regression curve
y
2.0 4.0 4.0 0.0
4
3.0 6.0 8.0 2.0
2
2.0 6.0 4.0 2.0
0
3.0 8.0 8.0 0.0 0 1 2 3 4
4
x
4
i
i 1 Figure. Regression curve for y=4x-4, y vs. x data
10 http://numericalmethods.eng.usf.edu
Linear Regression-Criteria#2
Using y=6 as a regression curve
x y ypredicted |ε| = |y – 8
ypredicted|
6
y
2.0 4.0 6.0 2.0
4
3.0 6.0 6.0 0.0
2
2.0 6.0 6.0 0.0 0
3.0 8.0 6.0 2.0 0 1 2 3 4
4 x
i 1
i 4
Figure. Regression curve for y=6, y vs. x data
11 http://numericalmethods.eng.usf.edu
Linear Regression-Criterion#2
4
i 1
i 4 for both regression models of y=4x-4 and y=6.
The sum of the errors has been made as small as possible, that
is 4, but the regression model is not unique.
Hence the above criterion of minimizing the sum of the absolute
value of the residuals is also a bad criterion.
12 http://numericalmethods.eng.usf.edu
Least Squares Criterion
The least squares criterion minimizes the sum of the square of the
residuals in the model, and also produces a unique line.
n n 2
Sr i yi a0 a1 xi
2
i 1 i 1
y
xi , yi
i yi a0 a1 xi xn , y n
x2 , y2
x3 , y3
x1 , y1
i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
13 http://numericalmethods.eng.usf.edu
Finding Constants of Linear Model
n n 2
i 1 i 1
To find a 0 and a1 we minimize Sr with respect to a1 and a0 .
S r n
2 yi a0 a1 xi 1 0
a0 i 1
S r n
2 yi a0 a1 xi xi 0
a1 i 1
giving
n n n
a a x y
i 1
0
i 1
1 i
i 1
i
n n n
(a0 y a1 x)
a x a x yi xi
2
0 i 1 i
i 1 i 1 i 1
14 http://numericalmethods.eng.usf.edu
Finding Constants of Linear Model
Solving for a 0 and a1 directly yields,
n n n
n xi y i xi y i
i 1 i 1 i 1
a1 2
2
n n
n xi xi
i 1 i 1
and
n n n n
x y x x y
2
i i i i i
a0 i 1 i 1 i 1 i 1
2 (a0 y a1 x)
n
n
n xi2 xi
i 1 i 1
15 http://numericalmethods.eng.usf.edu
Example 1
The torque, T needed to turn the torsion spring of a mousetrap through
an angle, is given below. Find the constants for the model given by
T k1 k 2
Angle, θ Torque, T
Torque (N-m)
0.3
Radians N-m
0.2
0.698132 0.188224
0.959931 0.209138
1.134464 0.230052 0.1
0.5 1 1.5 2
1.570796 0.250965
θ (radians)
1.919862 0.313707
Figure. Data points for Angle vs. Torque data
16 http://numericalmethods.eng.usf.edu
Example 1 cont.
The following table shows the summations needed for the calculations of
the constants in the regression model.
Table. Tabulation of data for calculation of important
summations
Using equations described for
T 2 T
a 0 and a1 with n 5
Radians N-m Radians2 N-m-Radians 5 5 5
0.698132 0.188224 0.487388 0.131405 n i Ti i Ti
0.959931 0.209138 0.921468 0.200758 k2 i 1 i 1 i 1
2
2
5 5
1.134464 0.230052 1.2870 0.260986
n i i
1.570796 0.250965 2.4674 0.394215 i 1 i 1
51.5896 6.28311.1921
1.919862 0.313707 3.6859 0.602274
5
i 1
6.2831 1.1921 8.8491 1.5896 58.8491 6.2831
2
9.6091102 N-m/rad
17 http://numericalmethods.eng.usf.edu
Example 1 cont.
Use the average torque and average angle to calculate k1
5 5
_ T i _ i
T i 1
i 1
n n
1.1921 6.2831
5 5
2.3842 10 1 1.2566
Using,
_ _
k1 T k 2
2.3842 101 (9.6091 102 )(1.2566)
1.1767 101 N-m
18 http://numericalmethods.eng.usf.edu
Example 1 Results
Using linear regression, a trend line is found from the data
Can you find the energy in the spring if it is twisted from 0 to 180 degrees?
19 http://numericalmethods.eng.usf.edu
Example 2
To find the longitudinal modulus of composite, the following data is
collected. Find the longitudinal modulus, E using the regression model
Table. Stress vs. Strain data
Strain Stress
E and the sum of the square of the
(%) (MPa)
residuals.
0 0 3.0E+09
0.183 306
0.36 612 Stress, σ (Pa)
2.0E+09
0.5324 917
0.702 1223
0.867 1529 1.0E+09
1.0244 1835
1.1774 2140
1.329 2446 0.0E+00
0 0.005 0.01 0.015 0.02
1.479 2752
Strain, ε (m/m)
1.5 2767
1.56 2896 Figure. Data points for Stress vs. Strain data
20 http://numericalmethods.eng.usf.edu
Example 2 cont.
Residual at each point is given by
i i E i
The sum of the square of the residuals then is
n
S r i2
i 1
n
i E i
2
i 1
i i
Therefore E i 1
n
2
i
i 1
21 http://numericalmethods.eng.usf.edu
Example 2 cont.
Table. Summation data for regression model
With
i ε σ ε2 εσ
12
1
2
0.0000
1.8300×10−3
0.0000
3.0600×108
0.0000
3.3489×10−6
0.0000
5.5998×105
i 1
i
2
1.2764 10 3
i 1
1.2764×10−3 2.3337×108 182.84 GPa
22 http://numericalmethods.eng.usf.edu
Example 2 Results
The equation 182.84 describes the data.
23 http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/linear_regr
ession.html
THE END
http://numericalmethods.eng.usf.edu