Professional Documents
Culture Documents
y
y i
, i 1, , n
n
St
Sy , St ( yi y ) 2
n 1
Mathematical Background (cont’d)
i y y
2 2
( y y ) 2
/n
S
2
or S 2
i i
n 1
y
n 1
y
Linear Regression
Fitting a straight line to a set of paired
observations: (x1, y1), (x2, y2),…,(xn, yn).
y = a0+ a1 x + e
a1 - slope
a0 - intercept
e - error, or residual, between the model and
the observations
Linear Regression: Residual
Linear Regression: Question
e1 e1= -e2
e2
Linear Regression: Criteria for a “Best” Fit
n n
min | e | | y
i 1
i
i 1
i a0 a1 xi |
Linear Regression: Criteria for a “Best” Fit
n
min max | ei || yi a0 a1 xi |
i 1
Linear Regression: Least Squares Fit
n n n
S r e ( yi , measured yi , model ) ( yi a0 a1 xi ) 2
2
i
2
i 1 i 1 i 1
n n
min S r ei
2
( yi a0 a1 xi ) 2
i 1 i 1
n n
min S r ei
2
( yi a0 a1 xi ) 2
i 1 i 1
S r
a 0
0
S r 0
a1
Linear Regression:
Determination of ao and a1
S r
2 ( yi ao a1 xi ) 0
ao
S r
2 ( yi ao a1 xi ) xi 0
a1
0 yi a 0 a1 xi
0 yi xi a 0 xi a1 xi2
a 0 na0
na0 xi a1 yi 2 equations with 2
unknowns, can be solved
ii 0i 1i
y x a x a x 2
simultaneously
Linear Regression:
Determination of ao and a1
n xi yi xi yi
a1
n x xi
2 2
i
a0 y a1 x
17
18
Error Quantification of Linear Regression
St
r2: coefficient of determination
r : correlation coefficient
Error Quantification of Linear Regression
n xi yi xi yi
a1
n x ( xi )
2 2
i
7 119.5 28 24
0.8392857
7 140 28 2
a0 y a1 x
3.428571 0.8392857 4 0.07142857
Y = 0.07142857 + 0.8392857 x
Least Squares Fit of a Straight Line: Example
(Error Analysis)
^
xi yi (yi y)
2
e ( yi y ) 2
2
i
i
22.7143
2
1 0.5 8.5765 0.1687 S t y y
2 2.5 0.8622 0.5625
S r ei 2.9911
2
3 2.0 2.0408 0.3473
4 4.0 0.3265 0.3265
5 3.5 0.0051 0.5896 St S r
6 6.0 6.6122 0.7972 r
2
0.868
St
7 5.5 4.2908 0.1993
28 24.0 22.7143 2.9911
r r 2 0.868 0.932
Least Squares Fit of a Straight Line:
Example (Error Analysis)
St 22.7143
sy 1.9457
n 1 7 1
•The standard error of estimate (quantifies the spread around the
regression line)
Sr 2.9911
sy / x 0.7735
n2 72
Because S y / x S y , the linear regression model has good fitness
Algorithm for linear regression
Linearization of Nonlinear Relationships
• The relationship between the dependent and
independent variables is linear.
• However, a few types of nonlinear functions
can be transformed into linear regression
problems.
The exponential equation.
The power equation.
The saturation-growth-rate equation.
Linearization of Nonlinear Relationships
1. The exponential equation.
y a1eb1x
ln y ln a1 b1 x
y* = a o + a1 x
Linearization of Nonlinear Relationships
2. The power equation
y a2 xb2
x
y a3
b3 x
y* = 1/y
1 1 b3 1
ao = 1/a3
a1 = b3/a3
y a3 a3 x x* = 1/x
Example
Fit the following Equation:
y a2 x b2
to the data in the following table: log y log( a2 xb2 )
xi yi X*=log xi Y*=logyi log y log a2 b2 log x
1 0.5 0 -0.301 let Y * log y, X * log x,
2 1.7 0.301 0.226 a0 log a2 , a1 b2
3 3.4 0.477 0.534
4 5.7 0.602 0.753 Y * a0 a1 X *
5 8.4 0.699 0.922
15 19.7 2.079 2.141
Example
a0 y a1x 0.4282 1.75 0.41584 0.334
Linearization of Nonlinear
Functions: Example
log y=-0.334+1.75log x
y 0.46x 1.75
Polynomial Regression
A parabola is preferable
Polynomial Regression (cont’d)
ei yi ao a1 xi a2 xi
2
Sr ei yi ao a1 xi a2 xi
2
2 2
Polynomial Regression (cont’d)
S r
2 ( yi ao a1 xi a2 xi2 ) 0
ao
S r
2 ( yi ao a1 xi a2 xi2 ) xi 0
a1
S r
2 ( yi ao a1 xi a2 xi2 ) xi2 0
a2
i
y n ao a1 i
x a 2 i
x 2
3 linear equations
with 3 unknowns
i i o i 1 i 2 i
x y a x a x 2
a x 3
(ao,a1,a2), can be
solved
i i o i 1 i 2 i
x 2
y a x 2
a x 3
a x 4
Polynomial Regression (cont’d)
Sr St S r
sy / x r
2
n3 St
Polynomial Regression (cont’d)
General:
The mth-order polynomial:
y ao a1 x a2 x 2 ..... am x m e
• A system of (m+1)x(m+1) linear equations must be solved for
determining the coefficients of the mth-order polynomial.
• The standard error:
Sr
sy / x
n m 1
St S r
• The coefficient of determination: r
2
St
Polynomial Regression- Example
Fit a second order polynomial to data:
xi yi xi2 xi3 xi4 xiyi xi2yi x i 15
0 2.1 0 0 0 0 0
1 7.7 1 1 1 7.7 7.7 y
xi3 225 i 152.6
2 13.6 4 8 16 27.2 54.4
i 55
x 2
x y i i 585.6
15 152.6
x 2.5, y 25.433
xi yi 2488.8
2
6 6
Polynomial Regression- Example
(cont’d)
• The system of simultaneous linear equations:
6 15 55 a0 152.6
15 55 225 a 585.6
1
55 225 979 2488.8
2
a
a0 2.47857, a1 2.35929, a2 1.86071
y 2.47857 2.35929 x 1.86071 x 2
S r ei 3.74657
2
St yi y 2513.39
2
Polynomial Regression- Example
(cont’d)
xi yi ymodel e i2 (yi-y`)2
0 2.1 2.4786 0.14332 544.42889
1 7.7 6.6986 1.00286 314.45929
2 13.6 14.64 1.08158 140.01989
3 27.2 26.303 0.80491 3.12229
4 40.9 41.687 0.61951 239.22809
5 61.1 60.793 0.09439 1272.13489
15 152.6 3.74657 2513.39333