Professional Documents
Culture Documents
1. Introduction
Finding one or more root of the equation
f (x) = 0
is one of the more commonly occurring problems of applied mathematics. In
most cases explicit solutions are not available and we must be satisfied with
being able to find a root to any specified degree of accuracy. The numerical
procedures for finding the roots are called iterative methods.
Definition 1.1 (Simple and multiple root). A root having multiplicity one
is called a simple root. For example, f (x) = (x 1)(x 2) has a simple root
at x = 1 and x = 2, but g(x) = (x 1)2 has a root of multiplicity 2 at x = 1,
which is therefore not a simple root.
A multiple root is a root with multiplicity m 2 is called a multiple point or
repeated root. For example, in the equation (x 1)2 = 0, x = 1 is multiple
(double) root.
If a polynomial has a multiple root, its derivative also shares that root.
Let be a root of the equation f (x) = 0, and imagine writing it in the
factored form
f (x) = (x )m (x)
with some integer m 1 and some continuous function (x) for which
() 6= 0. Then we say that is a root of f (x) of multiplicity m.
Definition 1.2 (Convergence). A sequence {xn } is said to be converge to a
point with order p if there is exist a constant c such that
|xn+1 |
n 0.
lim
p = c,
n |xn |
The constant c is known as asymptotic error constant.
Two cases are given special attention.
(i) If p = 1 (and c < 1), the sequence is linearly convergent.
(ii) If p = 2, the sequence is quadratically convergent.
Definition 1.3. Let {n } is a sequence which converges to zero and {xn } is
any sequence. If there exists a constant c > 0 and integer N > 0 such that
|xn | c|n |,
n N,
ak1
0
0
0
0.125
0.1875
bk1
1
0.5
0.25
0.25
0.25
ck
0.5
0.25
0.125
0.1875
0.21875
f (ak1 ) f (ck )
<0
<0
>0
>0
<0
Theorem 2.1. Suppose that f C[a, b] and f (a)f (b) < 0. The Bisection
method generates a sequence {ck } approximating a zero of f with linear
convergence.
Proof. Let [a0 , b0 ], [a1 , b1 ], ... denote the successive intervals produced
by the bisection algorithm. Thus
a = a0 a1 a2 b0 = b
b = b0 b1 b2 a0 = a.
This implies {an } and {bn } are monotonic and bounded and hence convergent.
Since
1
b1 a1 = (b0 a0 )
2
1
1
b2 a2 = (b1 a1 ) = 2 (b0 a0 )
2
2
........................
1
bn an = n (b0 a0 ).
2
Hence
lim (bn an ) = 0.
n
Take limit
lim an = lim bn = (say).
= f () = 0.
i.e. limit of {an } and {bn } is a zero of [a, b].
Let cn+1 = 21 (an + bn )
Then
1
1
|cn+1 | = | lim an (an +bn )| |bn (an +bn )|
n
2
2
(since an bn , n)
1
1
= |bn an | = n+1 |b0 a0 |.
2
2
By definition of convergence, we can say that the bisection method converges
linearly with rate 12 .
Note: 1. From the statement of the bisection algorithm, it is clear that the
algorithm always converges, however, can be very slow.
2. Computing ck : It might happen that at a certain iteration k, computation
ak + bk
of ck =
will give overflow. It is better to compute ck as:
2
bk ak
.
ck = ak +
2
Stopping Criteria: Since this is an iterative method, we must determine
some stopping criteria that will allow the iteration to stop. Criterion |f (ck )|
very small can be misleading since it is possible to have |f (ck )| very small,
even if ck is not close to the root.
Lets now find out what is the minimum number of iterations N needed
with the bisection method to achieve a certain desired accuracy. The interval
b0 a0
. So, to obtain an accuracy of , we must
length after N iterations is
2N
b0 a0
have
. That is,
2N
2N (b0 a0 ) ,
or
log(b0 a0 ) log
.
log 2
Note the number N depends only on the initial interval [a0 , b0 ] bracketing
the root.
N
Thus, a minimum of 11 iterations will be needed to obtain the desired accuracy using the bisection method.
3. Iteration method based on first degree equation
3.1. The Secant Method. Let f (x) = 0 be the given non-linear equation.
Let y = ax + b, be the equation of the secant line joining two points
f1 f0
x1 x0
x1 f0 x0 f1
.
x1 x0
The next approximation to the root is given by
b=
x2 =
x0 f1 x1 f0
b
=
a
f1 f0
x1 x0
f1 .
f1 f0
This is called the secant or chord method and successive iterations are given
by
xk xk1
xk+1 = xk
fk , k = 1, 2, . . .
fk fk1
x2 = x1
Algorithm:
1. Give inputs and take two initial guesses x0 and x1 .
2. Start iterations
x1 x0
f0 .
x2 = x1
f1 f0
3. If |f (x2 )| < (error tolerance) then stop and print the root.
4. Repeat the iterations (step 2). Also check if the number of iterations has
exceeded the maximum number of iterations.
Example 3. Apply secant method to find the root of the equation
cos x x ex = 0.
Sol. Let f (x) = cos x x ex = 0.
The successive iterations of the secant method are given by
xk+1 = xk
xk xk1
fk , k = 1, 2,
fk fk1
xk xk1
fk
fk fk1
k = 1, 2, . . .
k k1
f ( + k ).
f ( + k ) f ( + k1 )
1 f 00 ()
.
2 f 0 ()
This relation is called the error equation. Now by the definition of the order
of convergence, we expect a relation of the following type
k+1 = Cpk .
1/p
.
= pk = AC (1+1/p) k
Comparing the powers of k on both sides, we get
p = 1 + 1/p,
which gives two values of p, one is p = 1.618 and another one is negative
(and we neglect negative value of p as order of convergence is non-negative).
Therefore, order of convergence of secant method is less than 2.
3.3. Newton Method. Let f (x) = 0 be the given non-linear equation.
Let y = ax + b, be the equation of the tangent line at point (x0 , f (x0 )) on
the curve y = f (x).
Let intersection point of the tangent line with the x-axis is (x2 , 0).
= x2 = b/a, a 6= 0.
Here x0 is the approximations of the root and then we can determine a and
b. Then
f (x0 ) = ax0 + b.
Differentiating w.r.t. x, we obtain
f 0 (x0 ) = a.
Hence
b = f (x0 ) f 0 (x0 )x0 .
Now
x1 =
b
f (x0 ) f 0 (x0 )x0
=
a
f 0 (x0 )
f (x0 )
.
f 0 (x0 )
This is called the Newton method and successive iterations are given by
f (xk )
xk+1 = xk 0
, k = 0, 1, . . . .
f (xk )
The method can be obtained directly from the secant method by taking limit
xk1 xk . In the limiting case the chord joining the points (xk1 , fk1 )
and (xk , fk ) becomes the tangent at (xk , fk ).
In this case problem of finding the root of the equation is equivalent to
finding the point of intersection of the tangent to the curve y = f (x) at
point (xk , fk ) with the x-axis.
= x0
end
f (xk )
f 0 (xk )
If |xk+1 xk | is sufficiently small
then x = xk+1
return x
end
4. end (for main loop)
3. xk+1 = xk
2.
x2k 2
xk
1
=
+ .
2xk
2
xk
1 1
+ = 1.5
2 1
1
1.5
+
= 1.41666667
2
1.5
x3 = 1.41421569
x4 = 1.41421356
x5 = 1.41421356.
Since the fourth and fifth iterates agree in to eight decimal places, we assume
that 1.41421356 is a correct solution to f (x) = 0, to at least eight decimal
places.
Example 5. Perform four iterations to Newtons method to obtain the approximate value of (17)1/3 start with x0 = 2.0.
Sol. Let x = 171/3 which implies x3 = 17.
Let f (x) = x3 17 = 0.
Newton approximations are given by
xk+1 = xk
x3k 17
2x3 + 17
= k 2 ,
2
3xk
3xk
k = 0, 1, 2, . . . .
10
11
f ( + k )
f 0 ( + k )
k = 1, 2, . . .
12
|f (b)|
< b a.
|f 0 (b)|
13
m+1
m
k
k
f (m) () +
f (m+1) () + . . .
m!
(m + 1)!
f 0 (xk ) = f 0 ( + k ) =
m1
m
k
f (m) () + k f (m+1) () + . . .
(m 1)!
m!
Therefore
k+1 = k
f ( + k )
f 0 ( + k )
"
#
(m+1) ()
m
f
k
k (m)
f () 1 +
+ ...
m!
(m + 1) f (m) ()
#
"
= k
(m+1) ()
m1
f
k
k
+ ...
f (m) () 1 +
(m 1)!
m f (m) ()
# "
#1
"
k f (m+1) ()
k
k
f (m+1) ()
1+
= k
+ ...
+ ...
1+
m
(m + 1) f (m) ()
m f (m) ()
!
k
k f (m+1) ()
= k
+ ...
1
m
m f (m) ()
1
) + O(2k ).
m
This implies method has linear rate of convergence for multiple roots.
However when the multiplicity of the root is known in advance, we can
modify the method to increase the order of convergence.
We consider
f (xk )
xk+1 = xk e 0
f (xk )
where e is an arbitrary constant to be determined.
If is a multiple root with multiplicity m then error equation
= k (1
14
f (xk )
.
f 0 (xk )
15
x3k x2k xk + 1
.
3x2k 2xk 1
x3k x2k xk + 1
.
3x2k 2xk 1
n 0,
16
2
3
2.0
2.0
1.5
1.75
2.0 1.732147
1.5 1.73205
Now 3 = 1.73205 and it is clear that third choice is correct but why
other two are not working? Therefore which of the approximation is correct
or not, we will answer after the convergence result (which requires |g 0 () < 1|
in the neighborhood of ) for convergence.
Lemma 4.1. Let g(x) be a continuous function on [a, b] and assume that
a g(x) b, x [a, b] i.e. g([a, b]) [a, b] then x = g(x) has at least one
solution in [a, b].
Proof. Let g be a continuous function on [a, b].
Let assume that a g(x) b, x [a, b].
Now consider (x) = g(x) x.
If g(a) = a or g(b) = b then proof is trivial. Hence we assume that a 6= g(a)
and b 6= g(b).
Now since a g(x) b
= g(a) > a and g(b) < b.
Now
(a) = g(a) a > 0
and
(b) = g(b) b < 0.
Now is continuous and (a)(b) < 0, therefore by Intermediate Value
Theorem has at least one zero in [a, b], i.e. there exists some s.t.
g() = , [a, b].
Graphically, the roots are the intersection points of y = x & y = g(x) as
shown in the Figure.
Theorem 4.2 (Contraction Mapping Theorem). Let g & g 0 are continuous
functions on [a, b] and assume that g satisfy a g(x) b, x [a, b].
Furthermore, assume that there is a constant 0 < < 1 s.t.
= max g 0 (x).
axb
Then
1. x = g(x) has a unique solution of x = g(x) in the interval [a, b].
2. The iterates xn+1 = g(xn ), n 1 will converge to for any choice of
x0 [a, b].
17
3. | xn |
4.
n
|x1 x0 |, n 0.
1
| xn+1 |
= |g 0 ()|.
n | xn |
lim
18
1
|x1 x0 |
1
= n | x0 |
Therefore
| xn |
n
|x1 x0 |
1
n
|x1 x0 |
1
4. Now
lim
| xn+1 |
= = lim |g 0 (cn )|
n
| xn |
19
Corollary 4.3. Let g & g 0 are continuous on some interval c < x < d with
the fixed point contained in this interval. Moreover assume that
|g 0 ()| < 1.
Thus there is an interval [a, b] around for which the hypothesis and hence
conclusions of Theorem are true.
On the contrary if |g 0 ()| > 1, then the iteration method xn+1 = g(xn ) will
not converge to .
When |g 0 ()| = 1, no conclusion can be drawn and even if convergence occur,
the method would be far too slow for the iteration method to be practical.
Remark 4.2. The possible behavior of fixed-point iterates {xn } is shown in
Figure for various values of g 0 (). To see the convergence, consider the case
case of x1 = g(x0 ), the height of y = g(x) at x0 . We bring the number x1
back to the x-axis by using the line y = x and the height y = x1 . We continue
this with each iterate, obtaining a stair-step behavior when g 0 () > 0. When
g 0 () < 0, the iterates oscillates around the fixed point , as can be seen in
the Figure. In first figure (on top) iterations are monotonic convergence, in
second oscillatory convergent, in third figure iterations are divergent and in
the last figure iterations are oscillatory divergent.
Theorem 4.4. Let is a root of x = g(x), and g(x) is p times continuously
differentiable function for all x near to , for some p 2. Furthermore
assume
g 0 () = = g (p1) () = 0.
(4.1)
Then if the initial guess x0 is sufficiently close to , then iteration
xn+1 = g(xn ),
n0
lim
xn+1
g (p) (n )
p =
(xn )
p!
20
Figure 4. Convergent
xn+1 = g(xn )
=
and
non-convergent
sequences
(p)
xn+1
p1 g (n )
=
(1)
( xn )p
p!
f (x)
,
f 0 (x)
g 0 (x) =
f (x) f 00 (x)
[f 0 (x)]2
f 00 ()
6= 0,
f 0 ()
as f 0 () 6= 0 and f 00 () is either positive or negative.
g 0 () = 0,
g 00 () =
21
Sol. Using g(x) = tan x and x0 = 4 gives x1 = g(x0 ) = tan 4 = 1.158, which
is not in the interval [4, 5]. So we need a different fixed-point function.
If we note that x = tan x implies that
1
1
=
x
tan x
1
1
= x = x +
.
x tan x
1
1
Starting with x0 and taking g(x) = x +
,
x tan x
we obtain x1 = 4.61369, x2 = 4.49596, x3 = 4.49341, x4 = 4.49341.
As x3 and x4 agree to five decimals, it is reasonable to assume that these
values are sufficiently accurate.
Example 14. Consider the equation x3 7x + 2 = 0 in [0, 1]. Write a
fixed-point iteration which will converge to the solution.
1
Sol. We rewrite the equation in the form x = (x3 + 2) and define the the
7
1 3
fixed-point iteration xn+1 = (xn + 2).
7
1 3
Now g(x) = (x + 2)
7
then g : [0, 1] [0, 1] and |g 0 (x)| < 3/7 < 1, x [0, 1].
Hence by the Contraction Mapping Theorem the sequence {xn } defined
above will converge to the unique solution of given equation. Starting with
x0 = 0.5, we can compute the solution as following.
x1 = 0.303571429
x2 = 0.28971083
x3 = 0.289188016.
Therefore root correct to three decimals is 0.289.
Example 15. The iterates xn+1 = 2(1+c)xn +cx3n will converge to = 1
for some values of constant c (provided that x0 is sufficiently close to ).
Find the values of c for which convergence occurs? For what values of c, if
any, convergence is quadratic?
Sol. Fixed-point iteration
xn+1 = g(xn )
with
g(x) = 2 (1 + c)x + cx3 .
If = 1 is a fixed point then for convergence |g 0 ()| < 1
= | (1 + c) + 3c2 | < 1
= 0 < c < 1/2.
22
axn + x2
n +1
.
a+1
Sol. Let
lim xn = lim xn+1 = .
a + 2 + 1
=
a+1
= 3 2 1 = 0.
Therefore, the above formula is used to find the roots of the equation f (x) =
x3 x2 1 = 0.
Now substitute xn = + n , and xn+1 = + n+1 , we get
1
(a + 1)( + n+1 ) = a( + n ) + 2 (1 + /)2 + 1
which implies
(a + 1)n+1 = (a 2/3 )n + O(2n ).
Therefore, for fastest convergence, we have a = 2/3 . Here is the root of
the equation x3 x2 1 = 0 and can be computed by the Newton method.
Example 17. To compute the root of the equation
ex = 3 loge x,
using the formula
3 loge xn exp(xn )
,
p
show that p = 3 gives rapid convergence.
xn+1 = xn
23
3
+ e
24
f (x) =
x, x < 0
with the root = 0. What is the behavior of the iterates? Do they
converge, and if so, at what rate?
(b) Do the same as in (a), but with
3
x2 , x 0
f (x) =
3
x2 , x < 0.
(10) Find all positive roots of the equation
Z x
2
ex dt = 1
10
0
( a xn+1 )
lim
n ( a xn )3
assuming x0 has been chosen sufficiently close to the root.
(12) Show that the following two sequences
have convergence of the sec
ond order with the same limit a.
1
1
a
x2n
(i) xn+1 = xn 1 + 2
(ii) xn+1 = xn 3
.
2
xn
2
a
f (x)
.
f 0 (x)
25
Bibliography
[Gerald]