In numerical analysis, the order of convergence and the rate of convergence of a convergent sequence are quantities that represent how quickly the sequence approaches its limit. In the most common applications, a sequence that converges to is said to have order of convergence and rate of convergence if
Where greater methodological precision is required, these rates and orders of convergence are known specifically as the rates and orders of Q-convergence, short for quotient-convergence, since the limit in question is a quotient of error terms.[2] The rate of convergence is also called the asymptotic error constant. Note that this terminology is not standardized and some authors will use rate where this article uses order (e.g., [3]).
In practice, the rate and order of convergence provide useful insights when using iterative methods for calculating numerical approximations. If the order of convergence is higher, then typically fewer iterations are necessary to yield a useful approximation. Strictly speaking, however, the asymptotic behavior of a sequence does not give conclusive information about any finite part of the sequence.
Similar concepts are used for discretization methods. Ideally the solution of a discretized problem will converge to the solution of the continuous problem as the grid size goes to zero, and the speed of that convergence is one important characterization of the efficiency of a discretization method. However, the terminology of "rates of convergence" in this case is different from the terminology for iterative methods.
Series acceleration is a collection of techniques for improving the rate of convergence of a series discretization and possibly its order of convergence, also. These accelerations are commonly accomplished with sequence transformations.
Suppose that the sequence converges to the number . The sequence is said to converge with order to , and with a rate of convergence , if
for some positive constant if and if .[2][4][5] It is not necessary that be an integer. For example, the secant method, when converging to a regular, simple root, has an order of φ ≈ 1.618.[citation needed] This is technically called Q-convergence, short for quotient-convergence, and the rates and orders are called rates and orders of Q-convergence in certain technical settings where alternative rate definitions are more appropriate; see § R-convergence below.
Convergence with order
In addition, when for a sequence or for any sequence such that
that sequence is said to converge superlinearly to (i.e., faster than linearly).[2][6] A sequence is said to converge sublinearly to (i.e., slower than linearly) if it converges and
A sequence converges logarithmically to if the sequence converges sublinearly and [7]
The definitions of Q-convergence rates have a shortcoming in that they do not naturally capture the convergence behavior of sequences that do not converge with an asymptotically constant rate with every step, such as the staggered geometric progression below that gets closer to its limit only every other step.
In such cases, a closely related but more technical definition of rate of convergence called R-convergence is more appropriate; The "R-" prefix stands for "root."[2][8]: 620 A sequence that converges to is said to converge at least R-linearly if there exists an error-bounding sequence such that and converges Q-linearly to zero; analogous definitions hold for R-superlinear convergence, R-sublinear convergence, R-quadratic convergence, and so on.[2][9]
In order to define the rates and orders of R-convergence, one uses the rate and order of Q-convergence of am error-bounding sequence chosen such that no other error-bounding sequence could have been chosen that would converge with a faster rate and order; any provides a lower bound on the rate and order of R-convergence and the greatest lower bound gives the exact rate and order of R-convergence.
A practical method to calculate the order of convergence for a sequence generated by a fixed point iteration is to calculate the following sequence, which converges to the order :[10]
For numerical approximation of an exact value through a numerical method of order q see.[11]
The geometric progression converges to . Plugging the sequence into the definition of Q-linear convergence (i.e., order of convergence 1) shows that
Thus converges Q-linearly with a convergence rate of .
More generally, for any , a geometric progression converges linearly with rate and the sequence of partial sums of a geometric series also converges linearly with rate . The same holds also for geometric progressions and geometric series parameterized by any complex numbers
The staggered geometric progression using the floor function that gives the largest integer that is less than or equal to converges R-linearly to 0 with rate 1/2, but it does not converge Q-linearly. The defining Q-linear convergence limits do not exist for this sequence because one subsequence of error quotients (the sequence of quotients taken from odd steps) has a different limit than another subsequence (the sequence of quotients taken from even steps). Generally, for any staggered geometric progression , the sequence will not converge Q-linearly but will converge R-linearly with rate this example highlights why the "R" in R-linear convergence is short for "root."
The sequence converges to zero Q-superlinearly. In fact, it is quadratically convergent with a quadratic convergence rate of 1.
Finally, the sequence converges to zero Q-sublinearly and logarithmically.
A similar situation exists for discretization methods designed to approximate a function , which might be an integral being approximated by numerical quadrature, or the solution of an ordinary differential equation (see example below). The discretization method generates a sequence , where each successive is a function of along with the grid spacing between successive values of the independent variable . The important parameter here for the convergence speed to is the grid spacing , inversely proportional to the number of grid points, i.e. the number of points in the sequence required to reach a given value of .
In this case, the sequence is said to converge to the sequence with order q if there exists a constant C such that
This is written as using big O notation.
This is the relevant definition when discussing methods for numerical quadrature or the solution of ordinary differential equations (ODEs).[example needed]
A practical method to estimate the order of convergence for a discretization method is pick step sizes and and calculate the resulting errors and . The order of convergence is then approximated by the following formula:
which comes from writing the truncation error, at the old and new grid spacings, as
The error is, more specifically, a global truncation error (GTE), in that it represents a sum of errors accumulated over all iterations, as opposed to a local truncation error (LTE) over just one iteration.
Consider the ordinary differential equation
with initial condition . We can solve this equation using the Forward Euler scheme for numerical discretization:
which generates the sequence
In terms of , this sequence is as follows, from the Binomial theorem:
The exact solution to this ODE is , corresponding to the following Taylor expansion in for :
In this case, the truncation error is
so converges to with a convergence rate .
The sequence with was introduced above. This sequence converges with order 1 according to the convention for discretization methods.[why?]
The sequence with , which was also introduced above, converges with order q for every number q. It is said to converge exponentially using the convention for discretization methods. However, it only converges linearly (that is, with order 1) using the convention for iterative methods.[why?]
The case of recurrent sequences which occurs in dynamical systems and in the context of various fixed-point theorems is of particular interest. Assuming that the relevant derivatives of f are continuous, one can (easily) show that for a fixed point such that , one has at least linear convergence for any starting value sufficiently close to p. If and , then one has at least quadratic convergence, and so on. If , then one has a repulsive fixed point and no starting value will produce a sequence converging to p (unless one directly jumps to the point p itself).
Many methods exist to increase the rate of convergence of a given sequence, i.e., to transform one given sequence into a second one that converges more quickly to the same limit. Such techniques are in general known as "series acceleration" methods. These reduce the computational costs of approximating the limits of the transformed sequences. One example of series acceleration is Aitken's delta-squared process. These methods in general (and in particular Aitken's method) do not increase the order of convergence, and are useful only if initially the convergence is not faster than linear: if converges linearly, one gets a sequence that still converges linearly (except for pathologically designed special cases), but faster in the sense that . On the other hand, if the convergence is already of order ≥ 2, Aitken's method will bring no improvement.
The simple definition is used in
The extended definition is used in
The Big O definition is used in
The terms Q-linear and R-linear are used in