1.4. Taylor Series Expansion#
Note
Important things to retain from this block:
Understand how to compute Taylor series expansion (TSE) of a given function around a given point and its limitations
Understand how numerical derivatives are derived from TSE
Understand that the magnitude of the error depends on the expression used Things you do not need to know:
Any kind of Taylor expansion of a function by heart
Definition#
The Taylor Series Expansion (TSE) of an arbitrary function \(f(x)\) around \(x=x_i\) is a polynomial of varying order given by
This series is exact as long as we include infinite terms. We, however, are limited to a truncated expression: an approximation of the real function.
Using only 3 terms and defining \(\Delta x=x-x_i\), the TSE can be rewritten as
Here \(\Delta x\) is the distance between the point we “know” and the desired point. \(\mathcal{O}(\Delta x)^4\) means that we do not take into account the terms associated to \(\Delta x^4\) and therefore that is the truncation error order. From here we can also conclude that the larger the step \(\Delta x\), the larger the error!
Tip
We will use \(\Delta x\) more frequently from this point on, so it is good to recognize now, using the equations above, that it is a different way of representing the differential increment, for example, \(f(x_{i+1})=f(x_i+\Delta x)\) or \(f(x_{i+2})=f(x_i+2\Delta x)\).
Now let’s see an example.
Compute \(e^{0.2}\) using 3 terms of TSE around \(x_i=0\)
Solution
We want to evaluate \(e^x=e^{x_i+\Delta x}=e^{0.2}\). Therefore, \(\Delta x=0.2\). The value of \(f(x_i=0)=e^0=1\). For the case of this exponential, the derivatives have the same value \(f'(x_i=0)=f''(x_i=0)=f'''(x_i=0)=e^0=1\). The TSE looks like
Compute the TSE polynomial truncated until \(\mathcal{O}(\Delta x)^6\) for \(f(x)=\sin(x)\) around \(x_i=0\)
Solution
Applying the definition of TSE:
Note that \(x\) and \(\Delta x\) can be written interchangeably when \(x_i=0\) as \(\Delta x=x-x_i\). If this would not be the case, for example if \(x_i=\pi\) the result is completely different.
Compute the TSE polynomial truncated until \(\mathcal{O}(\Delta x)^6\) for \(f(x)=\sin(x)\) around \(x_i=\frac{\pi}{2}\)
Solution
Applying the definition of TSE:
How good is this polynomial? How does the result varies on the number of terms used?
Press rocket –>Live Code to interact with the figure. You should then click ‘run all’ in the cell below.
import numpy as np
import matplotlib.pyplot as plt
from ipywidgets import widgets, interact
def taylor_plt(order_aprox):
x = np.linspace(-2*np.pi,2*np.pi,100)
plt.plot(x, np.sin(x), label="sin(x)")
if order_aprox == '1st order':
plt.plot(x, x, label = "1st order approximation")
elif order_aprox == '3rd order':
plt.plot(x, x-(1/6*x**3), label = "3rd order approximation")
elif order_aprox == '5th order':
plt.plot(x, x-(1/6*x**3)+(1/120*x**5), label = "5th order approximation")
elif order_aprox == '7th order':
plt.plot(x, x-(1/6*x**3)+(1/120*x**5)-(1/5040*(x)**7), label = "7th order approximation")
plt.ylim(-5,5)
plt.axis('off')
plt.legend()
plt.show();
#run to interact with the figure
interact(taylor_plt, order_aprox = widgets.ToggleButtons(options=['1st order', '3rd order', '5th order','7th order']));
Relevant conclusions:
The 1st order, which depends only on the first derivative evaluation, is a straight line.
The more terms used (larger order) the smaller the error.
The further from the starting point (e.g., in the plots \(x_i=0\)), the larger the error.
Use of TSE to define the first derivative#
The TSE definition when \(\Delta x = x_{i+1} - x_i\) can be rewritten to solve for the first derivative:
By truncating the derivative to avoid the calculation of the second derivative, we find the forward difference
This is the same definition of the numerical derivative! Now we have the added knowledge that this comes with an error of the order of the step.
The backward difference can be found by redefining \(-\Delta x=x_i-x_{i-1}\)
The central difference can be found by summing the forward and backward difference expressions of the derivative and dividing it by 2:
The second derivative terms cancel each other out, therefore the order error is the step size squared! From here, it is obvious that the central difference is more accurate. You can notice it as well intuitively in the figure of the previous chapter.
Derive the forward difference for the first derivative \(f'(x_i)\) with a first order error \(\mathcal{O}(\Delta x)\) using Taylor series
Solution
Start from the TSE (tip: first derivative and first order error (1+1=2), we need to do a TSE truncated until the 2nd order):
Rearange to get:
Divide by \( - \Delta x \):
Use taylor series to derive the backward difference for the first derivative \(f'(x_i)\) with a first order error \(\mathcal{O}(\Delta x)\)
Solution
Start from the TSE
(tip: first derivative and first order error, 1+1=2, we need to do a TSE truncated until the 2nd order):
Rearrange to get:
Divide by \( \Delta x \):
Use taylor series to derive the Central difference for the first derivative \(f'(x_i)\) with a 2nd order error \(\mathcal{O}(\Delta x)^2\)
Solution
Start from the TSE of \(f(x_i+\Delta x)\) and \(f(x_i-\Delta x)\)
(tip: first derivative and second order error (1+2=3), we need to do a TSE truncated until the 3rd order):
Rearrange both equations:
Take the combination \(2\Delta x f'(x_i) + \Delta x f'(x_i)\) by adding equation 3 and 4:
Divide by \(2\Delta x \) :
TSE to define second derivatives#
There are equations that require second derivatives. One example is the diffusion equation. The 1-D diffusion equation reads:
For the moment we will use TSE to find only a numerical expression of the second derivative \(\frac{\partial^2 f}{\partial x^2}\).
The procedure is simple but cumbersome. The general idea is to isolate the second derivative in the TSE without there being a dependency on other derivatives. Below you can find more details about the algebraic manipulation (if you are curious) but you do not need to know it. Here is the result:
This is the forward difference approximation of the second derivative. Two aspects come to mind: one more point is needed to compute the second derivative and the error (of the simplest second derivative) is also of the order of the step. There are also backward and central approximations of the second derivative (not shown here).
Derivation of the forward difference approximation of the second derivative
First define a TSE for a point two steps farther from \(x_i\):
Now multiply the TSE by two for a point one step farher from \(x_i\):
By substracting the first expression from the second one the first derivative dissapears:
By solving for \(f''\) we obtain the forward expression:
Higher-accuracy Finite-Difference Approximations#
So far we have found expressions with a relative large magnitude error \(\mathcal{O}(\Delta x)\) with the exception of the central difference approximation. Sometimes a higher accuracy is desired for which better expressions can be found. The procedure is similar to the algebraic manipulation to find the forward approximation of the second derivative: a series of TSE are defined at varying distances from \(x_i\) and after algebraic manipulation a more accurate expression is found. For example, the forward approximation of the first derivative:
The error magnitude has improved to \(\mathcal{O}(\Delta x^2)\) at the expense of using one more point. The accuracy can be even better by using more points. It is important to note that central differences are more accurate than forward and backward differences when using the same number of points.
Derive a first derivative \(f('x)\) with an 2nd error order \(\mathcal{O}(\Delta x^2)\) finite-difference equations, using the following nodes \(x_i\), \(x_i+\Delta x\) and \(x_i+2\Delta x\) or \(x_i\), \(x_{i+1} x\) and \(x_{i+2}\)
The finite-difference equations will have the following form:
Use the Taylor series to find \(\alpha\), \(\beta\) and \(\gamma\)
Solution
Taylor series for \(f(x_i+\Delta x)\) and \(f(x_i+2\Delta x)\) (Tip: first order derivative and second error order, 1+2=3, meaning a truncation until 3rd order ):
Times \(f(x_i+\Delta x)\) by 4 and expand the term \(\frac{(2\Delta x)^2}{2!}f''(x_i)\) in TSE \(f(x_i+2\Delta x)\):
Now take \(4f(x_i+\Delta x)-f(x_i+2\Delta x)\):
Rearrange for \(f'(x_i)\):
Divide by \(-2 \Delta x\):
The solution is: \(\alpha=-\frac{3}{2}, \beta= 2, \gamma= -\frac{1}{2}\)