1. Mathematical Preliminaries#
These notes are intended as a supplement and alternative perspective to [Arfken et al., 2013].
TL;DR#
Seriously: You must read and work through enough problems until you are comfortable with these techniques! Here is an overall strategy for checking if a series \(S = \sum_{n} a_n\) converges or diverges:
Look at the asymptotic form for large \(n\) and compare with the integral:
\[\begin{gather*} \int^{\infty} a_n\d{n}. \end{gather*}\]The series converges / diverges with the integral. This immediately tells you that the Harmonic series is divergent, and that the Riemann Zeta function \(\zeta(s)\) converges for \(s>1\).
Demonstration.
\[\begin{align*} \int^{N} \frac{1}{n}\d{n} &= \ln N \rightarrow \infty.\\ \int^{N} \frac{1}{n^{s}}\d{n} = \int^{N} n^{-s}\d{n} &= \frac{N^{-s+1}}{1-s} \rightarrow \begin{cases} \infty & s < 1,\\ \infty & s = 0 \text{ (see above)}\\ \text{finite} & s > 1. \end{cases},\\ \int^{N} \frac{1}{r^n}\d{n} = \int^{N} e^{-n\ln r}\d{n} &= \frac{-e^{-N\ln r}}{\ln r} \rightarrow \begin{cases} \infty & r \leq 1,\\ \text{finite} & r > 1, \end{cases}. \end{align*}\]To derive various tests compare to slowly converging or diverging series. The more slowly diverging, the more sensitive (but complicated) the test. The most sensitive test we consider is comparison with \(u_n = 1/(n\ln ^s n)\):
\[\begin{gather*} \frac{u_{n+1}}{u_{n}} \rightarrow \frac{(n+1)\ln^s(n+1)}{n\ln^s n} \rightarrow 1 - \frac{1}{n} - \frac{s}{n\ln n} + O(\tfrac{1}{n^2}). \end{gather*}\]
Common Series#
It is useful to recognize the following series:
Harmonic (divergent):
\[\begin{gather*} \sum_{n=1}^{\infty} \frac{1}{n} = \frac{1}{1} + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} \dots. \end{gather*}\]Related is the following, which diverges even more slowly for \(s=1\):
\[\begin{gather*} \sum_{n=2}^{\infty} \frac{1}{n(\ln n)^{s}}. \end{gather*}\]This diverges for \(s \leq 1\) and converges for \(s>1\).
Geometric (convergent for \(0 \leq \abs{r} < 1\)):
\[\begin{gather*} \sum_{n=1}^{\infty} \frac{1}{r^n} = 1 + r + r^2 + r^3 = \dots = \frac{1}{1-r}. \end{gather*}\]Riemann Zeta function shows up in various places, and has known values for even integers:
\[\begin{gather*} \zeta(s) = \sum_{n=1}^{\infty} \frac{1}{n^s}, \qquad \zeta(2) = \frac{\pi^2}{6}, \qquad \zeta(4) = \frac{\pi^4}{90}, \end{gather*}\]and some other particular values.
Convergence Tests#
Here we consider absolute convergence of the series
The easiest way of testing convergence is by comparing with another series known to be convergence or divergent. Comparison with the geometric series \(u_n = r^n\) gives the Cauchy Root Test:
\[\begin{gather*} a_n^{1/n} \leq r < 1, \end{gather*}\]and (slightly less powerful) D’Alembert (or Cauchy) Ratio Test:
\[\begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow r, \qquad \begin{cases} r < 1 & \text{converge},\\ r = 1 & \text{inconclusive},\\ r > 1 & \text{diverge}. \end{cases} \end{gather*}\]The inconclusive case of \(r=1\) can be refined by comparing with \(\zeta(s)\) where \(u_n = n^s\), which converges (diverges) for \(s>1\) (\(s>1\)) giving a more sensitive test:
\[\begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{s}{n}, \qquad \begin{cases} s > 1 & \text{converge},\\ s = 1 & \text{inconclusive},\\ s < 1 & \text{diverge}. \end{cases} \end{gather*}\]Again, this is inconclusive for \(s=1\), but comparison with \(u_n = 1/(n\ln^t n)\) gives an even more sensitive test
\[\begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{1}{n} - \frac{t}{n \ln n}, \qquad \begin{cases} t > 1 & \text{converge},\\ t = 1 & \text{inconclusive},\\ t < 1 & \text{diverge}. \end{cases} \end{gather*}\]Or, in summary:
\[\begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow r - \frac{s}{n} - \frac{t}{n\ln n}, \qquad \begin{cases} r < 1 & \text{convergence}\\ r > 1 & \text{divergence}\\ r = 1 & \begin{cases} s > 1 & \text{convergence}\\ s < 1 & \text{divergence}\\ s = 1 & \begin{cases} t > 1 & \text{convergence}\\ t < 1 & \text{divergence}\\ t = 1 & \text{inconclusive} \end{cases} \end{cases} \end{cases} \end{gather*}\]Details
The geometric series \(u_n = r^n\) converges for \(\abs{r}<1\), so if \(a_n \leq r^n < 1\) for \(n>N\), then by comparison, the series is absolutely convergent. This directly gives the Cauchy Root Test and also proves the D’Alembert (or Cauchy) Ratio Test.
Comparing with \(u_n = 1/(n\ln^s n)\) where \(s>1\) gives
\[\begin{gather*} \lim_{n\rightarrow \infty} \frac{u_{n+1}}{u_n} = \frac{n}{n+1}\frac{\ln^s n}{\ln^s (n+1)} = \frac{1}{1 + \frac{1}{n}}\frac{1}{\Bigl(1 + \frac{\ln(1+\tfrac{1}{n})}{\ln n}\Bigr)^s}\\ = \left(1 - \frac{1}{n} + O(\tfrac{1}{n^2})\right) \left(1 - s\frac{1}{n \ln n } + O(\tfrac{1}{n^2\ln n})\right)\\ = 1 - \frac{1}{n} - s\frac{1}{n \ln n} + O(\tfrac{1}{n^2}). \end{gather*}\]We use these results to do the expansion:
\[\begin{gather*} \frac{1}{(1+\epsilon)^s} = 1 - s\epsilon + O(\epsilon^2),\\ \ln(n+1) = \ln n(1+\tfrac{1}{n}) = \ln n + \ln(1+\tfrac{1}{n}),\\ \ln(1+\epsilon) = -\epsilon + O(\epsilon^2). \end{gather*}\]Another easy way to check for convergence is to compare to an integral. If \(a_n = f(n)\) with \(f(n)\) monotonically decreasing, then \(S\) converges or diverges a the integral
\[\begin{gather*} \int^{\infty}f(n)\d{n}, \qquad 0 \leq S - \int^{\infty}_{n_0}f(n)\d{n} \leq a_{n_0}. \end{gather*}\]This allows us to establish the divergence of the harmonic series and the convergence of \(\zeta(s)\) and \(u_n = 1/(n\ln^s n)\) for \(s > 1\):
\[\begin{gather*} \int^{N}\frac{\d{n}}{n} \rightarrow \ln N,\qquad \int^{N}\frac{\d{n}}{n^s} \rightarrow \frac{1}{(1-s)N^{s-1}},\qquad \int^{N}\frac{\d{n}}{n(\ln n)^s} \rightarrow \frac{1}{(1-s)\ln^{s-1} N}. \end{gather*}\]Details
Let \(x = \ln n\), \(\d{x} = \d{n}/n\):
\[\begin{gather*} \int^{N}\d{n}\frac{1}{n(\ln n)^s} = \int^{\ln N}\d{x}\; x^{-s} \rightarrow \frac{\ln^{1-s}N}{1-s}. \end{gather*}\]Example: an even more stringent test
The previous examples suggest that we might construct a more test by choosing an integral that converges even more slowly:
\[\begin{gather*} \int^N \d{n}\; f(n) \rightarrow \frac{1}{\ln^{s-1}(\ln N)},\qquad s > 1,\\ u_n = \frac{1-s}{(n \ln{n})\bigl(\ln(\ln n)\bigr)^{s}}\\ \frac{u_{n+1}}{u_n} \rightarrow 1 - \frac{1}{n} - \frac{1}{n \ln n} - \frac{s}{(n \ln n) \bigl(\ln(\ln n)\bigr)}, \end{gather*}\]hence, the series \(S\) converges if
\[\begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{1}{n} - \frac{1}{n \ln n} - \frac{s}{(n \ln n) \bigl(\ln(\ln n)\bigr)} , \qquad s > 1. \end{gather*}\]
These tests should deal with most of the series you might encounter in physics.
Familiar Power Series#
You should be familiar with the following McLauren series:
Details
The following are used to derive these:
Less Familiar Power Series#
The Bernoulli numbers \(B_n\) appear in
Example: \(\tan x\) and \(\cot x\).
Let \(y = 2\I x\):
Here we use the fact that \(B_{1} = -1/2\) and \(B_{n} = 0\) for all other odd \(n\). The series for \(\tan x\) follows from
Manipulating Power Series#
Some useful techniques include relating a simple series to a power series evaluated at a specific point. The later might be manipulated by integration or differentiation.
Example: \(S = \frac{1}{2!} + \frac{2}{3!} + \frac{3}{4!} + \cdots\)
This looks almost like the series for \(e^x\). With a few manipulations we can get there quite easily:
Formal Power Series#
One is generally interested in power series that converge, however, one can consider manipulations and formulae related to the coefficients of a formal power series without consideration of convergence. These should be thought of as algebraic manipulations of the coefficients, expressing the results of manipulations like multiplying, series, inverting series, differentiating series, etc. Some of the results here and in the book can be studied in this framework.
Inversion#
We can invert a power series \(y = a_1x + a_2x^2 + \cdots\) locally if \(a_1 \neq 0\) by brute force as shown in the book
The book mentioned another technique based on complex analysis described in the 1st and 2nd editions, but I cannot find these to check. I expect this is based on the Lagrange inversion theorem which states that if
is analytic at \(w=0\) and \(f'(0) \neq 0\), then we can locally invert this to find a function \(g = f^{-1}\) where \(w = g(z)\):
Example: Lambert \(W\) function \(we^w = z\).
Though not a power series, the Lagrange inversion theorem is helpful for computing a series expansion for the Lambert \(W\) function \(W(z)\):
We need to compute
so we have
Improving Convergence#
Several strategies exist for improving convergence:
Subtracting a known series (comparing with an integral can sometimes help).
Expanding in partial fractions may allow you to identify parts of a series, summing them analytically.
Accelerating convergence. Please see Appendix A of [Bornemann et al., 2004] for a great description of many acceleration techniques. The method due to Euler describe in the text can be explained more generally as follows:
To accelerate a series \(f(x)\), consider the related series \(g(x)\) whose sum is known. We relate the two series by coefficients \(c_n\):
\[\begin{gather*} g(x) = \sum_{n=0}^{\infty} b_n x^n, \qquad f(x) = \sum_{n=0}^{\infty} c_nb_n x^n. \end{gather*}\]One can solve this terms by term to eliminate \(b_n\) to obtain
\[\begin{gather*} f(x) = c_0 g(x) + (c_1-c_0)xg'(x) + (c_1-2c_1+c_0)\frac{x^2g''(x)}{2!} + (c_3 - 3c_2 + 3c_1 - c_0)\frac{x^3 g''(x)}{3!} + \cdots. \end{gather*}\]The successive terms in this series are the leading differences in the difference table for the coefficients \(c_n\):
\[\begin{align*} c_0 \qquad & \\ & c_1 - c_0\\ c_1 \qquad & & c_2 - 2c_1 + c_0 &\\ & c_2 - c_1 & & \qquad c_3 - 3c_2 + 3c_1 - c_0\\ c_2 \qquad & & c_3 - 2c_2 + c_1 &\\ & c_3 - c_2 &\\ c_3 \qquad \end{align*}\]Comparison with \(g(x) = 1/(1+x)\) where \(b_n = (-1)^n\) yields Euler’s method described in the book.