--- jupytext: formats: ipynb,md:myst text_representation: extension: .md format_name: myst format_version: 0.13 jupytext_version: 1.13.6 kernelspec: display_name: Python 3 (phys-571) language: python name: phys-571 --- ```{code-cell} :tags: [hide-cell] %pylab inline import numpy as np, matplotlib.pyplot as plt ``` (sec:MathematicalPreliminaries)= # 1. Mathematical Preliminaries These notes are intended as a supplement and alternative perspective to {cite}`Arfken:2013`. ## TL;DR Seriously: You must read and work through enough problems until you are comfortable with these techniques! Here is an overall strategy for checking if a series $S = \sum_{n} a_n$ converges or diverges: 1. Look at the asymptotic form for large $n$ and compare with the integral: \begin{gather*} \int^{\infty} a_n\d{n}. \end{gather*} The series converges / diverges with the integral. This immediately tells you that the Harmonic series is divergent, and that the Riemann Zeta function $\zeta(s)$ converges for $s>1$. :::{admonition} Demonstration. :class: dropdown \begin{align*} \int^{N} \frac{1}{n}\d{n} &= \ln N \rightarrow \infty.\\ \int^{N} \frac{1}{n^{s}}\d{n} = \int^{N} n^{-s}\d{n} &= \frac{N^{-s+1}}{1-s} \rightarrow \begin{cases} \infty & s < 1,\\ \infty & s = 0 \text{ (see above)}\\ \text{finite} & s > 1. \end{cases},\\ \int^{N} \frac{1}{r^n}\d{n} = \int^{N} e^{-n\ln r}\d{n} &= \frac{-e^{-N\ln r}}{\ln r} \rightarrow \begin{cases} \infty & r \leq 1,\\ \text{finite} & r > 1, \end{cases}. \end{align*} ::: 2. To derive various tests compare to slowly converging or diverging series. The more slowly diverging, the more sensitive (but complicated) the test. The most sensitive test we consider is comparison with $u_n = 1/(n\ln ^s n)$: \begin{gather*} \frac{u_{n+1}}{u_{n}} \rightarrow \frac{(n+1)\ln^s(n+1)}{n\ln^s n} \rightarrow 1 - \frac{1}{n} - \frac{s}{n\ln n} + O(\tfrac{1}{n^2}). \end{gather*} ## Common Series It is useful to recognize the following series: * Harmonic *(divergent)*: \begin{gather*} \sum_{n=1}^{\infty} \frac{1}{n} = \frac{1}{1} + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} \dots. \end{gather*} Related is the following, which diverges even more slowly for $s=1$: \begin{gather*} \sum_{n=2}^{\infty} \frac{1}{n(\ln n)^{s}}. \end{gather*} This diverges for $s \leq 1$ and converges for $s>1$. * Geometric *(convergent for $0 \leq \abs{r} < 1$)*: :::{margin} Let $G = 1 + r + r^2 + \dots$: \begin{gather*} G = 1+\underbrace{rG}_{r + r^2 + r^3 + \dots}\\ G = \frac{1}{1-r}. \end{gather*} ::: \begin{gather*} \sum_{n=1}^{\infty} \frac{1}{r^n} = 1 + r + r^2 + r^3 = \dots = \frac{1}{1-r}. \end{gather*} * [Riemann Zeta function](https://www.3blue1brown.com/lessons/zeta) shows up in various places, and has known values for even integers: \begin{gather*} \zeta(s) = \sum_{n=1}^{\infty} \frac{1}{n^s}, \qquad \zeta(2) = \frac{\pi^2}{6}, \qquad \zeta(4) = \frac{\pi^4}{90}, \end{gather*} and some other [particular values](https://en.wikipedia.org/wiki/Particular_values_of_the_Riemann_zeta_function). ## Convergence Tests Here we consider absolute convergence of the series \begin{gather*} S = \sum_{n=n_0}^{\infty} a_n. \end{gather*} :::{margin} When not specified, the expressions are for sufficiently large $n>N$ and the conditions are for convergence. ::: 1. The easiest way of testing convergence is by comparing with another series known to be convergence or divergent. Comparison with the geometric series $u_n = r^n$ gives the **Cauchy Root Test**: \begin{gather*} a_n^{1/n} \leq r < 1, \end{gather*} and (slightly less powerful) **D'Alembert (or Cauchy) Ratio Test**: \begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow r, \qquad \begin{cases} r < 1 & \text{converge},\\ r = 1 & \text{inconclusive},\\ r > 1 & \text{diverge}. \end{cases} \end{gather*} The inconclusive case of $r=1$ can be refined by comparing with $\zeta(s)$ where $u_n = n^s$, which converges (diverges) for $s>1$ ($s>1$) giving a more sensitive test: \begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{s}{n}, \qquad \begin{cases} s > 1 & \text{converge},\\ s = 1 & \text{inconclusive},\\ s < 1 & \text{diverge}. \end{cases} \end{gather*} Again, this is inconclusive for $s=1$, but comparison with $u_n = 1/(n\ln^t n)$ gives an even more sensitive test \begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{1}{n} - \frac{t}{n \ln n}, \qquad \begin{cases} t > 1 & \text{converge},\\ t = 1 & \text{inconclusive},\\ t < 1 & \text{diverge}. \end{cases} \end{gather*} Or, in summary: \begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow r - \frac{s}{n} - \frac{t}{n\ln n}, \qquad \begin{cases} r < 1 & \text{convergence}\\ r > 1 & \text{divergence}\\ r = 1 & \begin{cases} s > 1 & \text{convergence}\\ s < 1 & \text{divergence}\\ s = 1 & \begin{cases} t > 1 & \text{convergence}\\ t < 1 & \text{divergence}\\ t = 1 & \text{inconclusive} \end{cases} \end{cases} \end{cases} \end{gather*} :::::{admonition} Details :class: dropdown The geometric series $u_n = r^n$ converges for $\abs{r}<1$, so if $a_n \leq r^n < 1$ for $n>N$, then by comparison, the series is absolutely convergent. This directly gives the **Cauchy Root Test** and also proves the **D'Alembert (or Cauchy) Ratio Test**. Comparing with $u_n = 1/(n\ln^s n)$ where $s>1$ gives \begin{gather*} \lim_{n\rightarrow \infty} \frac{u_{n+1}}{u_n} = \frac{n}{n+1}\frac{\ln^s n}{\ln^s (n+1)} = \frac{1}{1 + \frac{1}{n}}\frac{1}{\Bigl(1 + \frac{\ln(1+\tfrac{1}{n})}{\ln n}\Bigr)^s}\\ = \left(1 - \frac{1}{n} + O(\tfrac{1}{n^2})\right) \left(1 - s\frac{1}{n \ln n } + O(\tfrac{1}{n^2\ln n})\right)\\ = 1 - \frac{1}{n} - s\frac{1}{n \ln n} + O(\tfrac{1}{n^2}). \end{gather*} We use these results to do the expansion: \begin{gather*} \frac{1}{(1+\epsilon)^s} = 1 - s\epsilon + O(\epsilon^2),\\ \ln(n+1) = \ln n(1+\tfrac{1}{n}) = \ln n + \ln(1+\tfrac{1}{n}),\\ \ln(1+\epsilon) = -\epsilon + O(\epsilon^2). \end{gather*} ::::: 2. Another easy way to check for convergence is to compare to an integral. If $a_n = f(n)$ with $f(n)$ monotonically decreasing, then $S$ converges or diverges a the integral \begin{gather*} \int^{\infty}f(n)\d{n}, \qquad 0 \leq S - \int^{\infty}_{n_0}f(n)\d{n} \leq a_{n_0}. \end{gather*} This allows us to establish the divergence of the harmonic series and the convergence of $\zeta(s)$ and $u_n = 1/(n\ln^s n)$ for $s > 1$: \begin{gather*} \int^{N}\frac{\d{n}}{n} \rightarrow \ln N,\qquad \int^{N}\frac{\d{n}}{n^s} \rightarrow \frac{1}{(1-s)N^{s-1}},\qquad \int^{N}\frac{\d{n}}{n(\ln n)^s} \rightarrow \frac{1}{(1-s)\ln^{s-1} N}. \end{gather*} :::{admonition} Details :class: dropdown Let $x = \ln n$, $\d{x} = \d{n}/n$: \begin{gather*} \int^{N}\d{n}\frac{1}{n(\ln n)^s} = \int^{\ln N}\d{x}\; x^{-s} \rightarrow \frac{\ln^{1-s}N}{1-s}. \end{gather*} ::: :::{admonition} Example: an even more stringent test :class: dropdown The previous examples suggest that we might construct a more test by choosing an integral that converges even more slowly: \begin{gather*} \int^N \d{n}\; f(n) \rightarrow \frac{1}{\ln^{s-1}(\ln N)},\qquad s > 1,\\ u_n = \frac{1-s}{(n \ln{n})\bigl(\ln(\ln n)\bigr)^{s}}\\ \frac{u_{n+1}}{u_n} \rightarrow 1 - \frac{1}{n} - \frac{1}{n \ln n} - \frac{s}{(n \ln n) \bigl(\ln(\ln n)\bigr)}, \end{gather*} hence, the series $S$ converges if \begin{gather*} \Abs{\frac{a_{n+1}}{a_{n}}} \rightarrow 1 - \frac{1}{n} - \frac{1}{n \ln n} - \frac{s}{(n \ln n) \bigl(\ln(\ln n)\bigr)} , \qquad s > 1. \end{gather*} ::: These tests should deal with most of the series you might encounter in physics. ## Familiar Power Series You should be familiar with the following McLauren series: \begin{alignat*}{4} (1+x)^n &= 1 + nx + n(n-1)\frac{x^2}{2!} + \cdots &&= \sum_{m=0}^{\infty}\frac{n!}{(n-m)!}\frac{x^{m}}{m!} = \sum_{m=0}^{\infty}\binom{n}{m}x^{m},\\ e^{x} &= 1 + \frac{x}{1!} + \frac{x^2}{2!} + \cdots &&= \sum_{m=0}^{\infty}\frac{x^{m}}{m!}\\ \sin x &= x - \frac{x^3}{3!} + \frac{x^5}{5!} - \cdots &&= \sum_{m=0}^{\infty}\frac{(-1)^{m}x^{2m+1}}{(2m+1)!},\\ \cos x &= 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \cdots &&= \sum_{m=0}^{\infty}\frac{(-1)^{m}x^{2m}}{(2m)!},\\ \ln(1+x) &= x - \frac{x^2}{2} + \frac{x^3}{3} - \frac{x^4}{4} + \cdots &&= \sum_{m=1}^{\infty}\frac{(-x)^{m}}{m},\\ \tan^{-1}x &= x - \frac{x^3}{3} + \frac{x^5}{5} - \frac{x^7}{7} + \cdots &&= \sum_{m=0}^{\infty}\frac{(-1)^{m}x^{2m+1}}{(2m+1)}. \end{alignat*} :::::{admonition} Details :class: dropdown The following are used to derive these: \begin{gather*} \diff{e^x}{x} = e^{x}, \qquad e^0 = 1,\\ e^{\I x} = \underbrace{\left(1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \cdots\right)}_{\cos x} + \I\underbrace{\left(x - \frac{x^3}{3!} + \frac{x^5}{5!} - \cdots\right)}_{\sin x},\\ \ln(1+x) = \int_{0}^{x} \frac{1}{1+x}\d{x},\qquad \tan^{-1}x = \int_{0}^{x} \frac{1}{(1+x)^2}\d{x}. \end{gather*} ::::: ## Less Familiar Power Series :::{margin} This series appears in the Bose-Einstein occupancy factor: \begin{gather*} n_{B}(E) = \frac{1}{e^{E/k_B T} - 1}. \end{gather*} See problem 1.3.14 in {cite}`Arfken:2013`. ::: The [Bernoulli numbers](https://en.wikipedia.org/wiki/Bernoulli_number) $B_n$ appear in \begin{gather*} \frac{x}{e^{x}-1} = \sum_{n=0}^{\infty} B_n\frac{x^n}{n!}. \end{gather*} :::{admonition} Example: $\tan x$ and $\cot x$. :class: dropdown Let $y = 2\I x$: \begin{gather*} \cot x = \frac{\cos x}{\sin x} = \I \frac{e^{\I x} + e^{-\I x}}{e^{\I x} - e^{-\I x}} = \I \left(1 - 1 + \frac{e^{\I x} + e^{-\I x}}{e^{\I x} - e^{-\I x}}\right)\\ = \I \left(1 + \frac{2 e^{-\I x}}{e^{\I x} - e^{-\I x}}\right) = \I \left(1 + \frac{2}{e^{2\I x} - 1}\right) = \I \left(1 + \frac{2}{e^{y} - 1}\right)\\ = \frac{2\I}{y} \left(\frac{y}{2} + \frac{y}{e^{y} - 1}\right) = \frac{2\I}{y} \sum_{n \text{ even}}\frac{B_n y^n}{n!} = \frac{1}{x} \sum_{m=0}^{\infty} (-1)^{m}\frac{B_{2m} (2x)^{2m}}{(2m)!}. \end{gather*} Here we use the fact that $B_{1} = -1/2$ and $B_{n} = 0$ for all other odd $n$. The series for $\tan x$ follows from \begin{gather*} \tan x = \cot x - 2\cot 2x = \frac{\cos x}{ \sin x} - \frac{2\cos 2 x}{\sin 2x} = \frac{\cos x}{ \sin x} - \frac{2(\cos^2 x - \sin^2 x)}{2\cos x \sin x}\\ = \frac{2\cos^2 x - 2\cos^2 x + 2\sin^2 x)}{2\cos x \sin x} = \frac{\sin x}{\cos x}\\ = \frac{1}{x} \sum_{m=0}^{\infty} (-1)^{m}(1-2^{2m})B_{2m}\frac{(2x)^{2m}}{(2m)!}\\ = \sum_{m=1}^{\infty} \frac{(-1)^{m-1}4^{m}(4^{m}-1)B_{2m}}{(2m)!}x^{2m-1}\\ \end{gather*} ::: ## Manipulating Power Series Some useful techniques include relating a simple series to a power series evaluated at a specific point. The later might be manipulated by integration or differentiation. :::{admonition} Example: $S = \frac{1}{2!} + \frac{2}{3!} + \frac{3}{4!} + \cdots$ :class: dropdown This looks almost like the series for $e^x$. With a few manipulations we can get there quite easily: \begin{gather*} S = \frac{1}{2!} + \frac{2}{3!} + \frac{3}{4!} + \cdots\\ f(x) = \frac{1}{2!} + \frac{2x}{3!} + \frac{3x^2}{4!} + \cdots, \qquad S = f(1),\\ \int_0^{x} f(x) \d{x} = \frac{x}{2!} + \frac{x^2}{3!} + \frac{x^3}{4!} + \cdots = \frac{e^{x} - 1}{x}\\ f(x) = \diff{}{x}\frac{e^{x} - 1}{x} = \frac{xe^x-(e^x-1)}{x^2}, \qquad f(1) = \frac{1e^1-(e^1-1)}{1^2} = 1. \end{gather*} ::: ## Formal Power Series One is generally interested in power series that converge, however, one can consider manipulations and formulae related to the coefficients of a [formal power series][] without consideration of convergence. These should be thought of as algebraic manipulations of the coefficients, expressing the results of manipulations like multiplying, series, inverting series, differentiating series, etc. Some of the results here and in the book can be studied in this framework. :::{margin} The book considers inversion about an arbitrary point $y_0 = f(x_0)$: \begin{gather*} y-y_0 = \sum_{n=1}^{\infty} a_n(x-x_0)^n. \end{gather*} I.e. $a_0 = y_0$. To simplify notation here, we transform $x \rightarrow x-x_0$ and $y \rightarrow y - y_0$. Likewise, for the [Lagrange inversion theorem][] about $z = f(w)$ which is analytic about $w=a$, we take $a = 0$ and $f(a) = 0$. ::: ## Inversion We can invert a power series $y = a_1x + a_2x^2 + \cdots$ locally if $a_1 \neq 0$ by brute force as shown in the book \begin{gather*} y = \sum_{n=1}^{\infty} a_n x^n, \qquad x = \sum_{n=1}^{\infty} b_n y^n,\\ b_1 = \frac{1}{a_1}, \qquad b_2 = - \frac{a_2}{a_1^3}, \qquad \cdots. \end{gather*} The book mentioned another technique based on complex analysis described in the 1st and 2nd editions, but I cannot find these to check. I expect this is based on the [Lagrange inversion theorem][] which states that if \begin{gather*} z = f(w) \end{gather*} is analytic at $w=0$ and $f'(0) \neq 0$, then we can locally invert this to find a function $g = f^{-1}$ where $w = g(z)$: \begin{gather*} w = g(z) = \sum_{n=1}^{\infty} g_n \frac{z^n}{n!}, \qquad g_n = \lim_{w\rightarrow 0} \diff[n-1]{}{w}\left(\frac{w}{f(w)}\right)^n \end{gather*} :::::{admonition} Example: [Lambert $W$ function][] $we^w = z$. Though not a power series, the [Lagrange inversion theorem][] is helpful for computing a series expansion for the [Lambert $W$ function][] $W(z)$: \begin{gather*} w = W(z), \qquad z = f(w) = we^w. \end{gather*} We need to compute \begin{gather*} \diff[n-1]{}{w}\left(\frac{w}{we^w}\right)^n = \diff[n-1]{e^{-nw}}{w} = (-n)^{n-1}e^{-nw}, \end{gather*} so we have \begin{gather*} g_n = (-n)^{n-1}, \qquad W(z) = \sum_{n=1}^{\infty} \frac{(-n)^{n-1}z^{n}}{n!} = z - z^{2} + \frac{3z^{3}}{2} -\frac{8z^{4}}{3} + \dots \end{gather*} ::::: [formal power series]: [Lagrange inversion theorem]: [Lambert $W$ function]: ## Improving Convergence Several strategies exist for improving convergence: * Subtracting a known series (comparing with an integral can sometimes help). * Expanding in partial fractions may allow you to identify parts of a series, summing them analytically. :::{margin} I first learned about these in the book {cite}`Bornemann:2004rp`. I highly recommend trying to solve the problems in [Nick Trefethen's 100-digit Challenge](https://en.wikipedia.org/wiki/Hundred-dollar,_Hundred-digit_Challenge_problems) and then reading the book, which has solutions. ::: * Accelerating convergence. Please see [Appendix A][] of {cite}`Bornemann:2004rp` for a great description of many acceleration techniques. The method due to Euler describe in the text can be explained more generally as follows: To accelerate a series $f(x)$, consider the related series $g(x)$ whose sum is known. We relate the two series by coefficients $c_n$: \begin{gather*} g(x) = \sum_{n=0}^{\infty} b_n x^n, \qquad f(x) = \sum_{n=0}^{\infty} c_nb_n x^n. \end{gather*} One can solve this terms by term to eliminate $b_n$ to obtain \begin{gather*} f(x) = c_0 g(x) + (c_1-c_0)xg'(x) + (c_1-2c_1+c_0)\frac{x^2g''(x)}{2!} + (c_3 - 3c_2 + 3c_1 - c_0)\frac{x^3 g''(x)}{3!} + \cdots. \end{gather*} The successive terms in this series are the leading differences in the difference table for the coefficients $c_n$: \begin{align*} c_0 \qquad & \\ & c_1 - c_0\\ c_1 \qquad & & c_2 - 2c_1 + c_0 &\\ & c_2 - c_1 & & \qquad c_3 - 3c_2 + 3c_1 - c_0\\ c_2 \qquad & & c_3 - 2c_2 + c_1 &\\ & c_3 - c_2 &\\ c_3 \qquad \end{align*} Comparison with $g(x) = 1/(1+x)$ where $b_n = (-1)^n$ yields Euler's method described in the book. [Appendix A]: