Dirac Delta Function: Series Approximation

Weierstrass Theorem

Understanding the Dirac delta function as a limit of functions can lead to rich content, and these concepts are not far from rigorous proofs. For example, define: \delta_n(x)=\left\{\begin{aligned}&\frac{(1-x^2)^n}{I_n}, & x\in[-1,1]\\ &0, & \text{otherwise}\end{aligned}\right. where I_n = \int_{-1}^1 (1-x^2)^n dx. It is not difficult to prove that: \delta(x)=\lim_{n\to\infty}\delta_n(x) Thus, for a continuous function f(x) on [a,b], we obtain: f(x)=\int_{-1}^1 f(y)\delta(x-y)dy = \lim_{n\to\infty}\int_{-1}^1 f(y)\delta_n(x-y) dy Here -1 < a < b < 1. Although we have "non-rigorously" swapped the integral and limit signs, this is not particularly critical. What is important is the result: we can see that P_n(x)=\int_{-1}^1 f(y)\delta_n(x-y) dy is a polynomial in x of degree 2n. Therefore, the above expression indicates that f(x) is the limit of a polynomial of degree 2n! This leads to the famous "Weierstrass Theorem":

Any continuous function on a closed interval can be uniformly approximated by polynomials.

Rigorous readers might shake their heads: what kind of proof is this? Indeed, this is not a formal proof, but how far is it from one? For those interested in a rigorous proof, one might consider reading Professor Qi Minyou’s Revisiting Calculus. You will find that a rigorous proof actually involves a more detailed discussion and estimation (scaling) of the formulas above, and such a discussion is not difficult. Therefore, although the above is a non-rigorous guide, it is not far from a formal proof. All of this is thanks to our non-rigorous understanding of \delta(x) as the limit of \delta_n(x), rather than as a strict generalized function.

The process above actually demonstrates a framework that allows us to analyze what function bases can be used to approximate a function. Constructing a sequence of functions \{\delta_n(x)\} such that it serves as a limit for \delta(x) is not difficult: first is normalization, and then ensuring that the values on both sides of x=0 tend to 0 as n\to\infty. Here is another example: \delta_n(x)=\left\{\begin{aligned}&\frac{\cos^n x}{I_n}, & x\in\left[-\frac{\pi}{2},\frac{\pi}{2}\right]\\ &0, & \text{otherwise}\end{aligned}\right. where I_n = \int_{-\pi/2}^{\pi/2} \cos^n x dx. Similarly, we have: \delta(x)=\lim_{n\to\infty}\delta_n(x) What results does this limit bring? Following the previous discussion on polynomials, assume -\frac{\pi}{2} < a < b < \frac{\pi}{2}. For a continuous function f(x) on [a,b], we see: \int_{-\pi/2}^{\pi/2} f(y)\delta_n(x-y) dy = \frac{1}{I_n} \int_{-\pi/2}^{\pi/2} f(y)\cos^n (x-y) dy Expanding \cos^n(x-y) reveals that the above expression is actually a linear combination of \sin kx, \cos kx for k=0,1,\dots,n. Thus, we can also obtain:

Any continuous function on the closed interval [-\pi/2, \pi/2] can be uniformly approximated by trigonometric series \{\cos kx, \sin kx\}.

This is also known as the Weierstrass Theorem. Similarly, the above discussion is not far from a rigorous proof.

Does something feel slightly off? The period of \cos x, \sin x is 2\pi, yet we only approximated an interval of length \pi ([-\pi/2, \pi/2]). Isn’t that a bit wasteful? In fact, by slightly modifying the process above, one can prove:

For any \epsilon > 0, any continuous function on the closed interval [-\pi+\epsilon, \pi-\epsilon] can be uniformly approximated by trigonometric series \{\cos kx, \sin kx\}.

However, this does not hold for [-\pi, \pi]; in that case, it can only achieve convergence in measure or L^2 norm convergence. As is well known, the Fourier basis is not uniformly convergent for all continuous functions.

Orthonormal Basis

"Orthogonality" helps simplify problems. In geometry, we prefer orthogonal coordinate systems; for function problems, we prefer orthogonal bases.

The two previous Weierstrass theorems tell us that continuous functions on a closed interval can be uniformly approximated by both polynomials and trigonometric series. Furthermore, the process above actually provides a scheme for finding the approximation coefficients for each term. However, from a practical calculation standpoint, the above method is impractical because moving from an n-th order approximation to an n+1-th order approximation requires recalculating all coefficients (by contrast, consider Taylor series approximation, where moving from n to n+1 only requires calculating one additional term f^{(n+1)}(x), which is very economical; however, Taylor series requirements are too strong). Therefore, we need to seek more efficient calculation schemes.

We restrict our discussion to a closed interval [a,b]. For two functions f(x), g(x), we define their inner product as: \langle f, g\rangle=\int_a^b f(x)g(x)dx Suppose the sequence of functions \{e_k (x)\}, k=0,1,2,\dots is a set of basis functions satisfying: \langle e_m, e_n\rangle=\delta_{mn} This is called an orthonormal basis. With this, we can describe the "optimal approximation" of a function.

Suppose we have a function f(x) on the closed interval [a,b], and we know that f(x) can be uniformly approximated by the orthonormal basis \{e_k (x)\}, k=0,1,2,\dots. Let us consider a finite sum: \sum_{k=0}^n \alpha_k e_k(x) We want to use this finite series to optimally approximate f(x), meaning we want to minimize the error, defined as: E=\int_a^b \left(f(x)-\sum_{k=0}^n \alpha_k e_k(x)\right)^2 dx To be "optimal" means adjusting \alpha_0, \alpha_1, \dots, \alpha_n to minimize E. Since it is a finite series, we can directly take partial derivatives with respect to each \alpha_k and solve to find that E is minimized when: \alpha_k = \int_a^b f(x)e_k (x)dx Thus, we have obtained the optimal approximation. We can see that if we move from an n-th order approximation to an n+1-th order approximation, we only need to calculate one additional term \alpha_{n+1} = \int_a^b f(x)e_{n+1} (x)dx, without recalculating all previous coefficients.

Note that if \{e_k (x)\} is an arbitrary sequence of functions (not necessarily a basis or orthogonal), the same optimization process can be performed, but there is no guarantee that the final result will actually approximate the original function. That is, it is possible that no matter how much you increase n, the final approximation error does not decrease. However, if it has already been proven that \{e_k (x)\} can uniformly approximate the original function, then it is guaranteed that the error from the optimization process above tends to 0—because it is already the optimal solution; if the optimal solution does not tend to 0, how could there be uniform convergence to the original function?

Fourier Series

The question now is: how do we obtain a set of orthonormal bases? According to the Weierstrass Theorem, we already know that power functions 1, x, x^2, \dots and trigonometric functions 1, \sin x, \cos x, \sin 2x, \cos 2x, \dots can serve as bases. Let’s focus on the latter. In the interval [-\pi, \pi], one can verify that 1, \sin x, \cos x, \sin 2x, \cos 2x, \dots are orthogonal, i.e., for any natural numbers m, n: \left\{\begin{aligned}&\int_{-\pi}^{\pi} \cos mx \cos nx dx = 0 \quad (m\neq n)\\ &\int_{-\pi}^{\pi} \sin mx \sin nx dx = 0 \quad (m\neq n)\\ &\int_{-\pi}^{\pi} \sin mx \cos nx dx = 0 \end{aligned}\right. And we have: \int_{-\pi}^{\pi} \sin^2 mx dx = \int_{-\pi}^{\pi} \cos^2 mx dx = \pi Therefore: \sqrt{\frac{1}{2\pi}}, \sqrt{\frac{1}{\pi}}\sin x, \sqrt{\frac{1}{\pi}}\cos x, \sqrt{\frac{1}{\pi}}\sin 2x, \sqrt{\frac{1}{\pi}}\cos 2x, \dots is a set of orthonormal bases. Then we have: f(x) = \gamma + \sum_{n=1}^{\infty} \alpha_n \sin nx + \sum_{n=1}^{\infty} \beta_n \cos nx where: \begin{aligned}\gamma=&\frac{1}{2\pi}\int_{-\pi}^{\pi} f(x)dx,\\ \alpha_n=&\frac{1}{\pi}\int_{-\pi}^{\pi} f(x)\sin nx dx,\\ \beta_n=&\frac{1}{\pi}\int_{-\pi}^{\pi} f(x)\cos nx dx \end{aligned} This is the Fourier series. However, because we are considering the interval [-\pi, \pi], the Fourier series only converges in measure or in L^2 norm, rather than uniformly. In the interval [-\pi/2, \pi/2], the set 1, \sin x, \cos x, \sin 2x, \cos 2x, \dots is not orthogonal and would require an orthogonalization procedure.

Orthogonal Polynomials

We have already explored the approximation problem using trigonometric series, resulting in Fourier series. So, how do we handle the other set of bases from the Weierstrass Theorem—the power function basis? Trigonometric functions have natural orthogonality which simplifies the problem, but power functions do not. Therefore, we must perform an orthogonalization procedure.

The most well-known orthogonalization process is the Gram-Schmidt process, which is described in basic linear algebra textbooks and will not be repeated here. For the basis \{x^n\}, n=0,1,2,\dots, the result of performing Schmidt orthogonalization on the interval [-1,1] is:

These polynomials P_n(x) are called "Legendre polynomials." They are orthogonal but not yet normalized. After normalizing them to \hat{P}_n(x), we can write a result similar to the Fourier series: f(x)=\sum_{n=0}^{\infty} \alpha_n \hat{P}_n (x) where: \alpha_n = \int_{-1}^1 f(x)\hat{P}_n (x) dx

Other Series Approximations

Trigonometric approximation can also be viewed as an approximation using the complex exponential basis e^{ikx}, k=0,\pm 1,\pm 2,\dots. A natural question is: can we use real exponentials e^{kx} for approximation?

Clearly, based on previous experience, this depends on whether we can construct a Dirac delta sequence using exponential functions. We can consider the hyperbolic cosine \cosh x, which is an even function and behaves like an upward-opening parabola near x=0. If we consider the interval [-1, 1], we can define: \delta_n (x) = \frac{(\cosh 1 - \cosh x)^n}{I_n},\quad I_n = \int_{-1}^1 (\cosh 1 - \cosh x)^n dx It is not difficult to obtain \lim_{n\to\infty} \delta_n (x) = \delta (x). Thus, we can consider the function sequence: \frac{1}{I_n}\int_{-1}^1 f(y)[\cosh 1 - \cosh (x-y)]^n dy This sequence can be used to approximate f(x). Expanding it directly shows it is a linear combination of \cosh kx, \sinh kx for k = 0,1,2,\dots,n, which is equivalent to a linear combination of e^{kx}, k=0,\pm 1,\pm 2,\dots, \pm n. Therefore, approximation using real exponentials is possible.

It seems strange then: since both complex and real exponentials can be used for approximation, why is only complex exponential approximation (Fourier series) widely studied? I suspect the main reason is that real exponential approximation is neither elegant nor practical. These approximations are within a finite interval. Since Fourier series are periodic functions, studying a finite interval is equivalent to studying the whole domain. In contrast, real exponentials only study one interval and do not help with the global behavior; furthermore, they contain both e^x and e^{-x}, which leads to a divergent trend as x\to+\infty and x\to -\infty, offering no advantages.

When reposting, please include the original address of this article: https://kexue.fm/archives/4187

For more detailed reposting matters, please refer to: "Scientific Space FAQ"