Suppose we are listening to a song. After finishing the song, we have actually experienced a process where our ears received sound wave stimuli over a period of time, which in turn caused changes in our brain activity. This song, which is the sound wave over this duration, can be described as a function f(t) of time t. The interval of this function is finite, for example, t \in [0, T]. Now suppose another scenario—we want to use a computer to record ourselves singing. What kind of process is this? It is important to note that computer signals are discrete, while sound waves are continuous. Therefore, for a computer to record a song, it can only perform sampling and recording of the signal. In principle, the more points collected, the more realistically our singing can be restored. However, there is a question: how many points are enough? In information theory, a famous "Sampling Theorem" (also known as the Shannon Sampling Theorem or the Nyquist Sampling Theorem) tells us: We only need to collect a finite number of sample points to completely reconstruct our input signal!
How is it possible to reconstruct a continuous function by collecting only a finite number of points? Below, we will explain this theorem.
Given any function, generally speaking, we can perform a Fourier transform on it: F(\omega)=\int_{-\infty}^{+\infty} f(t)e^{i\omega t}dt \tag{1} Although our integration limits are written from negative to positive infinity, since f(t) is a function within a finite interval, the actual integration interval is finite. The inverse transform of the above equation is: f(t)=\frac{1}{2\pi}\int_{-\infty}^{+\infty} F(\omega)e^{-i\omega t}d\omega \tag{2}
Now we ask: what is the significance of the Fourier transform? In fact, it is a mutual transformation of the same function between the time domain and the frequency domain. In popular terms, the initial sound wave is described as a function of time; after the Fourier transform, the independent variable becomes frequency. That is, the transformed function tells us which frequencies of simple harmonic waves are superimposed to form the original sound wave. Thus, we reach a crucial step: the frequency range after transformation might be finite, meaning that outside a certain interval [-W, W], F(\omega) is identically zero.
Let us explain why this phenomenon occurs. The appearance of this phenomenon may be fundamental—that is, the transformed function really is identically zero outside some interval [-W, W]. However, more often, it is another situation—the frequencies we can receive are finite. For example, the frequency range of sounds our ears can hear is between 20 and 20,000 Hertz (Hz). So, when recording sound, why should we care about the parts of the sound below 20 Hz or above 20,000 Hz? Images are similar; the frequencies of electromagnetic waves we can see, namely visible light, also have a range. In other words, when we record a song, as long as it sounds exactly the same to us, we don’t need to care if it is truly identical. (If we switched to another device, such as an ultrasonic detector, we might find differences from the original sound wave, but when we listen to music, who pays attention to the sensations of an ultrasonic detector?)
Based on the above discussion, we can truncate the integral in equation (2): f(t)=\frac{1}{2\pi}\int_{-W}^{W} F(\omega)e^{-i\omega t}d\omega \tag{3}
Note that F(\omega) is now only a function on [-W, W]. A function on a finite interval can be transformed into a Fourier series (note that this is a Fourier series, not a Fourier transform): F(\omega)=\sum_{n=-\infty}^{+\infty}c_n e^{n\pi i\omega/W} \tag{4} Next, substitute (4) into (3) and swap the integration and summation signs to get: \begin{aligned} f(t)&=\frac{1}{2\pi}\int_{-W}^{W} \left(\sum_{n=-\infty}^{+\infty}c_n e^{n\pi i\omega/W}\right)e^{-i\omega t}d\omega\\ &=\frac{1}{2\pi}\sum_{n=-\infty}^{+\infty}c_n\int_{-W}^{W} e^{n\pi i\omega/W-i\omega t}d\omega\\ &=\frac{1}{2\pi}\sum_{n=-\infty}^{+\infty}c_n\left.\frac{e^{n\pi i\omega/W-i\omega t}}{i(n\pi /W- t)}\right|_{-W}^W\\ &=\frac{1}{2\pi}\sum_{n=-\infty}^{+\infty}c_n\frac{2\sin(n\pi -W t)}{n\pi /W- t}\\ &=\sum_{n=-\infty}^{+\infty}\left(\frac{c_n W}{\pi}\right)\frac{\sin(n\pi -W t)}{n\pi - Wt} \end{aligned} \tag{5} Now we can determine each c_n. We see that c_n is independent of t. We can take a special value, namely t \to \frac{n\pi}{W}, and then we can derive: \frac{c_n W}{\pi}=f\left(\frac{n\pi}{W}\right) Which is: f(t)=\sum_{n=-\infty}^{+\infty}f\left(\frac{n\pi}{W}\right)\frac{\sin(n\pi -W t)}{n\pi - Wt} \tag{6} Notice that f(t) is defined within a finite interval. When |n| is large enough, f\left(\frac{n\pi}{W}\right)=0. In other words, the above equation is actually a sum of finite terms!
Equation (6) is the entirety of the sampling theorem. It tells us that as long as we collect the values of a finite number of sample points f\left(\frac{n\pi}{W}\right), we can completely reconstruct f(t) according to equation (6)—at least it will be the same within the range we can perceive. This is one of the things I learned from "Feynman Lectures on Computation."
Some details in the proof process may not be strictly rigorous, but if you are a reader from the physics or engineering department, I think it should be sufficient; if you are a mathematics enthusiast pursuing rigor, then please formalize it.
When reprinting, please include the original address of this article: https://kexue.fm/archives/3266
For more detailed reprinting matters, please refer to: "Scientific Space FAQ"