From Markov Processes to the Master Equation (Derivation Process)

Markov Process

The master equation is used to describe Markov processes. A Markov process can be understood as the "memoryless" property of motion. In simple terms, the probability distribution at the next moment depends only on the current state and is independent of its history. Written as a probability formula (considering only continuous probabilities, where p is the probability density): \label{eq:maerkefu} p(x,\tau)=\int p(x,\tau|y,t) p(y,t) dy Here, the integration is over the entire space. p(x,\tau|y,t) is called the transition probability, which is the probability density of reaching position x at time \tau, given that the system was at position y at time t. The physical meaning of this equation is quite obvious, so no further explanation is needed.

Although Equation [eq:maerkefu] is intuitive, using it for modeling actually presents two difficult problems:

1. To model, one must write down p(x,\tau|y,t), but in practice, it is very difficult to directly formulate a reasonable transition probability.

2. Even if p(x,\tau|y,t) is formulated, the equation is an integral equation, and our understanding of integral equations is far less advanced than that of differential equations.

Therefore, it is necessary to derive its differential equation form.

Master Equation

Let \tau = t+\epsilon: \label{eq:maerkefu-2} p(x,t+\epsilon)=\int p(x,t+\epsilon|y,t) p(y,t) dy Considering the limit \epsilon \to 0 and retaining terms up to the first order of \epsilon, we have: \label{eq:yueqian} p(x,t+\epsilon|y,t)=\delta(x-y)+\epsilon \tilde{W}(x,y,t) where we use the fact that p(x,t|y,t)=\delta(x-y), and: \tilde{W}(x,y,t)=\left.\frac{\partial p(x,\tau|y,t)}{\partial \tau}\right|_{\tau=t} Expanding both sides of Equation [eq:maerkefu-2] to the first order of \epsilon, we get: p(x,t)+\epsilon\frac{\partial p(x,t)}{\partial t}=\int \left[\delta(x-y)+\epsilon \tilde{W}(x,y,t)\right] p(y,t) dy Thus: \label{eq:zhufangcheng-1} \frac{\partial p(x,t)}{\partial t}=\int \tilde{W}(x,y,t) p(y,t) dy

Facilitating Modeling

The reader may notice that Equation [eq:zhufangcheng-1] is not the form of the master equation we commonly see. This is because it is not convenient for modeling. Let’s look at Equation [eq:yueqian] again. Note that we have: \int p(x,t+\epsilon|y,t) dx = 1 Therefore, combined with Equation [eq:yueqian], it must be required that: \label{eq:yueshu} \int \tilde{W}(x,y,t)dx = 0 The modeling process in scientific research is reversed: one needs to write down the form of the master equation first and then solve it. In other words, to model using Equation [eq:zhufangcheng-1], one needs to write down \tilde{W}(x,y,t), and \tilde{W}(x,y,t) must satisfy the constraint of Equation [eq:yueshu]. However, it is difficult to construct a \tilde{W}(x,y,t) out of thin air that also satisfies this constraint.

Fortunately, we can use a trick to remove this constraint. First, we write down an arbitrary function W(x,y,t), and then consider: \begin{aligned} &\int W(x,y,t)dx\\ =& \iint \delta(y-z) W(x,z,t)dxdz \quad (\text{next, swap the positions of } x \text{ and } z)\\ =& \iint \delta(y-x) W(z,x,t)dzdx \end{aligned} Thus, we can let: \tilde{W}(x,y,t) = W(x,y,t) - \int \delta(y-x) W(z,x,t)dz Then it automatically satisfies Equation [eq:yueshu]. The final form of the master equation becomes: \label{eq:zhufangcheng-2} \frac{\partial p(x,t)}{\partial t}=\int \Big[W(x,y,t)p(y,t)-W(y,x,t)p(x,t)\Big] dy This is the form of the master equation seen in textbooks. At this point, there are no special constraints on W(x,y,t), making it convenient for modeling. As for the interpretation of the physical meaning of W(y,x,t), that is a separate matter. Similar derivation processes can be applied to the discrete master equation and will not be repeated here.

A Brief Evaluation

As seen from the process above, the reason the master equation takes the form we commonly see is that this form is more convenient for modeling. As for the interpretation of the results, they are essentially imposed after the fact and cannot be considered part of the derivation process. Most works on the derivation of the master equation focus on the physical interpretation of the results while neglecting the origin of the equation’s form. This is one of the major reasons for the difficulty in understanding. I hope the text here can serve as a supplement to that.