I wonder if readers have seen the article by QuantumBit earlier this year titled "The Strongest Writing AI Actually Learned Chess and Composition, Language Model Cross-over Operations Spark Heated Discussion, Seeking Challenges Online". It mentioned that a netizen used the GPT-2 model to train a model that plays International Chess. I have been thinking, how can such an interesting thing not have a Chinese version? For International Chess, its Chinese counterpart is naturally Chinese Chess (Xiangqi). Therefore, I have always wanted to replicate those results on Chinese Chess. After delaying for more than half a year, I finally completed this task in the last few days and would like to share it with everyone.
Chess Manual Formula
The General stays within the nine-palace square,
The Advisors follow and never leave the court.
The Elephants fly to the four corners of the camp,
The Horses move in a step and a diagonal jump.
The Cannons must fire over a piece to hit another,
The Chariots travel straight paths to the east or west.
Only the Soldiers can only take one step at a time,
Once across the river, they move sideways with no trace.
Memorizing Chess Manuals
In fact, by briefly looking at the QuantumBit article, one can understand the principle of GPT-2 playing chess: it is nothing more than "memorizing chess manuals." Simply put, a chess manual can be represented as a continuous text string, and GPT-2 is precisely good at memorizing text. Therefore, GPT-2 can be used to memorize human chess manuals. As for playing chess, it can be seen as reciting the next move in the manual based on the part of the manual already played. Thus, the entire task is theoretically achievable with GPT-2.
To complete this task, we need to understand how computers record chess games. Regarding the standards for recording moves, the more common ones are the ICCS notation and the FEN position representation. Details can be found in the articles "Chinese Chess Computer Application Specification (II): Move Representation" and "Chinese Chess Computer Application Specification (II): FEN File Format".
ICCS Notation
Simply put, ICCS notation represents the chessboard using horizontal and vertical coordinates as shown in the figure below. Each move only needs to record the starting and ending coordinates. For example, "h2e2" means moving the piece originally at coordinate (h, 2) to (e, 2). If the current position is the starting position, this corresponds to the move "Cannon Two moves to Five" (Pao Er Ping Wu). In this way, each move only requires 4 characters to record, and a manual of n moves becomes a string of length 4n. Of course, if it is to be input into a model, it does not necessarily have to follow this exact method. For example, I could represent "h2" with a single ID and "e2" with another ID, meaning each grid point is described by one coordinate instead of two. This would require only two IDs per move, thereby reducing the sequence length of the manual. There is no fixed rule; interested readers can improve this themselves.
FEN Position
As for the FEN (Forsyth-Edwards Notation) position representation, it
is used to describe which pieces are on the board and whose turn it is.
The manuals modeled in this article are actually full-game manuals, so
the model in this article does not actually need to use it (the
positions are all default starting positions). However, for the
convenience of interested readers to make improvements, I will briefly
introduce it here. The FEN notation mainly finds a way to represent
which pieces are in each row. For example, the opening is represented
as: rnbakabnr/9/1c5c1/p1p1p1p1p/9/9/P1P1P1P1P/1C5C1/9/RNBAKABNR w - - 0 1.
The red part represents the board position. Lowercase letters represent the Black side, and uppercase letters represent the Red side. Different rows are separated by "/". The meanings of the letters are shown in the table below. Thus, "rnbakabnr" indicates that the first row contains Black’s "Chariot, Horse, Elephant, Advisor, General, Advisor, Elephant, Horse, Chariot." "9" indicates that the second row has 9 empty points. "1c5c1" indicates that the third row is "1 empty space + 1 Black Cannon + 5 empty spaces + 1 Black Cannon + 1 empty space," and so on. The green part indicates which side’s turn it is to move; "w" stands for Red, and "b" stands for Black. The remaining parts are generally not very important; interested readers can check the links themselves.
| R/r | N/n | B/b | A/a | K/k | C/c | P/p | Digit m |
|---|---|---|---|---|---|---|---|
| Chariot | Horse | Elephant | Advisor | General | Cannon | Soldier | m empty spaces |
Building the Model
After reading the introduction to the notation representation above, we know that whether it is the moves of each step or the representation of the position, they have all been converted by us into a string of text. Since the deduction of chess is based on positions and moves as input and output, theoretically, the modeling of chess is entirely a "text processing" problem! This is the theoretical basis for GPT-2 playing chess. Essentially, GPT is just BERT plus a language model’s Attention Mask, so we can call this approach BERT playing chess or GPT playing chess.
Code Sharing
There isn’t much to write about the model principle. It was introduced in previous articles such as "From Language Models to Seq2Seq: Transformer is All About Masking". Related examples include "Conditional Text Generation Based on Conditional Layer Normalization" and "What Grade Can BERT Enter? Seq2Seq ’Hard-carrying’ Primary School Math Word Problems". Readers can look them up themselves. The processing in this article is actually very simple: only the full-game manuals are kept, the ICCS notation of the manual is treated as a long sentence, and then a language model is trained.
Project Link: https://github.com/bojone/gpt_cchess
The training process uses progressive training, which means gradually increasing the sequence length instead of using a uniform length from the start. Some articles call this approach "Curriculum Learning." This approach can improve the training speed and convergence speed of the model (initially the sequence is shorter, the training speed is faster, and it is easier to converge). Intuitively, it lets the model learn the "opening" first, then learn "opening + middle game," and finally learn "opening + middle game + endgame," gradually increasing the difficulty. BERT weights were loaded before training. Readers might wonder if BERT weights can have anything to do with chess manuals? Actually, there isn’t much connection, but in any case, using BERT weights is slightly better than completely random initialization, and convergence will be a bit faster.
Testing
The model script also includes an implementation for interactive chess play with the model. Readers can experience it themselves. This interactive play uses the Python cchess module for assistance, for which I am grateful. GPT itself is a generative model, but when deciding the next move, I did not use a generative method (because unconstrained generation might output invalid moves). Instead, I used a scoring method: I generate all valid moves for the current position, input them into the model for scoring, and pick the move with the highest score. This ensures that every step the model outputs is valid, guaranteeing that the game can continue until a win or loss is determined.
Readers might wonder if the calculation for enumerating all valid
moves is very large. In fact, for each position, there are not many
valid moves. It can be simply argued that it will not exceed 111 types
(is that surprising? The candidate moves for each step in Chinese Chess
do not exceed 111, rather than a very large number). Therefore, the
batch_size for this step will not exceed 111, which is
acceptable.
The derivation is simple: 1 Chariot or Cannon has at most 17 moves, so 2 Chariots and 2 Cannons have at most 68 moves; if all Soldiers have crossed the river, each Soldier has at most 3 moves, so 5 Soldiers have at most 15 moves; 1 Horse has at most 8 moves, so 2 Horses have 16; 2 Elephants have at most 6 moves (one in the center, one on the edge, 4+2); 1 Advisor in the center has at most 4 moves (2 Advisors actually block each other); finally, the General has at most 2 moves. Therefore, the result is 68+15+16+6+4+2=111. For specific position designs, one can refer to the thread "A Difficult Problem in Chinese Chess Position Design" on the Mathematics R&D Forum.
What everyone cares about most is likely how strong the model’s chess skills are. I did a simple test, and the general conclusion is: It can basically play a relatively good opening and has decent adaptability during the opening. However, once it reaches the middle game, its adaptability drops significantly. It is not very sensitive to capturing pieces; that is, when you capture its pieces randomly, it might not respond. It can be seen that these are actually the disadvantages of purely memorizing manuals. Of course, as mentioned before, every step’s output is valid, so you can keep playing with it until the game is finished.
Talking About Improvements
Some readers might ask if the model can improve its skills by playing against itself. Theoretically, of course, it can, but unfortunately, it is not implemented here. First, I didn’t have the heart to implement it, and second, I didn’t have the computing power. Additionally, increasing the model size should also further improve chess skills. Note that my results above only used the Base version (100 million parameters) of the model. The netizen mentioned at the beginning of the article used a 1.5 billion parameter GPT-2 for International Chess, which is 15 times ours. Another area for improvement is that in the modeling above, we directly learned the entire game manual. Logically, for better skills, we could learn only the moves of the winner and not the moves of the loser.
Of course, even if these methods bring improvements, they are likely limited. Ultimately, this is different from the principles of playing chess as we understand them. We play chess by projecting forward based on the situation of the board, but the language model approach above does not have the concept of a "position," or rather, its position needs to be determined by all the steps already taken. For the middle and late game, the number of historical steps is too high, which is indeed asking too much of the model. The improvement method is actually very simple: change it to "position as input, move as output." As mentioned earlier, a position can also be represented as text using FEN notation, so this is just a Seq2Seq task.
In addition, there are other approaches. For example, we can treat every position of the winner as a positive sample and every position of the loser as a negative sample, then train a binary classification model to judge the quality of a position. With this judgment function, we can also directly enumerate every valid move and choose the optimal move based on the results of the judgment function. Since a position can be represented as text, this means we have turned playing chess into a text classification task.
In short, thanks to the powerful modeling capabilities of the Transformer model for text, our thinking for modeling chess has also become simple and diverse.
Summary
This article attempted the practice of using GPT to play Chinese
Chess via bert4keras. The main idea is to give the model
the ability to predict the next move by "memorizing chess manuals with a
language model," and discussed some improvement ideas. Although the
approach in this article is not the standard way to model the task of
playing chess, through this method, we can further appreciate the power
of language models.
Everyone is welcome to report the chess intelligence of the models they have trained, haha!
When reposting, please include the original address of this article: https://kexue.fm/archives/7877
For more detailed reposting matters, please refer to: "Scientific Space FAQ"