As everyone knows, GPT-3 is currently in the spotlight. However, amidst all the hype surrounding GPT-3, do readers remember the actual title of the GPT-3 paper? In fact, the GPT-3 paper is titled “Language Models are Few-Shot Learners.” The title no longer contains the letters G, P, or T, but since it follows the same lineage as the original GPT, it is still referred to as GPT-3. As the name suggests, GPT-3 focuses on Few-Shot Learning. Additionally, another characteristic of GPT-3 is its size; the largest version boasts 175 billion parameters, which is over a thousand times larger than BERT Base.
Because of this, a recent paper on Arxiv titled “It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners” caught my attention. To paraphrase: “Who says it has to be big? Small models can also do few-shot learning.” Clearly, this title is aimed at GPT-3. I was curious to see who had the courage to challenge GPT-3 and what kind of small model could do it. After reading, I discovered that the authors proposed that with proper construction, BERT’s MLM model can also perform few-shot learning. After finishing the paper, I had a sudden realization of “So, it can be done this way!” I would like to share these insights with you.
The Rising MLM
MLM stands for “Masked Language Model.” It is essentially a cloze task where certain words in a text are randomly masked, and the model is required to predict the masked words. A schematic diagram is shown below:
The masked parts can be directly selected random tokens or random continuous tokens that form a whole word; the latter is known as WWM (Whole Word Masking).
Initially, MLM was only seen as a pre-training task for BERT—the kind that could be discarded once training was finished. Consequently, some open-source models did not even retain the weights for the MLM head, such as the brightmart version and the CLUE version of RoBERTa. Meanwhile, the RoBERTa-wwm-ext-large released by HFL randomly initialized the MLM weights for some reason. Therefore, if you want to reproduce the results mentioned later in this article, these versions are not suitable.
However, as research progressed, researchers found that not only is the BERT Encoder useful, but the MLM used for pre-training is also very useful. For example, the paper “BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model” pointed out that MLM can be used as a general generative model. The paper “Spelling Error Correction with Soft-Masked BERT” used MLM for text error correction. My previous experiments in “From Language Models to Seq2Seq: Transformer is All About Masking” also showed that MLM pre-training weights can be used as UniLM for Seq2Seq tasks. Furthermore, the article “Unsupervised Segmentation and Syntactic Analysis! BERT Can Also Be Used This Way” applied the idea of MLM to unsupervised word segmentation and syntactic analysis. It can be said that MLM is already shining brightly.
Converting Tasks into Cloze Tests
In this article, we will learn another brilliant application of MLM: using it for few-shot learning or semi-supervised learning, and in some scenarios, even achieving zero-shot learning.
How do we combine the task we want to perform with MLM? It’s simple: give the task a textual description and convert it into a cloze problem. For example, given the sentence “I felt this trip to Beijing was very good,” we can add a description to construct the following cloze test:
______ satisfied. I felt this trip to Beijing was very good.
Furthermore, we restrict the blank space to only be filled with words like “Very” or “Not.” The problem becomes clear: we need to judge satisfaction based on contextual consistency. If the probability of “Very” is greater than the probability of “Not,” it indicates a positive sentiment; otherwise, it is negative. In this way, we have converted a sentiment classification problem into a cloze problem. It can be predicted using an MLM model, and since the training of the MLM model does not require supervised data, this can theoretically achieve zero-shot learning.
Multi-classification problems can be converted similarly. For news topic classification, if the input sentence is “After eight months, we can finally see the women’s volleyball girls on the court again,” we can construct:
The following reports a piece of ______ news. After eight months, we can finally see the women’s volleyball girls on the court again.
Thus, we have converted news topic classification into a cloze problem. A good MLM model should be able to predict the word “Sports.”
Some simple reasoning tasks can also undergo this conversion. A common task is given two sentences, determine if they are compatible. For example, “I went to Beijing” and “I went to Shanghai” are contradictory, while “I went to Beijing” and “I am at Tiananmen Square” are compatible. The common approach is to concatenate the two sentences and input them into the model as a binary classification task. To convert this into a cloze test, how should we construct it? A natural way is:
I went to Beijing? ______, I went to Shanghai.
I went to Beijing? ______, I am at Tiananmen Square.
Where the candidate words for the blank are \{\text{Yes}, \text{No}\}.
Pattern-Exploiting
By now, readers should easily see the pattern: add a prefix or suffix description to the input text and mask certain tokens to convert it into a cloze problem. This conversion is called a Pattern in the original paper. This conversion should be as natural as possible when combined with the original sentence, as the pre-trained MLM model was trained on natural language. Obviously, the same problem can have many different patterns. For the sentiment classification example, the description can be placed at the end: “I felt this trip to Beijing was very good. ______ satisfied.” Or more words can be added: “How do you feel? ______ satisfied. I felt this trip to Beijing was very good.”
Next, we need to construct a candidate space for the predicted tokens and establish a mapping from tokens to actual categories. This is called a Verbalizer in the original paper. For the sentiment classification example, our candidate space is \{\text{Very}, \text{Not}\}, and the mapping is \text{Very} \to \text{Positive}, \text{Not} \to \text{Negative}. The mapping between the candidate space and actual categories does not have to be one-to-one. For instance, we could also include words like “Quite,” “Too,” or “Hardly,” and define \{\text{Very}, \text{Quite}, \text{Too}\} \to \text{Positive} and \{\text{Not}, \text{Hardly}\} \to \text{Negative}. It is easy to understand that many NLP tasks can undergo this conversion, but this conversion is generally only suitable for tasks with a finite candidate space—essentially, it is used for multiple-choice questions, most commonly text classification.
As mentioned, the same task can have multiple different patterns. The original paper handles them as follows:
For each pattern, fine-tune an MLM model separately using the training set.
Integrate the models corresponding to different patterns to obtain a fused model.
Use the fused model to predict pseudo-labels for unlabeled data.
Fine-tune a conventional (non-MLM) model using the pseudo-labeled data.
The specific integration method can be found in the paper; it is not the main focus here. This training mode is called Pattern-Exploiting Training (PET). It first appeared in the paper “Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference.” The paper introduced here further confirms and refines the value and results of PET and integrates multi-task learning, allowing its few-shot learning performance on the SuperGLUE benchmark to surpass GPT-3. Both papers share the same authors and represent a continuous body of work.
One point to critique is that in the figure above, the 223M parameters for PET refer to the ALBERT-xxlarge-v2 model. In fact, calling ALBERT a “small model” is somewhat misleading because its forward computation speed is not improved. ALBERT-xxlarge has 12 layers with shared parameters; in terms of forward computation, it is equivalent to a GPT with approximately 2700M (12 times) parameters.
Chinese Practice and Verification
To truly confirm the value of a method or model, looking at the experimental tables in a paper is not enough. No one can say for sure if the results in the paper can be reproduced, and even if they can be reproduced in English, it doesn’t mean they are valuable in Chinese. Therefore, the most practical approach is to conduct experiments personally. Below is my experimental code for reference:
Github Address: https://github.com/bojone/Pattern-Exploiting-Training
We will explore the feasibility of PET from the following perspectives:
How effective is it to use an off-the-shelf MLM model directly? (Zero-shot Learning 1)
How effective is it to fine-tune an off-the-shelf MLM model with “a large amount of unlabeled data”? (Zero-shot Learning 2)
How effective is it to fine-tune an off-the-shelf MLM model with “a small amount of labeled data”? (Few-shot Learning)
How effective is it to fine-tune an off-the-shelf MLM model with “a small amount of labeled data + a large amount of unlabeled data”? (Semi-supervised Learning)
Below are the experimental results for sentiment binary classification. There is also a multi-classification task for news topics, and the code is on Github; the results are similar, so I won’t repeat them here.
Zero-shot Learning 1
This explores the accuracy of predictions based directly on off-the-shelf MLM models after adding the corresponding patterns to the input text. Since the entire process does not involve supervised training with labeled data, it is considered “zero-shot learning.” We compare different patterns and different MLM models:
Below are the patterns used in the experiment, where the candidate words for the blank are “Very” and “Not”:
P1: ______ satisfied. I felt this trip to Beijing was very good.
P2: I felt this trip to Beijing was very good. ______ satisfied.
P3: ______ good. I felt this trip to Beijing was very good.
P4: ______ ideal. I felt this trip to Beijing was very good.
P5: How do you feel? ______ satisfied. I felt this trip to Beijing was very good.
The MLM models used are:
M1: Google’s official Chinese BERT Base;
M2: HFL’s RoBERTa-wwm-ext Base;
M3: Tencent UER’s BERT Base;
M4: Tencent UER’s BERT Large.
The experimental results are shown in the table below (Validation Set / Test Set):
| P1 | P2 | P3 | P4 | P5 | |
|---|---|---|---|---|---|
| M1 | 66.94 / 67.60 | 57.56 / 56.13 | 58.83 / 59.69 | 83.70 / 83.33 | 75.98 / 76.13 |
| M2 | 85.17 / 84.27 | 70.63 / 68.69 | 58.55 / 59.12 | 81.81 / 82.28 | 80.25 / 81.62 |
| M3 | 66.75 / 68.64 | 50.45 / 50.97 | 68.97 / 70.11 | 81.95 / 81.48 | 61.49 / 62.58 |
| M4 | 83.56 / 85.08 | 72.52 / 72.10 | 76.46 / 77.03 | 88.25 / 87.45 | 82.43 / 83.56 |
The best result actually reached 88%! In other words, by loading an off-the-shelf MLM and using appropriate patterns, we can correctly identify the sentiment of most samples without any labeled data. This forces us to look at the potential of MLM models with new respect.
It can be observed that there are certain differences between different patterns and pre-trained models. Overall, the Large version performs significantly better than the Base version, indicating that, like the progression from GPT to GPT-2 and then GPT-3, making the model larger is better. Furthermore, this might suggest that MLM has not yet been fully trained, perhaps because BERT’s masking approach is too inefficient. It might be better to use the improved MLM mentioned in the article “Modifying Transformer Structure to Design a Faster and Better MLM Model.”
Zero-shot Learning 2
After seeing the above results, readers might think: if I continue pre-training the MLM model with in-domain data, can I improve the results? The answer is: Yes! Below are our experimental results. Due to limited computing power, we only compared results based on RoBERTa-wwm-ext (M2 above; the model after continued pre-training is called M2^{+\text{unsupervised}}):
| P1 | P2 | P3 | P4 | P5 | |
|---|---|---|---|---|---|
| M2 | 85.17 / 84.27 | 70.63 / 68.69 | 58.55 / 59.12 | 81.81 / 82.28 | 80.25 / 81.62 |
| M2^{+\text{unsupervised}} | 88.05 / 87.53 | 71.01 / 68.78 | 81.05 / 81.24 | 86.40 / 85.65 | 87.26 / 87.40 |
Note that here we only used in-domain data to continue MLM training. This process is unsupervised and does not require label signals, so it is still considered “zero-shot learning.” At the same time, from the results so far, we can see that adding a “prefix” to the input text is more advantageous than a “suffix.”
Few-shot Learning
We just discussed the improvement from continuing MLM pre-training with unlabeled data. If we return to the target scenario of PET, what happens if we directly use a small amount of labeled data with specific patterns to train the MLM? This is true “few-shot learning.” Here, we kept about 200 labeled samples. When constructing samples, we first add the pattern to each sentence. In addition to the Mask position provided by the pattern itself, we also randomly mask another part to enhance regularization. The final experimental results are as follows:
| P1 | P2 | P3 | P4 | P5 | |
|---|---|---|---|---|---|
| M2 | 85.17 / 84.27 | 70.63 / 68.69 | 58.55 / 59.12 | 81.81 / 82.28 | 80.25 / 81.62 |
| M2^{+\text{few-shot}} | 89.29 / 89.18 | 84.71 / 82.76 | 88.91 / 89.05 | 89.31 / 89.13 | 89.07 / 88.75 |
The conclusion is that except for the “suffix” P2, other results are similar. This further indicates that “prefix” patterns are more competitive than “suffix” patterns. In terms of performance, directly fine-tuning a BERT model with the same data using conventional methods yields a result of about 88.93. Therefore, the few-shot learning method based on “MLM + Pattern” may bring a slight performance boost.
Semi-supervised Learning
Now that we’ve discussed unsupervised zero-shot learning and supervised few-shot learning, it’s naturally time for “semi-supervised learning,” which combines labeled and unlabeled data. In the same task, the ratio of labeled to unlabeled data is about 1:99. Labeled data includes patterns, while unlabeled data does not. Both are subjected to MLM pre-training with some tokens masked. The final measured results are as follows:
| P1 | P2 | P3 | P4 | P5 | |
|---|---|---|---|---|---|
| M2 | 85.17 / 84.27 | 70.63 / 68.69 | 58.55 / 59.12 | 81.81 / 82.28 | 80.25 / 81.62 |
| M2^{+\text{semi-supervised}} | 90.09 / 89.76 | 79.58 / 79.35 | 90.19 / 88.96 | 90.05 / 89.54 | 89.88 / 89.23 |
Again, the “suffix” is significantly worse than the “prefix,” and the “prefix” results are similar. In terms of specific effects, it confirms that additional unlabeled data is also useful. Intuitively, the “prefix” is better than the “suffix” likely because the Mask position in the “prefix” is more fixed, allowing weak supervision signals to be superimposed and enhanced. However, this doesn’t explain why the “prefix” is also better in zero-shot learning. It is estimated to be related to the learning difficulty of the model; perhaps the patterns in the beginning of a sentence are more obvious and relatively easier to learn, leading to more thorough learning. All of this is still speculation.
Summary and Conclusion
The results are summarized as follows:
| P1 | P2 | P3 | P4 | P5 | |
|---|---|---|---|---|---|
| M2 | 85.17 / 84.27 | 70.63 / 68.69 | 58.55 / 59.12 | 81.81 / 82.28 | 80.25 / 81.62 |
| M2^{+\text{unsupervised}} | 88.05 / 87.53 | 71.01 / 68.78 | 81.05 / 81.24 | 86.40 / 85.65 | 87.26 / 87.40 |
| M2^{+\text{few-shot}} | 89.29 / 89.18 | 84.71 / 82.76 | 88.91 / 89.05 | 89.31 / 89.13 | 89.07 / 88.75 |
| M2^{+\text{semi-supervised}} | 90.09 / 89.76 | 79.58 / 79.35 | 90.19 / 88.96 | 90.05 / 89.54 | 89.88 / 89.23 |
Readers can also compare these with the semi-supervised learning results using Virtual Adversarial Training (VAT) in our previous article “Random Thoughts on Generalization: From Random Noise and Gradient Penalty to Virtual Adversarial Training.” It can be seen that whether it is zero-shot, few-shot, or semi-supervised learning, the MLM-based approach can rival the results of VAT-based semi-supervised learning. Our results in short news multi-classification experiments were similar. Therefore, this shows that the MLM model can indeed be used as an excellent zero-shot/few-shot/semi-supervised learner.
Of course, there are still disadvantages to MLM-based models. For example, the independence assumption used by MLM limits its ability to predict longer texts (the blank space cannot be too long), and the inability to predict variable-length answers also constrains its scenarios (so it can currently only be used for multiple-choice questions, not generation). We look forward to the emergence of stronger MLM models, at which point it may be possible to compete with GPT-3 across all tasks.
Summary
This article introduced a novel application of BERT’s MLM model: converting tasks into cloze tests with specific descriptions and using the MLM model for zero-shot, few-shot, and semi-supervised learning. In the SuperGLUE experiments of the original paper, it achieved results comparable to GPT-3. I also conducted experiments on Chinese tasks, further confirming the effectiveness of this approach. The entire idea is quite unique and gives a sense of “So, it can be done this way!” I recommend everyone to learn from it.
When reposting, please include the original article address: https://kexue.fm/archives/7764
For more detailed reposting matters, please refer to: Scientific Space FAQ