Couplet Robot Based on CNN and Sequence Labeling

Analysis

However, if we think about it further, we find that compared to general sentence generation tasks, "writing couplets" is much more regular: 1. The upper and lower lines have the same number of characters; 2. Almost every character in the upper line has a corresponding relationship with the lower line. Thus, writing couplets can be directly viewed as a sequence labeling task, similar to the approach used for word segmentation or Named Entity Recognition (NER). This is the starting point of this article.

Speaking of which, this article doesn’t actually have much technical complexity. Sequence labeling is already a very common task, far simpler than general seq2seq. Sequence labeling refers to inputting a sequence of vectors and then outputting another sequence of vectors of the same length, and finally classifying "every frame" of this sequence. Related concepts can be further explored in the article "Brief Introduction to Conditional Random Fields (CRF) with Pure Keras Implementation".

Model

In this article, I will introduce the model while writing the code. For readers who need to further understand the underlying basic knowledge, you can also refer to: "[Chinese Word Segmentation Series] 4. seq2seq Character Labeling Based on Bidirectional LSTM", "[Chinese Word Segmentation Series] 6. Chinese Word Segmentation Based on Fully Convolutional Networks", and "Poetry Robot Based on CNN and VAE: Random Poetry Generation".

The model code we use is as follows:

x_in = Input(shape=(None,))
x = x_in
x = Embedding(len(chars)+1, char_size)(x)
x = Dropout(0.25)(x)

x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)
x = gated_resnet(x)

x = Dense(len(chars)+1, activation='softmax')(x)

model = Model(x_in, x)
model.compile(loss='sparse_categorical_crossentropy',
              optimizer='adam')

Where gated_resnet is the gated convolution module I defined (this module was also introduced in the article "DGCNN: A CNN-based Reading Comprehension Question Answering Model"):

def gated_resnet(x, ksize=3):
    # Gated Convolution + Residual
    x_dim = K.int_shape(x)[-1]
    xo = Conv1D(x_dim*2, ksize, padding='same')(x)
    return Lambda(lambda x: x[0] * K.sigmoid(x[1][..., :x_dim]) \
                            + x[1][..., x_dim:] * K.sigmoid(-x[1][..., :x_dim]))([x, xo])

And that’s it!

That’s all there is to it; the rest is just data preprocessing. Of course, readers can also try replacing gated_resnet with a standard bidirectional LSTM, but in my experiments, I found that the bidirectional LSTM did not perform as well as gated_resnet, and the LSTM is also relatively slower, so the LSTM was abandoned here.

Results

The training dataset comes from: https://github.com/wb14123/couplet-dataset. Thanks to the author for organizing it.

Complete Code:
https://github.com/bojone/seq2seq/blob/master/couplet_by_seq_tagging.py

Training Process:

Partial Results:

Upper line: The evening breeze shakes the trees, yet they stand firm
Lower line: The night rain taps the flowers, making them more fragrant

Upper line: The weather is nice today
Lower line: Yesterday’s human feelings were unclear

Upper line: Fish leap in the sea at this moment
Lower line: Birds chirp on what day for people

Upper line: Only the fragrance remains as before
Lower line: Not without the moon appearing as new

Upper line: Scientific Space
Lower line: Civilization Great Middle

It seems to have some flavor. Note that "The evening breeze shakes the trees, yet they stand firm" is an upper line from the training set; the standard lower line is "Morning dew moistens the flowers, making them redder," while the model gave "The night rain taps the flowers, making them more fragrant." This shows that the model is not simply memorizing the training set but has a certain level of understanding; I even feel that the lower line produced by the model is more vivid.

Overall, the basic character correspondence seems achievable, but it lacks a sense of overall cohesion. The overall effect is not as good as the two versions below, but as a small toy, it should be satisfactory.

Wang Bin’s AI Couplets: https://ai.binwang.me/couplet/
Microsoft Couplets: https://duilian.msra.cn/default.htm