Poetry Robot Based on CNN and VAE: Random Poem Generation

CNN + VAE

As far as the toy in this article is concerned, it is actually a relatively simple model, mainly combining 1D CNN and VAE. Since the length of the generated poetry is fixed, I used pure CNN for both the encoder and the decoder. The structure of the model is roughly as follows:

Specifically, each character is first embedded into a vector, then stacked CNNs are used for encoding, followed by pooling to obtain an encoder result. Based on this result, the mean and variance are calculated to generate a normal distribution and resample. In the decoding stage, since there is only one output from the encoder but multiple characters need to be output at the end, several different fully connected layers are first connected to obtain diverse outputs, followed by more fully connected layers.

GCNN

The CNN used here is not a standard CNN+ReLU, but the GCNN proposed by Facebook. It essentially involves creating two different CNNs with the same shape: one without an activation function and one using a sigmoid activation, then multiplying the results together. In this way, the sigmoid part acts as a “gate.”

I first saw GCNN in the paper “Language Modeling with Gated Convolutional Networks,” and then saw it again in “Convolutional Sequence to Sequence Learning.” I also briefly introduced it in my slide “Fancy Natural Language Processing.”

Based on actual testing, GCNN performs significantly better than ordinary CNN+ReLU on many NLP tasks.

Code

With the previous discussion and by combining the VAE example provided with Keras, implementing the entire model is not difficult. For demonstration purposes, I only selected the simplest five-character poetry (Wuyan) and did not require the generation of a complete poem, only a single couplet (10 characters). This actually makes it more like composing couplets.

Code: https://github.com/bojone/vae/blob/master/vae_shi.py

The training corpus used is the Complete Tang Poems (Quan Tang Shi), which has also been placed on GitHub. The model has not undergone extensive hyperparameter tuning; this is left for interested readers to experiment with and enhance.

Training

To observe the changes in the generated lines during the training process, I wrote an evaluator. The effect is shown below:

As can be seen, as the number of training iterations increases, the quality of the poetry lines indeed improves.

Testing

Below are some poetry lines randomly generated by the trained model. Strictly speaking, they are not exceptional; after all, this is just a mapping from random numbers to poetry lines—random poetry generation. It can be seen that these “poetry lines” are quite decent in terms of antithesis and tonal patterns.

A guest where no flowers bloom above, meeting for a single day’s time.
Following the strategy of the great carriage, the desolate canopy rests the carriage spear.
Now seeing the green clothes depart, a desolate chill in the white-leaf wind.
The imperial city is not at war today, the punishment for crime lies with the heavenly soldiers.
The crane looks up deep in the mountains, lingering as it exits the great return.
The painted pass leans over the water, the fading colors far surpass the poplars.
War exists on all upper roads, the four gates are not as good as the soldiers.
Mountain stream mist holds the dripping rain, wind shadows move the windy wind.
Returning to a setting scene, meeting again in one place.
Fragrant wine is not without intoxication, the long tower’s wine is even more like spring.
The heavenly moon fills the cloud pipe, the Chu maiden is heard long in the south.
Bright today, today it exists; one cannot say there is no life.
The morning light opens the green chrysanthemums, fragrant shadows fall from the red branches.
A place of mutual insignificance outside the world, carefree without old eyes.
Only hearing the color of contained jade, how can one see the moonlight’s glow.
The spring breeze brings the music festival, for whom is the wind-garment not yet entertained.
Where is the Lord of ten thousand years, this year’s wine is not yet new.
Turning the reins to summon each other, the song joins the envoy’s journey.
Today there are a thousand people drunk, otherwise a single son’s heart.
Today the wind floats at the cloud gathering, the clear day makes the white clothes new.

Note: The demonstration model only generates single lines; therefore, there is no correlation between the lines of these verses.