A Lightweight Deep Learning Word Segmentation System: NNCWS v0.1

A Few Words

What are the features of this program? Almost none! This article simply implements an n-grams based (7-grams used in the program) segmentation system by combining neural networks with character vectors. It does not use high-end models like those in [Chinese Word Segmentation Series] 4. Seq2seq Character Tagging Based on Bidirectional LSTM, nor can it perform unsupervised training like [Chinese Word Segmentation Series] 5. Unsupervised Word Segmentation Based on Language Models. This is purely a simple supervised model, and the training data is the 2014 People’s Daily annotated corpus.

So, what is the significance of this program? Two words: Lightweight! Current deep learning programs are quite massive and require various dependency libraries. Installing these libraries on certain platforms, such as Windows, is not always easy. Furthermore, the number of parameters can be exceptionally large, to the point where speed becomes an issue. Therefore, many deep learning projects are just for "playing around" in the lab; at most, they result in a paper and are far from real production applications. This program has been streamlined as much as possible: character vector dimensions and model scale have been reduced, and finally, the program was reproduced using NumPy. That is to say, I trained the model using Keras, extracted the model parameters, and then called these parameters using NumPy. Consequently, the final program only needs NumPy to run. Therefore, the main feature of the program is its light weight!

Of course, this kind of neural network-based segmentation system already possesses a preliminary level of semantic understanding. This is mainly reflected in its decent performance in segmenting ambiguous combinations and identifying entities such as personal names and geographical locations. Therefore, it is still worth using in general scenarios. Since it was trained on the People’s Daily corpus, it performs better on news-related text.

Download and Usage

GitHub Address: https://github.com/bojone/NNCWS

First, install the dependency NumPy, then run:

git clone https://github.com/bojone/NNCWS.git
cd NNCWS
python

Then you can use it directly:

from nncws import NNCWS
mycut = NNCWS()
s = u'As a leader of a small country, Castro was bound to be unable to be compared with Mao Zedong. However, in Castro, the glory of that great era was reflected after all. Today, we bid farewell to Castro; what we are bidding farewell to is that great era, but what we want to keep is the immortal spiritual core of that era: --- the firm pursuit of ideals. --- the firm belief in national independence. --- the "tough bone" spirit that is not afraid of threats!'
print(' '.join(mycut.cut_words(s)))

s = u'In January 2000, Robin Li founded Baidu. After more than ten years of development, Baidu has developed into the world\'s second largest independent search engine and the largest Chinese search engine. Baidu\'s success has also made China one of the only four countries in the world, besides the United States, Russia, and South Korea, to possess core search engine technology. In 2005, Baidu successfully listed on the NASDAQ in the United States and became the first Chinese company to enter the NASDAQ Component Index. Baidu has become one of China\'s most valuable brands.'
print(' '.join(mycut.cut_words(s)))

Output:

As a leader of a small country , Castro was bound to be unable to be compared with Mao Zedong . However , in Castro , the glory of that great era was reflected after all . Today , we bid farewell to Castro , what we are bidding farewell to is that great era , but what we want to keep is the immortal spiritual core of that era : — the firm pursuit of ideals . — the firm belief in national independence . — the " tough bone " spirit that is not afraid of threats !

2000 January , Robin Li founded Baidu . After more than ten years of development , Baidu has already developed into the world’s second largest independent search engine and the largest Chinese search engine . Baidu’s success , also made China become United States , Russia and South Korea besides , the world’s only 4 countries possessing search engine core technology one of . 2005 , Baidu in United States NASDAQ successfully listed , and became the first to enter NASDAQ component stock of Chinese company . Baidu has already become China’s most valuable brand one of .

Training Process

If you are interested in the model structure and training process, please look at the nncws_train.py file. It contains the processing of the 2014 People’s Daily corpus, the model structure, and the training. The file is very short and clear, so I won’t say much more. Training also requires Keras to be installed.

If you don’t want to look at the code, just place the script in the directory of the People’s Daily corpus and run python nncws_train.py. The 2014 People’s Daily corpus can be found and downloaded by searching online.

Concluding Remarks

In fact, this is just a toy, which is why it is version 0.1; it is likely still far from being truly practical. However, it represents a new attempt: to lower the barrier to applying technologies like deep learning (it doesn’t matter if the training barrier is high; the public cares about the application barrier), leveraging the effectiveness of new models while retaining convenience.

An obvious disadvantage of such a model is that if users find the model does not meet their requirements, they cannot adjust it themselves. In contrast, with traditional dictionary-based methods, users only need to add to the dictionary themselves, which is more flexible. Therefore, the next goal is to integrate supervised training, unsupervised new word discovery, and word lists. I already have some ideas for this, so please stay tuned.

Original Address: https://kexue.fm/archives/4114

For more details on reposting, please refer to: Scientific Space FAQ