Last year, I wrote "DGCNN: A CNN-based Reading Comprehension Question Answering Model", which introduced a simple, pure-convolutional question-answering model. At that time, it was implemented in TensorFlow and was not open-sourced. Over the past few days, I took some time to reproduce it using Keras and decided to open-source it.
Model Overview
I will not repeat the basic introduction of DGCNN here. The model released this time is not a direct duplicate of the previous implementation but includes several modifications. Here, I will only introduce the parts that have been changed.
The model released here achieves a score of approximately 0.72 on the offline validation set (previously it was about 0.75);
This model is character-based and uses the "Character-Word Hybrid Embedding" that I explored previously (the previous version was word-based);
This model completely removes manual features (the previous version used 8 manual features);
This model removes Position Embeddings (the previous version concatenated Position Embeddings to the input);
The model architecture and training details have been slightly fine-tuned.
Among these changes, using characters as the unit makes the model’s labeling more flexible (avoiding word segmentation errors); removing manual features also enhances the model’s flexibility and improves prediction speed. As for removing Position Embeddings, it was because several tests showed that Position Embeddings did not provide a significant improvement. Other adjustments include using the latest RAdam optimizer for training, and so on.
This release does not strictly pursue an increase in scores but is purely intended to provide a Keras version for everyone’s reference. I believe there is still plenty of room for improvement, and interested friends are welcome to debug and tune it themselves (both the code and the dataset have been made available).
Open Source Links
GitHub Address: https://github.com/bojone/dgcnn_for_reading_comprehension
(Running environment: Python 2.7 + TensorFlow 1.8 + Keras 2.2.4. Please
do not contact me regarding environment issues, thank you!)
Word Vectors: https://pan.baidu.com/s/1YYE2T3f-lPyLBrJuUowAsA,
Password: 5p0h
Dataset: https://pan.baidu.com/s/11C21BAupOpiYWoOx23J7Mg,
Password: dh9w
(If there is any impropriety in the open-sourcing of the dataset, please inform me by email, and I will delete it as soon as possible.)
Final Words
I hope everyone enjoys using it, and I welcome further exchange of ideas!
When reposting, please include the
original address of this article:
https://kexue.fm/archives/6906
For more detailed information
regarding reposting, please refer to:
"Scientific Space FAQ"