https://github.com/wut0n9/cnn_chinese_text_classification
运用cnn + highway network网络结构中文文本分类
https://github.com/wut0n9/cnn_chinese_text_classification
chinese-characters chinese-nlp cnn cnn-text-classification deep-learning highway-network mxnet
Last synced: 22 days ago
JSON representation
运用cnn + highway network网络结构中文文本分类
- Host: GitHub
- URL: https://github.com/wut0n9/cnn_chinese_text_classification
- Owner: wut0n9
- Created: 2017-09-25T15:31:28.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2017-09-25T15:44:24.000Z (over 7 years ago)
- Last Synced: 2024-11-08T18:46:09.451Z (6 months ago)
- Topics: chinese-characters, chinese-nlp, cnn, cnn-text-classification, deep-learning, highway-network, mxnet
- Language: Python
- Size: 2.07 MB
- Stars: 13
- Watchers: 3
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-MXNet - cnn+Highway Net
README
Implementing CNN + Highway Network for Chinese Text Classification in MXNet
============
Sentiment classification forked from [incubator-mxnet/cnn_text_classification/](https://github.com/apache/incubator-mxnet/tree/master/example/cnn_text_classification), i've implemented the [Highway Networks](https://arxiv.org/pdf/1505.00387.pdf) architecture.The final train model is CNN + Highway Network structure, and this version can achieve a best dev accuracy of 94.75% with the Chinese corpus.
It is a slightly simplified implementation of Kim's [Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882) paper in MXNet.
Recently, I have been learning mxnet for Natural Language Processing (NLP). I followed this nice blog ["Implementing a CNN for Text Classification in Tensorflow" blog post.](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/) to reimplement it by mxnet framework.
Data preprocessing code and corpus are directly borrowed from original author [cnn-text-classification-tf](https://github.com/dennybritz/cnn-text-classification-tf).
## Performance compared to original paper
I use the same pretrained word2vec [GoogleNews-vectors-negative300.bin](https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing) in Kim's paper. However, I don't implement L2-normalization of weights on penultimate layer, but provide a L2-normalization of gradients.
Finally, I got a best dev accuracy 80.1%, close to 81% that reported in the original paper.
## Data
Please download the corpus from this repository [cnn-text-classification-tf](https://github.com/dennybritz/cnn-text-classification-tf), :)
'data/rt.vec', this file was trained on the corpus by word2vec tool. I recommend to use GoogleNews word2vec, which could get better performance, since
this corpus is small (contains about 10K sentences).
When using GoogleNews word2vec, this code loads it with gensim tools [gensim](https://github.com/piskvorky/gensim/tree/develop/gensim/models).
## Remark
If I were wrong in CNN implementation via mxnet, please correct me.
## References
- ["Implementing a CNN for Text Classification in Tensorflow" blog post.](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/)
- [Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882)