{"id":14958663,"url":"https://github.com/yongzhuo/keras-textclassification","last_synced_at":"2025-05-15T03:07:50.172Z","repository":{"id":37664679,"uuid":"191784463","full_name":"yongzhuo/Keras-TextClassification","owner":"yongzhuo","description":"中文长文本分类、短句子分类、多标签分类、两句子相似度（Chinese Text Classification of Keras NLP, multi-label classify, or sentence classify, long or short），字词句向量嵌入层（embeddings）和网络层（graph）构建基类，FastText，TextCNN，CharCNN，TextRNN,  RCNN,  DCNN, DPCNN, VDCNN, CRNN, Bert, Xlnet, Albert, Attention, DeepMoji, HAN, 胶囊网络-CapsuleNet, Transformer-encode,  Seq2seq,  SWEM, LEAM, TextGCN","archived":false,"fork":false,"pushed_at":"2024-06-17T22:45:14.000Z","size":615,"stargazers_count":1803,"open_issues_count":3,"forks_count":404,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-05-15T03:07:44.387Z","etag":null,"topics":["albert","bert","capsule","charcnn","crnn","dcnn","dpcnn","embeddings","fasttext","han","keras","keras-textclassification","leam","nlp","rcnn","text-classification","textcnn","transformer","vdcnn","xlnet"],"latest_commit_sha":null,"homepage":"https://blog.csdn.net/rensihui","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yongzhuo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-13T15:02:31.000Z","updated_at":"2025-05-14T08:30:22.000Z","dependencies_parsed_at":"2024-05-03T20:47:22.906Z","dependency_job_id":"6724b62b-46a0-417a-a8e5-a97b723348ca","html_url":"https://github.com/yongzhuo/Keras-TextClassification","commit_stats":{"total_commits":135,"total_committers":3,"mean_commits":45.0,"dds":0.3851851851851852,"last_synced_commit":"af802f1dc6962cab2c47f93c123f18f169bacf1a"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yongzhuo%2FKeras-TextClassification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yongzhuo%2FKeras-TextClassification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yongzhuo%2FKeras-TextClassification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yongzhuo%2FKeras-TextClassification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yongzhuo","download_url":"https://codeload.github.com/yongzhuo/Keras-TextClassification/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264769,"owners_count":22041794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["albert","bert","capsule","charcnn","crnn","dcnn","dpcnn","embeddings","fasttext","han","keras","keras-textclassification","leam","nlp","rcnn","text-classification","textcnn","transformer","vdcnn","xlnet"],"created_at":"2024-09-24T13:17:46.655Z","updated_at":"2025-05-15T03:07:45.145Z","avatar_url":"https://github.com/yongzhuo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Keras-TextClassification](https://github.com/yongzhuo/Keras-TextClassification)\n\n[![PyPI](https://img.shields.io/pypi/v/Keras-TextClassification)](https://pypi.org/project/Keras-TextClassification/)\n[![Build Status](https://travis-ci.com/yongzhuo/Keras-TextClassification.svg?branch=master)](https://travis-ci.com/yongzhuo/Keras-TextClassification)\n[![PyPI_downloads](https://img.shields.io/pypi/dm/Keras-TextClassification)](https://pypi.org/project/Keras-TextClassification/)\n[![Stars](https://img.shields.io/github/stars/yongzhuo/Keras-TextClassification?style=social)](https://github.com/yongzhuo/Keras-TextClassification/stargazers)\n[![Forks](https://img.shields.io/github/forks/yongzhuo/Keras-TextClassification.svg?style=social)](https://github.com/yongzhuo/Keras-TextClassification/network/members)\n[![Join the chat at https://gitter.im/yongzhuo/Keras-TextClassification](https://badges.gitter.im/yongzhuo/Keras-TextClassification.svg)](https://gitter.im/yongzhuo/Keras-TextClassification?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\n\n\n# Install(安装)\n\n```bash\npip install Keras-TextClassification\n```\n\n```python\nstep2: download and unzip the dir of 'data.rar', 地址: 链接：https://pan.baidu.com/s/1pIDzGaGXCZ7cjng1XU_kPA   提取码：w6ps   压缩包密码: 2022\n       cover the dir of data to anaconda, like '/anaconda/3.5.1/envs/tensorflow13/Lib/site-packages/keras_textclassification/data'\nstep3: goto # Train\u0026Usage(调用) and Predict\u0026Usage(调用)\n```\n\n# keras_textclassification（代码主体,未完待续...）\n    - Electra-fineture(todo)\n    - Albert-fineture\n    - Xlnet-fineture\n    - Bert-fineture\n    - FastText\n    - TextCNN\n    - charCNN\n    - TextRNN\n    - TextRCNN\n    - TextDCNN\n    - TextDPCNN\n    - TextVDCNN\n    - TextCRNN\n    - DeepMoji\n    - SelfAttention\n    - HAN\n    - CapsuleNet\n    - Transformer-encode\n    - SWEM\n    - LEAM\n    - TextGCN(todo)\n\n\n# run(运行, 以FastText为例)\n    - 1. 进入keras_textclassification/m01_FastText目录，\n    - 2. 训练: 运行 train.py,   例如: python train.py\n    - 3. 预测: 运行 predict.py, 例如: python predict.py\n    - 说明: 默认不带pre train的random embedding，训练和验证语料只有100条，完整语料移步下面data查看下载\n\n# run(多标签分类/Embedding/test/sample实例)\n    - bert,word2vec,random样例在test/目录下, 注意word2vec(char or word), random-word,  bert(chinese_L-12_H-768_A-12)未全部加载,需要下载\n    - multi_multi_class/目录下以text-cnn为例进行多标签分类实例，转化为multi-onehot标签类别，分类则取一定阀值的类\n    - sentence_similarity/目录下以bert为例进行两个句子文本相似度计算,数据格式如data/sim_webank/目录下所示\n    - predict_bert_text_cnn.py\n    - tet_char_bert_embedding.py\n    - tet_char_bert_embedding.py\n    - tet_char_xlnet_embedding.py\n    - tet_char_random_embedding.py\n    - tet_char_word2vec_embedding.py\n    - tet_word_random_embedding.py\n    - tet_word_word2vec_embedding.py\n\n# keras_textclassification/data\n    - 数据下载\n      ** github项目中只是上传部分数据，需要的前往链接: https://pan.baidu.com/s/1I3vydhmFEQ9nuPG2fDou8Q 提取码: rket\n    - baidu_qa_2019（百度qa问答语料，只取title作为分类样本，17个类，有一个是空''，已经压缩上传）\n       - baike_qa_train.csv\n       - baike_qa_valid.csv\n    - byte_multi_news（今日头条2018新闻标题多标签语料，1070个标签，fate233爬取, 地址为: [byte_multi_news](https://github.com/fate233/toutiao-multilevel-text-classfication-dataset)）\n       -labels.csv\n       -train.csv\n       -valid.csv\n    - embeddings\n       - chinese_L-12_H-768_A-12/(取谷歌预训练好点的模型,已经压缩上传,\n                                  keras-bert还可以加载百度版ernie(需转换，[https://github.com/ArthurRizar/tensorflow_ernie](https://github.com/ArthurRizar/tensorflow_ernie)),\n                                  哈工大版bert-wwm(tf框架，[https://github.com/ymcui/Chinese-BERT-wwm](https://github.com/ymcui/Chinese-BERT-wwm))\n       - albert_base_zh/(brightmart训练的albert, 地址为https://github.com/brightmart/albert_zh)\n       - chinese_xlnet_base_L-12_H-768_A-12/(哈工大预训练的中文xlnet模型[https://github.com/ymcui/Chinese-PreTrained-XLNet],12层)\n       - term_char.txt(已经上传, 项目中已全, wiki字典, 还可以用新华字典什么的)\n       - term_word.txt(未上传, 项目中只有部分, 可参考词向量的)\n       - w2v_model_merge_short.vec(未上传, 项目中只有部分, 词向量, 可以用自己的)\n       - w2v_model_wiki_char.vec(已上传百度网盘, 项目中只有部分, 自己训练的维基百科字向量, 可以用自己的)\n    - model\n       - fast_text/预训练模型存放地址\n\n# 项目说明\n  - 1. 构建了base基类(网络(graph)、向量嵌入(词、字、句子embedding)),后边的具体模型继承它们，代码简单\n  - 2. keras_layers存放一些常用的layer, conf存放项目数据、模型的地址, data存放数据和语料, data_preprocess为数据预处理模块,\n\n\n# 模型与论文paper题与地址\n* FastText:   [Bag of Tricks for Efficient Text Classification](https://arxiv.org/abs/1607.01759)\n* TextCNN：   [Convolutional Neural Networks for Sentence Classification](https://arxiv.org/abs/1408.5882)\n* charCNN-kim：   [Character-Aware Neural Language Models](https://arxiv.org/abs/1508.06615)\n* charCNN-zhang:  [Character-level Convolutional Networks for Text Classification](https://arxiv.org/pdf/1509.01626.pdf)\n* TextRNN：   [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://www.ijcai.org/Proceedings/16/Papers/408.pdf)\n* RCNN：      [Recurrent Convolutional Neural Networks for Text Classification](http://www.nlpr.ia.ac.cn/cip/~liukang/liukangPageFile/Recurrent%20Convolutional%20Neural%20Networks%20for%20Text%20Classification.pdf)\n* DCNN:       [A Convolutional Neural Network for Modelling Sentences](https://arxiv.org/abs/1404.2188)\n* DPCNN:      [Deep Pyramid Convolutional Neural Networks for Text Categorization](https://www.aclweb.org/anthology/P17-1052)\n* VDCNN:      [Very Deep Convolutional Networks](https://www.aclweb.org/anthology/E17-1104)\n* CRNN:        [A C-LSTM Neural Network for Text Classification](https://arxiv.org/abs/1511.08630)\n* DeepMoji:    [Using millions of emojio ccurrences to learn any-domain represent ations for detecting sentiment, emotion and sarcasm](https://arxiv.org/abs/1708.00524)\n* SelfAttention: [Attention Is All You Need](https://arxiv.org/abs/1706.03762)\n* HAN: [Hierarchical Attention Networks for Document Classification](https://www.cs.cmu.edu/~diyiy/docs/naacl16.pdf)\n* CapsuleNet: [Dynamic Routing Between Capsules](https://arxiv.org/pdf/1710.09829.pdf)\n* Transformer(encode or decode): [Attention Is All You Need](https://arxiv.org/abs/1706.03762)\n* Bert:                  [BERT: Pre-trainingofDeepBidirectionalTransformersfor LanguageUnderstanding]()\n* Xlnet:                 [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://arxiv.org/abs/1906.08237)\n* Albert:                [ALBERT: A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS](https://arxiv.org/pdf/1909.11942.pdf)\n* RoBERTa:               [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://arxiv.org/abs/1907.11692)\n* ELECTRA:               [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB)\n* TextGCN:               [Graph Convolutional Networks for Text Classification](https://arxiv.org/abs/1809.05679)\n\n\n# 参考/感谢\n* 文本分类项目:   [https://github.com/mosu027/TextClassification](https://github.com/mosu027/TextClassification)\n* 文本分类看山杯: [https://github.com/brightmart/text_classification](https://github.com/brightmart/text_classification)\n* Kashgari项目: [https://github.com/BrikerMan/Kashgari](https://github.com/BrikerMan/Kashgari)\n* 文本分类Ipty : [https://github.com/lpty/classifier](https://github.com/lpty/classifier)\n* keras文本分类: [https://github.com/ShawnyXiao/TextClassification-Keras](https://github.com/ShawnyXiao/TextClassification-Keras)\n* keras文本分类: [https://github.com/AlexYangLi/TextClassification](https://github.com/AlexYangLi/TextClassification)\n* CapsuleNet模型: [https://github.com/bojone/Capsule](https://github.com/bojone/Capsule)\n* transformer模型: [https://github.com/CyberZHG/keras-transformer](https://github.com/CyberZHG/keras-transformer)\n* keras_albert_model: [https://github.com/TinkerMob/keras_albert_model](https://github.com/TinkerMob/keras_albert_model)\n\n\n# 训练简单调用:\n```python\nfrom keras_textclassification import train\ntrain(graph='TextCNN', # 必填, 算法名, 可选\"ALBERT\",\"BERT\",\"XLNET\",\"FASTTEXT\",\"TEXTCNN\",\"CHARCNN\",\n                       # \"TEXTRNN\",\"RCNN\",\"DCNN\",\"DPCNN\",\"VDCNN\",\"CRNN\",\"DEEPMOJI\",\n                       # \"SELFATTENTION\", \"HAN\",\"CAPSULE\",\"TRANSFORMER\"\n     label=17,         # 必填, 类别数, 训练集和测试集合必须一样\n     path_train_data=None, # 必填, 训练数据文件, csv格式, 必须含'label,ques'头文件, 详见keras_textclassification/data\n     path_dev_data=None, # 必填, 测试数据文件, csv格式, 必须含'label,ques'头文件, 详见keras_textclassification/data\n     rate=1,             # 可填, 训练数据选取比例\n     hyper_parameters=None) # 可填, json格式, 超参数, 默认embedding为'char','random'\n```\n\n# Reference\nFor citing this work, you can refer to the present GitHub project. For example, with BibTeX:\n```\n@misc{Keras-TextClassification,\n    howpublished = {\\url{https://github.com/yongzhuo/Keras-TextClassification}},\n    title = {Keras-TextClassification},\n    author = {Yongzhuo Mo},\n    publisher = {GitHub},\n    year = {2019}\n}\n```\n\n*希望对你有所帮助!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyongzhuo%2Fkeras-textclassification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyongzhuo%2Fkeras-textclassification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyongzhuo%2Fkeras-textclassification/lists"}