{"id":13578981,"url":"https://github.com/yoonkim/CNN_sentence","last_synced_at":"2025-04-05T20:32:47.910Z","repository":{"id":41497650,"uuid":"27521371","full_name":"yoonkim/CNN_sentence","owner":"yoonkim","description":"CNNs for sentence classification","archived":false,"fork":false,"pushed_at":"2018-04-26T11:15:57.000Z","size":506,"stargazers_count":2048,"open_issues_count":44,"forks_count":825,"subscribers_count":85,"default_branch":"master","last_synced_at":"2025-03-31T23:34:10.704Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yoonkim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-04T03:33:02.000Z","updated_at":"2025-03-20T10:14:17.000Z","dependencies_parsed_at":"2022-09-05T07:00:21.148Z","dependency_job_id":null,"html_url":"https://github.com/yoonkim/CNN_sentence","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoonkim%2FCNN_sentence","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoonkim%2FCNN_sentence/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoonkim%2FCNN_sentence/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoonkim%2FCNN_sentence/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yoonkim","download_url":"https://codeload.github.com/yoonkim/CNN_sentence/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247399818,"owners_count":20932875,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:35.566Z","updated_at":"2025-04-05T20:32:47.512Z","avatar_url":"https://github.com/yoonkim.png","language":"Python","funding_links":[],"categories":["Python","Uncategorized"],"sub_categories":["Uncategorized"],"readme":"## Convolutional Neural Networks for Sentence Classification\nCode for the paper [Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882) (EMNLP 2014).\n\nRuns the model on Pang and Lee's movie review dataset (MR in the paper).\nPlease cite the original paper when using the data.\n\n### Requirements\nCode is written in Python (2.7) and requires Theano (0.7).\n\nUsing the pre-trained `word2vec` vectors will also require downloading the binary file from\nhttps://code.google.com/p/word2vec/\n\n\n### Data Preprocessing\nTo process the raw data, run\n\n```\npython process_data.py path\n```\n\nwhere path points to the word2vec binary file (i.e. `GoogleNews-vectors-negative300.bin` file). \nThis will create a pickle object called `mr.p` in the same folder, which contains the dataset\nin the right format.\n\nNote: This will create the dataset with different fold-assignments than was used in the paper.\nYou should still be getting a CV score of \u003e81% with CNN-nonstatic model, though.\n\n### Running the models (CPU)\nExample commands:\n\n```\nTHEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -nonstatic -rand\nTHEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -static -word2vec\nTHEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -nonstatic -word2vec\n```\n\nThis will run the CNN-rand, CNN-static, and CNN-nonstatic models respectively in the paper.\n\n### Using the GPU\nGPU will result in a good 10x to 20x speed-up, so it is highly recommended. \nTo use the GPU, simply change `device=cpu` to `device=gpu` (or whichever gpu you are using).\nFor example:\n```\nTHEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python conv_net_sentence.py -nonstatic -word2vec\n```\n\n### Example output\nCPU output:\n```\nepoch: 1, training time: 219.72 secs, train perf: 81.79 %, val perf: 79.26 %\nepoch: 2, training time: 219.55 secs, train perf: 82.64 %, val perf: 76.84 %\nepoch: 3, training time: 219.54 secs, train perf: 92.06 %, val perf: 80.95 %\n```\nGPU output:\n```\nepoch: 1, training time: 16.49 secs, train perf: 81.80 %, val perf: 78.32 %\nepoch: 2, training time: 16.12 secs, train perf: 82.53 %, val perf: 76.74 %\nepoch: 3, training time: 16.16 secs, train perf: 91.87 %, val perf: 81.37 %\n```\n\n### Other Implementations\n#### TensorFlow\n[Denny Britz](http://www.wildml.com) has an implementation of the model in TensorFlow:\n\nhttps://github.com/dennybritz/cnn-text-classification-tf\n\nHe also wrote a [nice tutorial](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow) on it, as well as a general tutorial on [CNNs for NLP](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp).\n\n#### Torch\n[HarvardNLP](http://harvardnlp.github.io/) group has an implementation in Torch.\n\nhttps://github.com/harvardnlp/sent-conv-torch\n\n### Hyperparameters\nAt the time of my original experiments I did not have access to a GPU so I could not run a lot of different experiments.\nHence the paper is missing a lot of things like ablation studies and variance in performance, and some of the conclusions\nwere premature (e.g. regularization does not always seem to help).\n\nYe Zhang has written a [very nice paper](http://arxiv.org/abs/1510.03820) doing an extensive analysis of model variants (e.g. filter widths, k-max pooling, word2vec vs Glove, etc.) and their effect on performance.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoonkim%2FCNN_sentence","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyoonkim%2FCNN_sentence","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoonkim%2FCNN_sentence/lists"}