{"id":13765854,"url":"https://github.com/dinghanshen/SWEM","last_synced_at":"2025-05-10T21:32:09.426Z","repository":{"id":92718327,"uuid":"133176879","full_name":"dinghanshen/SWEM","owner":"dinghanshen","description":"The Tensorflow code for this ACL 2018 paper: \"Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms\"","archived":false,"fork":false,"pushed_at":"2022-12-17T18:53:21.000Z","size":8112,"stargazers_count":283,"open_issues_count":10,"forks_count":54,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-08-03T16:08:57.428Z","etag":null,"topics":["deep-learning","natural-language-processing","representation-learning","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dinghanshen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-05-12T19:23:46.000Z","updated_at":"2024-06-27T06:48:05.000Z","dependencies_parsed_at":"2023-08-22T09:18:04.396Z","dependency_job_id":null,"html_url":"https://github.com/dinghanshen/SWEM","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dinghanshen%2FSWEM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dinghanshen%2FSWEM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dinghanshen%2FSWEM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dinghanshen%2FSWEM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dinghanshen","download_url":"https://codeload.github.com/dinghanshen/SWEM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224989752,"owners_count":17403409,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","natural-language-processing","representation-learning","tensorflow"],"created_at":"2024-08-03T16:00:47.310Z","updated_at":"2024-11-17T01:31:36.737Z","avatar_url":"https://github.com/dinghanshen.png","language":"Python","funding_links":[],"categories":["Pooling Methods","Python"],"sub_categories":[],"readme":"# SWEM (Simple Word-Embedding-based Models)\n\nThis repository contains source code necessary to reproduce the results presented in the following paper:\n* [*Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms*](https://arxiv.org/abs/1805.09843) (ACL 2018)\n\nThis project is maintained by [Dinghan Shen](https://sites.google.com/view/dinghanshen). Feel free to contact dinghan.shen@duke.edu for any relevant issues.\n\n## Prerequisite: \n* CUDA, cudnn\n* Python 2.7\n* Tensorflow (version \u003e1.0). We used tensorflow 1.5.\n* Run: `pip install -r requirements.txt` to install requirements\n\n## Data: \n* For convenience, we provide pre-processed versions for the following datasets: DBpedia, SNLI, Yahoo. Data are prepared in pickle format, and each `.p` file has the same fields in the same order: \n\t* `train_text`, `val_text`, `test_text`, `train_label`, `val_label`, `test_label`, `dictionary(wordtoix)`, `reverse dictionary(ixtoword)`\n\n* These `.p` files can be downloaded from the links below. After downloading, you can put them into a `data` folder:\n\n \t* Ontology classification: [DBpedia (591MB)](https://drive.google.com/open?id=1EBmMise0LQu0QpO7T4a32WMFuTxAb6T0)\n\n \t* Natural language inference: [SNLI (101MB)](https://drive.google.com/open?id=1M13UswHThZYt-ARrHg6sN7Dlel-d6BB3),  [SNLI-word-embeddings (129MB)](https://drive.google.com/open?id=1qzClw-ZJzWZ4ce0eXAG4dDXYUNnmeX4f)\n\n \t* Topic categorization: [Yahoo (1.7GB)](https://drive.google.com/open?id=1Dorz_CWZkHHpojVS4K4YUEhhczVLQgRc)\n\n## Run \n* Run: `python eval_dbpedia_emb.py` for ontology classification on the DBpedia dataset\n* Run: `python eval_snli_emb.py` for natural language inference on the SNLI dataset\n* Run: `python eval_yahoo_emb.py` for topic categorization on the Yahoo! Answer dataset\n\n* Options: options can be made by changing `option` class in any of the above three files: \n- `opt.emb_size`: number of word embedding dimensions.\n- `opt.drop_rate`: the keep rate of dropout layer.\n- `opt.lr`: learning rate.\n- `opt.batch_size`: number of batch size.\n- `opt.H_dis`: the dimension of last hidden layer.\n\n* On a K80 GPU machine, training roughly takes about 3 minutes each epoch and 5 epochs for Debpedia to converge, 50 seconds each epoch and 20 epochs for SNLI, and 4 minutes each epoch and 5 epochs for the Yahoo dataset.\n\n## Subspace Training \u0026 Intrinsic Dimension\nTo measure the [*intrinsic dimension*](https://eng.uber.com/intrinsic-dimension/) of word-embedding-based text classification tasks, we compare SWEM and CNNs via subspace training in Section 5.1 of the paper. \n\nPlease follow the instructions in folder [`intrinsic_dimension`](./intrinsic_dimension) to reproduce the results.\n\n## Citation \nPlease cite our ACL paper in your publications if it helps your research:\n\n```latex\n@inproceedings{Shen2018Baseline, \ntitle={Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms}, \nauthor={Shen, Dinghan and Wang, Guoyin and Wang, Wenlin and Renqiang Min, Martin and Su, Qinliang and Zhang, Yizhe and Li, Chunyuan and Henao, Ricardo and Carin, Lawrence}, \nbooktitle={ACL}, \nyear={2018} \n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdinghanshen%2FSWEM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdinghanshen%2FSWEM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdinghanshen%2FSWEM/lists"}