{"id":19373652,"url":"https://github.com/4ai/agn","last_synced_at":"2025-04-23T17:32:06.764Z","repository":{"id":45963461,"uuid":"318686074","full_name":"4AI/AGN","owner":"4AI","description":"Official Code for Merging Statistical Feature via Adaptive Gate for Improved Text Classification (AAAI2021)","archived":false,"fork":false,"pushed_at":"2022-02-05T05:36:27.000Z","size":57,"stargazers_count":26,"open_issues_count":2,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T17:52:58.166Z","etag":null,"topics":["bert","deep-learning","nlp","text-classification"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/4AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-05T02:42:10.000Z","updated_at":"2024-05-21T08:31:45.000Z","dependencies_parsed_at":"2022-07-20T14:47:49.415Z","dependency_job_id":null,"html_url":"https://github.com/4AI/AGN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FAGN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FAGN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FAGN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FAGN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/4AI","download_url":"https://codeload.github.com/4AI/AGN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250480584,"owners_count":21437574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","deep-learning","nlp","text-classification"],"created_at":"2024-11-10T08:30:41.700Z","updated_at":"2025-04-23T17:32:06.482Z","avatar_url":"https://github.com/4AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AGN\nOfficial Code for [Merging Statistical Feature via Adaptive Gate for Improved Text Classification](https://ojs.aaai.org/index.php/AAAI/article/view/17569) (AAAI2021)\n\n## Prepare Data\n\n### Dataset\n\n|    Dataset   |                                URL                               |\n|:------------:|:----------------------------------------------------------------:|\n|     Subj     |      http://www.cs.cornell.edu/people/pabo/movie-review-data/     |\n|    SST-1/2   |                http://nlp.stanford.edu/sentiment/                |\n|     TREC     |                  https://trec.nist.gov/data.html                 |\n|   AG's News  | http://groups.di.unipi.it/~gulli/AG_corpus_of_news_articles      |\n| Yelp P. / F. |                   https://www.yelp.com/dataset/                  |\n\nYou first need to download datasets from official sites. Then convert the data into `JSONL` style, as follows:\n\n```json\n{\"label\": \"0\", \"text\": \"hoffman waits too long to turn his movie in an unexpected direction , and even then his tone retains a genteel , prep-school quality that feels dusty and leatherbound .\"}\n{\"label\": \"1\", \"text\": \"if you 're not deeply touched by this movie , check your pulse .\"}\n```\n\nEach line is a JSON object, in which two fields `text` and `label` are required.\n\n\n\n### Pretrained Language Model\n\nWe apply the pretrained `Uncased-Bert-Base` model in this paper, you can download it by [this url](https://storage.googleapis.com/bert_models/2020_02_20/uncased_L-12_H-768_A-12.zip) directly. \n\n\n## Setup Environment\n\nWe recommend you create a virtual environment to conduct experiments. \n\n```bash\n$ python -m venv agn\n$ source agn/bin/activate\n```\n\nYou should install TensorFlow in terms of your environment. Note that we only test the code under `tensorflow\u003c2.0`, greater versions may not be compatible.\n\nOur environments:\n\n```bash\n$ pip list | egrep \"tensorflow|Keras|langml\"\nKeras                            2.3.1\nlangml                           0.1.0\ntensorflow                       1.15.0\n```\n\nNext, You should install other python dependencies.\n\n```bash\n$ python -m pip install -r requirements.txt\n```\n\n\n## Train \u0026 Evaluate\n\nYou should first prepare a configure file to set data paths and hyperparameters.\n\nfor example:\n\n`sst2.json`\n\n```json\n{\n  \"max_len\": 80,\n  \"ae_epochs\": 100,\n  \"epochs\": 10,\n  \"batch_size\": 32,\n  \"learning_rate\": 0.00003,\n  \"pretrained_model_dir\": \"/path/to/pretrained-bert/uncased-bert-base\",\n  \"train_path\": \"/path/to/SST-2/train.jsonl\",\n  \"dev_path\": \"/path/to/SST-2/test.jsonl\",\n  \"save_dir\": \"/dir/to/save\",\n  \"epsilon\": 0.05,\n  \"dropout\": 0.3,\n  \"fgm_epsilon\": 0.3,\n  \"iterations\": 1,\n  \"verbose\": 1\n}\n```\n\n\n| Parameter             | Description                                      |\n|-----------------------|--------------------------------------------------|\n| max_len               | max length of input sequence                     |\n| ae_epochs             | epochs to train AutoEncoder                      |\n| epochs                | epochs to train classifier                       |\n| batch_size            | batch size                                       |\n| learning_rate         | learning rate                                    |\n| pretrained_model_dir  | file directory of the pre-trained language model |\n| save_dir              | dir to save model                                |\n| train_path            | data path of train set                           |\n| dev_path              | data path of develop set / test set              |\n| epsilon               | epsilon size of valve                            |\n| apply_fgm             | whether to apply fgm attack, default tru         |\n| fgm_epsilon           | epsilon of fgm, default 0.2                      |\n\n\nThen you can start to train and evaluate by following shell script.\n\n```bash\nexport TF_KERAS=1; CUDA_VISIBLE_DEVICES=0 python main.py /path/to/config.json\n```\n\nplease set `TF_KERAS=1` to use AdamW.\n\n\nAfter training is done, models will be stored in the specified `save_dir` folder.\n\n## Visualize Attention\n\nTo visualize attention, you should train a model first following the above instruction, then run `visualize_attn.py` as follows:\n\n```bash\nexport TF_KERAS=1; CUDA_VISIBLE_DEVICES=0 python visualize_attn.py /path/to/your_config.json\n```\n\nAfter inputting the text to the prompt box, the code will analyze the text and save the attention figure to `attn_visualize.png`.\n\nNote that, in previous settings, we pick up a most distinguished feature dimension from 2D attention and visualize the selected feature (1D) attention. In the latest version, we visualize the whole 2D attention rather than 1D attention.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ai%2Fagn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F4ai%2Fagn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ai%2Fagn/lists"}