{"id":13754277,"url":"https://github.com/INK-USC/MHGRN","last_synced_at":"2025-05-09T22:31:39.854Z","repository":{"id":52576720,"uuid":"259581196","full_name":"INK-USC/MHGRN","owner":"INK-USC","description":"Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (EMNLP 2020)","archived":false,"fork":false,"pushed_at":"2021-10-23T16:52:14.000Z","size":2365,"stargazers_count":248,"open_issues_count":11,"forks_count":46,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-11-16T07:33:17.449Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/INK-USC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-28T08:46:41.000Z","updated_at":"2024-11-04T19:56:44.000Z","dependencies_parsed_at":"2022-09-23T12:51:47.361Z","dependency_job_id":null,"html_url":"https://github.com/INK-USC/MHGRN","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/INK-USC%2FMHGRN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/INK-USC%2FMHGRN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/INK-USC%2FMHGRN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/INK-USC%2FMHGRN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/INK-USC","download_url":"https://codeload.github.com/INK-USC/MHGRN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253335720,"owners_count":21892718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T09:01:52.917Z","updated_at":"2025-05-09T22:31:36.886Z","avatar_url":"https://github.com/INK-USC.png","language":"Python","funding_links":[],"categories":["知识图谱问答KBQA、多跳推理","Python"],"sub_categories":["其他_文本生成、文本对话"],"readme":"# Multi-Hop Graph Relation Networks (EMNLP 2020)\n\n\u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/9/96/Pytorch_logo.png/800px-Pytorch_logo.png\" width=\"10%\"\u003e [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nThis is the repo of our EMNLP'20 [paper](https://arxiv.org/abs/2005.00646):\n\n```\nScalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering\nYanlin Feng*, Xinyue Chen*, Bill Yuchen Lin, Peifeng Wang, Jun Yan and Xiang Ren.\nEMNLP 2020.\n*=equal contritbution\n```\n\nThis repository also implements other graph encoding models for question answering (including vanilla LM finetuning).\n\n- **RelationNet**\n- **R-GCN**\n- **KagNet** \n- **GConAttn**\n- **KVMem**\n- **MHGRN (or. MultiGRN)**\n\nEach model supports the following text encoders:\n\n- **LSTM**\n- **GPT**\n- **BERT** \n- **XLNet** \n- **RoBERTa**\n\n\n\n## Resources\n\nWe provide preprocessed ConceptNet and pretrained entity embeddings for your own usage. These resources are independent of the source code.\n\n***Note that the following reousrces can be download [here](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws).***\n\n### ConceptNet (5.6.0)\n\n| Description                  | Downloads                                                    | Notes                                                        |\n| ---------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |\n| Entity Vocab                 | [entity-vocab](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws) | one entity per line, space replaced by '_'                   |\n| Relation Vocab               | [relation-vocab](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws) | one relation per line, merged                                |\n| ConceptNet (CSV format)      | [conceptnet-5.6.0-csv](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws) | English tuples extracted from the full conceptnet with merged relations |\n| ConceptNet (NetworkX format) | [conceptnet-5.6.0-networkx](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws) | NetworkX pickled format, pruned by filtering out stop words  |\n\n### Entity Embeddings (Node Features)\n\nEntity embeddings are packed into a matrix of shape (#ent, dim) and stored in numpy format. Use `np.load` to read the file. You may need to download the vocabulary files first.\n\n| Embedding Model | Dimensionality | Description                                               | Downloads                                                    |\n| --------------- | -------------- | --------------------------------------------------------- | ------------------------------------------------------------ |\n| TransE          | 100            | Obtained using OpenKE with optim=sgd, lr=1e-3, epoch=1000 | [entities](\u003chttps://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws\u003e) [relations](\u003chttps://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws\u003e) |\n| NumberBatch     | 300            | \u003chttps://github.com/commonsense/conceptnet-numberbatch\u003e   | [entities](\u003chttps://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws\u003e) |\n| BERT-based      | 1024           | Provided by Zhengwei                                      | [entities](https://drive.google.com/drive/folders/155codqEnsKazO8-BchF3rO_cP3EyYdws) |\n\n\n\n## Dependencies\n\n- [Python](\u003chttps://www.python.org/\u003e) \u003e= 3.6\n- [PyTorch](\u003chttps://pytorch.org/get-started/locally/\u003e) == 1.1.0\n- [transformers](\u003chttps://github.com/huggingface/transformers/tree/v2.0.0\u003e) == 2.0.0\n- [tqdm](\u003chttps://github.com/tqdm/tqdm\u003e)\n- [dgl](\u003chttps://github.com/dmlc/dgl\u003e) == 0.3.1 (GPU version)\n- [networkx](\u003chttps://networkx.github.io/\u003e) == 2.3\n\nRun the following commands to create a conda environment (assume CUDA10):\n\n```bash\nconda create -n krqa python=3.6 numpy matplotlib ipython\nsource activate krqa\nconda install pytorch=1.1.0 torchvision cudatoolkit=10.0 -c pytorch\npip install dgl-cu100==0.3.1\npip install transformers==2.0.0 tqdm networkx==2.3 nltk spacy==2.1.6\npython -m spacy download en\n```\n\n\n\n## Usage\n\n### 1. Download Data\n\nFirst, you need to download all the necessary data in order to train the model:\n\n```bash\ngit clone https://github.com/INK-USC/MHGRN.git\ncd MHGRN\nbash scripts/download.sh\n```\n\nThe script will:\n\n- Download the [CommonsenseQA](\u003chttps://www.tau-nlp.org/commonsenseqa\u003e) dataset\n- Download [ConceptNet](\u003chttp://conceptnet.io/\u003e)\n- Download pretrained TransE embeddings\n\n### 2. Preprocess\n\nTo preprocess the data, run:\n\n```bash\npython preprocess.py\n```\n\nBy default, all available CPU cores will be used for multi-processing in order to speed up the process. Alternatively, you can use \"-p\" to specify the number of processes to use:\n\n```bash\npython preprocess.py -p 20\n```\n\nThe script will:\n\n- Convert the original datasets into .jsonl files (stored in `data/csqa/statement/`)\n- Extract English relations from ConceptNet, merge the original 42 relation types into 17 types\n- Identify all mentioned concepts in the questions and answers\n- Extract subgraphs for each q-a pair\n\nThe preprocessing procedure takes approximately 3 hours on a 40-core CPU server. Most intermediate files are in .jsonl or .pk format and stored in various folders. The resulting file structure will look like:\n\n```plain\n.\n├── README.md\n└── data/\n    ├── cpnet/                 (prerocessed ConceptNet)\n    ├── glove/                 (pretrained GloVe embeddings)\n    ├── transe/                (pretrained TransE embeddings)\n    └── csqa/\n        ├── train_rand_split.jsonl\n        ├── dev_rand_split.jsonl\n        ├── test_rand_split_no_answers.jsonl\n        ├── statement/             (converted statements)\n        ├── grounded/              (grounded entities)\n        ├── paths/                 (unpruned/pruned paths)\n        ├── graphs/                (extracted subgraphs)\n        ├── ...\n```\n\n### 3. Hyperparameter Search (optional)\n\nTo search the parameters for RoBERTa-Large on CommonsenseQA:\n\n```bash\nbash scripts/param_search_lm.sh csqa roberta-large\n```\n\nTo search the parameters for BERT+RelationNet on CommonsenseQA:\n\n```bash\nbash scripts/param_search_rn.sh csqa bert-large-uncased\n```\n\n### 4. Training \n\nEach graph encoding model is implemented in a single script:\n\n| Graph Encoder                                                | Script      | Description                                                  |\n| ------------------------------------------------------------ | ----------- | ------------------------------------------------------------ |\n| None                                                         | lm.py       | w/o knowledge graph                                          |\n| [Relation Network](\u003chttps://papers.nips.cc/paper/7082-a-simple-neural-network-module-for-relational-reasoning.pdf\u003e) | rn.py       |                                                              |\n| [R-GCN](\u003chttps://arxiv.org/pdf/1703.06103.pdf\u003e)              | rgcn.py     | Use `--gnn_layer_num ` and `--num_basis` to specify #layer and #basis |\n| [KagNet](https://arxiv.org/abs/1909.02151)                   | kagnet.py   | Adapted from \u003chttps://github.com/INK-USC/KagNet\u003e, still tuning |\n| Gcon-Attn                                                    | gconattn.py |                                                              |\n| KV-Memory                                                    | kvmem.py    |                                                              |\n| MHGRN                                                        | grn.py      |                                                              |\n\nSome important command line arguments are listed as follows (run `python {lm,rn,rgcn,...}.py -h` for a complete list):\n\n| Arg                             | Values                                                     | Description                      | Notes                                                        |\n| ------------------------------- | ---------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------ |\n| `--mode`                        | {train, eval, ...}                                         | Training or Evaluation           | default=train                                                |\n| `-enc, --encoder`               | {lstm, openai-gpt, bert-large-unased, roberta-large, ....} | Text Encoer                      | Model names (except for lstm) are the ones used by [huggingface-transformers](\u003chttps://github.com/huggingface/transformers\u003e), default=bert-large-uncased |\n| `--optim`                       | {adam, adamw, radam}                                       | Optimizer                        | default=radam                                                |\n| `-ds, --dataset`                | {csqa, obqa}                                               | Dataset                          | default=csqa                                                 |\n| `-ih, --inhouse`                | {0, 1}                                                     | Run In-house Split               | default=1, only applicable to CSQA                           |\n| `--ent_emb`                     | {transe, numberbatch, tzw}                                 | Entity Embeddings                | default=tzw (BERT-based node features)                       |\n| `-sl, --max_seq_len`            | {32, 64, 128, 256}                                         | Maximum Sequence Length          | Use 128 or 256 for datasets that contain long sentences! default=64 |\n| `-elr, --encoder_lr`            | {1e-5, 2e-5, 3e-5, 6e-5, 1e-4}                             | Text Encoder LR                  | dataset specific and text encoder specific, default values in `utils/parser_utils.py` |\n| `-dlr, --decoder_lr`            | {1e-4, 3e-4, 1e-3, 3e-3}                                   | Graph Encoder LR                 | dataset specific and model specific, default values in `{model}.py` |\n| `--lr_schedule`                 | {fixed, warmup_linear, warmup_constant}                    | Learning Rate Schedule           | default=fixed                                                |\n| `-me, --max_epochs_before_stop` | {2, 4, 6}                                                  | Early Stopping Patience          | default=2                                                    |\n| `--unfreeze_epoch`              | {0, 3}                                                     | Freeze Text Encoder for N epochs | model specific                                               |\n| `-bs, --batch_size`             | {16, 32, 64}                                               | Batch Size                       | default=32                                                   |\n| `--save_dir`                    | str                                                        | Checkpoint Directory             | model specific                                               |\n| `--seed`                        | {0, 1, 2, 3}                                               | Random Seed                      | default=0                                                    |\n\nFor example, run the following command to train a RoBERTa-Large model on CommonsenseQA:\n\n```bash\npython lm.py --encoder roberta-large --dataset csqa\n```\n\nTo train a RelationNet with BERT-Large-Uncased as the encoder:\n\n```bash\npython rn.py --encoder bert-large-uncased\n```\n\nTo **reproduce the reported results of MultiGRN** on CommonsenseQA official set:\n\n```\nbash scripts/run_grn_csqa.sh\n```\n\n\n### 5. Evaluation\n\nTo evaluate a trained model (you need to specify `--save_dir` if the checkpoint is not stored in the default directory):\n\n```bash\npython {lm,rn,rgcn,...}.py --mode eval [ --save_dir path/to/directory/ ]\n```\n\n\n\n## Use Your Own Dataset\n\n- Convert your dataset to  `{train,dev,test}.statement.jsonl`  in .jsonl format (see `data/csqa/statement/train.statement.jsonl`)\n- Create a directory in `data/{yourdataset}/` to store the .jsonl files\n- Modify `preprocess.py` and perform subgraph extraction for your data\n- Modify `utils/parser_utils.py` to support your own dataset\n- Tune `encoder_lr`,`decoder_lr` and other important hyperparameters, modify `utils/parser_utils.py` and `{model}.py` to record the tuned hyperparameters\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FINK-USC%2FMHGRN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FINK-USC%2FMHGRN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FINK-USC%2FMHGRN/lists"}