{"id":13754276,"url":"https://github.com/haozheji/multigen","last_synced_at":"2025-05-09T22:31:42.699Z","repository":{"id":44497696,"uuid":"296016859","full_name":"haozheji/multigen","owner":"haozheji","description":"EMNLP2020 - Language Generation with Multi-hop Reasoning on Commonsense Knowledge Graph","archived":false,"fork":false,"pushed_at":"2024-02-26T16:11:58.000Z","size":6365,"stargazers_count":130,"open_issues_count":7,"forks_count":31,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-02T03:32:01.662Z","etag":null,"topics":["kg","nlg","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haozheji.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-16T11:49:52.000Z","updated_at":"2024-11-02T09:45:17.000Z","dependencies_parsed_at":"2024-08-03T09:17:18.051Z","dependency_job_id":null,"html_url":"https://github.com/haozheji/multigen","commit_stats":null,"previous_names":["haozheji/multigen","cdjhz/multigen"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haozheji%2Fmultigen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haozheji%2Fmultigen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haozheji%2Fmultigen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haozheji%2Fmultigen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haozheji","download_url":"https://codeload.github.com/haozheji/multigen/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253335729,"owners_count":21892720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kg","nlg","pytorch"],"created_at":"2024-08-03T09:01:52.888Z","updated_at":"2025-05-09T22:31:37.687Z","avatar_url":"https://github.com/haozheji.png","language":"Python","funding_links":[],"categories":["知识图谱问答KBQA、多跳推理"],"sub_categories":["其他_文本生成、文本对话"],"readme":"# Language Generation with Multi-hop Reasoning on Commonsense Knowledge Graph\n\n## Introduction\n\nThis is the pytorch implementation of our paper \"*Language Generation with Multi-hop Reasoning on Commonsense Knowledge Graph*\". \nThe arxiv version of the paper could be found [here](https://arxiv.org/pdf/2009.11692.pdf).\n\n\n## Requirements\n\n```\npython version \u003e= 3\ntorch version \u003e= 1.4.0\ntransformers == 2.8.0\nnltk == 3.4.5\nnetworkx == 2.1\nspacy == 2.2.1\ntorch-scatter == 2.0.5+${CUDA}\n```\n\nFor `torch-scatter`, `${CUDA}` should be replaced by either `cpu`, `cu92`, `cu101` or `cu102` depending on your PyTorch installation. \nFor more information check [here](https://github.com/rusty1s/pytorch_scatter).\n\n\n## Preprocessing\n\nPreprocessed datasets can be downloaded from [here](https://drive.google.com/file/d/15ckbKsyq5sMU-RJh0n-mB9NgyW1WhTsF/view?usp=sharing).\n\nUnzip the file and move it to `data`.\n\nExtract English ConceptNet and build graph.\n\n```bash\ncd data\nwget https://s3.amazonaws.com/conceptnet/downloads/2018/edges/conceptnet-assertions-5.6.0.csv.gz\ngzip -d conceptnet-assertions-5.6.0.csv.gz\ncd ../preprocess\npython extract_cpnet.py\npython graph_construction.py\n```\n\nPreprocessing multi-hop relational paths for the model. Set `$DATA` to either `anlg`, `eg`, `story`.\n\n```bash\nexport DATA=eg\npython ground_concepts_simple.py $DATA\npython find_neighbours.py $DATA\npython filter_triple.py $DATA\n```\n\nDownload the pre-trained GPT-2 model.\n\n```bash\nmkdir -p models\ncd models\nmkdir -p gpt2-small\nwget -O pytorch_model.bin https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-pytorch_model.bin\nwget -O vocab.json https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json\nwget -O merges.txt https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt\nwget -O config.json https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json\n```\n\nAdd special tokens to vocabulary.\n\n```bash\ncd scripts\npython add_special_tokens.py\n```\n\n## Usage\n\n### Training \n\nThe following command is an example to train the model on the trarining set and evaluate on the development set. Set `$DATA_TYPE` to either `anlg`, `eg`, `story`.\n\n```bash\nexport DATA_TYPE={anlg, eg, story}\nexport ROOT_PATH=..\nexport DEVICE=1\nCUDA_VISIBLE_DEVICES=${DEVICE} \\\npython3 main.py \\\n--train_data_file ${ROOT_PATH}/data/${DATA_TYPE}/train \\\n--dev_data_file ${ROOT_PATH}/data/${DATA_TYPE}/dev \\\n--test_data_file ${ROOT_PATH}/data/${DATA_TYPE}/test \\\n--graph_path 2hops_100_directed_triple_filter.json \\\n--output_dir ${ROOT_PATH}/models/${DATA_TYPE}/grf-${DATA_TYPE} \\\n--source_length 32 \\\n--target_length 32 \\\n--model_type gpt2 \\\n--model_name_or_path ${ROOT_PATH}/models/gpt2-small \\\n--do_train \\\n--per_gpu_train_batch_size 16 \\\n--per_gpu_eval_batch_size 16 \\\n--workers 7 \\\n--seed 42 \\\n--evaluate_metrics bleu \\\n--overwrite_output_dir \\\n--num_train_epochs 3 \\\n--learning_rate 1e-5 \\\n--aggregate_method max \\\n--alpha 3 \\\n--beta 5 \\\n--gamma 0.5 \\\n--weight_decay 0.0 \\\n--warmup_ratio 0.0 \\\n--logging_steps 20 \\\n```\n\n**Note:** Some scatter operations in `torch_scatter` are currently non-deterministic and not controlled by the random seed due to the usage of atomic operations. \nSee the discussions [here](https://discuss.pytorch.org/t/possible-solution-to-the-reproducibility-issues-using-scatter-add-operation/48989). \nSo different runs may vary slightly. \n\nTo reproduce our results, you can directly download our [checkpoints](https://drive.google.com/drive/folders/1yx6WTjR1mXYZ6W7Pd-_MI7XkWfyV78c9?usp=sharing) and evaluate the model.\n\n### Inference\n\nThe following command is an example to run the inference on the test set. Set `$DATA_TYPE` to either `anlg`, `eg`, `story`.\n\n```bash\nexport DATA_TYPE={anlg, eg, story}\nexport ROOT_PATH=..\nexport DEVICE=1\nCUDA_VISIBLE_DEVICES=${DEVICE} \\\npython3 main.py \\\n--train_data_file ${ROOT_PATH}/data/${DATA_TYPE}/train \\\n--dev_data_file ${ROOT_PATH}/data/${DATA_TYPE}/dev \\\n--test_data_file ${ROOT_PATH}/data/${DATA_TYPE}/test \\\n--graph_path 2hops_100_directed_triple_filter.json \\\n--output_dir ${ROOT_PATH}/models/${DATA_TYPE}/grf-${DATA_TYPE} \\\n--source_length 32 \\\n--target_length 32 \\\n--model_type gpt2 \\\n--model_name_or_path ${ROOT_PATH}/models/gpt2-small \\\n--do_eval \\\n--per_gpu_train_batch_size 16 \\\n--per_gpu_eval_batch_size 16 \\\n--workers 7 \\\n--seed 42 \\\n--evaluate_metrics bleu \\\n--overwrite_output_dir \\\n--aggregate_method max \\\n--gamma 0.5 \\\n```\n\nWe also provide generated results in `results/` on the three tasks and you can directly evaluate the results with our evaluation scripts.\n\n### Evaluation\n\nTo evaluate the generated results on `anlg` and `eg`, run the following commands. Set `$DATA` to either `anlg`, `eg`.\n\n```bash\nexport DATA={eg, anlg}\npython evaluation/eval.py --dataset ${DATA} --output_dir results/grf-${DATA}.txt\n```\n\nTo evaluate the generated results on `story`, run the following commands.\n\n```bash\npython eval_story.py results/grf-story.txt\n```\n\n## Citation\n\n```\n@inproceedings{ji2020language,\n    title = \"Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph\",\n    author = \"Ji, Haozhe and Ke, Pei and Huang, Shaohan and Wei, Furu and Zhu, Xiaoyan and Huang, Minlie\",\n    booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing\",\n    year = \"2020\",\n}\n```\n\n**Please kindly cite our paper if you find this paper and the codes helpful.**\n\n## Thanks\n\nMany thanks to the Github repository of [Transformers](https://github.com/huggingface/transformers), [fairseq](https://github.com/pytorch/fairseq) and [KagNet](https://github.com/INK-USC/KagNet). Part of our codes are modified based on their codes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaozheji%2Fmultigen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaozheji%2Fmultigen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaozheji%2Fmultigen/lists"}