{"id":18949968,"url":"https://github.com/salesforce/multihopkg","last_synced_at":"2025-05-16T07:02:45.758Z","repository":{"id":33361653,"uuid":"150488867","full_name":"salesforce/MultiHopKG","owner":"salesforce","description":"Multi-hop knowledge graph reasoning learned via policy gradient with reward shaping and action dropout ","archived":false,"fork":false,"pushed_at":"2025-03-04T08:16:22.000Z","size":25422,"stargazers_count":308,"open_issues_count":8,"forks_count":80,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-01T15:11:44.979Z","etag":null,"topics":["action-dropout","knowledge-graph","multi-hop-reasoning","policy-gradient","pytorch","reinforcement-learning","reward-shaping"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1808.10568","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salesforce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-26T20:53:30.000Z","updated_at":"2025-03-25T00:48:27.000Z","dependencies_parsed_at":"2025-01-05T16:03:39.737Z","dependency_job_id":"fdc39e60-f98c-4505-b687-1f5796f41860","html_url":"https://github.com/salesforce/MultiHopKG","commit_stats":{"total_commits":24,"total_committers":3,"mean_commits":8.0,"dds":0.08333333333333337,"last_synced_commit":"16cb1d48170d27981e5cee15f68c4095296f69a9"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2FMultiHopKG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2FMultiHopKG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2FMultiHopKG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salesforce%2FMultiHopKG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salesforce","download_url":"https://codeload.github.com/salesforce/MultiHopKG/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247878022,"owners_count":21011158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["action-dropout","knowledge-graph","multi-hop-reasoning","policy-gradient","pytorch","reinforcement-learning","reward-shaping"],"created_at":"2024-11-08T13:20:05.295Z","updated_at":"2025-04-08T16:07:08.857Z","avatar_url":"https://github.com/salesforce.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Hop Knowledge Graph Reasoning with Reward Shaping\n\n⚠️ REPO NO LONGER MAINTAINED, RESEARCH CODE PROVIDED AS IT IS\n\nThis is the official code release of the following paper: \n\nXi Victoria Lin, Richard Socher and Caiming Xiong. [Multi-Hop Knowledge Graph Reasoning with Reward Shaping](https://arxiv.org/abs/1808.10568). EMNLP 2018.\n\n\u003cimg src=\"http://victorialin.net/img/multihopkg.png\" alt=\"multihopkg_architecture\" width=\"700\" class=\"center\"\u003e\n\n## Quick Start\n\n### Environment variables \u0026 dependencies\n#### Use Docker\nBuild the docker image\n```\ndocker build -\u003c Dockerfile -t multi_hop_kg:v1.0\n```\n\nSpin up a docker container and run experiments inside it.\n```\nnvidia-docker run -v `pwd`:/workspace/MultiHopKG -it multi_hop_kg:v1.0\n```\n*The rest of the readme assumes that one works interactively inside a container. If you prefer to run experiments outside a container, please change the commands accordingly.*\n\n#### Mannually set up \nAlternatively, you can install Pytorch (\u003e=0.4.1) manually and use the Makefile to set up the rest of the dependencies. \n```\nmake setup\n```\n\n### Process data\nFirst, unpack the data files \n```\ntar xvzf data-release.tgz\n```\nand run the following command to preprocess the datasets.\n```\n./experiment.sh configs/\u003cdataset\u003e.sh --process_data \u003cgpu-ID\u003e\n```\n\n`\u003cdataset\u003e` is the name of any dataset folder in the `./data` directory. In our experiments, the five datasets used are: `umls`, `kinship`, `fb15k-237`, `wn18rr` and `nell-995`. \n`\u003cgpu-ID\u003e` is a non-negative integer number representing the GPU index.\n\n### Train models\nThen the following commands can be used to train the proposed models and baselines in the paper. By default, dev set evaluation results will be printed when training terminates.\n\n1. Train embedding-based models\n```\n./experiment-emb.sh configs/\u003cdataset\u003e-\u003cemb_model\u003e.sh --train \u003cgpu-ID\u003e\n```\nThe following embedding-based models are implemented: `distmult`, `complex` and `conve`.\n\n2. Train RL models (policy gradient)\n```\n./experiment.sh configs/\u003cdataset\u003e.sh --train \u003cgpu-ID\u003e\n```\n\n3. Train RL models (policy gradient + reward shaping)\n```\n./experiment-rs.sh configs/\u003cdataset\u003e-rs.sh --train \u003cgpu-ID\u003e\n```\n\n* Note: To train the RL models using reward shaping, make sure 1) you have pre-trained the embedding-based models and 2) set the file path pointers to the pre-trained embedding-based models correctly ([example configuration file](configs/umls-rs.sh)).\n\n### Evaluate pretrained models\nTo generate the evaluation results of a pre-trained model, simply change the `--train` flag in the commands above to `--inference`. \n\nFor example, the following command performs inference with the RL models (policy gradient + reward shaping) and prints the evaluation results (on both dev and test sets).\n```\n./experiment-rs.sh configs/\u003cdataset\u003e-rs.sh --inference \u003cgpu-ID\u003e\n```\n\nTo print the inference paths generated by beam search during inference, use the `--save_beam_search_paths` flag:\n```\n./experiment-rs.sh configs/\u003cdataset\u003e-rs.sh --inference \u003cgpu-ID\u003e --save_beam_search_paths\n```\n\n* Note for the NELL-995 dataset: \n\n  On this dataset we split the original training data into `train.triples` and `dev.triples`, and the final model to test has to be trained with these two files combined. \n  1. To obtain the correct test set results, you need to add the `--test` flag to all data pre-processing, training and inference commands.  \n    ```\n    # You may need to adjust the number of training epochs based on the dev set development.\n\n    ./experiment.sh configs/nell-995.sh --process_data \u003cgpu-ID\u003e --test\n    ./experiment-emb.sh configs/nell-995-conve.sh --train \u003cgpu-ID\u003e --test\n    ./experiment-rs.sh configs/NELL-995-rs.sh --train \u003cgpu-ID\u003e --test\n    ./experiment-rs.sh configs/NELL-995-rs.sh --inference \u003cgpu-ID\u003e --test\n    ```    \n  2. Leave out the `--test` flag during development.\n\n### Change the hyperparameters\nTo change the hyperparameters and other experiment set up, start from the [configuration files](configs).\n\n### More on implementation details\nWe use mini-batch training in our experiments. To save the amount of paddings (which can cause memory issues and slow down computation for knowledge graphs that contain nodes with large fan-outs),\nwe group the action spaces of different nodes into buckets based on their sizes. Description of the bucket implementation can be found\n[here](https://github.com/salesforce/MultiHopKG/blob/master/src/rl/graph_search/pn.py#L193) and \n[here](https://github.com/salesforce/MultiHopKG/blob/master/src/knowledge_graph.py#L164).\n\n## Citation\nIf you find the resource in this repository helpful, please cite\n```\n@inproceedings{LinRX2018:MultiHopKG, \n  author = {Xi Victoria Lin and Richard Socher and Caiming Xiong}, \n  title = {Multi-Hop Knowledge Graph Reasoning with Reward Shaping}, \n  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural\n               Language Processing, {EMNLP} 2018, Brussels, Belgium, October\n               31-November 4, 2018},\n  year = {2018} \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalesforce%2Fmultihopkg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalesforce%2Fmultihopkg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalesforce%2Fmultihopkg/lists"}