{"id":22019185,"url":"https://github.com/jishnujayakumar/mlrc2020-embedkgqa","last_synced_at":"2026-03-10T16:32:18.836Z","repository":{"id":67247749,"uuid":"314187212","full_name":"jishnujayakumar/MLRC2020-EmbedKGQA","owner":"jishnujayakumar","description":"This is the code for the MLRC2020 challenge w.r.t. the ACL 2020 paper Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings","archived":false,"fork":false,"pushed_at":"2021-10-25T17:54:58.000Z","size":642,"stargazers_count":24,"open_issues_count":2,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-07T03:42:58.010Z","etag":null,"topics":["embeddings","knowledge-graph","machine-learning","multi-hop-reasoning","neural-embeddings","nlp"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jishnujayakumar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-11-19T08:45:49.000Z","updated_at":"2025-04-01T05:34:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"e36e2424-c11a-4811-96ef-8fff8180f9ca","html_url":"https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jishnujayakumar/MLRC2020-EmbedKGQA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jishnujayakumar%2FMLRC2020-EmbedKGQA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jishnujayakumar%2FMLRC2020-EmbedKGQA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jishnujayakumar%2FMLRC2020-EmbedKGQA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jishnujayakumar%2FMLRC2020-EmbedKGQA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jishnujayakumar","download_url":"https://codeload.github.com/jishnujayakumar/MLRC2020-EmbedKGQA/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jishnujayakumar%2FMLRC2020-EmbedKGQA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30342177,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T15:55:29.454Z","status":"ssl_error","status_checked_at":"2026-03-10T15:54:58.440Z","response_time":106,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embeddings","knowledge-graph","machine-learning","multi-hop-reasoning","neural-embeddings","nlp"],"created_at":"2024-11-30T05:16:13.801Z","updated_at":"2026-03-10T16:32:18.807Z","avatar_url":"https://github.com/jishnujayakumar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![SWH](https://archive.softwareheritage.org/badge/origin/https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA/)](https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA) [![SWH](https://archive.softwareheritage.org/badge/swh:1:dir:c95bc4fec7023c258c7190975279b5baf6ef6725/)](https://archive.softwareheritage.org/swh:1:dir:c95bc4fec7023c258c7190975279b5baf6ef6725;origin=https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA;visit=swh:1:snp:d08d258bc04ace7627e00bf0f3b12c499297e84d;anchor=swh:1:rev:9af249590ec26d122bd82eb2dc10a0c3545e7c1c)\n\n# EmbedKGQA: Reproduction and Extended Study \n- This is the code for the [MLRC2020 challenge](https://paperswithcode.com/rc2020) w.r.t. the [ACL 2020](https://acl2020.org/) paper [Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings](https://malllabiisc.github.io/publications/papers/final_embedkgqa.pdf)[1]\n- The code is build upon [1]:[5d8fdbd4](https://github.com/malllabiisc/EmbedKGQA/tree/5d8fdbd4be77fdcb2e67a0dc8a7115844606175a)\n- Minor modifications have been made to [5d8fdbd4](https://github.com/malllabiisc/EmbedKGQA/tree/5d8fdbd4be77fdcb2e67a0dc8a7115844606175a) in order to perform the ablation study. In case of any query relating to the original code[1], please contact [Apoorv](https://apoorvumang.github.io/).\n# Additional Experiments\n- Knowledge Graph Embedding model\n     - [TuckER](https://arxiv.org/abs/1901.09590)\n     - Tested on {MetaQA_full, MetaQA_half} datasets \n- Question embedding models\n    - [ALBERT](https://arxiv.org/abs/1909.11942)\n    - [XLNet](https://arxiv.org/abs/1906.08237)\n    - [Longformer](https://arxiv.org/abs/2004.05150)\n    - [SentenceBERT](https://arxiv.org/abs/1908.10084) (SentenceTransformer)\n    - Tested on {fbwq_full, fbwq_half} datasets \n\n# Requirements\n- Python \u003e= 3.7.5, pip\n- zip, unzip\n- Docker (Recommended)\n- Pytorch version [1.3.0a0+24ae9b5](https://github.com/pytorch/pytorch/tree/24ae9b504094937fbc7c24012fbe5c601e024bcd). For more info, visit [here](https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_19-10.html).\n\n# Helpful pointers\n- Docker Image: [Cuda-Python[2]](https://hub.docker.com/r/qts8n/cuda-python/) can be used. Use the `runtime` tag.\n    - ```bash\n      docker run -itd --rm --runtime=nvidia -v /raid/kgdnn/:/raid/kgdnn/ --name embedkgqa__4567 -e NVIDIA_VISIBLE_DEVICES=4,5,6,7  -p 7777:7777 qts8n/cuda-python:runtime\n      ```\n- Alternatively, Docker Image: [Embed_KGQA[3]](https://hub.docker.com/r/jishnup/embed_kgqa) can be used as well. It's build upon [2] and contains all the packages for conducting the experiments. \n    - Use `env` tag for image without models.\n    - Use `env-models` tag for image with models.\n    - ```bash\n      docker run -itd --rm --runtime=nvidia -v /raid/kgdnn/:/raid/kgdnn/ --name embedkgqa__4567 -e NVIDIA_VISIBLE_DEVICES=4,5,6,7  -p 7777:7777 jishnup/embed_kgqa:env\n      ```\n    - All the required packages and models (from the extended study with better performance) are readily available in [3].\n        - Model location within the docker container: `/raid/mlrc2020models/`\n            - `/raid/mlrc2020models/embeddings/` contain the KG embedding models.\n            - `/raid/mlrc2020models/qa_models/` contain the QA models.\n- The experiments have been done using [2]. The requirements.txt packages' version have been set accordingly. This may vary w.r.t. [1].\n- `KGQA/LSTM` and `KGQA/RoBERTa` directory nomenclature hasn't been changed to avoid unnecessary confusion w.r.t. the original codebase[1].\n\n- `fbwq_full` and `fbwq_full_new` are the same but independent existence is required because\n    - Pretrained `ComplEx` model uses `fbwq_full_new` as the dataset name\n    - Trained `SimplE` model uses `fbwq_full` as the dataset name\n- No `fbwq_full_new` dataset was found in the data shared by the author[1], so went ahead with this setting.\n\n- Also, pretrained qa_models were absent in the data shared. The reproduction results are based on training scheme used by us.\n\n- For training QA datasets, use ```batch_size \u003e= 2```. \n\n# Get started\n```bash\n# Clone the repo\ngit clone https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA \u0026\u0026 cd \"$_\"\n\n# Set a new env variable called EMBED_KGQA_DIR with MLRC2020-EmbedKGQA/ directory's absolute path as value\n# If using bash shell, run \necho 'export EMBED_KGQA_DIR=`pwd`' \u003e\u003e ~/.bash_profile \u0026\u0026 source ~/.bash_profile\n\n# Change script permissions\nchmod -R 700 scripts/\n\n# Initial setup\n./scripts/initial_setup.sh\n\n# Download and unzip, data and pretrained_models from the original EmbedKGQA paper\n./scripts/download_artifacts.sh\n\n# Install LibKGE\n./scripts/install_libkge.sh\n```\n\n# Train KG Embeddings\n- [Steps](https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA/tree/main/train_embeddings#steps-to-train-knowledge-graph-embedding-models) to train KG embeddings.\n\n# Train QA Datasets\nHyperparameters in the following commands are set w.r.t. [[1]](https://github.com/malllabiisc/EmbedKGQA#metaqa).\n### MetaQA\n```bash\n# Method: 1\ncd $EMBED_KGQA_DIR/KGQA/LSTM;\npython main.py  --mode train \\\n            --nb_epochs 100 \\\n            --relation_dim 200 \\\n            --hidden_dim 256 \\\n            --gpu 0 \\ #GPU-ID\n            --freeze 0 \\\n            --batch_size 64 \\\n            --validate_every 4 \\\n            --hops \u003c1/2/3\u003e \\ #n-hops\n            --lr 0.0005 \\\n            --entdrop 0.1 \\ \n            --reldrop 0.2 \\\n            --scoredrop 0.2 \\\n            --decay 1.0 \\\n            --model \u003cComplEx/TuckER\u003e \\ #KGE models\n            --patience 10 \\\n            --ls 0.0 \\\n            --use_cuda True \\ #Enable CUDA\n            --kg_type \u003chalf/full\u003e\n\n        \n# Method: 2\n# Modify the hyperparameters in the script file w.r.t. your usecase\n$EMBED_KGQA_DIR/scripts/train_metaQA.sh \\\n    \u003cComplEX/TuckER\u003e \\\n    \u003chalf/full\u003e \\\n    \u003c1/2/3\u003e \\\n    \u003cbatch_size\u003e \\\n    \u003cgpu_id\u003e \\\n    \u003crelation_dim\u003e\n```\n\n### WebQuestionsSP\n```bash\n# Method: 1\ncd $EMBED_KGQA_DIR/KGQA/RoBERTa;\npython main.py  --mode train \\\n                --relation_dim 200 \\\n                --que_embedding_model RoBERTa \\\n                --do_batch_norm 0 \\\n                --gpu 0 \\\n                --freeze 1 \\\n                --batch_size 16 \\\n                --validate_every 10 \\\n                --hops webqsp_half \\\n                --lr 0.00002 \\\n                --entdrop 0.0 \n                --reldrop 0.0 \\\n                --scoredrop 0.0 \\\n                --decay 1.0 \\\n                --model ComplEx \\\n                --patience 20 \\\n                --ls 0.0 \\\n                --l3_reg 0.001 \\\n                --nb_epochs 200 \\\n                --outfile delete\n\n# Method: 2\n# Modify the hyperparameters in the script file w.r.t. your usecase\n$EMBED_KGQA_DIR/scripts/train_webqsp.sh \\\n    \u003cComplEx/SimplE\u003e \\\n    \u003cRoBERTa/ALBERT/XLNet/Longformer/SentenceTransformer\u003e \\\n    \u003chalf/full\u003e \\\n    \u003cbatch_size\u003e \\\n    \u003cgpu_id\u003e \\\n    \u003crelation_dim\u003e\n```\n\n# Test QA Datasets\nSet the mode parameter as `test` (keep the other hyperparameters same as used in training)\n\n# Helpful links\n- [Details](https://github.com/malllabiisc/EmbedKGQA#instructions) about data and pretrained weights.\n- [Details](https://github.com/malllabiisc/EmbedKGQA#dataset-creation) about dataset creation.\n- [Presentation](https://slideslive.com/38929421/improving-multihop-question-answering-over-knowledge-graphs-using-knowledge-base-embeddings) for [1] by [Apoorv](https://apoorvumang.github.io/).\n\n\n### Citation:\nPlease cite the following if you incorporate our work.\n\n```bibtex\n@article{P:2021,\n  author = {P, Jishnu Jaykumar and Sardana, Ashish},\n  title = {{[Re] Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings}},\n  journal = {ReScience C},\n  year = {2021},\n  month = may,\n  volume = {7},\n  number = {2},\n  pages = {{#15}},\n  doi = {10.5281/zenodo.4834942},\n  url = {https://zenodo.org/record/4834942/files/article.pdf},\n  code_url = {https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA},\n  code_doi = {},\n  code_swh = {swh:1:dir:c95bc4fec7023c258c7190975279b5baf6ef6725},\n  data_url = {},\n  data_doi = {},\n  review_url = {https://openreview.net/forum?id=VFAwCMdWY7},\n  type = {Replication},\n  language = {Python},\n  domain = {ML Reproducibility Challenge 2020},\n  keywords = {knowledge graph, embeddings, multi-hop, question-answering, deep learning}\n}\n```\n\nFollowing 3 options are available for any clarification, comments or suggestions\n- Join the [discussion forum](https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA/discussions/).\n- Create an [issue](https://github.com/jishnujayakumar/MLRC2020-EmbedKGQA/issues).\n- Contact [Jishnu](https://jishnujayakumar.github.io/) or [Ashish](mailto:asardana@nvidia.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjishnujayakumar%2Fmlrc2020-embedkgqa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjishnujayakumar%2Fmlrc2020-embedkgqa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjishnujayakumar%2Fmlrc2020-embedkgqa/lists"}