{"id":28098797,"url":"https://github.com/rmanluo/reasoning-on-graphs","last_synced_at":"2025-05-13T17:58:51.378Z","repository":{"id":197917273,"uuid":"699108524","full_name":"RManLuo/reasoning-on-graphs","owner":"RManLuo","description":"Official Implementation of ICLR 2024 paper: \"Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning\"","archived":false,"fork":false,"pushed_at":"2025-03-05T11:37:00.000Z","size":17200,"stargazers_count":379,"open_issues_count":3,"forks_count":47,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-05T12:29:23.670Z","etag":null,"topics":["kg","knowledge","large-language-models","llm","reasoning","reasoning-on-graph"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2310.01061","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RManLuo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-01T23:47:43.000Z","updated_at":"2025-03-05T11:37:04.000Z","dependencies_parsed_at":"2023-10-03T13:51:19.091Z","dependency_job_id":"3fe6e09b-4ae6-49e8-9d59-0321028cc95e","html_url":"https://github.com/RManLuo/reasoning-on-graphs","commit_stats":null,"previous_names":["rmanluo/reasoning-on-graphs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RManLuo%2Freasoning-on-graphs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RManLuo%2Freasoning-on-graphs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RManLuo%2Freasoning-on-graphs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RManLuo%2Freasoning-on-graphs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RManLuo","download_url":"https://codeload.github.com/RManLuo/reasoning-on-graphs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253999904,"owners_count":21997352,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kg","knowledge","large-language-models","llm","reasoning","reasoning-on-graph"],"created_at":"2025-05-13T17:58:51.129Z","updated_at":"2025-05-13T17:58:51.344Z","avatar_url":"https://github.com/RManLuo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reasoning on Graphs (RoG)\nOfficial Implementation of \"[Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://arxiv.org/abs/2310.01061)\".\n\n\u003cimg src=\"resources/rog.png\" width = \"800\" /\u003e\n\nReasoning on graphs (RoG) synergizes LLMs with KGs to enable faithful and interpretable reasoning. We present a planning-retrieval-reasoning framework, where RoG first generates relation paths grounded by KGs as faithful plans. These plans are then used to retrieve valid reasoning paths from the KGs for LLMs to conduct faithful reasoning and generate interpretable results.\n\n## News 🎉\n* Check our the first graph foundation model-powered RAG pipeline ([GFM-RAG](https://github.com/RManLuo/gfm-rag)) that combines the power of GNNs+KGs with LLMs to enhance reasoning. [Paper](https://www.arxiv.org/abs/2502.01113)\n* Check out our latest work on KG + LLM reasoning: [Graph-constrained Reasoning](https://github.com/RManLuo/graph-constrained-reasoning) \n\n## Requirements\n```\npip install -r requirements.txt\n```\n\n## Pre-trained weights\n\n\u003e Our code will automatically download the model weight from the huggingface.\n\nYou can find the pre-trained weights [here](https://huggingface.co/rmanluo/RoG).\n\n## Datasets\n\n\u003e Our code will automatically download the data from the huggingface.\n\n[RoG-WebQSP](https://huggingface.co/datasets/rmanluo/RoG-webqsp)   \n[RoG-CWQ](https://huggingface.co/datasets/rmanluo/RoG-cwq)\n\u003cdetails\u003e \u003csummary\u003eSubgraph Extraction\u003c/summary\u003e\n\nWe extract the subgraphs from the Freebase following previous studies. The code can be found [here](https://github.com/RichardHGL/WSDM2021_NSM/tree/main/preprocessing/Freebase).   \n\u003c/details\u003e\n\n## Inference\nRequirements: Any GPU with at least 12GB memory.\n### Step1: Planning (Generate relation paths)\n\nRun: `./scripts/planning.sh`\n\n```bash\npython src/qa_prediction/gen_rule_path.py \\\n        --model_name RoG \\\n        --model_path rmanluo/RoG \\\n        -d {RoG-webqsp,RoG-cwq} \\\n        --split test \\\n        --n_beam 3\n```\n\nGenerated rules will be saved at: `results/gen_rule_path/{dataset}/{model_name}/{split}`\n\n### Step2: Reasoning (Generate answers with RoG)\n\nRun: `./scripts/rog-reasoning.sh`\n\n```bash\npython src/qa_prediction/predict_answer.py \\\n        --model_name RoG \\\n        --model_path rmanluo/RoG \\\n        -d {RoG-webqsp,RoG-cwq} \\\n        --prompt_path prompts/llama2_predict.txt \\\n        --add_rul \\\n        --rule_path {rule_path} \\\n```\n\nAnswers will be saved at: `results/KGQA/{dataset}/{model_name}/{split}`\n\n### Plug-and-play Reasoning (Generate answers with different LLMs)\n\u003e\n\u003e Note: you need to set your openai key at `.env` to use ChatGPT.\n\nRun: `./scripts/plug-and-play.sh`\n\n```bash\npython src/qa_prediction/predict_answer.py \\\n        --model_name {gpt-3.5-turbo,alpaca,llama2-chat-hf,flan-t5} \\\n        -d {RoG-webqsp,RoG-cwq} \\\n        --prompt_path {prompt_path} \\\n        --add_rule \\\n        --rule_path {rule_path}\n```\n### Interpretable Reasoning\nRun: `python scripts/interpretable_example.py`\n\n```python\nfrom transformers import pipeline, AutoTokenizer\nimport torch\n\nMODEL_PATH_OR_NAME=\"rmanluo/RoG\"\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_PATH_OR_NAME, use_fast=False)\nmodel = pipeline(\"text-generation\", model=MODEL_PATH_OR_NAME, tokenizer=tokenizer, device_map=\"auto\", torch_dtype=torch.float16)\n\nprint(\"====EXAMPLE 1: ====\")\n\nINPUT_TEXT_1 = \"\"\"Based on the reasoning paths, please answer the given question and explain why \n\nReasoning Paths: \nNorthern District -\u003e location.administrative_division.first_level_division_of -\u003e Israel -\u003e government.form_of_government.countries -\u003e Parliamentary system\n\nQuestion: \nWhat type of government is used in the country with Northern District?\"\"\"\n\noutputs = model(INPUT_TEXT_1, return_full_text=False)\nprint(outputs[0]['generated_text'])\n```\n\n## Training\n\n### Training Datasets\nYou can download the processed datasets from [RoG_train_data.tar.tz](datasets/RoG_train_data.tar.tz). Unzip the files and put them under `datasets/` folder.\n\u003cdetails\u003e \u003csummary\u003eProcess datasets\u003c/summary\u003e\n\n1. Build question to relation path pairs.\n\n```bash\npython src/align_kg/build_align_qa_dataset.py -d {RoG-webqsp,RoG-cwq} --split {train,validation,test}\n```\n2. Build joint-training datasets.\n\n```bash\npython src/joint_training/preprocess_align.py\npython src/joint_training/preprocess_qa.py\n```\n\n3. Build interpretable examples.\n```bash\npython src/joint_training/generate_explanation_results.py\n```\n\n\u003c/details\u003e\n\n### Training RoG\n2 A100-80GB GPUs are required for training RoG.\n\nRun: `./scripts/train.sh`\n\n## Results\n\n\u003cimg src=\"resources/results.png\" width = \"600\" /\u003e\n\u003cimg src=\"resources/plug-and-play.png\" width = \"600\" /\u003e\n\u003cimg src=\"resources/lack_of_knowledge.png\" width = \"600\" /\u003e\n\u003cimg src=\"resources/hallucination.png\" width = \"600\" /\u003e\n\n## Bibinfo\nIf you found this repo helpful, please help us by citing this paper:\n```\n@inproceedings{luo2024rog,\ntitle={Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning},\nauthor={Luo, Linhao and Li, Yuan-Fang and Haffari, Gholamreza and Pan, Shirui},\nbooktitle={International Conference on Learning Representations},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmanluo%2Freasoning-on-graphs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frmanluo%2Freasoning-on-graphs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmanluo%2Freasoning-on-graphs/lists"}