{"id":18887145,"url":"https://github.com/thunlp-mt/skr","last_synced_at":"2025-09-04T08:33:00.033Z","repository":{"id":205258495,"uuid":"706015447","full_name":"THUNLP-MT/SKR","owner":"THUNLP-MT","description":"Self-Knowledge Guided Retrieval Augmentation for Large Language Models (EMNLP Findings 2023)","archived":false,"fork":false,"pushed_at":"2023-12-08T08:08:56.000Z","size":2764,"stargazers_count":22,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-08T07:40:52.704Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUNLP-MT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-17T06:27:34.000Z","updated_at":"2024-10-11T02:52:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"5bb22ed6-a031-4504-918d-896e63d7b444","html_url":"https://github.com/THUNLP-MT/SKR","commit_stats":null,"previous_names":["thunlp-mt/skr"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FSKR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FSKR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FSKR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FSKR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUNLP-MT","download_url":"https://codeload.github.com/THUNLP-MT/SKR/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231945469,"owners_count":18449892,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T07:34:45.461Z","updated_at":"2024-12-31T05:19:00.513Z","avatar_url":"https://github.com/THUNLP-MT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Overview\n[**Self-Knowledge Guided Retrieval Augmentation for Large Language Models**](https://aclanthology.org/2023.findings-emnlp.691.pdf) (EMNLP Findings 2023)\n\n![Method_overview](figs/skr.png)\n\n## Data\nThe Temporal dataset we use is in the fold `data/`.\n- `Question`: The question.\n- `Gold answer`: The answer.\n- `passages`: The retrieved passages from wikipedia.\n\n## Chain-of-Thought Results\n- The CoT and retrieval-augmented CoT results are given in the fold `results/`, where the `chain_of_thought_gpt3` indicates the responses.\n\n## Steps\n- For SKR_prompt and SKR_icl, we use the prompts shown in the paper to elicit the self-knowledge of the dev data directly.\n\n- For SKR_cls, we use the training data and train a [BERT classifier](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification) to elicit the self-knowledge of the dev data. We use the settings with `lr=2e-5` and `epochs=10`.\n\n- For SKR_knn, the steps are as follows:\n    - cd `source/` , collect the self-knowledge of the training data, run `skr.py` and get the `train_skr.json` file.\n    - run `knn.py` to use the self-knowledge to the dev data and get the `dev_skr_knn.json` file.\n    - run `eval_skr.py` to evaluate the results.\n\n## Citation\n\n```bibtex\n@inproceedings{wang-etal-2023-self-knowledge,\n    title = \"Self-Knowledge Guided Retrieval Augmentation for Large Language Models\",\n    author = \"Wang, Yile  and Li, Peng  and Sun, Maosong  and Liu, Yang\",\n    booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2023\",\n    month = dec,\n    year = \"2023\",\n    address = \"Singapore\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2023.findings-emnlp.691\",\n    pages = \"10303--10315\",\n}\n```\n\n## Acknowledgement\n- [Rethinking with Retrieval](https://github.com/HornHehhf/RR)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fskr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthunlp-mt%2Fskr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fskr/lists"}