{"id":28676512,"url":"https://github.com/zjunlp/unlearn","last_synced_at":"2025-10-29T02:03:16.869Z","repository":{"id":278112768,"uuid":"633779841","full_name":"zjunlp/unlearn","owner":"zjunlp","description":"Knowledge Unlearning for Large Language Models","archived":false,"fork":false,"pushed_at":"2025-05-05T06:05:16.000Z","size":518,"stargazers_count":25,"open_issues_count":1,"forks_count":6,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-05T07:22:55.532Z","etag":null,"topics":["artificial-intelligence","copyright","knowledge-editing","knowledge-unlearning","large-language-models","machine-unlearning","natural-language-processing","privacy","semeval-2025","unlearning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zjunlp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-28T09:01:25.000Z","updated_at":"2025-05-05T06:05:20.000Z","dependencies_parsed_at":"2025-05-05T07:21:34.112Z","dependency_job_id":"d96d033f-b857-4d3b-9c9e-5d5bc980903c","html_url":"https://github.com/zjunlp/unlearn","commit_stats":null,"previous_names":["zjunlp/unlearn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zjunlp/unlearn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2Funlearn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2Funlearn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2Funlearn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2Funlearn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zjunlp","download_url":"https://codeload.github.com/zjunlp/unlearn/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2Funlearn/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259732772,"owners_count":22903087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","copyright","knowledge-editing","knowledge-unlearning","large-language-models","machine-unlearning","natural-language-processing","privacy","semeval-2025","unlearning"],"created_at":"2025-06-13T23:04:58.429Z","updated_at":"2025-10-29T02:03:16.796Z","avatar_url":"https://github.com/zjunlp.png","language":"Python","readme":"# Knowledge Unlearning for Large Language Models\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/2502.11190\"\u003e📄arXiv\u003c/a\u003e •\n  \u003ca href=\"https://huggingface.co/papers/2502.11190\"\u003e🤗HFPaper\u003c/a\u003e •\n  \u003ca href=\"https://huggingface.co/collections/zjunlp/relearn-67bbd781c4d637c75db8e577\"\u003e🤗HF Collection\u003c/a\u003e\n\u003c/p\u003e\n\nThis repository provides the official PyTorch implementation of our paper:\n\n\u003e **ReLearn: Unlearning via Learning for Large Language Models**\n\u003e\n\u003e Haoming Xu\u003csup\u003e1\u003c/sup\u003e, Ningyuan Zhao\u003csup\u003e2\u003c/sup\u003e, Liming Yang\u003csup\u003e3\u003c/sup\u003e, Sendong Zhao\u003csup\u003e4\u003c/sup\u003e, Shumin Deng\u003csup\u003e5\u003c/sup\u003e, Mengru Wang\u003csup\u003e1\u003c/sup\u003e, Bryan Hooi\u003csup\u003e5\u003c/sup\u003e, Nay Oo\u003csup\u003e5\u003c/sup\u003e, Huajun Chen\u003csup\u003e1\u003c/sup\u003e, Ningyu Zhang\u003csup\u003e1\u003c/sup\u003e\n\u003e\n\u003e \u003csup\u003e1\u003c/sup\u003eZhejiang University, \u003csup\u003e2\u003c/sup\u003eXiamen University, \u003csup\u003e3\u003c/sup\u003eTsinghua University, \u003csup\u003e4\u003c/sup\u003eHarbin Institute of Technology, \u003csup\u003e5\u003c/sup\u003eNational University of Singapore\n\n## 🎉 News\n\n🏆 Our team won 2nd place in the [**SEMEval 2025 Challenge on Unlearning Sensitive Content from Large Language Models**!](https://llmunlearningsemeval2025.github.io/) Check out our implementation in the `Semeval25` directory.\n\n## 🌟 Overview\n\n![Introduction](images/intro.jpg)\n\n## 📦 Installation\n\n```bash\n# Create and activate conda environment\nconda create -n relearn python=3.10.15\nconda activate relearn\n\n# Install PyTorch with CUDA support\nconda install pytorch pytorch-cuda=11.8 -c pytorch -c nvidia\nconda install -c \"nvidia/label/cuda-11.8.0\" cuda-toolkit\n\n# Install dependencies\npip install -r requirements.txt\npip install flash-attn --no-build-isolation\n```\n\n## 🚀 Quick Start\n\n### 1. Data Augmentation\n```bash\ncd dataAugument\nbash augu.sh\n```\n\n### 2. Model Training\nCurrently supports:\n- Llama3-8b instruct\n- Gemma2-2b-it\n- Llama2-7b chat\n\n```bash\ncd baselines/pretrain_scripts/\nbash kud-pt.sh\n```\n\n### 3. Unlearning Process\n```bash\ncd baselines/unlearn_scripts/\nbash kud-relearn.sh\n```\n\n### 4. Evaluation\n```bash\ncd evals\nbash merge_all.sh\nbash inf_all.sh\nbash eval_all.sh\n```\n**Note:** If you plan to use KFR and KRR, please configure the API in [`dataAugment/utils.py`](https://github.com/zjunlp/unlearn/blob/main/dataAugument/utils.py).\n\n## 🔧 Supported Methods\n\n| Method      | Script                                               | \n| ----------- | ---------------------------------------------------- | \n| GA / NPO    | `unlearn/baselines/unlearn_scripts/kud-baselines.sh` |\n| SURE        | `unlearn/baselines/unlearn_scripts/kud-baselines.sh` |\n| Memflex (Iterative version)     | `unlearn/baselines/unlearn_scripts/kud-baselines.sh` |\n| ReLearn     | `unlearn/baselines/unlearn_scripts/kud-relearn.sh`   | \n| ReLearn_dpo | `unlearn/baselines/unlearn_scripts/kud-relearn.sh`   | \n\n## 📂 Open Resources\n\n### Pretrained Models\n- **Llama-2-7b-chat-KnowUnDo-Privacy (Vanilla)**  \n  [🔗 ModelScope](https://www.modelscope.cn/models/haomingx/Llama-2-7b-chat-KnowUnDo-Privacy/files)\n\n- **Llama-2-7b-chat-TOFU-Forget10-ReLearn**  \n  [🔗 Google Drive](https://drive.google.com/drive/folders/1wsPKpF2IZ4RC52_PI7ILhYsegtqZG25Y?usp=drive_link)\n\n- **Llama-2-7b-chat-KnowUnDo-Privacy-ReLearn**  \n  [🔗 Google Drive](https://drive.google.com/drive/folders/1R7wSu1kegr0Ui4x_R-5L5vg4vuoFhskM?usp=drive_link)\n\n### Datasets\n- **Augmented KnowUnDo Privacy Dataset**  \n  [🔗 Google Drive](https://drive.google.com/file/d/1lct2s3Xs8JKv4CL-LlBZHXTP9H1AKeg5/view?usp=drive_link)\n- **Augmented ToFU Forget01 Dataset**  \n  [🔗 Google Drive](https://drive.google.com/file/d/16NtfMeB_4ISApuVrJnQHo26EKjT9xzvz/view?usp=sharing)\n\n### Inference \u0026 Eval Results\n- **Llama-2-7b-chat KnowUnDo Privacy**\n  [🔗 Google Drive](https://drive.google.com/drive/folders/169E1HDgZGcDTKAJcKJX17SoQtpkkd1pV?usp=drive_link)\n## 🙏 Acknowledgements\nWe would like to express our heartfelt gratitude for the contribution of [KnowUnDo](https://github.com/zjunlp/KnowUnDo), [TOFU](https://github.com/locuslab/tofu), [MUSE](https://github.com/jaechan-repo/muse_bench), [SURE](https://github.com/zzwjames/FailureLLMUnlearning) [Open-Unlearning](https://github.com/locuslab/open-unlearning) to our project, as we have utilized portions of their source code in our project.\n\n## 📝 Citation\n\nIf you find this work useful for your research, please cite [our paper](https://arxiv.org/abs/2502.11190):\n\n```bibtex\n@article{xu2025relearnunlearninglearninglarge,\n      title={ReLearn: Unlearning via Learning for Large Language Models}, \n      author={Haoming Xu and Ningyuan Zhao and Liming Yang and Sendong Zhao and \n              Shumin Deng and Mengru Wang and Bryan Hooi and Nay Oo and \n              Huajun Chen and Ningyu Zhang},\n      journal={arXiv preprint arXiv:2502.11190},\n      year={2025}\n}\n\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2Funlearn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzjunlp%2Funlearn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2Funlearn/lists"}