{"id":28676536,"url":"https://github.com/zjunlp/knowledgecircuits","last_synced_at":"2025-06-13T23:05:04.317Z","repository":{"id":242362265,"uuid":"743879445","full_name":"zjunlp/KnowledgeCircuits","owner":"zjunlp","description":"Knowledge Circuits  in Pretrained Transformers","archived":false,"fork":false,"pushed_at":"2024-09-18T06:35:24.000Z","size":6220,"stargazers_count":46,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-09-18T09:04:36.399Z","etag":null,"topics":["artificial-intelligence","circuit","hallucination","interpretability","knowledge-circuit","knowledge-editing","knowledge-edting","large-language-models","model-editing","natural-language-processing","transformer"],"latest_commit_sha":null,"homepage":"http://knowledgecircuits.zjukg.cn/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zjunlp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-16T07:23:21.000Z","updated_at":"2024-09-18T06:35:27.000Z","dependencies_parsed_at":"2024-06-02T15:00:35.801Z","dependency_job_id":"4527223b-cfc1-49d6-84e3-38c79a42bd78","html_url":"https://github.com/zjunlp/KnowledgeCircuits","commit_stats":null,"previous_names":["zjunlp/knowledgecircuits"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zjunlp/KnowledgeCircuits","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FKnowledgeCircuits","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FKnowledgeCircuits/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FKnowledgeCircuits/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FKnowledgeCircuits/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zjunlp","download_url":"https://codeload.github.com/zjunlp/KnowledgeCircuits/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjunlp%2FKnowledgeCircuits/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259732771,"owners_count":22903087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","circuit","hallucination","interpretability","knowledge-circuit","knowledge-editing","knowledge-edting","large-language-models","model-editing","natural-language-processing","transformer"],"created_at":"2025-06-13T23:05:03.681Z","updated_at":"2025-06-13T23:05:04.283Z","avatar_url":"https://github.com/zjunlp.png","language":"Python","readme":"\u003ch1 align=\"center\"\u003e Knowledge Circuits \u003c/h1\u003e\n\u003ch3 align=\"center\"\u003e Knowledge Circuits in Pretrained Transformers \u003c/h3\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/2405.17969\"\u003e📄arXiv\u003c/a\u003e •\n  \u003ca href=\"http://knowledgecircuits.zjukg.cn/\"\u003e🌐Demo\u003c/a\u003e •\n  \u003ca href=\"https://www.youtube.com/watch?v=qDgCLeDs4Kg\"\u003e \u003cimg src=\"https://img.icons8.com/?size=100\u0026id=19318\u0026format=png\u0026color=000000\" height=\"20\" width=\"20\"\u003e Youtube\u003c/a\u003e • \n    \u003ca href=\"https://x.com/zxlzr/status/1797261767674138924\"\u003e𝕏 Blog\u003c/a\u003e\n\u003c/p\u003e\n\n[![Awesome](https://awesome.re/badge.svg)](https://github.com/zjunlp/KnowledgeCircuits) \n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n![](https://img.shields.io/github/last-commit/zjunlp/KnowledgeCircuits?color=green) \n\n## 🔔News\n\n- \u003cstrong\u003e[2025-02-16]\u003c/strong\u003e We release our new paper [How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training](https://arxiv.org/abs/2502.11196), analyzing the evolution of knowledge circuits throughout continual pre-training. Check it out:)\n- \u003cstrong\u003e[2024-09-26]\u003c/strong\u003e Our paper [Knowledge Circuits in Pretrained Transformers](https://arxiv.org/abs/2405.17969) is accepetd by \u003cstrong\u003eNeurIPS 2024\u003c/strong\u003e!\n- \u003cstrong\u003e[2024-05-28]\u003c/strong\u003e We release our paper [Knowledge Circuits in Pretrained Transformers](https://arxiv.org/abs/2405.17969).\n\n\n## Table of Contents\n- 🌟[Overview](#overview)\n- 🔧[Installation](#installation)\n- 📚[Get the circuit](#get-the-circuit)\n- 🧐[Analyze Component](#analyze-component)\n- 🌻[Acknowledgement](#acknowledgement)\n- 🚩[Citation](#citation)\n\n---\n\n\n## 🌟Overview\n\nThis work aims to build the circuits in the pretrained language models that are responsible for the specific knowledge and analyze the behavior of these components.\nWe construct a [demo](http://knowledgecircuits.zjukg.cn/) to see the discovered circuit.\n* A new method [EAP-IG](https://arxiv.org/abs/2403.17806) is integrated in the eap folder. This method takes less time than the ACDC method and you can use it in the `knowledge_eap.ipynb`. If you are using the LLaMA2-7B-Chat model, running this file on a single GPU will require approximately 57,116M of GPU memory and 3-4 minutes. \n\n## 🔧Installation\n\nThe filtered data for each kind of model is at [here](https://pan.zju.edu.cn/share/7c613d16095c504605f83eba72). Please download it and put it in the data folder.\n\nBuild the environement:\n```\nconda create -n knowledgecircuit python=3.10\npip install -r requirements.txt\n```\n❗️The code may fail under torch 2.x.x. We recommend torch 1.x.x\n\n## 📚Get the circuit\n\nJust run the following commond:\n```\ncd acdc\nsh run.sh\n```\nHere is an example to run the circuit for the `country_capital_city` in `GPT2-Medium`.\n```\nMODEL_PATH=/path/to/the/model\nKT=factual \nKNOWLEDGE=country_capital_city\nNUM_EXAMPLES=20\nMODEL_NAME=gpt2-medium\n\npython main.py --task=knowledge \\\n--zero-ablation \\\n--threshold=0.01 \\\n--device=cuda:0 \\\n--metric=match_nll \\\n--indices-mode=reverse \\\n--first-cache-cpu=False \\\n--second-cache-cpu=False \\\n--max-num-epochs=10000 \\\n--specific-knowledge=$KNOWLEDGE \\\n--num-examples=$NUM_EXAMPLES \\\n--relation-reverse=False \\\n--knowledge-type=$KT \\\n--model-name=$MODEL_NAME \\\n--model-path=$MODEL_PATH\n```\n\nYou would get the results in `acdc/factual_results/gpt2-medium` and the  `final_graph.pdf` is the computed circuits.\n\n## 🧐Analyze component\n\nRun the component.ipynb in notebook.\n\n## 🌻Acknowledgement\n\nWe thank for the project of [transformer_lens](https://github.com/TransformerLensOrg/TransformerLens), [ACDC](https://github.com/ArthurConmy/Automatic-Circuit-Discovery) and [LRE](https://lre.baulab.info/).\nThe code in this work is built on top of these three projects' codes.\n\n\n## 🚩Citation\n\nPlease cite our repository if you use Knowledge Circuit in your work. Thanks!\n\n```bibtex\n@article{DBLP:journals/corr/abs-2405-17969,\n  author       = {Yunzhi Yao and\n                  Ningyu Zhang and\n                  Zekun Xi and\n                  Mengru Wang and\n                  Ziwen Xu and\n                  Shumin Deng and\n                  Huajun Chen},\n  title        = {Knowledge Circuits in Pretrained Transformers},\n  journal      = {CoRR},\n  volume       = {abs/2405.17969},\n  year         = {2024},\n  url          = {https://doi.org/10.48550/arXiv.2405.17969},\n  doi          = {10.48550/ARXIV.2405.17969},\n  eprinttype    = {arXiv},\n  eprint       = {2405.17969},\n  timestamp    = {Fri, 21 Jun 2024 22:39:09 +0200},\n  biburl       = {https://dblp.org/rec/journals/corr/abs-2405-17969.bib},\n  bibsource    = {dblp computer science bibliography, https://dblp.org}\n}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2Fknowledgecircuits","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzjunlp%2Fknowledgecircuits","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjunlp%2Fknowledgecircuits/lists"}