{"id":19422836,"url":"https://github.com/kwaikeg/coggpt","last_synced_at":"2025-08-13T11:46:53.649Z","repository":{"id":215547422,"uuid":"736488997","full_name":"KwaiKEG/CogGPT","owner":"KwaiKEG","description":"Unleashing the Power of Cognitive Dynamics on Large Language Models","archived":false,"fork":false,"pushed_at":"2024-09-24T07:33:04.000Z","size":4116,"stargazers_count":61,"open_issues_count":0,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-06T19:06:37.846Z","etag":null,"topics":["agi","ai","chatgpt","cognitive-science","gpt","gpt-4","llms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KwaiKEG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-28T03:43:20.000Z","updated_at":"2025-04-06T07:12:40.000Z","dependencies_parsed_at":"2024-11-10T13:36:01.213Z","dependency_job_id":"ffeb99c1-dae7-4fa5-986e-d5d9c68ee4ef","html_url":"https://github.com/KwaiKEG/CogGPT","commit_stats":null,"previous_names":["kwaikeg/coggpt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/KwaiKEG/CogGPT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwaiKEG%2FCogGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwaiKEG%2FCogGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwaiKEG%2FCogGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwaiKEG%2FCogGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KwaiKEG","download_url":"https://codeload.github.com/KwaiKEG/CogGPT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwaiKEG%2FCogGPT/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270234901,"owners_count":24550173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agi","ai","chatgpt","cognitive-science","gpt","gpt-4","llms"],"created_at":"2024-11-10T13:35:28.253Z","updated_at":"2025-08-13T11:46:53.596Z","avatar_url":"https://github.com/KwaiKEG.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [EMNLP 2024] CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models\n\n\u003cp align=\"left\"\u003e\n    English | \u003ca href=\"README_ZH.md\"\u003e中文\u003c/a\u003e\n\u003c/p\u003e\n\u003cbr\u003e\u003cbr\u003e\n\nCode and data for the paper \"\u003ca href=\"https://arxiv.org/abs/2401.08438\"\u003eCogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models\u003c/a\u003e\".\n\n## CogBench\n\n**\u003ca href=\"https://huggingface.co/datasets/kwaikeg/CogBench\"\u003eCogBench\u003c/a\u003e** is a bilingual benchmark specifically designed to evaluate the cognitive dynamics of Large Language Models (LLMs) in both Chinese and English. CogBench is divided into two parts based on the type of information flow: CogBench\u003csub\u003ea\u003c/sub\u003e for articles and CogBench\u003csub\u003ev\u003c/sub\u003e for short videos.\n\nIn this benchmark, both an LLM and a human are assigned the same initial profile and receive identical information flows over 10 iterations. After each iteration, they are required to complete the same cognitive questionnaire. This questionnaire, using a five-point Likert scale, allows participants to express their attitudes towards the current questions.\n\nCogBench aims to assess the cognitive alignment between the LLM and the human. The evaluation metrics include:\n\n1. **Authenticity**: Measures the consistency of ratings between the LLM and the human.\n2. **Rationality**: Assesses the reasoning provided by the LLM.\n\n## CogGPT\n\n**CogGPT** is an LLM-driven agent, designed to showcase the cognitive dynamics of LLMs. Confronted with ever-changing information flows, CogGPT regularly updates its profile and methodically stores preferred knowledge in its long-term memory. This unique capability enables CogGPT to sustain role-specific cognitive dynamics, facilitating lifelong learning.\n\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\n   \u003cimg src=\"blob/model.png\" alt=\"CogGPT\"/\u003e\n\u003c/p\u003e\n\n## News\n\n* 2024.01.17 - [Paper](https://arxiv.org/abs/2401.08438) released.\n* 2024.01.12 - [CogBench](https://huggingface.co/datasets/kwaikeg/CogBench) released.\n* 2024.01.05 - Project initially released.\n\n## User Guide\n\n### Setup\n\nFollow these steps to build CogBench:\n\n1. **Clone the Repository**: Clone this repository to your local environment.\n2. **Switch Directory**: Use the `cd` command to enter the repository directory.\n3. **Download Data**: Download the [CogBench](https://huggingface.co/datasets/kwaikeg/CogBench) and save it in the `dataset` directory.\n4. **Run Experiments**: Implement your method using `cogbench_a.json` and `cogbench_v.json` for CogBench\u003csub\u003ea\u003c/sub\u003e and CogBench\u003csub\u003ev\u003c/sub\u003e, respectively, and record your experimental results.\n5. **Evaluate Results**: Fill in the `eval_cogbench_a.json` and `eval_cogbench_v.json` files with your experimental results for evaluations.\n\n### Using CogGPT\n\n1. Declare environment variables to use the GPT-4 API:\n\n```bash\nexport OPENAI_API_KEY=sk-xxxxx\n```\n\n2. Run CogGPT with default settings:\n\n```bash\npython coggpt/agent.py\n```\n\n### Evaluation\n\nTo evaluate your method based on the authenticity and rationality metrics, we recommend running the following commands:\n\n```bash\npython evaluation.py --file_path \u003cYOUR_FILE_PATH\u003e --method \u003cYOUR_METHOD_NAME\u003e --authenticity --rationality\n```\n\nFor example, to evaluate the `CoT` method on CogBench\u003csub\u003ev\u003c/sub\u003e, run:\n\n```bash\npython evaluation.py --file_path dataset/english/eval_cogbench_v.json --method CoT --authenticity --rationality\n```\n\nThe evaluation scores will be displayed as follows:\n\n```bash\n======= CoT Authenticity =======\nAverage authenticity: 0.15277666156947955\n5th iteration authenticity: 0.3023255813953488\n10th iteration authenticity: 0.13135593220338992\n======= CoT Rationality =======\nAverage rationality: 3.058333333333333\n5th iteration rationality: 3.7666666666666666\n10th iteration rationality: 3.0833333333333335\n```\n\nPlease refer to \u003ca href=\"https://huggingface.co/datasets/kwaikeg/CogBench\"\u003eCogBench\u003c/a\u003e for more details.\n\n## Citation\n```\n@misc{lv2024coggpt,\n      title={CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models}, \n      author={Yaojia Lv and Haojie Pan and Ruiji Fu and Ming Liu and Zhongyuan Wang and Bing Qin},\n      year={2024},\n      eprint={2401.08438},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwaikeg%2Fcoggpt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwaikeg%2Fcoggpt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwaikeg%2Fcoggpt/lists"}