{"id":28615881,"url":"https://github.com/habedi/cogitator","last_synced_at":"2025-06-12T02:10:40.446Z","repository":{"id":290921696,"uuid":"974969380","full_name":"habedi/cogitator","owner":"habedi","description":"A Python toolkit for chain-of-thought prompting 🐍","archived":false,"fork":false,"pushed_at":"2025-06-08T11:54:10.000Z","size":132,"stargazers_count":167,"open_issues_count":6,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-08T12:33:09.884Z","etag":null,"topics":["ai-research","ai-toolkit","chain-of-thought","cot-prompting","explainable-ai","large-language-models","llms","machine-learning","prompt-engineering","python","python-library"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/habedi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-29T15:20:28.000Z","updated_at":"2025-06-06T13:38:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"8b4809f1-d662-487b-973d-a77d63923f93","html_url":"https://github.com/habedi/cogitator","commit_stats":null,"previous_names":["habedi/cogitator"],"tags_count":3,"template":false,"template_full_name":"habedi/template-python-library","purl":"pkg:github/habedi/cogitator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Fcogitator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Fcogitator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Fcogitator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Fcogitator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/habedi","download_url":"https://codeload.github.com/habedi/cogitator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/habedi%2Fcogitator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259382356,"owners_count":22848845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-research","ai-toolkit","chain-of-thought","cot-prompting","explainable-ai","large-language-models","llms","machine-learning","prompt-engineering","python","python-library"],"created_at":"2025-06-12T02:10:36.399Z","updated_at":"2025-06-12T02:10:40.433Z","avatar_url":"https://github.com/habedi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003cimg alt=\"Cogitator Logo\" src=\"logo.svg\" height=\"30%\" width=\"30%\"\u003e\n  \u003c/picture\u003e\n\u003cbr\u003e\n\n\u003ch2\u003eCogitator\u003c/h2\u003e\n\n[![Tests](https://img.shields.io/github/actions/workflow/status/habedi/cogitator/tests.yml?label=tests\u0026style=flat\u0026labelColor=333333\u0026logo=github\u0026logoColor=white)](https://github.com/habedi/cogitator/actions/workflows/tests.yml)\n[![Code Coverage](https://img.shields.io/codecov/c/github/habedi/cogitator?style=flat\u0026label=coverage\u0026labelColor=333333\u0026logo=codecov\u0026logoColor=white)](https://codecov.io/gh/habedi/cogitator)\n[![Code Quality](https://img.shields.io/codefactor/grade/github/habedi/cogitator?style=flat\u0026label=code%20quality\u0026labelColor=333333\u0026logo=codefactor\u0026logoColor=white)](https://www.codefactor.io/repository/github/habedi/cogitator)\n[![Python Version](https://img.shields.io/badge/python-%3E=3.10-3776ab?style=flat\u0026labelColor=333333\u0026logo=python\u0026logoColor=white)](https://github.com/habedi/cogitator)\n[![PyPI Version](https://img.shields.io/pypi/v/cogitator.svg?style=flat\u0026label=pypi\u0026labelColor=333333\u0026logo=pypi\u0026logoColor=white\u0026color=3775a9)](https://pypi.org/project/cogitator)\n[![Downloads](https://img.shields.io/pypi/dm/cogitator.svg?style=flat\u0026label=downloads\u0026labelColor=333333\u0026logo=pypi\u0026logoColor=white\u0026color=cc8400)](https://github.com/habedi/cogitator)\n\u003cbr\u003e\n[![License](https://img.shields.io/badge/license-MIT-00acc1?style=flat\u0026labelColor=333333\u0026logo=open-source-initiative\u0026logoColor=white)](https://github.com/habedi/cogitator/blob/main/LICENSE)\n[![Docs](https://img.shields.io/badge/docs-latest-8ca0d7?style=flat\u0026labelColor=333333\u0026logo=readthedocs\u0026logoColor=white)](https://habedi.github.io/cogitator)\n[![DOI](https://img.shields.io/badge/doi-10.5281/zenodo.15331821-6f42c1.svg?style=flat\u0026labelColor=333333\u0026logo=zenodo\u0026logoColor=white)](https://doi.org/10.5281/zenodo.15331821)\n\nA Python Toolkit for Chain-of-Thought Prompting\n\n\u003c/div\u003e\n\n---\n\nCogitator is a Python toolkit for experimenting and working with\n[chain-of-thought (CoT) prompting](https://arxiv.org/abs/2201.11903)\nmethods in large language models (LLMs).\nCoT prompting improves LLM performance on complex tasks (like question-answering, reasoning, and problem-solving)\nby guiding the models to generate intermediate reasoning steps before arriving at the final answer.\nAdditionally, it can be used to improve the interpretability of LLMs by providing insight into the model's reasoning process.\nThe toolkit aims to make it easier to use popular CoT strategies and frameworks for research or integrating them into AI\napplications.\n\n### Features\n\n* Provides unified sync/async API for CoT strategies\n* Supports using OpenAI and Ollama as LLM providers\n* Supports structured model output with Pydantic validation\n* Includes a customizable benchmarking framework (see [benches](benches))\n* Includes implementations of popular CoT strategies and frameworks like\n    - [Self-Consistency CoT (ICLR 2023)](https://arxiv.org/abs/2203.11171)\n    - [Automatic CoT (ICLR 2023)](https://arxiv.org/abs/2210.03493)\n    - [Least-to-Most Prompting (ICLR 2023)](https://arxiv.org/abs/2205.10625)\n    - [Tree of Thoughts (NeurIPS 2023)](https://arxiv.org/abs/2305.10601)\n    - [Graph of Thoughts (AAAI 2024)](https://arxiv.org/abs/2308.09687)\n    - [Clustered Distance-Weighted CoT (AAAI 2025)](https://arxiv.org/abs/2501.12226)\n\n---\n\n### Getting Started\n\nYou can install Cogitator with\n\n```bash\npip install cogitator\n```\n\nOr, if you want to install from the latest version with examples and benchmarks included\n\n```bash\ngit clone https://github.com/habedi/cogitator \u0026\u0026 cd cogitator\n\n# Set up Python environment\npip install poetry\npoetry install --with dev\n\n# Run the tests to make sure everything is working (optional)\npoetry run pytest\n```\n\n#### Examples\n\nBelow is a simple example of using the Self-Consistency CoT with Ollama.\n\n```python\nimport logging\nfrom cogitator import SelfConsistency, OllamaLLM\n\n# Step 1: Configure logging (optional, but helpful)\nlogging.basicConfig(level=logging.INFO)\nlogging.getLogger(\"httpx\").setLevel(logging.WARNING)  # Suppress HTTPX logs\n\n# Step 2: Initialize the LLM (using Ollama)\n# Needs Ollama running locally with the model pulled (e.g., `ollama pull gemma3:4b`)\ntry:\n    llm = OllamaLLM(model=\"gemma3:4b\")\nexcept Exception as e:\n    print(f\"Error initializing Ollama LLM: {e}\")\n    print(\"Please make sure Ollama is running and the model is pulled.\")\n    exit(1)\n\n# Step 3: Choose a CoT strategies (Self-Consistency in this case)\n# Self-Consistency generates multiple reasoning paths and finds the most common answer\nsc_strategy = SelfConsistency(\n    llm,\n    n_samples=5,  # Number of reasoning paths to generate\n    temperature=0.7  # Higher temperature can lead to more diverse answers\n)\n\n# Step 4: Define the prompt (with a basic CoT trigger)\nquestion = \"A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?\"\nprompt = f\"Q: {question}\\nA: Let's think step by step.\"\n\n# Step 5: Run the CoT prompting sc_strategy\nprint(f\"\\nQuestion: {question}\")\nprint(\"Running Self-Consistency CoT...\")\nfinal_answer = sc_strategy.run(prompt)  # Returns the most consistent (repeated) answer\n\n# Expected output: $0.05 or 0.05 (may vary slightly based on model and temperature)\nprint(f\"\\nCogitator's Answer (Self-Consistency): {final_answer}\")\n```\n\nCheck out the [examples](examples) directory for more examples.\n\n---\n\n### Documentation\n\nCogitator documentation is available [here](https://habedi.github.io/cogitator).\n\n---\n\n### Benchmarking Framework\n\nThis project includes a customizable and extensible benchmarking framework to evaluate the performance of different\nCoT strategies on various datasets like [GSM8K](https://arxiv.org/abs/2110.14168) and\n[StrategyQA](https://arxiv.org/abs/2101.02235).\n\nCheck out the [benches](benches) directory for more details about the framework and how it could be used.\n\n---\n\n### Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to make a contribution.\n\n### Citations\n\nIf you find this project useful, please give it a star!\nIf you have any questions or feedback, please use the discussion section of the repository or open an issue.\nIf you use this project in your research, please consider citing using the following information:\n\n```bibtex\n@software{abedi_cogitator_2025,\n    author = {Abedi Firouzjaei, Hassan},\n    title = {{Cogitator: A Python Toolkit for Chain-of-Thought Prompting}},\n    year = {2025--},\n    publisher = {Zenodo},\n    doi = {10.5281/zenodo.15331821},\n    url = {https://github.com/habedi/cogitator}\n}\n```\n\n### Logo\n\nThe logo is named \"Cognition\" and was originally created by [vectordoodle](https://www.svgrepo.com/author/vectordoodle).\n\n### License\n\nCogitator is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhabedi%2Fcogitator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhabedi%2Fcogitator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhabedi%2Fcogitator/lists"}