{"id":13994137,"url":"https://github.com/reasoning-machines/pal","last_synced_at":"2025-07-22T18:33:32.284Z","repository":{"id":63613761,"uuid":"567774780","full_name":"reasoning-machines/pal","owner":"reasoning-machines","description":"PaL: Program-Aided Language Models (ICML 2023)","archived":false,"fork":false,"pushed_at":"2023-06-30T12:57:39.000Z","size":2256,"stargazers_count":488,"open_issues_count":8,"forks_count":60,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-12T19:17:59.147Z","etag":null,"topics":["commonsense-reasoning","few-shot-learning","language-generation","language-model","large-language-models","reasoning"],"latest_commit_sha":null,"homepage":"https://reasonwithpal.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reasoning-machines.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-18T14:48:18.000Z","updated_at":"2025-04-08T13:37:06.000Z","dependencies_parsed_at":"2024-01-18T05:07:26.630Z","dependency_job_id":"30c02292-7d6c-4a0d-b9b8-1a6e28f75d56","html_url":"https://github.com/reasoning-machines/pal","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/reasoning-machines/pal","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reasoning-machines%2Fpal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reasoning-machines%2Fpal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reasoning-machines%2Fpal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reasoning-machines%2Fpal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reasoning-machines","download_url":"https://codeload.github.com/reasoning-machines/pal/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reasoning-machines%2Fpal/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266552590,"owners_count":23947179,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["commonsense-reasoning","few-shot-learning","language-generation","language-model","large-language-models","reasoning"],"created_at":"2024-08-09T14:02:43.568Z","updated_at":"2025-07-22T18:33:27.265Z","avatar_url":"https://github.com/reasoning-machines.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话","Python"],"sub_categories":["大语言对话模型及数据"],"readme":"# PaL:  Program-Aided Language Model\nRepo for the paper [PaL: Program-Aided Language Models](https://arxiv.org/pdf/2211.10435.pdf).\n\nIn PaL, Large Language Model solves reasoning problems that involve complex arithmetic and procedural tasks by generating reasoning chains of **text and code**.  This offloads the execution of the code to a program runtime, in our case, a Python interpreter. In our paper, we implement PaL using a few-shot prompting approach. \n\n\u003cimg width=\"879\" alt=\"image\" src=\"https://user-images.githubusercontent.com/15002544/202954503-b3fade57-87ff-4beb-81de-72405577b2b4.png\"\u003e\n\n\nThis repo provides an interactive implementation of PAL.\n\n## News 📢\n[Mar 2023] We have added supports for ChatGPT APIs (e.g., gpt-3.5-turbo). We expect a smooth transition for PAL over the codex API shutdown. Checkout a beta script `scripts/gsm_chatgpt.py` for Math reasoning.\n\n[Jan 2023] We release [GSM-hard](https://github.com/reasoning-machines/pal/blob/main/datasets/gsmhardv2.jsonl), a harder version of GSM8k we created. Also avaliable on [Huggingface 🤗](https://huggingface.co/datasets/reasoning-machines/gsm-hard)\n```python\nimport datasets\ngsm_hard = datasets.load_dataset(\"reasoning-machines/gsm-hard\")\n\n```\n## Installation\nClone this repo and install with `pip`.\n```\ngit clone https://github.com/luyug/pal\npip install -e ./pal\n```\n\nBefore running the scripts, set the OpenAI key,\n```export OPENAI_API_KEY='sk-...'```\n## Interactive Usage\nThe core components of the `pal` package are the Interface classes. Specifically, `ProgramInterface` connects the LLM backend, a Python backend and user prompts.\n```\nimport pal\nfrom pal.prompt import math_prompts\n\ninterface = pal.interface.ProgramInterface(\n  model='code-davinci-002',\n  stop='\\n\\n\\n', # stop generation str for Codex API\n  get_answer_expr='solution()' # python expression evaluated after generated code to obtain answer \n)\n\nquestion = 'xxxxx'\nprompt = math_prompts.MATH_PROMPT.format(question=question)\nanswer = interface.run(prompt)\n```\nHere, the `interface` 's `run`  method will run generation with the OpenAI API, run the generated snippet and then evaluate `get_answer_expr` (here `solution()`) to obtain the final answer.  \n\nUser should set `get_answer_expr` based on the prompt.\n\n## Inference Loop\nWe provide simple inference loops in the `scripts/` folder.\n```\nmkdir eval_results\npython scripts/{colored_objects|gsm|date_understanding|penguin}_eval.py\n``` \n\nWe have a beta release of a **ChatGPT** dedicated script for math reasoning.\n```\npython scripts/gsm_chatgpt.py\n``` \n\nFor running bulk inference, we used the generic prompting library [prompt-lib](https://github.com/madaan/prompt-lib) and recommend it for running CoT inferenence on all tasks used in our work.\n\n## Results\n\n\u003cimg width=\"831\" alt=\"image\" src=\"https://user-images.githubusercontent.com/15002544/202954755-bf89aab6-6467-436e-98d6-2ca378a20116.png\"\u003e\n\n\u003cimg width=\"1166\" alt=\"image\" src=\"https://user-images.githubusercontent.com/15002544/202954780-7e1221f1-3008-46d9-877b-b26df9f98d66.png\"\u003e\n\n\u003cimg width=\"597\" alt=\"image\" src=\"https://user-images.githubusercontent.com/15002544/202954797-68e8d45d-3435-4abf-96e4-f10371b55e38.png\"\u003e\n\nFor the complete details of the results, see the [paper](https://arxiv.org/pdf/2211.10435.pdf) .\n\n## Citation\n```\n@article{gao2022pal,\n  title={PAL: Program-aided Language Models},\n  author={Gao, Luyu and Madaan, Aman and Zhou, Shuyan and Alon, Uri and Liu, Pengfei and Yang, Yiming and Callan, Jamie and Neubig, Graham},\n  journal={arXiv preprint arXiv:2211.10435},\n  year={2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freasoning-machines%2Fpal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freasoning-machines%2Fpal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freasoning-machines%2Fpal/lists"}