{"id":13429729,"url":"https://github.com/1989Ryan/llm-mcts","last_synced_at":"2025-03-16T04:31:02.770Z","repository":{"id":168513516,"uuid":"643714050","full_name":"1989Ryan/llm-mcts","owner":"1989Ryan","description":"[NeurIPS 2023] We use large language models as commonsense world model and heuristic policy within Monte-Carlo Tree Search, enabling better-reasoned decision-making for daily task planning problems. ","archived":false,"fork":false,"pushed_at":"2024-02-25T11:36:29.000Z","size":764,"stargazers_count":64,"open_issues_count":0,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-06T17:36:29.580Z","etag":null,"topics":["large-language-models","neurips-2023","task-planning"],"latest_commit_sha":null,"homepage":"https://llm-mcts.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1989Ryan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-05-22T02:10:03.000Z","updated_at":"2024-04-23T12:56:35.213Z","dependencies_parsed_at":"2023-10-17T05:32:21.229Z","dependency_job_id":"67ff7fe3-b410-4aba-8a28-0e6d75ca8f3d","html_url":"https://github.com/1989Ryan/llm-mcts","commit_stats":null,"previous_names":["llm-mcts/llm-mcts"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1989Ryan%2Fllm-mcts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1989Ryan%2Fllm-mcts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1989Ryan%2Fllm-mcts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1989Ryan%2Fllm-mcts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1989Ryan","download_url":"https://codeload.github.com/1989Ryan/llm-mcts/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243826788,"owners_count":20354220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["large-language-models","neurips-2023","task-planning"],"created_at":"2024-07-31T02:00:44.336Z","updated_at":"2025-03-16T04:31:02.228Z","avatar_url":"https://github.com/1989Ryan.png","language":"Python","funding_links":[],"categories":["3 Reasoning Tasks"],"sub_categories":["3.1 Commonsense Reasoning"],"readme":"# llm-mcts\n\nThis repository contains the code for NeurIPS'23 paper: [Large language models as commonsense knowledge for large-scale task planning](https://arxiv.org/abs/2305.14078). \n\nWe use Large Language Models as both the commonsense world model and the heuristic policy within Monte Carlo Tree Search. LLM's world model provides with MCTS a commonsense prior belief of states for reasoned decision-making. The LLM's heuristic policy guides the search to relevant parts of the tree, substantially reducing the search complexity.\n\n![Figure](media/llm-mcts.jpeg)\n\n## Updates\n\n* [25 Feb 2024] We have updated the code to use the latest version of the OpenAI API. \n\n## Cite\n\n```\n@inproceedings{\n  zhao2023large,\n  title={Large Language Models as Commonsense Knowledge for Large-Scale Task Planning},\n  author={Zirui Zhao and Wee Sun Lee and David Hsu},\n  booktitle={Thirty-seventh Conference on Neural Information Processing Systems},\n  year={2023},\n  url={https://openreview.net/forum?id=Wjp1AYB8lH}\n}\n```\n\n## Install\n\nInstall the repo: \n```\ngit clone --recurse-submodules https://github.com/1989Ryan/llm-mcts.git\n```\n\nYou need to first install virtual home. Please follow with the link at [here](./vh/vh_sim/README_Download.md) as well as the official repository at [here](https://github.com/xavierpuigf/virtualhome) to install.\n\nTo intall the dependencies in our method, run\n```\npip install -r requirement.txt\n```\n\n## Generate Data\n\nWe use the code from [here](https://github.com/xavierpuigf/watch_and_help) to generate the data. You can also use the script at [here](./scripts/gene_data.sh) to generate the data. \n\nTo generate data, you need to generate the goal of a domain first, using the command\n```\npython vh/data_gene/gen_data/vh_init.py \\\n    --port \"{Port Number}\" \\\n    --task {choose your task} \\\n    --mode {choose one difficulty} \\\n    --usage {training or testing} \\\n    --num-per-apartment {a number} \n```\n\nThen, to generate expert data, you need to use\n```\npython vh/data_gene/testing_agents/gene_data.py \\\n    --mode {difficulty} \\\n    --dataset_path {the path to the file generated in the previous step}\\\n    --base-port {port number}\n```\n\nAfter that, we need to pre-process the expert data\n```\npython mcts/virtualhome/expert_data.py\n```\n\n## Run\n\nAdd your openai api key in both `./mcts/virtualhome/llm_model.py` and `./mcts/virtualhome/llm_policy.py`. \n\nGenerate the world model by LLM:\n```\npython mcts/virtualhome/llm_model.py\n```\n\nTo run the code for LLM-MCTS, use\n```\npython mcts/virtualhome/mcts_agent.py \\\n    --exploration_constant 24 \\\n    --max_episode_len 50 \\\n    --max_depth 20 \\\n    --round 0 \\\n    --simulation_per_act 2 \\\n    --simulation_num 100 \\\n    --discount_factor 0.95  \\\n    --uct_type PUCT \\\n    --mode simple \\\n    --seen_item \\\n    --seen_apartment\\\n    --model gpt-3.5-turbo-0125 \\\n    --seen_comp\n```\n\n## Acknowledge\n\nThis repository is built upon a number of prior opensource works. \n* Our data generation and testing settings are adapted from https://github.com/xavierpuigf/watch_and_help. \n* The baseline (fine-tuned GPT2 policy) is adapted from https://github.com/ShuangLI59/Pre-Trained-Language-Models-for-Interactive-Decision-Making. Their training code is available in the supplementary materials at https://openreview.net/forum?id=FWMQYjFso-a. \n* GPT3.5 baseline is adapted from https://github.com/huangwl18/language-planner. \n* Our MCTS implementation is adapted from https://github.com/jys5609/MC-LAVE-RL. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1989Ryan%2Fllm-mcts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1989Ryan%2Fllm-mcts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1989Ryan%2Fllm-mcts/lists"}