{"id":20298167,"url":"https://github.com/trotsky1997/MathBlackBox","last_synced_at":"2025-05-07T20:34:33.389Z","repository":{"id":244188167,"uuid":"814499551","full_name":"trotsky1997/MathBlackBox","owner":"trotsky1997","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-17T12:37:59.000Z","size":40,"stargazers_count":978,"open_issues_count":0,"forks_count":99,"subscribers_count":26,"default_branch":"main","last_synced_at":"2024-12-17T13:30:26.601Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/trotsky1997.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-13T06:21:58.000Z","updated_at":"2024-12-17T12:38:03.000Z","dependencies_parsed_at":"2024-06-13T10:24:42.137Z","dependency_job_id":"81aadae8-86a6-454a-be39-b6e3b08d9885","html_url":"https://github.com/trotsky1997/MathBlackBox","commit_stats":null,"previous_names":["trotsky1997/mathblackbox"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trotsky1997%2FMathBlackBox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trotsky1997%2FMathBlackBox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trotsky1997%2FMathBlackBox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trotsky1997%2FMathBlackBox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/trotsky1997","download_url":"https://codeload.github.com/trotsky1997/MathBlackBox/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252953716,"owners_count":21830890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T16:02:21.260Z","updated_at":"2025-05-07T20:34:33.372Z","avatar_url":"https://github.com/trotsky1997.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"# MCTSr: Mathematic as a Blackbox for LLM\n## News\n- 2024-11-4\n  LLaMA-O1 is Opensource Now! Call for Contributors!\n  https://github.com/SimpleBerry/LLaMA-O1\nhttps://huggingface.co/datasets/qq8933/OpenLongCoT-Pretrain\nhttps://huggingface.co/datasets/qq8933/OpenLongCoT-SFT\n\n- 2024-10-12\n  🚀 Exciting News! At the end of October, we’re announcing the next phase of our work on the open-source reimplementation of OpenAI O1, codenamed TiC!\n\n- 2024-10-11\nNew Preprint! \nLLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning\nhttps://arxiv.org/abs/2410.02884\n\n- 2024-09-30\n🚀 Exciting News! At the end of September, we're releasing an upgraded version of the MCTSr algorithm, LLaMA-Berry, as an open-source fork of the OpenAI Strawberry project. This enhanced tool specializes in tackling the most challenging Mathematical Olympiad tasks\n\n\n## Envoirment\n\n### Server\nWe need VLLM or other Openai compatible method.\n```\npip install vllm\n```\n### Clients\nWe need Huggingface toolkit and Openai for inference.\n```\npip install datasets transformers openai\n```\n\n## Usage\n\nThe script relies on Slurm, If you run it on non-slurm environments,\n\nJust use VLLM to create a openai compatible server, and insert to 'server.csv'\n\n```\nIP,PORT,MODEL_NAME\n```\n\nIf you run it on slurm environment, change the `partition name` to your own partition in `make_n_server.py`\n\nthen, you can run the `run_with_earlystopping.py` for datasets.\n\n```\npython run_with_earlystopping.py MODEL_NAME DATA_DIR_NAME\n```\n\n### Support Datasets\n\ndatasets were given by the first part of `DATA_DIR_NAME` arguments, like ` gsm8k-llama3-8b-new-mcts-8` for `gsm8k` , can selected in,\n\n```\n        'gsm8k-llama3-8b-new-mcts-8',\n         'gsmhard-llama3-8b-new-mcts-8',\n         'olympiadbench-llama3-8b-new-mcts-8',\n         'GAIC-llama3-8b-new-mcts-8',\n         'MATH-llama3-8b-new-mcts-8',\n         'AIME-llama3-8b-mcts-2'\n```\n\nUsing `run_olympics.py` to run all of them.\n\n`check` function was used to early stopping the search process if ground truth answer was reached.\n\nIf you have enough computing resource, you can comment lines including `check` function to disable early stopping.\n\nAlert: That would consume a long time.\n\n## Citation\n```\n@misc{zhang2024accessing,\n      title={Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B}, \n      author={Di Zhang and Xiaoshui Huang and Dongzhan Zhou and Yuqiang Li and Wanli Ouyang},\n      year={2024},\n      eprint={2406.07394},\n      archivePrefix={arXiv},\n      primaryClass={id='cs.AI' full_name='Artificial Intelligence' is_active=True alt_name=None in_archive='cs' is_general=False description='Covers all areas of AI except Vision, Robotics, Machine Learning, Multiagent Systems, and Computation and Language (Natural Language Processing), which have separate subject areas. In particular, includes Expert Systems, Theorem Proving (although this may overlap with Logic in Computer Science), Knowledge Representation, Planning, and Uncertainty in AI. Roughly includes material in ACM Subject Classes I.2.0, I.2.1, I.2.3, I.2.4, I.2.8, and I.2.11.'}\n}\n\n@misc{zhang2024llamaberrypairwiseoptimizationo1like,\n      title={LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning}, \n      author={Di Zhang and Jianbo Wu and Jingdi Lei and Tong Che and Jiatong Li and Tong Xie and Xiaoshui Huang and Shufei Zhang and Marco Pavone and Yuqiang Li and Wanli Ouyang and Dongzhan Zhou},\n      year={2024},\n      eprint={2410.02884},\n      archivePrefix={arXiv},\n      primaryClass={cs.AI},\n      url={https://arxiv.org/abs/2410.02884}, \n}\n```\n\n\n## Disclaimer\n\nThis project was still in a very early stage for explore, pay attentions for the algorithm's output, and do not deploying it to real-world product without fully test.\n\nThis repository was for personal experimentation only and has no connection with any third-party organization or institution.\n\n\n## Read More\n\nhttps://arxiv.org/abs/2406.07394\n\n## Re-implementations\n\nhttps://github.com/BrendanGraham14/mcts-llm\n\n[Jupyter Notebook](https://github.com/trotsky1997/MathBlackBox/issues/2)\n\n[GPT-4O Results](https://github.com/SidU/MathBlackBox)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrotsky1997%2FMathBlackBox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrotsky1997%2FMathBlackBox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrotsky1997%2FMathBlackBox/lists"}