{"id":13429921,"url":"https://github.com/google-deepmind/alphageometry","last_synced_at":"2025-05-14T15:06:24.608Z","repository":{"id":217586858,"uuid":"702719235","full_name":"google-deepmind/alphageometry","owner":"google-deepmind","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-25T23:44:29.000Z","size":180,"stargazers_count":4474,"open_issues_count":126,"forks_count":508,"subscribers_count":57,"default_branch":"main","last_synced_at":"2025-04-29T13:12:42.198Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-09T21:47:43.000Z","updated_at":"2025-04-28T23:19:01.000Z","dependencies_parsed_at":"2025-01-14T22:10:52.538Z","dependency_job_id":"195e707b-bb2d-4d23-9885-c98a5f0e7dd6","html_url":"https://github.com/google-deepmind/alphageometry","commit_stats":null,"previous_names":["google-deepmind/alphageometry"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Falphageometry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Falphageometry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Falphageometry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Falphageometry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/alphageometry/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254169047,"owners_count":22026207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T02:00:47.832Z","updated_at":"2025-05-14T15:06:24.586Z","avatar_url":"https://github.com/google-deepmind.png","language":"Python","funding_links":[],"categories":["3 Reasoning Tasks","🤖 Research Agents \u0026 Autonomous Workflows","Python","Repos"],"sub_categories":["3.2 Mathematical Reasoning","Domain-Specific Research Agents"],"readme":"\n# Solving Olympiad Geometry without Human Demonstrations\n\n\nThis repository contains the code necessary to\nreproduce DDAR and AlphaGeometry,\nthe two geometry theorem provers\nintroduced in the [Nature 2024](https://www.nature.com/articles/s41586-023-06747-5) paper:\n\n*\u003ccenter\u003e\"Solving Olympiad Geometry without Human Demonstrations\".\u003c/center\u003e*\n\n\n\u003c/br\u003e\n\n\n\u003ccenter\u003e\n\u003cimg alt=\"fig1\" width=\"800px\" src=\"fig1.svg\"\u003e\n\u003c/center\u003e\n\n\n## Dependencies\n\nFor the instructions presented below,\nwe use Python 3.10.9, and dependencies with their exact\nversion numbers listed in `requirements.txt`.\n\nOur code depends on `meliad`, which is\nnot a registered package with `pip`. See instructions below\nfor how to manually install `meliad`.\n\nNote that one can still run the DDAR solver\nwithout the `meliad` and `sentencepiece` dependencies.\n\n## Run the instructions\n\nAll instructions in this `README.md` can be run in one go by:\n\n```\nbash run.sh\n```\n\nBelow, we explain these instructions step-by-step.\n\n## Install dependencies, download weights and vocabulary.\n\nInstallation is done in a virtual environment:\n\n```\nvirtualenv -p python3 .\nsource ./bin/activate\npip install --require-hashes -r requirements.txt\n```\n\nDownload weights and vocabulary:\n\n```\nbash download.sh\nDATA=ag_ckpt_vocab\n```\n\nFinally, install `meliad` separately as it is not\nregistered with `pip`:\n\n```\nMELIAD_PATH=meliad_lib/meliad\nmkdir -p $MELIAD_PATH\ngit clone https://github.com/google-research/meliad $MELIAD_PATH\nexport PYTHONPATH=$PYTHONPATH:$MELIAD_PATH\n```\n\n## Set up common flags\n\nBefore running the python scripts,\nlet us first prepare some commonly used flags.\nThe symbolic engine needs definitions and deduction rules to operate.\nThese definitions and rules are provided in two text files\n`defs.txt` and `rules.txt`.\n\n```shell\nDDAR_ARGS=(\n  --defs_file=$(pwd)/defs.txt \\\n  --rules_file=$(pwd)/rules.txt \\\n);\n```\n\nNext, we define the flags relevant to the proof search.\nTo reproduce the simple examples below,\nwe use lightweight values for the proof search parameters:\n\n```shell\nBATCH_SIZE=2\nBEAM_SIZE=2\nDEPTH=2\n\nSEARCH_ARGS=(\n  --beam_size=$BEAM_SIZE\n  --search_depth=$DEPTH\n)\n```\n\nNOTE: The results in our paper can be obtained by setting\n`BATCH_SIZE=32`, `BEAM_SIZE=512`, `DEPTH=16`\nas described in section Methods.\nTo stay under IMO time limits, 4 V100-GPUs and 250 CPU workers\nare needed as shown in Extended Data - Figure 1.\nNote that we also strip away other memory/speed optimizations\ndue to internal dependencies and to promote code clarity.\n\nAssume the downloaded checkpoint and vocabulary is placed in `DATA`,\nand the installed `meliad` source code is at `MELIAD_PATH`.\nWe make use of the `gin` library to manage model configurations,\nfollowing `meliad` conventions. We now define the flags relevant to the\nlanguage model:\n\n```shell\nLM_ARGS=(\n  --ckpt_path=$DATA \\\n  --vocab_path=$DATA/geometry.757.model\n  --gin_search_paths=$MELIAD_PATH/transformer/configs,$(pwd) \\\n  --gin_file=base_htrans.gin \\\n  --gin_file=size/medium_150M.gin \\\n  --gin_file=options/positions_t5.gin \\\n  --gin_file=options/lr_cosine_decay.gin \\\n  --gin_file=options/seq_1024_nocache.gin \\\n  --gin_file=geometry_150M_generate.gin \\\n  --gin_param=DecoderOnlyLanguageModelGenerate.output_token_losses=True \\\n  --gin_param=TransformerTaskConfig.batch_size=$BATCH_SIZE \\\n  --gin_param=TransformerTaskConfig.sequence_length=128 \\\n  --gin_param=Trainer.restore_state_variables=False\n);\n```\n\nTIP: Note that you can still run the DDAR solver\nwithout defining `SEARCH_ARGS` and `LM_ARGS`.\nIn such case, simply disable the import of the `lm_inference` module\ninside `alphageometry.py`.\n\n## Run DDAR\n\nThe script loads a problem by reading a list of problems\nfrom a text file and solves the specific problem in the list according\nto its name. We pass these two pieces of information through the flags\n`--problems_file` and `--problem_name`.\nWe use `--mode=ddar` to indicate that we want to use the DDAR solver.\n\nBelow we showed this solver solving IMO 2000 P1:\n\n```shell\npython -m alphageometry \\\n--alsologtostderr \\\n--problems_file=$(pwd)/imo_ag_30.txt \\\n--problem_name=translated_imo_2000_p1 \\\n--mode=ddar \\\n\"${DDAR_ARGS[@]}\"\n```\n\nExpect the following output\n\n```shell\ngraph.py:468] translated_imo_2000_p1\ngraph.py:469] a b = segment a b; g1 = on_tline g1 a a b; g2 = on_tline g2 b b a; m = on_circle m g1 a, on_circle m g2 b; n = on_circle n g1 a, on_circle n g2 b; c = on_pline c m a b, on_circle c g1 a; d = on_pline d m a b, on_circle d g2 b; e = on_line e a c, on_line e b d; p = on_line p a n, on_line p c d; q = on_line q b n, on_line q c d ? cong e p e q\nddar.py:41] Depth 1/1000 time = 1.7772269248962402\nddar.py:41] Depth 2/1000 time = 5.63526177406311\nddar.py:41] Depth 3/1000 time = 6.883412837982178\nddar.py:41] Depth 4/1000 time = 10.275688409805298\nddar.py:41] Depth 5/1000 time = 12.048273086547852\nalphageometry.py:190]\n==========================\n * From theorem premises:\nA B G1 G2 M N C D E P Q : Points\nAG_1 ⟂ AB [00]\nBA ⟂ G_2B [01]\nG_2M = G_2B [02]\nG_1M = G_1A [03]\n\n...\n[log omitted]\n...\n\n036. ∠QEB = ∠(QP-EA) [46] \u0026 ∠(BE-QP) = ∠AEP [55] ⇒  ∠EQP = ∠QPE [56]\n037. ∠PQE = ∠EPQ [56] ⇒  EP = EQ\n\n==========================\n```\n\nThe output first includes a list of relevant premises that it uses,\nand then proof steps that gradually build up the proof.\nAll predicates are numbered to track how they are derived\nfrom the premises, and to show that the proof is fully justified.\n\nTIP: Additionally passing the flag `--out_file=path/to/output/text/file.txt`\nwill write the proof to a text file.\n\nRunning on all problems in `imo_ag_30.txt` will yield solutions to\n14 of them, as reported in Table 1 in our paper.\n\n## Run AlphaGeometry:\n\nAs a simple example, we load `--problem_name=orthocenter`\nfrom `--problem_file=examples.txt`.\nThis time, we pass `--mode=alphageometry` to use the AlphaGeometry solver\nand pass the `SEARCH_ARGS` and `LM_ARGS` flags.\n\n```shell\npython -m alphageometry \\\n--alsologtostderr \\\n--problems_file=$(pwd)/examples.txt \\\n--problem_name=orthocenter \\\n--mode=alphageometry \\\n\"${DDAR_ARGS[@]}\" \\\n\"${SEARCH_ARGS[@]}\" \\\n\"${LM_ARGS[@]}\"\n```\n\nExpect the following output:\n\n```shell\n...\n[log omitted]\n...\ntraining_loop.py:725] Total parameters: 152072288\ntraining_loop.py:739] Total state size: 0\ntraining_loop.py:492] Training loop: creating task for mode beam_search\n\ngraph.py:468] orthocenter\ngraph.py:469] a b c = triangle a b c; d = on_tline d b a c, on_tline d c a b ? perp a d b c\nddar.py:41] Depth 1/1000 time = 0.009987592697143555 branch = 4\nddar.py:41] Depth 2/1000 time = 0.00672602653503418 branch = 0\nalphageometry.py:221] DD+AR failed to solve the problem.\nalphageometry.py:457] Depth 0. There are 1 nodes to expand:\nalphageometry.py:460] {S} a : ; b : ; c : ; d : T a b c d 00 T a c b d 01 ? T a d b c {F1} x00\nalphageometry.py:465] Decoding from {S} a : ; b : ; c : ; d : T a b c d 00 T a c b d 01 ? T a d b c {F1} x00\n...\n[log omitted]\n...\nalphageometry.py:470] LM output (score=-1.102287): \"e : C a c e 02 C b d e 03 ;\"\nalphageometry.py:471] Translation: \"e = on_line e a c, on_line e b d\"\n\nalphageometry.py:480] Solving: \"a b c = triangle a b c; d = on_tline d b a c, on_tline d c a b; e = on_line e a c, on_line e b d ? perp a d b c\"\ngraph.py:468]\ngraph.py:469] a b c = triangle a b c; d = on_tline d b a c, on_tline d c a b; e = on_line e a c, on_line e b d ? perp a d b c\nddar.py:41] Depth 1/1000 time = 0.021120786666870117\nddar.py:41] Depth 2/1000 time = 0.033370018005371094\nddar.py:41] Depth 3/1000 time = 0.04297471046447754\nalphageometry.py:140]\n==========================\n * From theorem premises:\nA B C D : Points\nBD ⟂ AC [00]\nCD ⟂ AB [01]\n\n * Auxiliary Constructions:\nE : Points\nE,B,D are collinear [02]\nE,C,A are collinear [03]\n\n * Proof steps:\n001. E,B,D are collinear [02] \u0026 E,C,A are collinear [03] \u0026 BD ⟂ AC [00] ⇒  ∠BEA = ∠CED [04]\n002. E,B,D are collinear [02] \u0026 E,C,A are collinear [03] \u0026 BD ⟂ AC [00] ⇒  ∠BEC = ∠AED [05]\n003. A,E,C are collinear [03] \u0026 E,B,D are collinear [02] \u0026 AC ⟂ BD [00] ⇒  EC ⟂ EB [06]\n004. EC ⟂ EB [06] \u0026 CD ⟂ AB [01] ⇒  ∠(EC-BA) = ∠(EB-CD) [07]\n005. E,C,A are collinear [03] \u0026 E,B,D are collinear [02] \u0026 ∠(EC-BA) = ∠(EB-CD) [07] ⇒  ∠BAE = ∠CDE [08]\n006. ∠BEA = ∠CED [04] \u0026 ∠BAE = ∠CDE [08] (Similar Triangles)⇒  EB:EC = EA:ED [09]\n007. EB:EC = EA:ED [09] \u0026 ∠BEC = ∠AED [05] (Similar Triangles)⇒  ∠BCE = ∠ADE [10]\n008. EB:EC = EA:ED [09] \u0026 ∠BEC = ∠AED [05] (Similar Triangles)⇒  ∠EBC = ∠EAD [11]\n009. ∠BCE = ∠ADE [10] \u0026 E,C,A are collinear [03] \u0026 E,B,D are collinear [02] \u0026 ∠EBC = ∠EAD [11] ⇒  AD ⟂ BC\n==========================\n\nalphageometry.py:505] Solved.\n```\n\nNOTE: Point `H` is automatically renamed to `D`,\nas the LM is trained on synthetic problems\nwhere the points are named alphabetically, and so it expects\nthe same during test time.\n\nNOTE: In this implementation of AlphaGeometry,\nwe removed all optimizations that are dependent on\ninternal infrastructure, e.g.,\nparallelized model inference on multi GPUs,\nparallelized DDAR on multiple CPUs,\nparallel execution of LM and DDAR,\nshared pool of CPU workers across different problems, etc.\nWe also removed some memory/speed optimizations and code\nabstractions in favor of code clarity.\n\nAs can be seen in the output, initially DDAR failed to solve the problem.\nThe LM proposes two auxiliary constructions (because `BATCH_SIZE=2`):\n\n* `e = eqdistance e c a b, eqdistance e b a c`, i.e.,\nconstruct `E` as the intersection of circle (center=C, radius=AB) and\ncircle (center=B, radius=AC). This construction has a score of `-1.186`.\n* `e = on_line e a c, on_line e b d`, i.e.,\n`E` is the intersection of `AC` and `BD`.\nThis construction has a higher score (`-1.102287`) than the previous.\n\nSince the second construction has a higher score, DDAR attempted the second\nconstruction first and found the solution right away.\nThe proof search therefore terminates and there is no second iteration.\n\n## Results\n\nBefore attempting to reproduce the AlphaGeometry numbers in our paper,\nplease make sure to pass all tests in the prepared test suite:\n\n```\nbash run_tests.sh\n```\n\nNOTE: [Issues#14](https://github.com/google-deepmind/alphageometry/issues/14) reports that although the top beam decodes are still the same, the LM is not giving the same score for different users.\n\nThen, pass the corresponding values for `--problem_file` (column)\nand `--mode` (row), and\niterate on all problems to obtain the following results:\n\n\u003ccenter\u003e\n\n\u003cb\u003eNumber of solved problems:\u003c/b\u003e\n\n|          | `imo_ag_30.txt`  | `jgex_ag_231.txt` |\n|----------|------------------|-------------------|\n| `ddar`   | 14               | 198               |\n| `alphageometry`     | 25               | 228               |\n\n\u003c/center\u003e\n\n## Source code description\n\nFiles in this repository include python modules/scripts to run the solvers and\nresource files necessary for the script to execute. We listed below\neach of them and their description.\n\n| File name              | Description                                                                        |\n|------------------------|------------------------------------------------------------------------------------|\n| `geometry.py`          | Implements nodes (Point, Line, Circle, etc) in the proof state graph.              |\n| `numericals.py`        | Implements the numerical engine in the dynamic geometry environment.               |\n| `graph_utils.py`       | Implements utilities for the proof state graph.                                    |\n| `graph.py`             | Implements the proof state graph.                                                  |\n| `problem.py`           | Implements the classes that represent the problem premises, conclusion, DAG nodes. |\n| `dd.py`                | Implements DD and its traceback.                                                   |\n| `ar.py`                | Implements AR and its traceback.                                                   |\n| `trace_back.py`        | Implements the recursive traceback and dependency difference algorithm.            |\n| `ddar.py`              | Implements the combination DD+AR.                                                  |\n| `beam_search.py`       | Implements beam decoding of a language model in JAX.                               |\n| `models.py`            | Implements the transformer model.                                                  |\n| `transformer_layer.py` | Implements the transformer layer.                                                  |\n| `decoder_stack.py`     | Implements the transformer decoder stack.                                          |\n| `lm_inference.py`      | Implements an interface to a trained LM to perform decoding.                       |\n| `alphageometry.py`                | Main script that loads problems, calls DD+AR or AlphaGeometry solver, and prints solutions.   |\n| `pretty.py`            | Pretty formating the solutions output by solvers.                                  |\n| `*_test.py`            | Tests for the corresponding module.                                                |\n| `download.sh`          | Script to download model checkpoints and LM                                        |\n| `run.sh`               | Script to execute instructions in README.                                          |\n| `run_tests.sh`         | Script to execute the test suite.                                                  |\n\n\nResource files:\n\n| Resource file name     | Description                                                                        |\n|------------------------|------------------------------------------------------------------------------------|\n| `defs.txt`             | Definitions of different geometric construction actions.                           |\n| `rules.txt`            | Deduction rules for DD.                                                            |\n| `geometry_150M_generate.gin`| Gin config of the LM implemented in meliad.                                   |\n| `imo_ag_30.txt`        | Problems in IMO-AG-30.                                                             |\n| `jgex_ag_231.txt`      | Problems in JGEX-AG-231.                                                           |\n\n\n\n## Citing this work\n\n```bibtex\n@Article{AlphaGeometryTrinh2024,\n  author  = {Trinh, Trieu and Wu, Yuhuai and Le, Quoc and He, He and Luong, Thang},\n  journal = {Nature},\n  title   = {Solving Olympiad Geometry without Human Demonstrations},\n  year    = {2024},\n  doi     = {10.1038/s41586-023-06747-5}\n}\n```\n\n## Acknowledgements\n\nThis research is a collaboration between the Google Brain team\n(now Google Deepmind) and\nthe Computer Science Department of New York University.\nWe thank Rif A. Saurous, Denny Zhou, Christian Szegedy, Delesley Hutchins,\nThomas Kipf, Hieu Pham, Petar Veličković, Debidatta Dwibedi,\nKyunghyun Cho, Lerrel Pinto, Alfredo Canziani,\nThomas Wies, He He’s research group,\nEvan Chen (the USA’s IMO team coach),\nMirek Olsak, Patrik Bak,\nand all three Nature's referees for their help and support.\n\nThe code of AlphaGeometry communicates with and/or references the following\nseparate libraries and packages:\n\n*   [Abseil](https://github.com/abseil/abseil-py)\n*   [JAX](https://github.com/google/jax/)\n*   [matplotlib](https://matplotlib.org/)\n*   [NumPy](https://numpy.org)\n*   [SciPy](https://scipy.org)\n*   [TensorFlow](https://github.com/tensorflow/tensorflow)\n*   [Meliad](https://github.com/google-research/meliad)\n*   [Flax](https://github.com/google/flax)\n*   [Gin](https://github.com/google/gin-config)\n*   [T5](https://github.com/google-research/text-to-text-transfer-transformer)\n*   [SentencePiece](https://github.com/google/sentencepiece)\n\n\n\nWe thank all their contributors and maintainers!\n\n\n## Disclaimer\n\nThis is not an officially supported Google product.\n\nThis research code is provided \"as-is\" to the broader research community.\nGoogle does not promise to maintain or otherwise support this code in any way.\n\n## Code License\n\nCopyright 2023 DeepMind Technologies Limited\n\nAll software is licensed under the Apache License, Version 2.0 (Apache 2.0);\nyou may not use this file except in compliance with the Apache 2.0 license.\nYou may obtain a copy of the Apache 2.0 license at:\nhttps://www.apache.org/licenses/LICENSE-2.0\n\nAll other materials are licensed under the Creative Commons Attribution 4.0\nInternational License (CC-BY). You may obtain a copy of the CC-BY license at:\nhttps://creativecommons.org/licenses/by/4.0/legalcode\n\nUnless required by applicable law or agreed to in writing, all software and\nmaterials distributed here under the Apache 2.0 or CC-BY licenses are\ndistributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,\neither express or implied. See the licenses for the specific language governing\npermissions and limitations under those licenses.\n\n## Model Parameters License\n\nThe AlphaGeometry checkpoints and vocabulary are made available\nunder the terms of the Creative Commons Attribution 4.0\nInternational (CC BY 4.0) license.\nYou can find details at:\nhttps://creativecommons.org/licenses/by/4.0/legalcode\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Falphageometry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Falphageometry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Falphageometry/lists"}