{"id":18621338,"url":"https://github.com/4ment/gradient-benchmark","last_synced_at":"2025-04-11T02:31:42.191Z","repository":{"id":40164036,"uuid":"291630068","full_name":"4ment/gradient-benchmark","owner":"4ment","description":"automatic/analytical differentiation benchmark ","archived":false,"fork":false,"pushed_at":"2023-01-24T04:49:10.000Z","size":86,"stargazers_count":3,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T08:23:07.503Z","etag":null,"topics":["bayesian-inference","gradient-computation","phylogenetics","phylostan","pytorch","stan","tensorflow","torchtree","variational"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/4ment.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-31T06:10:17.000Z","updated_at":"2023-06-09T21:19:47.000Z","dependencies_parsed_at":"2023-02-13T17:46:17.596Z","dependency_job_id":null,"html_url":"https://github.com/4ment/gradient-benchmark","commit_stats":null,"previous_names":["4ment/gradient-benchmark"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4ment%2Fgradient-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4ment%2Fgradient-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4ment%2Fgradient-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4ment%2Fgradient-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/4ment","download_url":"https://codeload.github.com/4ment/gradient-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248329612,"owners_count":21085568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-inference","gradient-computation","phylogenetics","phylostan","pytorch","stan","tensorflow","torchtree","variational"],"created_at":"2024-11-07T04:10:51.823Z","updated_at":"2025-04-11T02:31:42.162Z","avatar_url":"https://github.com/4ment.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# autodiff-experiments\n\n[![Docker Image CI](https://github.com/4ment/autodiff-experiments/actions/workflows/docker-image.yml/badge.svg)](https://github.com/4ment/autodiff-experiments/actions/workflows/docker-image.yml)\n\nThis repository contains the pipeline and data sets supporting the results of the following article:\n\nFourment M, Swanepoel CJ, Galloway JG, Ji X, Gangavarapu K, Suchard MA, Matsen IV FA. Automatic differentiation is no panacea for phylogenetic gradient computation. [arXiv:2211.02168](https://arxiv.org/abs/2211.02168)\n\nThis benchmark compares the efficiency (memory usage and speed) of several gradient implementations of phylogenetic models (e.g., tree likelikelihood and coalescent model). The goal of this study is to compare the efficiency of automatic differentiation (AD) and analytic gradient. The pipeline reuses parts of the treetime validation [workflow](https://github.com/neherlab/treetime_validation).\n\n\n| Program      | Language  | Framework    | Gradient | [BITO] support |\n| ------------ | --------- | ------------ | :-------:| :-----:|\n| [physher]    | C         |              | analytic |     |\n| [phylostan]  | Stan      | [Stan]       | AD       |     |\n| [phylojax]   | python    | [JAX]        | AD       |     |\n| [torchtree]  | python    | [PyTorch]    | AD       | :white_check_mark: |\n| [treeflow]   | python    | [TensorFlow] | AD       |  |\n\nThe gradient of the tree likelihood is optionaly computed by [BITO], an efficient C++ library that analytically calculate\nthe gradient using the [BEAGLE] library. torchtree uses the [torchtree-bito] plugin to access BITO.\n\n## Dependencies\nYou will need to install [nextflow](https://www.nextflow.io) and [docker](https://www.docker.com) to run this benchmark.\nDocker is not required but it is highly recommended to use it due to the numerous dependencies.\n\n## Installation\n\n    git clone 4ment/autodiff-experiments.git\n\n### Initialize treetime_validation\n\n    git submodule update --init --recursive\n\n## Running the pipeline with docker\n\n    nextflow run 4ment/autodiff-experiments -profile docker -with-trace\n\nSince the pipeline will take weeks to run to completion one should use a high performance computer. Examples of configuration files for pbspro and slurm can be found in the [configs](configs/) folder.\n\n## Summarizing results\n\nBefore generating the figures, we need to extract memory usage information from the `trace.txt` file and `work` directory:\n\n    python scripts/parse-trace.py work/ trace.txt \u003e results/trace.csv\n\nGenerate figures in a single pdf:\n\n    Rscript -e 'rmarkdown::render(\"plot.Rmd\")'\n\n## Library versions\n\nFor reproducbility, we provide below the version or commit hash of each library/program used in the benchmark.\n\n| Library      | Version  |\n| ------------ | -------- |\n| [jax]                  | 0.2.24   |\n| jaxlib                 | 0.3.7    |\n| numpy                  | 1.22     |\n| [pystan]               | 2.19.1.1 |\n| [tensorflow]           | 2.10.0   |\n| tensorflow-probability | 0.18.0   |\n| [pytorch]              | 1.12.1   |\n\n | Program | Version/hash |\n | ------------ | -------- |\n | [bito] |         cc0806abcd0b9f2fab604e800c674c9a5c5afebe |\n | [phylojax]       | a1612cae36292af76e8d24cc40d6544162c987aa |\n | [phylostan]      | 1.0.5 |\n | [physher]        | b19ff2f9422f29ba1ab31306a3fe29ab6a6f607b |\n | [torchtree]      | f3831650a807e74cc2e9478009e57a41f47bed8d |\n | [torchtree-bito] | e2a95cefb13968f95f6e5520bd0a52d726ee7fc9 |\n | [treeflow]       | e3414dcc9e764d06abc3e19c1d0f55110499e2ea |\n \n\n\n[physher]: https://github.com/4ment/physher\n[phylostan]: https://github.com/4ment/phylostan\n[phylojax]: https://github.com/4ment/phylojax\n[torchtree]: https://github.com/4ment/torchtree\n[torchtree-bito]: https://github.com/4ment/torchtree-bito\n[treeflow]: https://github.com/christiaanjs/treeflow\n\n[BITO]: https://github.com/phylovi/bito\n[BEAGLE]: https://github.com/beagle-dev/beagle-lib\n\n[Stan]: https://mc-stan.org\n[JAX]: https://github.com/google/jax\n[PyTorch]: https://pytorch.org\n[TensorFlow]: https://www.tensorflow.org\n[pystan]: https://github.com/stan-dev/pystan2","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ment%2Fgradient-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F4ment%2Fgradient-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ment%2Fgradient-benchmark/lists"}