{"id":19175884,"url":"https://github.com/astropy/astropy-benchmarks","last_synced_at":"2025-04-15T06:50:33.248Z","repository":{"id":12916717,"uuid":"15594155","full_name":"astropy/astropy-benchmarks","owner":"astropy","description":"Benchmarks for the astropy project","archived":false,"fork":false,"pushed_at":"2025-02-06T15:23:21.000Z","size":113541,"stargazers_count":9,"open_issues_count":13,"forks_count":28,"subscribers_count":26,"default_branch":"main","last_synced_at":"2025-04-12T02:21:59.338Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://spacetelescope.github.io/bench/astropy-benchmarks/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/astropy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"numfocus","custom":"https://numfocus.org/donate-to-astropy"}},"created_at":"2014-01-02T21:25:44.000Z","updated_at":"2025-02-04T17:46:10.000Z","dependencies_parsed_at":"2024-11-06T18:47:04.949Z","dependency_job_id":null,"html_url":"https://github.com/astropy/astropy-benchmarks","commit_stats":{"total_commits":147,"total_committers":22,"mean_commits":6.681818181818182,"dds":0.5918367346938775,"last_synced_commit":"5f6f488c1f8ce1fc2f1930e1d05ce7418db5dfe7"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astropy%2Fastropy-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astropy%2Fastropy-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astropy%2Fastropy-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astropy%2Fastropy-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/astropy","download_url":"https://codeload.github.com/astropy/astropy-benchmarks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249023708,"owners_count":21199958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T10:26:03.413Z","updated_at":"2025-04-15T06:50:33.229Z","avatar_url":"https://github.com/astropy.png","language":"Python","funding_links":["https://github.com/sponsors/numfocus","https://numfocus.org/donate-to-astropy"],"categories":[],"sub_categories":[],"readme":"Astropy performance benchmarks\n==============================\n\nAbout\n-----\n\nThis repository includes a set of benchmarks to regularly test the performance of\nvarious parts of the astropy core package. These benchmarks are run for every\nnew commit in the core package repository and the results are published using a\nuser-friendly [web interface](https://spacetelescope.github.io/bench/astropy-benchmarks/). The\nbenchmarks are run using [airspeed velocity](https://asv.readthedocs.io) on\ndedicated physical machines belonging to members of the Astropy developer community.\n\nRunning the benchmarks locally\n------------------------------\n\nIf you want to try and run the benchmarks locally you will first need to install asv\n\n```shell\npip install asv\n```\n\nthen clone the benchmarks repository\n\n```shell\ngit clone git@github.com:astropy/astropy-benchmarks.git --single-branch\ncd astropy-benchmarks\n```\n\nNote that the `--single-branch` option is to avoid downloading the `results`\nbranch, which is large.\n\nThe easiest/fastest way to try out the benchmarks is to make sure you have\neither a stable or a developer version of astropy installed, then run\n\n```shell\nasv run --python=same --quick --show-stderr --dry-run\n```\n\nThis will run the benchmarks against the local astropy version and will do some\nbasic timing, but the timing will not be very accurate, because this only runs\neach benchmark once instead of taking an average of many runs. Nevertheless,\nthis is the first step to make sure things are running correctly and can still\ngive order of magnitude timings.\n\nTo run asv properly on the latest commit in the upstream astropy main, you can\ndo\n\n```shell\nasv run\n```\n\nThis will set up a temporary environment in which astropy will be installed, and\nthe benchmark functions will be run multiple times and averaged to get accurate\ntimings. The results from this will be saved into a `results/\u003cmachine-name\u003e`\ndirectory, which will store one file per commit (or more than one file if the\ntests are set up to run for multiple python or numpy versions, which they are\nnot configured to do by default for Astropy).\n\nYou can specify the commit(s) to run the benchmarks for by using the same syntax\nas you would for the `git log` command. For example, to run the benchmarks for\na single specific commit, you can do\n\n```shell\nasv run 88fbbc33^!\n```\n\nreplacing 88fbbc33 by the commit you want to test (the `^!` Indicates to just run\nthis commit, not all commits up to that point). If\nyou want to run a range of commits, use\n\n```shell\nasv run 827f322b..729abcf3\n```\n\nFor the best accuracy, if you are using Linux, you can prevent core-swapping\n(which can introduce some noise in the timings) by running `asv` with\n\n```shell\ntaskset -c 0 asv ...\n```\n\nThis will ensure that the whole process runs on the same core.\n\nYou can generate a user-friendly web interface for your results locally by\nrunning\n\n```shell\nasv publish\nasv preview\n```\n\nThe `asv preview` command will give the URL of the local web server (e.g.\n`http://127.0.0.1:21331`) - go to this address in your favorite browser to see\nthe results.\n\nRunning benchmarks with specific versions of dependencies\n---------------------------------------------------------\n\nIf you would like to run the benchmarks against specific version(s) of an\ninstalled dependency (or dependencies), modify `asv.conf.json` to add \n`matrix.req` with the desired version(s):\n\n```diff\n{\n  \"version\": 1,\n  \"project\": \"astropy\",\n  \"project_url\": \"http://www.astropy.org/\",\n  \"repo\": \"https://github.com/astropy/astropy.git\",\n  \"install_command\": [\n    \"pip install . matplotlib scipy\"\n  ],\n+ \"matrix\": {\n+   \"req\": {\n+     \"matplotlib\": [\n+       \"3.5.1\"\n+     ],\n+     \"numpy\": [\n+       \"1.26.0\",\n+       \"2.0.0rc1\"\n+     ]\n+   }\n+ },\n  \"branches\": [\"main\"],\n  \"show_commit_url\": \"http://github.com/astropy/astropy/commit/\",\n  \"pythons\": [\"3.11\"],\n  \"environment_type\": \"virtualenv\"\n}\n```\n\nWriting a benchmark\n-------------------\n\nTo write a new benchmark, fork this repository and take a look inside the\n`benchmarks` folder - each module in astropy that already has benchmarks has a\ncorresponding file or directory with files for different parts of the module.\nEither add your benchmark to one of the existing files or create a new file as\nneeded.\n\nA benchmark is a Python function whose name starts with `time_`. A function\nshould test as little as possible and therefore be as short/simple as possible.\nAny required imports should be done outside the function. Here is an example of\na benchmark to test unit conversion:\n\n```python\nfrom astropy import units as u\n\ndef time_my_benchmark():\n    (u.m / u.s).to(u.km / u.h)\n```\n\nOnce you have added a benchmark, you can make sure it runs by running\n\n```shell\nasv run --python=same --quick --show-stderr --dry-run\n```\n\nAs mentioned in [Running the benchmarks locally], this will run all the\nbenchmarks in fast mode (running each function once).\n\nYou can select just the benchmark you have written using the `--bench` option\n\n```shell\nasv run --python=same --quick --show-stderr --dry-run --bench time_my_benchmark\n```\n\nRunning benchmarks against a local commit\n-----------------------------------------\n\nSay you have a local clone of `astropy`, and you would like to run benchmarks \nagainst your local changes. To do this, edit `asv.conf.json` to switch the \n`repo` entry from the upstream repository to your local repository:\n\n```diff\n{\n  \"version\": 1,\n  \"project\": \"astropy\",\n  \"project_url\": \"http://www.astropy.org/\",\n- \"repo\": \"https://github.com/astropy/astropy.git\",\n+ \"repo\": \"./astropy/\",\n  \"install_command\": [\n    \"pip install . matplotlib scipy\"\n  ],\n  \"branches\": [\"main\"],\n  \"show_commit_url\": \"http://github.com/astropy/astropy/commit/\",\n  \"pythons\": [\"3.11\"],\n  \"environment_type\": \"virtualenv\"\n}\n```\n\nYou will then be able to run the benchmarks against a commit in your local \nrepository, e.g.\n\n```shell\nasv run 827f322b^!\n```\n\nComparing commits\n-----------------\n\nTo compare two commits (e.g. the latest upstream commit and a local commit), \nuse `asv compare`:\n\n```shell\nasv compare 88fbbc33 827f322b\n```\n\nThis will show a table with a comparison of the benchmark times for the two\ncommits:\n\n```diff\nAll benchmarks:\n\n       before           after         ratio\n     [e8f1432a]       [c378f912]\n+        1.17±0ms           1.69ms     1.44  coordinates.FrameBenchmarks.time_init_array\n+       388±0.3μs            543μs     1.40  coordinates.FrameBenchmarks.time_init_nodata\n+         932±2μs           1.34ms     1.44  coordinates.FrameBenchmarks.time_init_scalar\n            1.08s            1.17s     1.08  coordinates.SkyCoordBenchmarks.time_icrs_to_galactic_array\n+     28.6±0.03ms           43.0ms     1.50  coordinates.SkyCoordBenchmarks.time_icrs_to_galactic_scalar\n-        48.2±7ms           37.4ms     0.78  coordinates.SkyCoordBenchmarks.time_init_array\n+     2.84±0.01ms           5.22ms     1.84  coordinates.SkyCoordBenchmarks.time_init_scalar\n+         168±2ms            29.5s   175.69  coordinates.SkyCoordBenchmarks.time_iter_array\n-       118±0.2ms           46.4ms     0.39  coordinates.SkyCoordBenchmarks.time_repr_array\n-         876±1μs            760μs     0.87  coordinates.SkyCoordBenchmarks.time_repr_scalar\n...\n```\n\nContributing benchmarks\n-----------------------\n\nOnce you are happy with your benchmark(s), open a pull request to the\nastropy-benchmarks repository. You do not need to add any result files for the\nbenchmarks - we have machines that automatically do this every night.\n\nNotes to maintainers (outdated)\n-------------------------------\n\nThe `main` branch in this repository should not contain any results or built\nwebsite. Results should be added to the `results` branch, and commits to the\n`results` branch trigger a build to the `gh-pages` branch.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastropy%2Fastropy-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fastropy%2Fastropy-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastropy%2Fastropy-benchmarks/lists"}