{"id":26518757,"url":"https://github.com/P403n1x87/pytest-austin","last_synced_at":"2025-03-21T10:02:12.934Z","repository":{"id":62583985,"uuid":"283814492","full_name":"P403n1x87/pytest-austin","owner":"P403n1x87","description":"Python Performance Testing with Austin","archived":false,"fork":false,"pushed_at":"2023-11-18T14:27:29.000Z","size":238,"stargazers_count":119,"open_issues_count":2,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-18T13:47:28.258Z","etag":null,"topics":["hacktoberfest","performance","profiling","python","testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/P403n1x87.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"p403n1x87","patreon":"P403n1x87","custom":"https://www.buymeacoffee.com/Q9C1Hnm28"}},"created_at":"2020-07-30T15:42:39.000Z","updated_at":"2025-02-05T15:25:42.000Z","dependencies_parsed_at":"2022-11-03T21:50:59.092Z","dependency_job_id":null,"html_url":"https://github.com/P403n1x87/pytest-austin","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P403n1x87%2Fpytest-austin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P403n1x87%2Fpytest-austin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P403n1x87%2Fpytest-austin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/P403n1x87%2Fpytest-austin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/P403n1x87","download_url":"https://codeload.github.com/P403n1x87/pytest-austin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244776322,"owners_count":20508506,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","performance","profiling","python","testing"],"created_at":"2025-03-21T10:02:11.038Z","updated_at":"2025-03-21T10:02:12.901Z","avatar_url":"https://github.com/P403n1x87.png","language":"Python","funding_links":["https://github.com/sponsors/p403n1x87","https://patreon.com/P403n1x87","https://www.buymeacoffee.com/Q9C1Hnm28","https://www.paypal.me/gtornetta/1"],"categories":["Python"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cbr\u003e\u003cimg src=\"art/logo.png\" alt=\"pytest-austin\" /\u003e\u003cbr\u003e\n\u003c/p\u003e\n\n\u003ch3 align=\"center\"\u003ePython Performance Testing with Austin\u003c/h3\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/3/3a/Tux_Mono.svg\"\n       height=\"24px\" /\u003e\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/f/fa/Apple_logo_black.svg\"\n       height=\"24px\" /\u003e\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/2/2b/Windows_logo_2012-Black.svg\"\n       height=\"24px\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/P403n1x87/pytest-austin/actions?workflow=Tests\"\u003e\n    \u003cimg src=\"https://github.com/P403n1x87/pytest-austin/workflows/Tests/badge.svg\"\n         alt=\"GitHub Actions: Tests\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://travis-ci.com/P403n1x87/pytest-austin\"\u003e\n    \u003cimg src=\"https://travis-ci.com/P403n1x87/pytest-austin.svg?token=fzW2yzQyjwys4tWf9anS\"\n         alt=\"Travis CI\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/P403n1x87/pytest-austin\"\u003e\n    \u003cimg src=\"https://codecov.io/gh/P403n1x87/pytest-austin/branch/master/graph/badge.svg\"\n         alt=\"Codecov\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/pytest-austin/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/pytest-austin.svg\"\n         alt=\"PyPI\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/P403n1x87/pytest-austin/blob/master/LICENSE.md\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/license-GPLv3-ff69b4.svg\"\n         alt=\"LICENSE\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#synopsis\"\u003e\u003cb\u003eSynopsis\u003c/b\u003e\u003c/a\u003e\u0026nbsp;\u0026bull;\n  \u003ca href=\"#installation\"\u003e\u003cb\u003eInstallation\u003c/b\u003e\u003c/a\u003e\u0026nbsp;\u0026bull;\n  \u003ca href=\"#usage\"\u003e\u003cb\u003eUsage\u003c/b\u003e\u003c/a\u003e\u0026nbsp;\u0026bull;\n  \u003ca href=\"#compatibility\"\u003e\u003cb\u003eCompatibility\u003c/b\u003e\u003c/a\u003e\u0026nbsp;\u0026bull;\n  \u003ca href=\"#contribute\"\u003e\u003cb\u003eContribute\u003c/b\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca\n    href=\"https://www.buymeacoffee.com/Q9C1Hnm28\"\n    target=\"_blank\"\u003e\n  \u003cimg\n    src=\"https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png\"\n    alt=\"Buy Me A Coffee\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n# Synopsis\n\nThe pytest-austin plugin for [pytest](https://docs.pytest.org/en/stable/) brings\nPython performance testing right into your CI pipelines. It uses\n[Austin](https://github.com/P403n1x87/austin) to profile your test runs without\nany instrumentation. All you have to do is simply mark the tests on which you\nwant to execute checks for preventing performance regressions.\n\n~~~ Python\nimport pytest\n\n\n@pytest.mark.total_time(td(milliseconds=50), function=\"fibonacci\")\n@pytest.mark.total_time(\"99%\", line=8)\n@pytest.mark.total_time(\"50.3141592653%\", line=9)\ndef test_hello_default():\n    fibonacci(27)\n    fibonacci(25)\n~~~\n\nAny failed tests will be reported by pytest at the end of the session. All the\ncollected statistics are written on a file prefixed with `austin_` and followed\nby a truncated timestamp, inside pytest rootdir. You can drop it onto\n[Speedscope](https://speedscope.app) for a quick visual representation of your\ntests.\n\n~~~\n================================================================ test session starts ================================================================\nplatform linux -- Python 3.6.9, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /home/gabriele/.cache/pypoetry/virtualenvs/pytest-austin-yu27Ep_e-py3.6/bin/python3.6\ncachedir: .pytest_cache\nrootdir: /tmp/pytest-of-gabriele/pytest-226/test_austin_time_checks0\nplugins: cov-2.10.0, austin-0.1.0\ncollecting ... collected 3 items\n\ntest_austin_time_checks.py::test_lines PASSED\ntest_austin_time_checks.py::test_check_fails PASSED\ntest_austin_time_checks.py::test_check_succeeds PASSED\n\n=================================================================== Austin report ===================================================================\naustin 2.0.0\nCollected stats written on /tmp/pytest-of-gabriele/pytest-226/test_austin_time_checks0/.austin_97148135487643.aprof\n\n🕑 Sampling time (min/avg/max) : 376/3327/18019 μs\n🐢 Long sampling rate : 87/87 (100.00 %) samples took longer than the sampling interval\n💀 Error rate : 0/87 (0.00 %) invalid samples\n\ntest_austin_time_checks.py::test_lines test_lines:19 (test_austin_time_checks.py) -16.0 ms (-78.2% of 20.5 ms)\ntest_austin_time_checks.py::test_lines test_lines:18 (test_austin_time_checks.py) -4.1 ms (-10.1% of 40.3 ms)\ntest_austin_time_checks.py::test_lines fibonacci (test_austin_time_checks.py) -9.3 ms (-18.6% of 50.0 ms)\ntest_austin_time_checks.py::test_check_fails test_check_fails (test_austin_time_checks.py) +99.8 ms (9978.6% of 1000.0 μs)\ntest_austin_time_checks.py::test_check_succeeds test_check_succeeds (test_austin_time_checks.py) -9.0 ms (-8.1% of 110.0 ms)\n\n================================================================== 1 check failed ===================================================================\n\n================================================================= 3 passed in 0.35s =================================================================\n~~~\n\n\n# Installation\n\npytest-austin can be installed directly from PyPI with\n\n~~~ bash\npip install pytest-austin --upgrade\n~~~\n\n**NOTE** In order for the plugin to work, the Austin binary needs to be on the\n``PATH`` environment variable. See [Austin\ninstallation](https://github.com/P403n1x87/austin#installation) instructions to\nsee how you can easily install Austin on your platform.\n\nFor platform-specific issues and remarks, refer to the\n[Compatibility](#compatibility) section below.\n\n\n# Usage\n\nOnce installed, the plugin will try to attach Austin to the pytest process in\norder to sample it every time you run pytest. If you want to prevent Austin from\nprofiling your tests, you have to steal its mojo. You can do so with the\n`--steal-mojo` command line argument.\n\n\n## Time checks\n\nThe plugin looks for the `total_time` marker on collected test items, which\ntakes a mandatory argument `time` and three optional ones: `function`, `module`\nand `line`.\n\nIf you simply want to check that the duration of a test item doesn't take longer\nthan `time`, you can mark it with `@pytest.mark.total_time(time)`. Here, `time`\ncan either be a `float` (in seconds) or an instance of `datetime.timedelta`.\n\n~~~ python\nfrom datetime import timedelta as td\n\nimport pytest\n\n\n@pytest.mark.total_time(td(milliseconds=50))\ndef test_hello_default():\n    ...\n~~~\n\nIn some cases, you would want to make sure that a function or method called on a\ncertain line in your test script executes in under a certain amount of time, say\n5% of the total test time. You can achieve this like so\n\n~~~ python\nimport pytest\n\n\n@pytest.mark.total_time(\"5%\", line=7)\ndef test_hello_default():\n    somefunction()\n    fastfunction()  # \u003c- this is line no. 7 in the test script\n    someotherfunction()\n~~~\n\nIn many cases, however, one would want to test that a function or a method\ncalled either directly or indirectly by a test doesn't take more than a certain\noverall time to run. This is where the remaining arguments of the ``total_test``\nmarker come into play. Suppose that you want to profile the procedure ``bar``\nthat is called by method ``foo`` of an object  of type ``Snafu``. To ensure that\n``bar`` doesn't take longer than, say, 50% of the overall test duration, you can\nwrite\n\n~~~ python\nimport pytest\n\n\n@pytest.mark.total_time(\"50%\", function=\"bar\")\ndef test_snafu():\n    ...\n    snafu = Snafu()\n    ...\n    snafu.foo()\n    ...\n~~~\n\nYou can use the `module` argument to resolve function name clashes. For example,\nif the definition of the function/method `bar` occurs within the modules\n``somemodule.py`` and ``someothermodule.py``, but you are only interested in the\none defined in ``somemodule.py``, you can change the above into\n\n~~~ python\nimport pytest\n\n\n@pytest.mark.total_time(\"50%\", function=\"bar\", module=\"somemodule.py\")\ndef test_snafu():\n    ...\n    snafu = Snafu()\n    ...\n    snafu.foo()\n    ...\n~~~\n\nAnd whilst you can also specify a line number, this is perhaps not very handy\nand practical outside of test scripts themselves, unless the content of the\nmodule is stable enough that line numbers don't need to be updated very\nfrequently.\n\nWhen the pluing runs, it will produce an output containing lines of the form\n\n~~~\ntest_austin_time_checks.py::test_lines test_lines:19 (test_austin_time_checks.py) -16.0 ms (-78.2% of 20.5 ms)\n~~~\n\nIn this case, a negative number, such as `-16.0 ms`, indicates that the total\ntime spent on `test_lines:19 (test_austin_time_checks.py)` was 16.0 ms less\nthan the total allowed time specified with the `total_time` marker, which in\nthis example is 20.5 ms. Hence, negative numbers indicate a successful check.\n\nFailing tests will have a positive delta reported, e.g.\n\n~~~\ntest_austin_time_checks.py::test_check_fails test_check_fails (test_austin_time_checks.py) +99.8 ms (9978.6% of 1000.0 μs)\n~~~\n\nThis indicates that the total time spent on\n`test_check_fails (test_austin_time_checks.py)` was 99.8 ms more than the\nrequired threshold, which was set to 1 ms.\n\n## Memory checks\n\nOne can perform memory allocation checks with the `total_memory` marker. The\nfirst argument is ``size``, which can be a percentage of the total memory\nallocation of the marked test case, as well as an absolute measure of the\nmaximum amount of memory, e.g., ``\"24 MB\"``. The ``function``, ``module`` and\n``line`` are the same as for the ``total_time`` marker. The extra ``net``\nargument can be set to ``True`` to check for the total _net_ memory usage, that\nis the difference between memory allocations and deallocations.\n\n~~~ python\nimport pytest\n\n\n@pytest.mark.total_memory(\"24 MB\")\ndef test_snafu():\n    allota_memory()\n~~~\n\nIn order to perform memory checks, you need to specify either the ``memory`` or\n``all`` profile mode via the ``--profile-mode`` option.\n\nThe negative and positive memory deltas reported by the plugin in the report\nbehave like the time deltas described in the previous section. That is, a\nnegative memory delta indicates a successful check, whereas a positive delta\nindicates a check that has failed.\n\n## Mixed checks\n\nWhen in the ``all`` profile mode, you can perform both time and memory checks by\nstacking ``total_time`` and ``total_memory`` markers.\n\n~~~ python\nimport pytest\n\n\n@pytest.mark.total_time(5.15)\n@pytest.mark.total_memory(\"24 MB\")\ndef test_snafu():\n    allota_memory_and_time()\n~~~\n\n\n## Multi-processing\n\nIf your tests spawn other Python processes, you can ask pytest-austin to profile\nthem too with the ``--minime`` option. Note that if your tests are spawning too\nmany non-Python processes, the sampling rate might be affected because of the\nway that Austin tries to discover Python child processes.\n\n## Reporting\n\nThis plugins generate a report on terminal and dumps the collected profiling\nstatistics on the file system as well, for later analysis and visualisation. The\nverbosity of the terminal report can be controlled with the ``--austin-report``\noption. By default, it is set to ``minimal``, which means that only checks that\nhave failed will be reported. Use ``full`` to see the results for all the checks\nthat have been detected and executed by the plugin.\n\nRegarding the dump of the profiling statistics, the generated file is in the\nAustin format by default (this is a generalisation of the collapsed stack\nformat). If you want the plugin to dump the data in either the ``pprof`` or\n``speedscope`` format, you can set the ``--profile-format`` option accordingly.\n\n\n# Compatibility\n\nThis plugin has been tested on Linux, MacOS and Windows. Given that it relies on\n[Austin](https://github.com/P403n1x87/austin) for sampling the frame stacks of\nthe pytest process, its compatibility considerations apply to pytest-austin as\nwell.\n\nOn Linux, the use of ``sudo`` is required, unless the ``CAP_SYS_PTRACE``\ncapability is granted to the Austin binary with, e.g.\n\n~~~ bash\nsudo setcap cap_sys_ptrace+ep `which austin`\n~~~\n\nThen the use of ``sudo`` is no longer required to allow Austin to attach and\nsample pytest.\n\nOn MacOS, the use of ``sudo`` is also mandatory, unless the user that is\ninvoking pytest belongs to the ``procmod`` group.\n\n\n# Contribute\n\nIf you like pytest-austin and you find it useful, there are ways for you to\ncontribute.\n\nIf you want to help with the development, then have a look at the open issues\nand have a look at the [contributing guidelines](CONTRIBUTING.md) before you\nopen a pull request.\n\nYou can also contribute to the development of the pytest-austin by becoming a\nsponsor and/or by [buying me a coffee](https://www.buymeacoffee.com/Q9C1Hnm28)\non BMC or by chipping in a few pennies on\n[PayPal.Me](https://www.paypal.me/gtornetta/1).\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.buymeacoffee.com/Q9C1Hnm28\"\n     target=\"_blank\"\u003e\n  \u003cimg src=\"https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png\"\n       alt=\"Buy Me A Coffee\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FP403n1x87%2Fpytest-austin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FP403n1x87%2Fpytest-austin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FP403n1x87%2Fpytest-austin/lists"}