{"id":13460284,"url":"https://github.com/sumerc/yappi","last_synced_at":"2025-05-14T07:08:22.154Z","repository":{"id":686243,"uuid":"330227","full_name":"sumerc/yappi","owner":"sumerc","description":"Yet Another Python Profiler, but this time multithreading, asyncio and gevent aware.","archived":false,"fork":false,"pushed_at":"2024-07-09T12:16:42.000Z","size":1660,"stargazers_count":1472,"open_issues_count":23,"forks_count":73,"subscribers_count":15,"default_branch":"master","last_synced_at":"2024-10-29T15:10:38.224Z","etag":null,"topics":["asgi","asynchronous","asyncio","coroutine","cpu","gevent","greenlet","multi-threaded-applications","multithreading","performance","profile","profilers","python","thread"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sumerc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2009-10-07T20:46:30.000Z","updated_at":"2024-10-29T12:03:07.000Z","dependencies_parsed_at":"2023-02-10T08:16:01.932Z","dependency_job_id":"f1cc1675-1cfe-4ea1-8afc-b382f043c4c1","html_url":"https://github.com/sumerc/yappi","commit_stats":{"total_commits":628,"total_committers":29,"mean_commits":"21.655172413793103","dds":0.4299363057324841,"last_synced_commit":"38afdacf526410f970afc58e147c7377c6c7112c"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumerc%2Fyappi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumerc%2Fyappi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumerc%2Fyappi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumerc%2Fyappi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sumerc","download_url":"https://codeload.github.com/sumerc/yappi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254092659,"owners_count":22013290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asgi","asynchronous","asyncio","coroutine","cpu","gevent","greenlet","multi-threaded-applications","multithreading","performance","profile","profilers","python","thread"],"created_at":"2024-07-31T10:00:38.691Z","updated_at":"2025-05-14T07:08:22.131Z","avatar_url":"https://github.com/sumerc.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/sumerc/yappi/master/Misc/logo.png\" alt=\"yappi\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eYappi\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    A tracing profiler that is \u003cb\u003emultithreading, asyncio and gevent\u003c/b\u003e aware.\n\u003c/p\u003e\n\n[![FreePalestine.Dev](https://freepalestine.dev/header/1)](https://freepalestine.dev)\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/sumerc/yappi/actions/workflows/main.yml\"\u003e\u003cimg src=\"https://github.com/sumerc/yappi/workflows/CI/badge.svg?branch=master\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/yappi/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/yappi.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/yappi/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/dw/yappi.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/yappi/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/pyversions/yappi.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/sumerc/yappi/commits/\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/sumerc/yappi.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/sumerc/yappi/blob/master/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/sumerc/yappi.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://freepalestine.dev\"\u003e\u003cimg src=\"https://freepalestine.dev/badge?t=d\u0026u=0\u0026r=1\" alt=\"From the river to the sea, Palestine will be free\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Highlights\n\n- **Fast**: Yappi is fast. It is completely written in C and lots of love and care went into making it fast.\n- **Unique**: Yappi supports multithreaded, [asyncio](https://github.com/sumerc/yappi/blob/master/doc/coroutine-profiling.md) and [gevent](https://github.com/sumerc/yappi/blob/master/doc/greenlet-profiling.md) profiling. Tagging/filtering multiple profiler results has interesting [use cases](https://github.com/sumerc/yappi/blob/master/doc/api.md#set_tag_callback).\n- **Intuitive**: Profiler can be started/stopped and results can be obtained from any time and any thread.\n- **Standards Compliant**: Profiler results can be saved in [callgrind](http://valgrind.org/docs/manual/cl-format.html) or [pstat](http://docs.python.org/3.4/library/profile.html#pstats.Stats) formats.\n- **Rich in Feature set**: Profiler results can show either [Wall Time](https://en.wikipedia.org/wiki/Elapsed_real_time) or actual [CPU Time](http://en.wikipedia.org/wiki/CPU_time) and can be aggregated from different sessions. Various flags are defined for filtering and sorting profiler results.\n- **Robust**: Yappi has been around for years.\n\n## Motivation\n\nCPython standard distribution comes with three deterministic profilers. `cProfile`, `Profile` and `hotshot`. `cProfile` is implemented as a C module based on `lsprof`, `Profile` is in pure Python and `hotshot` can be seen as a small subset of a cProfile. The major issue is that all of these profilers lack support for multi-threaded programs and CPU time.\n\nIf you want to profile a  multi-threaded application, you must give an entry point to these profilers and then maybe merge the outputs. None of these profilers are designed to work on long-running multi-threaded applications. It is also not possible to profile an application that start/stop/retrieve traces on the fly with these profilers. \n\nNow fast forwarding to 2019: With the latest improvements on `asyncio` library and asynchronous frameworks, most of the current profilers lacks the ability to show correct wall/cpu time or even call count information per-coroutine. Thus we need a different kind of approach to profile asynchronous code. Yappi, with v1.2 introduces the concept of `coroutine profiling`. With `coroutine-profiling`, you should be able to profile correct wall/cpu time and call count of your coroutine. (including the time spent in context switches, too). You can see details [here](https://github.com/sumerc/yappi/blob/master/doc/coroutine-profiling.md).\n\n\n## Installation\n\nCan be installed via PyPI\n\n```\n$ pip install yappi\n```\n\nOR from the source directly.\n\n```\n$ pip install git+https://github.com/sumerc/yappi#egg=yappi\n```\n\n## Examples\n\n### A simple example:\n\n```python\nimport yappi\n\ndef a():\n    for _ in range(10000000):  # do something CPU heavy\n        pass\n\nyappi.set_clock_type(\"cpu\") # Use set_clock_type(\"wall\") for wall time\nyappi.start()\na()\n\nyappi.get_func_stats().print_all()\nyappi.get_thread_stats().print_all()\n'''\n\nClock type: CPU\nOrdered by: totaltime, desc\n\nname                                  ncall  tsub      ttot      tavg      \ndoc.py:5 a                            1      0.117907  0.117907  0.117907\n\nname           id     tid              ttot      scnt        \n_MainThread    0      139867147315008  0.118297  1\n'''\n```\n\n### Profile a multithreaded application:\n\nYou can profile a multithreaded application via Yappi and can easily retrieve\nper-thread profile information by filtering on `ctx_id` with `get_func_stats` API.\n\n```python\nimport yappi\nimport time\nimport threading\n\n_NTHREAD = 3\n\n\ndef _work(n):\n    time.sleep(n * 0.1)\n\n\nyappi.start()\n\nthreads = []\n# generate _NTHREAD threads\nfor i in range(_NTHREAD):\n    t = threading.Thread(target=_work, args=(i + 1, ))\n    t.start()\n    threads.append(t)\n# wait all threads to finish\nfor t in threads:\n    t.join()\n\nyappi.stop()\n\n# retrieve thread stats by their thread id (given by yappi)\nthreads = yappi.get_thread_stats()\nfor thread in threads:\n    print(\n        \"Function stats for (%s) (%d)\" % (thread.name, thread.id)\n    )  # it is the Thread.__class__.__name__\n    yappi.get_func_stats(ctx_id=thread.id).print_all()\n'''\nFunction stats for (Thread) (3)\n\nname                                  ncall  tsub      ttot      tavg\n..hon3.7/threading.py:859 Thread.run  1      0.000017  0.000062  0.000062\ndoc3.py:8 _work                       1      0.000012  0.000045  0.000045\n\nFunction stats for (Thread) (2)\n\nname                                  ncall  tsub      ttot      tavg\n..hon3.7/threading.py:859 Thread.run  1      0.000017  0.000065  0.000065\ndoc3.py:8 _work                       1      0.000010  0.000048  0.000048\n\n\nFunction stats for (Thread) (1)\n\nname                                  ncall  tsub      ttot      tavg\n..hon3.7/threading.py:859 Thread.run  1      0.000010  0.000043  0.000043\ndoc3.py:8 _work                       1      0.000006  0.000033  0.000033\n'''\n```\n\n### Different ways to filter/sort stats:\n\nYou can use `filter_callback` on `get_func_stats` API to filter on functions, modules\nor whatever available in `YFuncStat` object.\n\n```python\nimport package_a\nimport yappi\nimport sys\n\ndef a():\n    pass\n\ndef b():\n    pass\n\nyappi.start()\na()\nb()\npackage_a.a()\nyappi.stop()\n\n# filter by module object\ncurrent_module = sys.modules[__name__]\nstats = yappi.get_func_stats(\n    filter_callback=lambda x: yappi.module_matches(x, [current_module])\n)  # x is a yappi.YFuncStat object\nstats.sort(\"name\", \"desc\").print_all()\n'''\nClock type: CPU\nOrdered by: name, desc\n\nname                                  ncall  tsub      ttot      tavg\ndoc2.py:10 b                          1      0.000001  0.000001  0.000001\ndoc2.py:6 a                           1      0.000001  0.000001  0.000001\n'''\n\n# filter by function object\nstats = yappi.get_func_stats(\n    filter_callback=lambda x: yappi.func_matches(x, [a, b])\n).print_all()\n'''\nname                                  ncall  tsub      ttot      tavg\ndoc2.py:6 a                           1      0.000001  0.000001  0.000001\ndoc2.py:10 b                          1      0.000001  0.000001  0.000001\n'''\n\n# filter by module name\nstats = yappi.get_func_stats(filter_callback=lambda x: 'package_a' in x.module\n                             ).print_all()\n'''\nname                                  ncall  tsub      ttot      tavg\npackage_a/__init__.py:1 a             1      0.000001  0.000001  0.000001\n'''\n\n# filter by function name\nstats = yappi.get_func_stats(filter_callback=lambda x: 'a' in x.name\n                             ).print_all()\n'''\nname                                  ncall  tsub      ttot      tavg\ndoc2.py:6 a                           1      0.000001  0.000001  0.000001\npackage_a/__init__.py:1 a             1      0.000001  0.000001  0.000001\n'''\n```\n\n### Profile an asyncio application:\n\nYou can see that coroutine wall-time's are correctly profiled.\n\n```python\nimport asyncio\nimport yappi\n\nasync def foo():\n    await asyncio.sleep(1.0)\n    await baz()\n    await asyncio.sleep(0.5)\n\nasync def bar():\n    await asyncio.sleep(2.0)\n\nasync def baz():\n    await asyncio.sleep(1.0)\n\nyappi.set_clock_type(\"WALL\")\nwith yappi.run():\n    asyncio.run(foo())\n    asyncio.run(bar())\nyappi.get_func_stats().print_all()\n'''\nClock type: WALL\nOrdered by: totaltime, desc\n\nname                                  ncall  tsub      ttot      tavg      \ndoc4.py:5 foo                         1      0.000030  2.503808  2.503808\ndoc4.py:11 bar                        1      0.000012  2.002492  2.002492\ndoc4.py:15 baz                        1      0.000013  1.001397  1.001397\n'''\n```\n\n### Profile a gevent application:\n\nYou can use yappi to profile greenlet applications now!\n\n```python\nimport yappi\nfrom greenlet import greenlet\nimport time\n\nclass GreenletA(greenlet):\n    def run(self):\n        time.sleep(1)\n\nyappi.set_context_backend(\"greenlet\")\nyappi.set_clock_type(\"wall\")\n\nyappi.start(builtins=True)\na = GreenletA()\na.switch()\nyappi.stop()\n\nyappi.get_func_stats().print_all()\n'''\nname                                  ncall  tsub      ttot      tavg\ntests/test_random.py:6 GreenletA.run  1      0.000007  1.000494  1.000494\ntime.sleep                            1      1.000487  1.000487  1.000487\n'''\n```\n\n## Documentation\n\n- [Introduction](https://github.com/sumerc/yappi/blob/master/doc/introduction.md)\n- [Clock Types](https://github.com/sumerc/yappi/blob/master/doc/clock_types.md)\n- [API](https://github.com/sumerc/yappi/blob/master/doc/api.md)\n- [Coroutine Profiling](https://github.com/sumerc/yappi/blob/master/doc/coroutine-profiling.md) _(new in 1.2)_\n- [Greenlet Profiling](https://github.com/sumerc/yappi/blob/master/doc/greenlet-profiling.md) _(new in 1.3)_\n\n  Note: Yes. I know I should be moving docs to readthedocs.io. Stay tuned!\n\n\n## Related Talks\n\n  Special thanks to A.Jesse Jiryu Davis:\n- [Python Performance Profiling: The Guts And The Glory (PyCon 2015)](https://www.youtube.com/watch?v=4uJWWXYHxaM)\n\n## PyCharm Integration\n\nYappi is the default profiler in `PyCharm`. If you have Yappi installed, `PyCharm` will use it. See [the official](https://www.jetbrains.com/help/pycharm/profiler.html) documentation for more details.\n\n","funding_links":[],"categories":["Python","System Monitoring \u0026 Profiling","语言资源库"],"sub_categories":["python"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumerc%2Fyappi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsumerc%2Fyappi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumerc%2Fyappi/lists"}