{"id":18412246,"url":"https://github.com/akkefa/pycon-python-performance-profiling","last_synced_at":"2025-10-12T03:14:22.501Z","repository":{"id":130772235,"uuid":"157355188","full_name":"akkefa/pycon-python-performance-profiling","owner":"akkefa","description":"Examples of using common Python profiling techniques","archived":false,"fork":false,"pushed_at":"2019-05-24T18:13:08.000Z","size":1473,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-22T17:03:59.196Z","etag":null,"topics":["cprofile","profiling","pycon","python"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akkefa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-13T09:35:21.000Z","updated_at":"2023-05-15T07:13:09.000Z","dependencies_parsed_at":"2023-07-08T01:00:39.339Z","dependency_job_id":null,"html_url":"https://github.com/akkefa/pycon-python-performance-profiling","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akkefa%2Fpycon-python-performance-profiling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akkefa%2Fpycon-python-performance-profiling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akkefa%2Fpycon-python-performance-profiling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akkefa%2Fpycon-python-performance-profiling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akkefa","download_url":"https://codeload.github.com/akkefa/pycon-python-performance-profiling/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247644288,"owners_count":20972263,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cprofile","profiling","pycon","python"],"created_at":"2024-11-06T03:40:31.291Z","updated_at":"2025-10-12T03:14:17.456Z","avatar_url":"https://github.com/akkefa.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Performance Profiling\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/akkefa/pycon-python-performance-profiling/blob/master/imgs/1.png?raw=true\" alt=\"Python performance profiling\"/\u003e\n\u003c/p\u003e\n\n## HELLO!\n\nI am **Ikram Ali**\n\n+ Data Scientist @ Arbisoft\n+ Working on Deep learning projects for Kayak\n+ Github.com/akkefa\n+ Linkedin.com/in/akkefa\n\n\n# What is Profiling?\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/akkefa/pycon-python-performance-profiling/blob/master/imgs/5.jpg?raw=true\" alt=\"Python performance profiling\"/\u003e\n\u003c/p\u003e\n\n### Profiling Definition?\n\n+ Measuring the execution time.\n+ Insight of run time performance of a given piece of code.\n+ Frequently used to optimize execution time.\n+ Used to analyze other characteristics such as memory consumption.\n\n\n# What is Python Profiling?\n+ Measure Performance\n\n\n# Why Profile?\n```\nYou can use a profiler to answer questions like these:\n```\n+ Why is this program slow?\n+ Why does it slow my computer to a crawl?\n+ What is actually happening when this code executes?\n+ Is there anything I can improve?\n+ How much memory consumed by program?\n+ How much time taken by each function execution?\n\n# Why You should care about Performance\n\n+ “If You Can’t Measure It, You Can’t Manage It.”\n\n+ Writing efficient code saves money in modern \"cloud economy\" (e.g. you need fewer VM instances).\n\n+ Even if you don't use clouds, a particular problem domain can have strict performance requirements (e.g. when you have to process a chunk of data in time before the next chunk arrives).\n\n# Available options for measuring Performance\n\n#### Command Line\n\n#### time Module\n\n####  timeit Module\n\n#### cProfile Module\n\n\n# Command Line\n\nThe **time** command is available in *nix systems.\n```\n$ time python some_program.py\n```\n\n```\nreal 0m4.536s\nuser 0m3.411s\nsys 0m0.979s\n```\n\n+ Easy to use\n+  Very limited information\n+  Not very deterministic\n+  Not available on Windows\n\n# Python time Module\n\ntime.time() statements\n\n```\nimport time\ninitial_time = time.time()\ntime.sleep(1)\nfinal_time = time.time()\nprint('Duration: {}'.format(final_time - initial_time))\n```\n\n```\nDuration: 1.0898\n```\n\n+ Easy to use\n+ Simple to understand\n+ Very limited information\n+ Not very deterministic\n+ Manual code modification and analysis\n\n# Python timeit Module\n\n```\nimport timeit\n\nprint('Plus:', timeit.timeit(\"['Hello world: ' + str(n) for n in range(100)]\", number=1000))\nprint('Format:', timeit.timeit(\"['Hello world: {0}'.format(n) for n in range(100)]\",\nnumber=1000))\nprint('Percent:', timeit.timeit(\"['Hello world: %s' % n for n in range(100)]\", number=1000))\n```\n\n+ Easy to use\n+ Simple to understand\n+ Measure execution time of small code snippets\n+ Simple code only\n+ Not very deterministic\n+ Have to manually create runnable code snippets\n+ Manual analysis\n\n\n# cProfile Module\n\nBest approach: cProfile\n+ Python comes with two profiling tools, profile and cProfile.\n+ Both share the same API, and should act the same.\n\n```\n\u003e\u003e\u003e import cProfile\n\u003e\u003e\u003e cProfile.run('2 + 2')\n```\n\n```\n3 function calls in 0.000 seconds\nOrdered by: standard name\nncalls tottime percall cumtime percall filename:lineno(function)\n1 0.000 0.000 0.000 0.000 \u003cstring\u003e:1(\u003cmodule\u003e)\n1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler'}\n```\n\n### Running a script with cProfile\n```\n# slow.py\nimport time\ndef main():\n    sum = 0\n    for i in range(10):\n    sum += expensive(i // 2)\n    return sum\n\ndef expensive(t):\n    time.sleep(t)\n    return t\n\nif __name__ == '__main__':\n    print(main())\n```\n\n```\npython -m cProfile slow.py\n```\n\n```\n25 function calls in 20.030 seconds\n\nOrdered by: standard name\n\nncalls tottime percall cumtime percall filename:lineno(function)\n10 0.000 0.000 20.027 2.003 slow.py:11(expensive)\n1 0.002 0.002 20.030 20.030 slow.py:2(\u003cmodule\u003e)\n1 0.000 0.000 20.027 20.027 slow.py:5(main)\n1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler'objects}\n1 0.000 0.000 0.000 0.000 {print}\n1 0.000 0.000 0.000 0.000 {range}\n10 20.027 2.003 20.027 2.003 {time.sleep}\n```\n\n## cProfile sort by options\n\n**ncalls**\n    Total the number of calls of a function\n   \n\n**tottime**\n     for the total time spent in the given function\n\n**cumtime**\n    is the cumulative time spent in this and all sub functions.\n\n\n**filename:lineno(function)**\n    provides the respective data of each function\n\n\n## cProfile result sorted by tottime\n\n```\npython -m cProfile -s tottime slow.py\n```\n\n```\n25 function calls in 20.015 seconds\n\nOrdered by: **internal time**\n\nncalls **tottime** percall cumtime percall filename:lineno(function)\n10 **20.015** 2.001 20.015 2.001 {built-in method time.sleep}\n1 **0.000** 0.000 0.000 0.000 {built-in method builtins.print}\n1 **0.000** 0.000 20.015 20.015 slow.py:6(main)\n10 **0.000** 0.000 20.015 2.001 slow.py:13(expensive)\n1 **0.000** 0.000 20.015 20.015 slow.py:3(\u003cmodule\u003e)\n1 **0.000** 0.000 20.015 20.015 {built-in method builtins.exec}\n1 **0.000** 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n```\n\n### cProfile result sorted by ncalls\n\n```\npython -m cProfile -s ncalls slow.py\n```\n\n```\n25 function calls in 20.015 seconds\n\nOrdered by: **call count**\n\n**ncalls** tottime percall cumtime percall filename:lineno(function)\n**10** 20.020 2.002 20.020 2.002 {built-in method time.sleep}\n**10** 0.000 0.000 20.020 2.002 slow.py:13(expensive)\n**1** 0.000 0.000 20.020 20.020 {built-in method builtins.exec}\n**1** 0.000 0.000 0.000 0.000 {built-in method builtins.print}\n**1** 0.000 0.000 20.020 20.020 slow.py:6(main)\n**1** 0.000 0.000 20.020 20.020 slow.py:3(\u003cmodule\u003e)\n**1** 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n```\n\n\n### Easiest way to profile Python code\n\n```\ndef main():\n    sum = 0\n    for i in range(10):\n    sum += expensive(i // 2)\n    return sum\ndef expensive(t):\n    time.sleep(t)\n    return t\n    \nif __name__ == '__main__':\n     pr = cProfile.Profile()\n     pr.enable()\n     main()\n     pr.disable()\n     pr.print_stats()\n```\n\n```\n25 function calls in 20.030 seconds\n\nOrdered by: standard name\n\nncalls tottime percall cumtime percall filename:lineno(function)\n10 0.000 0.000 20.027 2.003 slow.py:11(expensive)\n1 0.002 0.002 20.030 20.030 slow.py:2(\u003cmodule\u003e)\n1 0.000 0.000 20.027 20.027 slow.py:5(main)\n1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler'objects}\n1 0.000 0.000 0.000 0.000 {print}\n1 0.000 0.000 0.000 0.000 {range}\n10 20.027 2.003 20.027 2.003 {time.sleep}\n```\n\n### We can also save the output!\n```\nif __name__ == '__main__':\npr = cProfile.Profile()\npr.enable()\nmain()\npr.disable()\n**pr.dump_stats(\"profile.output\")**\n```\n\n\n# How do we use the profiling information?\n\n### pstats Module\n\n+ You can use pstats to format the output in various ways.\n+ pstats provides sorting options. **( calls, time, cumulative )**\n\n```\nimport pstats\n\np = pstats.Stats(\"profile.output\")\np.strip_dirs().sort_stats(\"calls\").print_stats()\n```\n\n```\n23 function calls in 20.019 seconds\n\nOrdered by: call count\n\nncalls tottime percall cumtime percall filename:lineno(function)\n10 20.019 2.002 20.019 2.002 {built-in method time.sleep}\n10 0.000 0.000 20.019 2.002 slow.py:14(expensive)\n1 0.000 0.000 0.000 0.000 {built-in method builtins.print}\n1 0.000 0.000 20.019 20.019 slow.py:7(main)\n```\n\n\n# An easy way to visualize cProfile results\n\n### SNAKEVIZ library\n```\npip install snakeviz\n```\n\n```\n$ snakeviz profile.output\n```\n\n+ Snakeviz provides two ways to explore profiler data\n+ Summaries Times\n+ You can choose the sorting criterion in the output table\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/akkefa/pycon-python-performance-profiling/blob/master/imgs/4.png?raw=true\" alt=\"Python performance profiling\"/\u003e\n\u003c/p\u003e\n\n\n\n\n\n### PyCallGraph library\n```\npip install pycallgraph\n```\n\n```\n$ pycallgraph graphviz -- python slow.py\n```\n\n+ Visual extension of cProfile.\n+ Understand code structure and Flow\n+ Summaries Times\n+ Darker color represent more time spent.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/akkefa/pycon-python-performance-profiling/blob/master/imgs/2.png?raw=true\" alt=\"Python performance profiling\"/\u003e\n\u003c/p\u003e\n\n\n\n# Other profiling options\n\n### Line profiler\n+ line_profiler will profile the time individual lines of code take to execute.\n+ https://github.com/rkern/line_profiler\n\n### Memory profiler\n+ Monitoring memory consumption of a process.\n+ line-by-line analysis of memory consumption.\n+ https://pypi.org/project/memory_profiler/\n\n# Live Example\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/akkefa/pycon-python-performance-profiling/blob/master/imgs/3.gif?raw=true\" alt=\"Python performance profiling\"/\u003e\n\u003c/p\u003e\n\n```\nhttps://github.com/akkefa/pycon-python-performance-profiling/profiling-demo.ipynb\n```\n\n# Thank you.\n```\nLinkedin.com/in/akkefa\nContact : mrikram1989@gmail.com\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakkefa%2Fpycon-python-performance-profiling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakkefa%2Fpycon-python-performance-profiling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakkefa%2Fpycon-python-performance-profiling/lists"}