{"id":13609491,"url":"https://github.com/fabianp/memory_profiler","last_synced_at":"2025-04-12T20:32:01.752Z","repository":{"id":53334839,"uuid":"139026109","full_name":"fabianp/memory_profiler","owner":"fabianp","description":"Monitor Memory usage of Python code","archived":false,"fork":true,"pushed_at":"2022-10-19T02:12:45.000Z","size":485,"stargazers_count":89,"open_issues_count":0,"forks_count":13,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-07T15:45:17.770Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://pypi.python.org/pypi/memory_profiler","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"pythonprofilers/memory_profiler","license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabianp.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-28T14:01:33.000Z","updated_at":"2024-07-25T15:54:30.000Z","dependencies_parsed_at":"2023-01-20T09:00:48.101Z","dependency_job_id":null,"html_url":"https://github.com/fabianp/memory_profiler","commit_stats":null,"previous_names":[],"tags_count":38,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianp%2Fmemory_profiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianp%2Fmemory_profiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianp%2Fmemory_profiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianp%2Fmemory_profiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabianp","download_url":"https://codeload.github.com/fabianp/memory_profiler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248629771,"owners_count":21136309,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T19:01:35.313Z","updated_at":"2025-04-12T20:32:01.306Z","avatar_url":"https://github.com/fabianp.png","language":"Python","readme":".. image:: https://travis-ci.org/pythonprofilers/memory_profiler.svg?branch=master\n    :target: https://travis-ci.org/pythonprofilers/memory_profiler\n\n=================\n Memory Profiler\n=================\n\nThis is a python module for monitoring memory consumption of a process\nas well as line-by-line analysis of memory consumption for python\nprograms. It is a pure python module which depends on the `psutil\n\u003chttp://pypi.python.org/pypi/psutil\u003e`_ module.\n\n\n==============\n Installation\n==============\nTo install through easy_install or pip::\n\n    $ easy_install -U memory_profiler # pip install -U memory_profiler\n\nTo install from source, download the package, extract and type::\n\n    $ python setup.py install\n\n\n=======\n Usage\n=======\n\n\nline-by-line memory usage\n=========================\n\nThe line-by-line memory usage mode is used much in the same way of the\n`line_profiler \u003chttps://pypi.python.org/pypi/line_profiler/\u003e`_: first\ndecorate the function you would like to profile with ``@profile`` and\nthen run the script with a special script (in this case with specific\narguments to the Python interpreter).\n\nIn the following example, we create a simple function ``my_func`` that\nallocates lists ``a``, ``b`` and then deletes ``b``::\n\n\n    @profile\n    def my_func():\n        a = [1] * (10 ** 6)\n        b = [2] * (2 * 10 ** 7)\n        del b\n        return a\n\n    if __name__ == '__main__':\n        my_func()\n\n\nExecute the code passing the option ``-m memory_profiler`` to the\npython interpreter to load the memory_profiler module and print to\nstdout the line-by-line analysis. If the file name was example.py,\nthis would result in::\n\n    $ python -m memory_profiler example.py\n\nOutput will follow::\n\n    Line #    Mem usage  Increment   Line Contents\n    ==============================================\n         3                           @profile\n         4      5.97 MB    0.00 MB   def my_func():\n         5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)\n         6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)\n         7     13.61 MB -152.59 MB       del b\n         8     13.61 MB    0.00 MB       return a\n\n\nThe first column represents the line number of the code that has been\nprofiled, the second column (*Mem usage*) the memory usage of the\nPython interpreter after that line has been executed. The third column\n(*Increment*) represents the difference in memory of the current line\nwith respect to the last one. The last column (*Line Contents*) prints\nthe code that has been profiled.\n\nDecorator\n=========\nA function decorator is also available.  Use as follows::\n\n    from memory_profiler import profile\n\n    @profile\n    def my_func():\n        a = [1] * (10 ** 6)\n        b = [2] * (2 * 10 ** 7)\n        del b\n        return a\n\nIn this case the script can be run without specifying ``-m\nmemory_profiler`` in the command line.\n\nIn function decorator, you can specify the precision as an argument to the\ndecorator function.  Use as follows::\n\n    from memory_profiler import profile\n\n    @profile(precision=4)\n    def my_func():\n        a = [1] * (10 ** 6)\n        b = [2] * (2 * 10 ** 7)\n        del b\n        return a\n\nIf a python script with decorator ``@profile`` is called using ``-m\nmemory_profiler`` in the command line, the ``precision`` parameter is ignored.\n\nTime-based memory usage\n==========================\nSometimes it is useful to have full memory usage reports as a function of\ntime (not line-by-line) of external processes (be it Python scripts or not).\nIn this case the executable ``mprof`` might be useful. Use it like::\n\n    mprof run \u003cexecutable\u003e\n    mprof plot\n\nThe first line run the executable and record memory usage along time,\nin a file written in the current directory.\nOnce it's done, a graph plot can be obtained using the second line.\nThe recorded file contains a timestamps, that allows for several\nprofiles to be kept at the same time.\n\nHelp on each `mprof` subcommand can be obtained with the `-h` flag,\ne.g. `mprof run -h`.\n\nIn the case of a Python script, using the previous command does not\ngive you any information on which function is executed at a given\ntime. Depending on the case, it can be difficult to identify the part\nof the code that is causing the highest memory usage.\n\nAdding the `profile` decorator to a function and running the Python\nscript with\n\n    mprof run \u003cscript\u003e\n\nwill record timestamps when entering/leaving the profiled function. Running\n\n    mprof plot\n\nafterward will plot the result, making plots (using matplotlib) similar to these:\n\n.. image:: https://camo.githubusercontent.com/3a584c7cfbae38c9220a755aa21b5ef926c1031d/68747470733a2f2f662e636c6f75642e6769746875622e636f6d2f6173736574732f313930383631382f3836313332302f63623865376337382d663563632d313165322d386531652d3539373237623636663462322e706e67\n   :target: https://github.com/scikit-learn/scikit-learn/pull/2248\n   :height: 350px\n\nA discussion of these capabilities can be found `here \u003chttp://fa.bianp.net/blog/2014/plot-memory-usage-as-a-function-of-time/\u003e`_.\n\n.. warning:: If your Python file imports the memory profiler `from memory_profiler import profile` these timestamps will not be recorded. Comment out the import, leave your functions decorated, and re-run.\n\nThe available commands for `mprof` are:\n\n  - ``mprof run``: running an executable, recording memory usage\n  - ``mprof plot``: plotting one the recorded memory usage (by default,\n    the last one)\n  - ``mprof list``: listing all recorded memory usage files in a\n    user-friendly way.\n  - ``mprof clean``: removing all recorded memory usage files.\n  - ``mprof rm``: removing specific recorded memory usage files\n\nTracking forked child processes\n===============================\nIn a multiprocessing context the main process will spawn child processes whose\nsystem resources are allocated separately from the parent process. This can\nlead to an inaccurate report of memory usage since by default only the parent\nprocess is being tracked. The ``mprof`` utility provides two mechanisms to\ntrack the usage of child processes: sum the memory of all children to the\nparent's usage and track each child individual.\n\nTo create a report that combines memory usage of all the children and the\nparent, use the ``include_children`` flag in either the ``profile`` decorator or\nas a command line argument to ``mprof``::\n\n    mprof run --include-children \u003cscript\u003e\n\nThe second method tracks each child independently of the main process,\nserializing child rows by index to the output stream. Use the ``multiprocess``\nflag and plot as follows::\n\n    mprof run --multiprocess \u003cscript\u003e\n    mprof plot\n\nThis will create a plot using matplotlib similar to this:\n\n.. image:: https://cloud.githubusercontent.com/assets/745966/24075879/2e85b43a-0bfa-11e7-8dfe-654320dbd2ce.png\n    :target: https://github.com/pythonprofilers/memory_profiler/pull/134\n    :height: 350px\n\nYou can combine both the ``include_children`` and ``multiprocess`` flags to show\nthe total memory of the program as well as each child individually. If using\nthe API directly, note that the return from ``memory_usage`` will include the\nchild memory in a nested list along with the main process memory.\n\nSetting debugger breakpoints\n=============================\nIt is possible to set breakpoints depending on the amount of memory used.\nThat is, you can specify a threshold and as soon as the program uses more\nmemory than what is specified in the threshold it will stop execution\nand run into the pdb debugger. To use it, you will have to decorate\nthe function as done in the previous section with ``@profile`` and then\nrun your script with the option ``-m memory_profiler --pdb-mmem=X``,\nwhere X is a number representing the memory threshold in MB. For example::\n\n    $ python -m memory_profiler --pdb-mmem=100 my_script.py\n\nwill run ``my_script.py`` and step into the pdb debugger as soon as the code\nuses more than 100 MB in the decorated function.\n\n.. TODO: alternatives to decoration (for example when you don't want to modify\n    the file where your function lives).\n\n=====\n API\n=====\nmemory_profiler exposes a number of functions to be used in third-party\ncode.\n\n\n\n``memory_usage(proc=-1, interval=.1, timeout=None)`` returns the memory usage\nover a time interval. The first argument, ``proc`` represents what\nshould be monitored.  This can either be the PID of a process (not\nnecessarily a Python program), a string containing some python code to\nbe evaluated or a tuple ``(f, args, kw)`` containing a function and its\narguments to be evaluated as ``f(*args, **kw)``. For example,\n\n\n    \u003e\u003e\u003e from memory_profiler import memory_usage\n    \u003e\u003e\u003e mem_usage = memory_usage(-1, interval=.2, timeout=1)\n    \u003e\u003e\u003e print(mem_usage)\n\t[7.296875, 7.296875, 7.296875, 7.296875, 7.296875]\n\n\nHere I've told memory_profiler to get the memory consumption of the\ncurrent process over a period of 1 second with a time interval of 0.2\nseconds. As PID I've given it -1, which is a special number (PIDs are\nusually positive) that means current process, that is, I'm getting the\nmemory usage of the current Python interpreter. Thus I'm getting\naround 7MB of memory usage from a plain python interpreter. If I try\nthe same thing on IPython (console) I get 29MB, and if I try the same\nthing on the IPython notebook it scales up to 44MB.\n\n\nIf you'd like to get the memory consumption of a Python function, then\nyou should specify the function and its arguments in the tuple ``(f,\nargs, kw)``. For example::\n\n\n    \u003e\u003e\u003e # define a simple function\n    \u003e\u003e\u003e def f(a, n=100):\n        ...     import time\n        ...     time.sleep(2)\n        ...     b = [a] * n\n        ...     time.sleep(1)\n        ...     return b\n        ...\n    \u003e\u003e\u003e from memory_profiler import memory_usage\n    \u003e\u003e\u003e memory_usage((f, (1,), {'n' : int(1e6)}))\n\nThis will execute the code `f(1, n=int(1e6))` and return the memory\nconsumption during this execution.\n\n=========\nREPORTING\n=========\n\nThe output can be redirected to a log file by passing IO stream as\nparameter to the decorator like @profile(stream=fp)\n\n    \u003e\u003e\u003e fp=open('memory_profiler.log','w+')\n    \u003e\u003e\u003e @profile(stream=fp)\n    \u003e\u003e\u003e def my_func():\n        ...     a = [1] * (10 ** 6)\n        ...     b = [2] * (2 * 10 ** 7)\n        ...     del b\n        ...     return a\n\n    For details refer: examples/reporting_file.py\n\n``Reporting via logger Module:``\n\nSometime it would be very convenient to use logger module specially\nwhen we need to use RotatingFileHandler.\n\nThe output can be redirected to logger module by simply making use of\nLogFile of memory profiler module.\n\n    \u003e\u003e\u003e from memory_profiler import LogFile\n    \u003e\u003e\u003e import sys\n    \u003e\u003e\u003e sys.stdout = LogFile('memory_profile_log')\n\n``Customized reporting:``\n\nSending everything to the log file while running the memory_profiler\ncould be cumbersome and one can choose only entries with increments\nby passing True to reportIncrementFlag, where reportIncrementFlag is\na parameter to LogFile class of memory profiler module.\n\n    \u003e\u003e\u003e from memory_profiler import LogFile\n    \u003e\u003e\u003e import sys\n    \u003e\u003e\u003e sys.stdout = LogFile('memory_profile_log', reportIncrementFlag=False)\n\n    For details refer: examples/reporting_logger.py\n\n=====================\n IPython integration\n=====================\nAfter installing the module, if you use IPython, you can use the `%mprun`, `%%mprun`,\n`%memit` and `%%memit` magics.\n\nFor IPython 0.11+, you can use the module directly as an extension, with\n``%load_ext memory_profiler``\n\nTo activate it whenever you start IPython, edit the configuration file for your\nIPython profile, ~/.ipython/profile_default/ipython_config.py, to register the\nextension like this (If you already have other extensions, just add this one to\nthe list)::\n\n    c.InteractiveShellApp.extensions = [\n        'memory_profiler',\n    ]\n\n(If the config file doesn't already exist, run ``ipython profile create`` in\na terminal.)\n\nIt then can be used directly from IPython to obtain a line-by-line\nreport using the `%mprun` or `%%mprun` magic command. In this case, you can skip\nthe `@profile` decorator and instead use the `-f` parameter, like\nthis. Note however that function my_func must be defined in a file\n(cannot have been defined interactively in the Python interpreter)::\n\n    In [1]: from example import my_func, my_func_2\n\n    In [2]: %mprun -f my_func my_func()\n\nor in cell mode::\n\n    In [3]: %%mprun -f my_func -f my_func_2\n       ...: my_func()\n       ...: my_func_2()\n\nAnother useful magic that we define is `%memit`, which is analogous to\n`%timeit`. It can be used as follows::\n\n    In [1]: %memit range(10000)\n    peak memory: 21.42 MiB, increment: 0.41 MiB\n\n    In [2]: %memit range(1000000)\n    peak memory: 52.10 MiB, increment: 31.08 MiB\n\nor in cell mode (with setup code)::\n\n    In [3]: %%memit l=range(1000000)\n       ...: len(l)\n       ...:\n    peak memory: 52.14 MiB, increment: 0.08 MiB\n\nFor more details, see the docstrings of the magics.\n\nFor IPython 0.10, you can install it by editing the IPython configuration\nfile ~/.ipython/ipy_user_conf.py to add the following lines::\n\n    # These two lines are standard and probably already there.\n    import IPython.ipapi\n    ip = IPython.ipapi.get()\n\n    # These two are the important ones.\n    import memory_profiler\n    memory_profiler.load_ipython_extension(ip)\n\n============================\n Frequently Asked Questions\n============================\n    * Q: How accurate are the results ?\n    * A: This module gets the memory consumption by querying the\n      operating system kernel about the amount of memory the current\n      process has allocated, which might be slightly different from\n      the amount of memory that is actually used by the Python\n      interpreter. Also, because of how the garbage collector works in\n      Python the result might be different between platforms and even\n      between runs.\n\n    * Q: Does it work under windows ?\n    * A: Yes, thanks to the\n      `psutil \u003chttp://pypi.python.org/pypi/psutil\u003e`_ module.\n\n\n\n===========================\n Support, bugs \u0026 wish list\n===========================\nFor support, please ask your question on `stack overflow\n\u003chttp://stackoverflow.com/\u003e`_ and add the `*memory-profiling* tag \u003chttp://stackoverflow.com/questions/tagged/memory-profiling\u003e`_.\nSend issues, proposals, etc. to `github's issue tracker\n\u003chttps://github.com/pythonprofilers/memory_profiler/issues\u003e`_ .\n\nIf you've got questions regarding development, you can email me\ndirectly at fabian@fseoane.net\n\n.. image:: http://fseoane.net/static/tux_memory_small.png\n\n\n=============\n Development\n=============\nLatest sources are available from github:\n\n    https://github.com/pythonprofilers/memory_profiler\n\n===============================\nProjects using memory_profiler\n===============================\n\n`Benchy \u003chttps://github.com/python-recsys/benchy\u003e`_\n\n`IPython memory usage \u003chttps://github.com/ianozsvald/ipython_memory_usage\u003e`_\n\n`PySpeedIT \u003chttps://github.com/peter1000/PySpeedIT\u003e`_ (uses a reduced version of memory_profiler)\n\n`pydio-sync \u003chttps://github.com/pydio/pydio-sync\u003e`_ (uses custom wrapper on top of memory_profiler)\n\n=========\n Authors\n=========\nThis module was written by `Fabian Pedregosa \u003chttp://fseoane.net\u003e`_\nand `Philippe Gervais \u003chttps://github.com/pgervais\u003e`_\ninspired by Robert Kern's `line profiler\n\u003chttp://packages.python.org/line_profiler/\u003e`_.\n\n`Tom \u003chttp://tomforb.es/\u003e`_ added windows support and speed improvements via the\n`psutil \u003chttp://pypi.python.org/pypi/psutil\u003e`_ module.\n\n`Victor \u003chttps://github.com/octavo\u003e`_ added python3 support, bugfixes and general\ncleanup.\n\n`Vlad Niculae \u003chttp://vene.ro/\u003e`_ added the `%mprun` and `%memit` IPython magics.\n\n`Thomas Kluyver \u003chttps://github.com/takluyver\u003e`_ added the IPython extension.\n\n`Sagar UDAY KUMAR \u003chttps://github.com/sagaru\u003e`_ added Report generation feature and examples.\n\n`Dmitriy Novozhilov \u003chttps://github.com/demiurg906\u003e`_ and `Sergei Lebedev \u003chttps://github.com/superbobry\u003e`_ added support for `tracemalloc \u003chttps://docs.python.org/3/library/tracemalloc.html\u003e`_.\n\n`Benjamin Bengfort \u003chttps://github.com/bbengfort\u003e`_ added support for tracking the usage of individual child processes and plotting them.\n\n`Muhammad Haseeb Tariq \u003chttps://github.com/mhaseebtariq\u003e`_ fixed issue #152, which made the whole interpreter hang on functions that launched an exception.\n\n`Juan Luis Cano \u003chttps://github.com/Juanlu001\u003e`_ modernized the infrastructure and helped with various things.\n\n=========\n License\n=========\nBSD License, see file COPYING for full text.\n","funding_links":[],"categories":["Debugging Tools","调试工具","Debugging Tools [🔝](#readme)","Awesome Python"],"sub_categories":["Debugging Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianp%2Fmemory_profiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabianp%2Fmemory_profiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianp%2Fmemory_profiler/lists"}