{"id":13648103,"url":"https://github.com/joblib/threadpoolctl","last_synced_at":"2025-05-15T01:07:28.806Z","repository":{"id":34374707,"uuid":"177753680","full_name":"joblib/threadpoolctl","owner":"joblib","description":"Python helpers to limit the number of threads used in native libraries that handle their own internal threadpool (BLAS and OpenMP implementations)","archived":false,"fork":false,"pushed_at":"2025-05-08T12:02:54.000Z","size":276,"stargazers_count":372,"open_issues_count":18,"forks_count":31,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-05-13T22:53:55.526Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joblib.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-03-26T09:17:31.000Z","updated_at":"2025-05-08T12:02:59.000Z","dependencies_parsed_at":"2023-10-10T15:59:51.918Z","dependency_job_id":"666a00b6-4311-4bf5-876a-a0abe7c16946","html_url":"https://github.com/joblib/threadpoolctl","commit_stats":{"total_commits":189,"total_committers":12,"mean_commits":15.75,"dds":0.6613756613756614,"last_synced_commit":"7eff7ebb6337e368bc58614933df500db6106bad"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joblib%2Fthreadpoolctl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joblib%2Fthreadpoolctl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joblib%2Fthreadpoolctl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joblib%2Fthreadpoolctl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joblib","download_url":"https://codeload.github.com/joblib/threadpoolctl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254051343,"owners_count":22006463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:03:58.351Z","updated_at":"2025-05-15T01:07:23.790Z","avatar_url":"https://github.com/joblib.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Thread-pool Controls [![Build Status](https://github.com/joblib/threadpoolctl/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/joblib/threadpoolctl/actions?query=branch%3Amaster) [![codecov](https://codecov.io/gh/joblib/threadpoolctl/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/threadpoolctl)\n\nPython helpers to limit the number of threads used in the\nthreadpool-backed of common native libraries used for scientific\ncomputing and data science (e.g. BLAS and OpenMP).\n\nFine control of the underlying thread-pool size can be useful in\nworkloads that involve nested parallelism so as to mitigate\noversubscription issues.\n\n## Installation\n\n- For users, install the last published version from PyPI:\n\n  ```bash\n  pip install threadpoolctl\n  ```\n\n- For contributors, install from the source repository in developer\n  mode:\n\n  ```bash\n  pip install -r dev-requirements.txt\n  flit install --symlink\n  ```\n\n  then you run the tests with pytest:\n\n  ```bash\n  pytest\n  ```\n\n## Usage\n\n### Command Line Interface\n\nGet a JSON description of thread-pools initialized when importing python\npackages such as numpy or scipy for instance:\n\n```\npython -m threadpoolctl -i numpy scipy.linalg\n[\n  {\n    \"filepath\": \"/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so\",\n    \"prefix\": \"libmkl_rt\",\n    \"user_api\": \"blas\",\n    \"internal_api\": \"mkl\",\n    \"version\": \"2019.0.4\",\n    \"num_threads\": 2,\n    \"threading_layer\": \"intel\"\n  },\n  {\n    \"filepath\": \"/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so\",\n    \"prefix\": \"libiomp\",\n    \"user_api\": \"openmp\",\n    \"internal_api\": \"openmp\",\n    \"version\": null,\n    \"num_threads\": 4\n  }\n]\n```\n\nThe JSON information is written on STDOUT. If some of the packages are missing,\na warning message is displayed on STDERR.\n\n### Python Runtime Programmatic Introspection\n\nIntrospect the current state of the threadpool-enabled runtime libraries\nthat are loaded when importing Python packages:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import threadpool_info\n\u003e\u003e\u003e from pprint import pprint\n\u003e\u003e\u003e pprint(threadpool_info())\n[]\n\n\u003e\u003e\u003e import numpy\n\u003e\u003e\u003e pprint(threadpool_info())\n[{'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so',\n  'internal_api': 'mkl',\n  'num_threads': 2,\n  'prefix': 'libmkl_rt',\n  'threading_layer': 'intel',\n  'user_api': 'blas',\n  'version': '2019.0.4'},\n {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so',\n  'internal_api': 'openmp',\n  'num_threads': 4,\n  'prefix': 'libiomp',\n  'user_api': 'openmp',\n  'version': None}]\n\n\u003e\u003e\u003e import xgboost\n\u003e\u003e\u003e pprint(threadpool_info())\n[{'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libmkl_rt.so',\n  'internal_api': 'mkl',\n  'num_threads': 2,\n  'prefix': 'libmkl_rt',\n  'threading_layer': 'intel',\n  'user_api': 'blas',\n  'version': '2019.0.4'},\n {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so',\n  'internal_api': 'openmp',\n  'num_threads': 4,\n  'prefix': 'libiomp',\n  'user_api': 'openmp',\n  'version': None},\n {'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libgomp.so.1.0.0',\n  'internal_api': 'openmp',\n  'num_threads': 4,\n  'prefix': 'libgomp',\n  'user_api': 'openmp',\n  'version': None}]\n```\n\nIn the above example, `numpy` was installed from the default anaconda channel and comes\nwith MKL and its Intel OpenMP (`libiomp5`) implementation while `xgboost` was installed\nfrom pypi.org and links against GNU OpenMP (`libgomp`) so both OpenMP runtimes are\nloaded in the same Python program.\n\nThe state of these libraries is also accessible through the object oriented API:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import ThreadpoolController, threadpool_info\n\u003e\u003e\u003e from pprint import pprint\n\u003e\u003e\u003e import numpy\n\u003e\u003e\u003e controller = ThreadpoolController()\n\u003e\u003e\u003e pprint(controller.info())\n[{'architecture': 'Haswell',\n  'filepath': '/home/jeremie/miniconda/envs/dev/lib/libopenblasp-r0.3.17.so',\n  'internal_api': 'openblas',\n  'num_threads': 4,\n  'prefix': 'libopenblas',\n  'threading_layer': 'pthreads',\n  'user_api': 'blas',\n  'version': '0.3.17'}]\n\n\u003e\u003e\u003e controller.info() == threadpool_info()\nTrue\n```\n\n### Setting the Maximum Size of Thread-Pools\n\nControl the number of threads used by the underlying runtime libraries\nin specific sections of your Python program:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import threadpool_limits\n\u003e\u003e\u003e import numpy as np\n\n\u003e\u003e\u003e with threadpool_limits(limits=1, user_api='blas'):\n...     # In this block, calls to blas implementation (like openblas or MKL)\n...     # will be limited to use only one thread. They can thus be used jointly\n...     # with thread-parallelism.\n...     a = np.random.randn(1000, 1000)\n...     a_squared = a @ a\n```\n\nThe threadpools can also be controlled via the object oriented API, which is especially\nuseful to avoid searching through all the loaded shared libraries each time. It will\nhowever not act on libraries loaded after the instantiation of the\n`ThreadpoolController`:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import ThreadpoolController\n\u003e\u003e\u003e import numpy as np\n\u003e\u003e\u003e controller = ThreadpoolController()\n\n\u003e\u003e\u003e with controller.limit(limits=1, user_api='blas'):\n...     a = np.random.randn(1000, 1000)\n...     a_squared = a @ a\n```\n\n### Restricting the limits to the scope of a function\n\n`threadpool_limits` and `ThreadpoolController` can also be used as decorators to set\nthe maximum number of threads used by the supported libraries at a function level. The\ndecorators are accessible through their `wrap` method:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import ThreadpoolController, threadpool_limits\n\u003e\u003e\u003e import numpy as np\n\u003e\u003e\u003e controller = ThreadpoolController()\n\n\u003e\u003e\u003e @controller.wrap(limits=1, user_api='blas')\n... # or @threadpool_limits.wrap(limits=1, user_api='blas')\n... def my_func():\n...     # Inside this function, calls to blas implementation (like openblas or MKL)\n...     # will be limited to use only one thread.\n...     a = np.random.randn(1000, 1000)\n...     a_squared = a @ a\n...\n```\n\n### Switching the FlexiBLAS backend\n\n`FlexiBLAS` is a BLAS wrapper for which the BLAS backend can be switched at runtime.\n`threadpoolctl` exposes python bindings for this feature. Here's an example but note\nthat this part of the API is experimental and subject to change without deprecation:\n\n```python\n\u003e\u003e\u003e from threadpoolctl import ThreadpoolController\n\u003e\u003e\u003e import numpy as np\n\u003e\u003e\u003e controller = ThreadpoolController()\n\n\u003e\u003e\u003e controller.info()\n[{'user_api': 'blas',\n  'internal_api': 'flexiblas',\n  'num_threads': 1,\n  'prefix': 'libflexiblas',\n  'filepath': '/usr/local/lib/libflexiblas.so.3.3',\n  'version': '3.3.1',\n  'available_backends': ['NETLIB', 'OPENBLASPTHREAD', 'ATLAS'],\n  'loaded_backends': ['NETLIB'],\n  'current_backend': 'NETLIB'}]\n\n# Retrieve the flexiblas controller\n\u003e\u003e\u003e flexiblas_ct = controller.select(internal_api=\"flexiblas\").lib_controllers[0]\n\n# Switch the backend with one predefined at build time (listed in \"available_backends\")\n\u003e\u003e\u003e flexiblas_ct.switch_backend(\"OPENBLASPTHREAD\")\n\u003e\u003e\u003e controller.info()\n[{'user_api': 'blas',\n  'internal_api': 'flexiblas',\n  'num_threads': 4,\n  'prefix': 'libflexiblas',\n  'filepath': '/usr/local/lib/libflexiblas.so.3.3',\n  'version': '3.3.1',\n  'available_backends': ['NETLIB', 'OPENBLASPTHREAD', 'ATLAS'],\n  'loaded_backends': ['NETLIB', 'OPENBLASPTHREAD'],\n  'current_backend': 'OPENBLASPTHREAD'},\n {'user_api': 'blas',\n  'internal_api': 'openblas',\n  'num_threads': 4,\n  'prefix': 'libopenblas',\n  'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so',\n  'version': '0.3.8',\n  'threading_layer': 'pthreads',\n  'architecture': 'Haswell'}]\n\n# It's also possible to directly give the path to a shared library\n\u003e\u003e\u003e flexiblas_controller.switch_backend(\"/home/jeremie/miniforge/envs/flexiblas_threadpoolctl/lib/libmkl_rt.so\")\n\u003e\u003e\u003e controller.info()\n[{'user_api': 'blas',\n  'internal_api': 'flexiblas',\n  'num_threads': 2,\n  'prefix': 'libflexiblas',\n  'filepath': '/usr/local/lib/libflexiblas.so.3.3',\n  'version': '3.3.1',\n  'available_backends': ['NETLIB', 'OPENBLASPTHREAD', 'ATLAS'],\n  'loaded_backends': ['NETLIB',\n   'OPENBLASPTHREAD',\n   '/home/jeremie/miniforge/envs/flexiblas_threadpoolctl/lib/libmkl_rt.so'],\n  'current_backend': '/home/jeremie/miniforge/envs/flexiblas_threadpoolctl/lib/libmkl_rt.so'},\n {'user_api': 'openmp',\n  'internal_api': 'openmp',\n  'num_threads': 4,\n  'prefix': 'libomp',\n  'filepath': '/home/jeremie/miniforge/envs/flexiblas_threadpoolctl/lib/libomp.so',\n  'version': None},\n {'user_api': 'blas',\n  'internal_api': 'openblas',\n  'num_threads': 4,\n  'prefix': 'libopenblas',\n  'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so',\n  'version': '0.3.8',\n  'threading_layer': 'pthreads',\n  'architecture': 'Haswell'},\n {'user_api': 'blas',\n  'internal_api': 'mkl',\n  'num_threads': 2,\n  'prefix': 'libmkl_rt',\n  'filepath': '/home/jeremie/miniforge/envs/flexiblas_threadpoolctl/lib/libmkl_rt.so.2',\n  'version': '2024.0-Product',\n  'threading_layer': 'gnu'}]\n```\n\nYou can observe that the previously linked OpenBLAS shared object stays loaded by\nthe Python program indefinitely, but FlexiBLAS itself no longer delegates BLAS calls\nto OpenBLAS as indicated by the `current_backend` attribute.\n### Writing a custom library controller\n\nCurrently, `threadpoolctl` has support for `OpenMP` and the main `BLAS` libraries.\nHowever it can also be used to control the threadpool of other native libraries,\nprovided that they expose an API to get and set the limit on the number of threads.\nFor that, one must implement a controller for this library and register it to\n`threadpoolctl`.\n\nA custom controller must be a subclass of the `LibController` class and implement\nthe attributes and methods described in the docstring of `LibController`. Then this\nnew controller class must be registered using the `threadpoolctl.register` function.\nAn complete example can be found [here](\n  https://github.com/joblib/threadpoolctl/blob/master/tests/_pyMylib/__init__.py).\n\n### Sequential BLAS within OpenMP parallel region\n\nWhen one wants to have sequential BLAS calls within an OpenMP parallel region, it's\nsafer to set `limits=\"sequential_blas_under_openmp\"` since setting `limits=1` and\n`user_api=\"blas\"` might not lead to the expected behavior in some configurations\n(e.g. OpenBLAS with the OpenMP threading layer\nhttps://github.com/xianyi/OpenBLAS/issues/2985).\n\n### Known Limitations\n\n- `threadpool_limits` can fail to limit the number of inner threads when nesting\n  parallel loops managed by distinct OpenMP runtime implementations (for instance\n  libgomp from GCC and libomp from clang/llvm or libiomp from ICC).\n\n  See the `test_openmp_nesting` function in [tests/test_threadpoolctl.py](\n  https://github.com/joblib/threadpoolctl/blob/master/tests/test_threadpoolctl.py)\n  for an example. More information can be found at:\n  https://github.com/jeremiedbb/Nested_OpenMP\n\n  Note however that this problem does not happen when `threadpool_limits` is\n  used to limit the number of threads used internally by BLAS calls that are\n  themselves nested under OpenMP parallel loops. `threadpool_limits` works as\n  expected, even if the inner BLAS implementation relies on a distinct OpenMP\n  implementation.\n\n- Using Intel OpenMP (ICC) and LLVM OpenMP (clang) in the same Python program\n  under Linux is known to cause problems. See the following guide for more details\n  and workarounds:\n  https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md\n\n- Setting the maximum number of threads of the OpenMP and BLAS libraries has a global\n  effect and impacts the whole Python process. There is no thread level isolation as\n  these libraries do not offer thread-local APIs to configure the number of threads to\n  use in nested parallel calls.\n\n\n## Maintainers\n\nTo make a release:\n\n- Create a PR to bump the version number (`__version__`) in `threadpoolctl.py` and\n  update the release date in `CHANGES.md`.\n\n- Merge the PR and check that the `Publish threadpoolctl distribution to TestPyPI` job\n  of the `publish-to-pypi.yml` workflow successfully uploaded the wheel and source\n  distribution to Test PyPI.\n\n- If everything is fine create a tag for the release and push it to github:\n\n  ```bash\n  git tag -a X.Y.Z\n  git push git@github.com:joblib/threadpoolctl.git X.Y.Z\n  ```\n\n- Check that the `Publish threadpoolctl distribution to PyPI` job of the\n  `publish-to-pypi.yml` workflow successfully uploaded the wheel and source distribution\n  to PyPI this time.\n\n- Create a PR for the release on the [conda-forge feedstock](https://github.com/conda-forge/threadpoolctl-feedstock) (or wait for the bot to make it).\n\n- Publish the release on github.\n\nIf for some reason the steps above can't be achieved and a munual upload of the wheel\nand source distribution is needed:\n\n- Build the distribution archives:\n\n  ```bash\n  pip install flit\n  flit build\n  ```\n\n- Upload the wheels and source distribution to PyPI using flit. Since PyPI doesn't\n  allow password authentication anymore, the username needs to be changed to the\n  generic name `__token__`:\n\n  ```bash\n  FLIT_USERNAME=__token__ flit publish\n  ```\n\n  and a PyPI token has to be passed in place of the password.\n\n### Credits\n\nThe initial dynamic library introspection code was written by @anton-malakhov\nfor the smp package available at https://github.com/IntelPython/smp .\n\nthreadpoolctl extends this for other operating systems. Contrary to smp,\nthreadpoolctl does not attempt to limit the size of Python multiprocessing\npools (threads or processes) or set operating system-level CPU affinity\nconstraints: threadpoolctl only interacts with native libraries via their\npublic runtime APIs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoblib%2Fthreadpoolctl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoblib%2Fthreadpoolctl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoblib%2Fthreadpoolctl/lists"}