{"id":29435944,"url":"https://github.com/diegojromerolopez/backgroundlog","last_synced_at":"2026-01-20T16:48:27.231Z","repository":{"id":303892556,"uuid":"1014671650","full_name":"diegojromerolopez/backgroundlog","owner":"diegojromerolopez","description":"Thread-based log handler for better performance","archived":false,"fork":false,"pushed_at":"2025-07-10T05:22:42.000Z","size":29,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-10T10:19:12.355Z","etag":null,"topics":["library","log","logger","logging","non-blocking","performance","python","threading"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/diegojromerolopez.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-06T07:15:47.000Z","updated_at":"2025-07-10T05:22:45.000Z","dependencies_parsed_at":"2025-07-10T10:08:27.438Z","dependency_job_id":"12bfb620-1201-4d5d-81be-b679ee298ed7","html_url":"https://github.com/diegojromerolopez/backgroundlog","commit_stats":null,"previous_names":["diegojromerolopez/backgroundlog","diegojromerolopez/blackgroundlog"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/diegojromerolopez/backgroundlog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegojromerolopez%2Fbackgroundlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegojromerolopez%2Fbackgroundlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegojromerolopez%2Fbackgroundlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegojromerolopez%2Fbackgroundlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/diegojromerolopez","download_url":"https://codeload.github.com/diegojromerolopez/backgroundlog/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegojromerolopez%2Fbackgroundlog/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265087540,"owners_count":23709355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["library","log","logger","logging","non-blocking","performance","python","threading"],"created_at":"2025-07-13T04:11:09.949Z","updated_at":"2025-10-14T09:39:34.488Z","avatar_url":"https://github.com/diegojromerolopez.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# backgroundlog\nThread-based log handler for better performance\n\n![test](https://github.com/diegojromerolopez/backgroundlog/actions/workflows/test.yml/badge.svg)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/diegojromerolopez/backgroundlog/graphs/commit-activity)\n[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort/)\n[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n[![PyPI pyversions](https://img.shields.io/pypi/pyversions/backgroundlog.svg)](https://pypi.python.org/pypi/backgroundlog/)\n[![PyPI version backgroundlog](https://badge.fury.io/py/backgroundlog.svg)](https://pypi.python.org/pypi/backgroundlog/)\n[![PyPI status](https://img.shields.io/pypi/status/backgroundlog.svg)](https://pypi.python.org/pypi/backgroundlog/)\n[![PyPI download month](https://img.shields.io/pypi/dm/backgroundlog.svg)](https://pypi.python.org/pypi/backgroundlog/)\n\nDo not have your python program slowed down by your logging.\n\n## Introduction\nMost of the time we log on disk, and all I/O is time consuming.\nBy leveraging a thread, it is possible to speed up your python application\nconsiderably.\n\n## Use\n\n### Default use\n\n```python\nimport logging\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\n# Setting up the logging thread handler\nfile_handler = logging.FileHandler('/var/log/myapp.log', mode=\"a\", encoding=\"utf-8\")\nthread_handler = ThreadHandler(file_handler)\n\n# Creating a new logger\nbg_logger = logging.getLogger('bg_logger')\nbg_logger.setLevel(logging.INFO)\n\n# Adding the thread handler\nbg_logger.addHandler(thread_handler)\n\n# Using the logger\nbg_logger.info('This is a log message')\n```\n\n### Options\n\n#### Set a queue size\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, queue_size=5000)\n```\n\nBy default, the queue size is 1000.\n\n#### Set a blocking policy by logging record levels\n\nWhen putting the records in the queue, it could reach the queue size.\nWe provide a way to deal with this issue: set a blocking policy\nby logging record level, and in the case of a non-blocking policy,\nthe record will be discarded and we will increment a dropped log record.\n\n##### Only info, error and critical records are blocking:\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR, INFO\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={INFO, ERROR, CRITICAL})\n```\n\n##### Only error and critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL, ERROR\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={ERROR, CRITICAL})\n```\n\n##### Only critical records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\nfrom logging import CRITICAL\n\nthread_handler = ThreadHandler(file_handler, blocking_levels={CRITICAL})\n```\n\n##### No records are blocking\n\n```python\nfrom backgroundlog.handlers.thread_handler import ThreadHandler\n\nthread_handler = ThreadHandler(file_handler, blocking_levels=None)\n```\n\nBy default, the error and critical records are blocking, the rest are not.\n\n## Performance testing\n\nWe have done several local testing with different logging handlers.\nSee the file\n[run_performance_comparison.py](/backgroundlog/performance/run_performance_comparison.py) for\na full catalog of the performance tests we run.\n\nWe used two versions of Python depending on if they had or not the global interpret lock:\n- Python 3.15.3 (with GIL)\n- Python 3.15.3t (without GIL)\n\nAll tests are 100_000 iterations of creating the same logging message,\nand were run in a Macbook Pro M1 with 16 GB of RAM.\n\n### Python 3.15.3 (with GIL)\n\n| Logging Handler               | Spent Time     |              | vs. Baseline |\n|-------------------------------|----------------|--------------|--------------|\n|                               | Mean Time (ms) | Std Dev (ms) |              |\n| StreamHandler                 | 0.685          | 0.006        | baseline     |\n| FileHandler                   | 0.685          | 0.018        | +0.03%       |\n| ThreadHandler (StreamHandler) | 0.487          | 0.03         | -28.911%     |\n| ThreadHandler (FileHandler)   | 0.475          | 0.002        | -30.66%      |\n\nThere is a ~30% of improvement when running the thread handler.\n\n### Python 3.15.3t (without GIL)\n\n| Logging Handler               | Spent Time     |              | vs. Baseline |\n|-------------------------------|----------------|--------------|--------------|\n|                               | Mean Time (ms) | Std Dev (ms) |              |\n| StreamHandler                 | 0.539          | 0.004        | baseline     |\n| FileHandler                   | 0.545          | 0.013        | +1.109%      |\n| ThreadHandler (StreamHandler) | 0.344          | 0.002        | -36.301%     |\n| ThreadHandler (FileHandler)   | 0.339          | 0.001        | -37.118%     |\n\nThere is a ~36% of improvement when running the thread handler. +6% with respect of the\nPython version with GIL.\n\n### Conclusions\nNot blocking the main flow of your program by making use of backgroundlog\ngives you a 30% speed gain.\n\n## Dependencies\nThis package has no dependencies.\n\n## Python version support\nMinimum version support is 3.10.\n\n## License\n[MIT](LICENSE) license, but if you need any other contact me.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiegojromerolopez%2Fbackgroundlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiegojromerolopez%2Fbackgroundlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiegojromerolopez%2Fbackgroundlog/lists"}