{"id":18011265,"url":"https://github.com/sloisel/streamcapture","last_synced_at":"2025-03-26T15:32:39.887Z","repository":{"id":57471722,"uuid":"351519968","full_name":"sloisel/streamcapture","owner":"sloisel","description":"Capture the outputs of Python streams, such as sys.stdout and sys.stderr","archived":false,"fork":false,"pushed_at":"2024-07-26T03:42:14.000Z","size":140,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-30T04:51:34.536Z","etag":null,"topics":["logging","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sloisel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-25T17:25:31.000Z","updated_at":"2024-07-26T03:42:17.000Z","dependencies_parsed_at":"2024-11-18T22:19:59.727Z","dependency_job_id":"2702b711-ece4-4d53-b112-13c63586b579","html_url":"https://github.com/sloisel/streamcapture","commit_stats":{"total_commits":22,"total_committers":4,"mean_commits":5.5,"dds":0.5909090909090908,"last_synced_commit":"a38984f6c64f8f9f0605c80bb0f14ccc85955495"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sloisel%2Fstreamcapture","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sloisel%2Fstreamcapture/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sloisel%2Fstreamcapture/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sloisel%2Fstreamcapture/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sloisel","download_url":"https://codeload.github.com/sloisel/streamcapture/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245681533,"owners_count":20655215,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","python"],"created_at":"2024-10-30T03:08:46.973Z","updated_at":"2025-03-26T15:32:39.558Z","avatar_url":"https://github.com/sloisel.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n## streamcapture: capture the outputs of Python streams, such as sys.stdout and sys.stderr\n\n### Author: Sébastien Loisel\n\n[![Build Status](https://github.com/sloisel/streamcapture/actions/workflows/CI.yml/badge.svg?branch=master)](https://github.com/sloisel/streamcapture/actions/workflows/CI.yml?query=branch%3Amaster)\n[![codecov](https://codecov.io/gh/sloisel/streamcapture/graph/badge.svg?token=NO478ARVX6)](https://codecov.io/gh/sloisel/streamcapture)\n\n# Installation\n\n`pip install streamcapture`\n\n# Example usage\n\n```python\nimport streamcapture, sys, os\nprint(\"This does not get saved to the log file\")\nwith streamcapture.StreamCapture(sys.stdout,open('logfile.txt','wb')):\n        os.write(sys.stdout.fileno(),b\"Hello, captured world!\\n\")\n        os.system('echo Hello from the shell')\n        print(\"More capturing\")\nprint(\"This also does not get saved to the log file\")\n```\n\nAfter execution, this will create a file `logfile.txt` in the current directory, containing\nthe relevant captured outputs.\n\n# Documentation\n\nClass `StreamCapture(stream, writer, echo=True, monkeypatch=None)` is able to capture,\nat the operating system level, the data being written to the given `stream`.\nA typical use is to capture all outputs to `sys.stdout` and `sys.stderr`\nand log them to a file. This will even capture the outputs of spawned shell commands.\n\n`StreamCapture` works by essentially using `os.dup2` to send `stream.fileno()` to a `os.pipe()`.\nA separate thread is used to listen to that `os.pipe` and send the outputs to the destination\n`writer` stream. `StreamCapture` also uses `os.dup` to duplicate the original filedescriptor to be able\nto restore it at the end. This duplicated filedescriptor is stored in `StreamCapture.dup_fd`, and\nwriting to this filedescriptor results in writing to the original file, before it was redirected.\nFor example, when redirecting `sys.stdout`, one can still write to the terminal by writing directly\nto `StreamCapture.dup_fd` with `os.write()`.\n\nOn Windows, `sys.stdout` and `sys.stderr` do not take kindly to their `fileno()` being\nredirected with `os.dup2`. `StreamCapture` features an optional workaround, enabled by the\n`monkeypatch` optional parameter to the constructor. When enabled, the workaround\noverwrites `stream.write(...)` by an implementation that sends everything to `os.write(self.fd,...)`.\nThis workaround is enabled when `monkeypatch=True` and disabled when `monkeypatch=False`.\nThe default is `monkeypatch=None`, in which case monkeypatching is enabled only when \n`platform.system()=='Windows'`.\n\nWhen writing to multiple streams and file descriptors, sometimes the order in which the writes\nappear can be surprising. For example, when writing to stderr and stdout, these outputs do not\nnecessarily appear in the order in which they occurred during the program execution, because\nof various levels of buffering that occur in Python, the C library or the operating system.\n\nAt the Python level, streams can be `flush()`ed to attempt to reduce the delay before a `write()`\nhas delivered its payload. Furthermore, `os.fsync()` can be used on some, but not all, file descriptors.\nHowever, `os.fsync()` usually causes an exception if it is called on `sys.stdout.fileno()` or on a\n`os.pipe()`. In principle, the operating system should promtly flush any buffers when a file descriptor\nis `os.close()`d, but there is no guarantee. To complicate matters, although one usually prefers minimal\nbuffering for outputs that go to the console, Python tries very hard to force some sort of buffering on\ntext-mode files.\n\nWe have tried to prevent most forms of buffering at the Python level and at the operating system levels,\nbut when multiple file descriptors are used, or at the boundary when a `StreamCapture` starts or stops\ncapturing the underlying stream, some outputs that go to the console may appear in an unexpected order.\n\nMore sophisticated behaviors can be handled by implementing a custom stream-like object.\nThe `writer` object should implement functions `writer.write(data)`, where `data` is a byte string,\nand `writer.close()`.\n\nThe `echo` flag can be set at construction time `StreamCapture(...,echo=True)` and defaults to `True`.\nIn this mode, all captured outputs are sent both to the `writer` and also to `StreamCapture.dup_fd`.\nThis allows one to send, e.g. `stdout` to a log file while simultaneously printing it to the console,\nsimilar to the `tee` console command in Unix. The `echo` flag can be set to `False` to disable this.\n\nOne can call `StreamCapture.close()` to cleanly unwind the captured streams. This is automatically\ndone if `StreamCapture` is used in a `with` block.\n\nOne may also wish to capture a filedescriptor without the overhead of a wrapping Python stream.\nTo that end, one may use `FDCapture(fd,writer,echo=True)`. The parameter `fd` is an integer filedescriptor\nto be captured. `StreamCapture` is a thin wrapper around `FDCapture`, it mainly adds the monkeypatching\ncapability.\n\n`streamcapture.Writer` is a thin wrapper around an underlying stream, that allows sharing a stream\nbetween multiple threads in a thread-safe manner, guaranteeing that the underlying stream is closed\nonly when all threads have called `close`. `Writer` objects are constructed by\n`streamcapture.Writer(stream,count,lock_write = False)`.\n\n`stream`: is a stream that is being wrapped, e.g. `stream = open('logfile.txt','wb')`\n\n`count`: is the number of times that `Writer.close()` will be called before the writer\nis finally closed. This is so that a single stream can be used from multiple threads.\n\n`lock_write`: set this to `True` if you want calls to `stream.write()` to be serialized.\nThis causes `Writer.write` to acquire `Writer.lock` before calling `stream.write`.\nIf `lock_write=False` then `Writer.lock` is not acquired. Use this when `stream.write` is\nthread-safe. `lock_write=False` is the default.\n\nExample usage:\n```python\nimport sys, streamcapture\nwriter = streamcapture.Writer(open('logfile.txt','wb'),2)\nwith streamcapture.StreamCapture(sys.stdout,writer), streamcapture.StreamCapture(sys.stderr,writer):\n\tprint(\"This goes to stdout and is captured to logfile.txt\")\n\tprint(\"This goes to stderr and is also captured to logfile.txt\",file=sys.stderr)\n```\n\nIn the above example, writer will be closed twice: once from the `StreamCapture(sys.stdout,...)`\nobject, and once from the `StreamCapture(sys.stderr,...)` object. Correspondingly, the `count` parameter\nof the `streamcapture.Writer` was set to `2`, so that the underlying stream is only closed after 2\ncalls to `writer.close()`.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsloisel%2Fstreamcapture","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsloisel%2Fstreamcapture","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsloisel%2Fstreamcapture/lists"}