{"id":22371318,"url":"https://github.com/leenr/gzip-stream","last_synced_at":"2025-04-06T19:11:24.300Z","repository":{"id":45562136,"uuid":"198476939","full_name":"leenr/gzip-stream","owner":"leenr","description":"Streaming GZIP compression for Python","archived":false,"fork":false,"pushed_at":"2024-10-17T11:14:16.000Z","size":23,"stargazers_count":31,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-30T18:09:56.748Z","etag":null,"topics":["compression","gzip","python","python-library","python3"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leenr.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-23T17:21:00.000Z","updated_at":"2024-11-06T02:13:43.000Z","dependencies_parsed_at":"2024-12-16T16:29:50.234Z","dependency_job_id":"dfce4abd-1a48-4900-8488-969d68afe1ae","html_url":"https://github.com/leenr/gzip-stream","commit_stats":{"total_commits":13,"total_committers":4,"mean_commits":3.25,"dds":0.3076923076923077,"last_synced_commit":"5161a19b67015d819109666f53fd2152f1e7029f"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leenr%2Fgzip-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leenr%2Fgzip-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leenr%2Fgzip-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leenr%2Fgzip-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leenr","download_url":"https://codeload.github.com/leenr/gzip-stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247535517,"owners_count":20954576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","gzip","python","python-library","python3"],"created_at":"2024-12-04T20:18:53.429Z","updated_at":"2025-04-06T19:11:24.277Z","avatar_url":"https://github.com/leenr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"===========\ngzip-stream\n===========\n\n`gzip-stream` is a super-tiny library that will help you compress by GZIP\non-the-fly.\n\n`GZIPCompressedStream` class instance acting like an any other stream (in fact,\n`GZIPCompressedStream` inherits `io.RawIOBase \u003chttps://docs.python.org/3/library/io.html#io.RawIOBase\u003e`_),\nbut wraps another stream and compress it on-the-fly.\n\n.. code-block:: python\n\n    from gzip_stream import GZIPCompressedStream\n    from my_upload_lib import MyUploadClient\n\n    upload_client = MyUploadClient()\n    with open('my_very_big_1tb_file.txt') as file_to_upload:\n        compressed_stream = GZIPCompressedStream(\n            file_to_upload,\n            compression_level=7\n        )\n        upload_client.upload_fileobj(compressed_stream)\n\n`GZIPCompressedStream` does not read entire stream, but instead read it\nby chunks, until compressed output size will not satisfy read size.\n\n`AsyncGZIPDecompressedStream` class can async read from another source\nwith zlib and gzip decompression on-the-fly\n\n.. code-block:: python\n\n    # aiobotocore example\n\n    import aiobotocore\n\n    from gzip_stream import AsyncGZIPDecompressedStream\n\n    AWS_ACCESS_KEY_ID = \"KEY_ID\"\n    AWS_SECRET_ACCESS_KEY = \"ACCESS_KEY\"\n    BUCKET = \"AWESOME_BUCKET\"\n\n    upload_client = MyAsyncUploadClient()\n    session = aiobotocore.get_session()\n    async with session.create_client(\n        service_name=\"s3\",\n        endpoint_url=\"s3_endpoint\",\n        aws_secret_access_key=AWS_SECRET_ACCESS_KEY,\n        aws_access_key_id=AWS_ACCESS_KEY_ID,\n    ) as client:\n        response = await client.get_object(Bucket=BUCKET, Key='my_very_big_1tb_file.txt.gz')\n        async for decompressed_chunk in AsyncGZIPDecompressedStream(response[\"Body\"])):\n            await upload_client.upload_fileobj(decompressed_chunk)\n\n\nModule works on Python ~= 3.5.\n\nInstallation\n------------\n.. code-block:: bash\n\n    pip install gzip-stream\n\n\nLicense\n-------\nPublic Domain: `CC0 1.0 Universal \u003chttps://creativecommons.org/publicdomain/zero/1.0/\u003e`_.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleenr%2Fgzip-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleenr%2Fgzip-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleenr%2Fgzip-stream/lists"}