{"id":27979389,"url":"https://github.com/shivkun/corstream","last_synced_at":"2025-05-08T02:52:45.274Z","repository":{"id":291791034,"uuid":"978796412","full_name":"shivkun/corstream","owner":"shivkun","description":"A coroutine composition framework for declarative async pipelines in Python","archived":false,"fork":false,"pushed_at":"2025-05-06T19:57:27.000Z","size":36,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-08T02:52:40.683Z","etag":null,"topics":["async","composition","coroutine","declarative","framework","pipeline","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shivkun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"shivkun"}},"created_at":"2025-05-06T14:18:56.000Z","updated_at":"2025-05-06T19:57:29.000Z","dependencies_parsed_at":"2025-05-06T15:46:10.213Z","dependency_job_id":"deba615a-9dbf-421d-95c0-794d0315ae61","html_url":"https://github.com/shivkun/corstream","commit_stats":null,"previous_names":["shivkun/corstream"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shivkun%2Fcorstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shivkun%2Fcorstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shivkun%2Fcorstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shivkun%2Fcorstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shivkun","download_url":"https://codeload.github.com/shivkun/corstream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252989963,"owners_count":21836667,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","composition","coroutine","declarative","framework","pipeline","python"],"created_at":"2025-05-08T02:52:44.724Z","updated_at":"2025-05-08T02:52:45.264Z","avatar_url":"https://github.com/shivkun.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\r\n  \u003ca href=\"\" rel=\"noopener\"\u003e\r\n \u003cimg src=\"https://4jvunv3uhj.ufs.sh/f/j3njteUqvFKuYLYN4feQPcuZI9TeKXw4Cvol0qBhSHkiFJxN\" alt=\"CorStream Logo\"\u003e\u003c/a\u003e\r\n\u003c/p\u003e\r\n\r\n\u003ch3 align=\"center\"\u003eCorStream\u003c/h3\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n\r\n[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/)\r\n[![Status](https://img.shields.io/badge/status-active-success.svg)]()\r\n[![Last Commit](https://img.shields.io/github/last-commit/shivkun/corstream/main)]()\r\n[![Contributors](https://img.shields.io/github/contributors/shivkun/corstream)]()\r\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](/LICENSE)\r\n\r\n\r\n\u003c/div\u003e\r\n\r\n---\r\n\r\n\u003cp align=\"center\"\u003e\r\n    A coroutine composition framework for Python that enables declarative, streaming-style async pipelines.\r\n\u003c/p\u003e\r\n\r\n## 📝 Table of Contents\r\n\r\n- [About](#about)\r\n- [Getting Started](#getting_started)\r\n- [Deployment](#deployment)\r\n- [Usage](#usage)\r\n- [Built Using](#built_using)\r\n- [TODO](../TODO.md)\r\n- [Contributing](../CONTRIBUTING.md)\r\n- [Authors](#authors)\r\n- [Acknowledgments](#acknowledgement)\r\n\r\n## 🧐 About \u003ca name = \"about\"\u003e\u003c/a\u003e\r\n\r\n**CorStream** is a Python library that lets you build elegant, composable pipelines using asynchronous iterables and coroutine-based operators. Think of it like a blend of `asyncio`, `RxPy`, and Unix pipes — but designed for readability, type safety, and modern Python development.\r\n\r\nWith CorStream, you can easily:\r\n- Transform and filter async data flows\r\n- Batch, throttle, or log stream data\r\n- Apply async functions with concurrency control\r\n- Collect or reduce outputs with simple syntax\r\n\r\n## 🏁 Getting Started \u003ca name = \"getting_started\"\u003e\u003c/a\u003e\r\n\r\nThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.\r\n\r\n### Prerequisites\r\n\r\nYou’ll need Python 3.9 or higher and [Poetry](https://python-poetry.org/) installed:\r\n\r\n```\r\npython3 --version\r\n# Should be 3.9+\r\n\r\ncurl -sSL https://install.python-poetry.org | python3 -\r\n```\r\n\r\n### Installing\r\n\r\nClone the repo and install dependencies with Poetry:\r\n\r\n```\r\ngit clone https://github.com/shivkun/corstream.git\r\ncd corstream\r\npoetry install\r\n```\r\n\r\nCorStream is also published on PyPI, you can install it directly with:\r\n\r\n```\r\npip install corstream\r\n# or use Poetry\r\npoetry add corstream\r\n```\r\n\r\nYou can now run tests, examples, or start building pipelines!\r\n\r\n## 🔧 Running the tests \u003ca name = \"tests\"\u003e\u003c/a\u003e\r\n\r\nTo run all tests:\r\n\r\n```\r\npoetry run pytest\r\n```\r\n\r\n### Break down into end-to-end tests\r\n\r\nEach operator and sink has its own test file under `tests/`.\r\n\r\nExample:\r\n\r\n```\r\ntests/test_map.py          # Tests for .map()\r\ntests/test_batch.py        # Tests for .batch()\r\ntests/test_to_list.py      # Tests for .to_list()\r\n```\r\n\r\n### And coding style tests\r\n\r\nCorStream follows strict linting and formatting with `black`, `mypy`, and `ruff`:\r\n\r\n```\r\npoetry run black corstream/\r\npoetry run ruff check corstream/\r\npoetry run mypy corstream/\r\n```\r\n\r\n## 🎈 Usage \u003ca name=\"usage\"\u003e\u003c/a\u003e\r\n\r\nHere's a simple pipeline:\r\n\r\n```python\r\nfrom corstream import Stream\r\n\r\nasync def get_email(user_id: int) -\u003e str:\r\n    return f\"user{user_id}@example.com\"\r\n\r\nasync def send_batch(batch: list[str]) -\u003e None:\r\n    print(\"Sending:\", batch)\r\n\r\nawait (\r\n    Stream\r\n    .from_iterable(range(1, 11))\r\n    .filter(lambda x: x % 2 == 0)\r\n    .map_async(get_email, max_concurrent=3)\r\n    .batch(5)\r\n    .log(label=\"batch\")\r\n    .for_each(send_batch)\r\n)\r\n```\r\n\r\n## ⛏️ Built Using \u003ca name = \"built_using\"\u003e\u003c/a\u003e\r\n\r\n- [Python 3.9+](https://www.python.org/)\r\n- [Poetry](https://python-poetry.org/)\r\n- [AsyncIO](https://docs.python.org/3/library/asyncio.html)\r\n- [Pytest](https://docs.pytest.org/)\r\n\r\n## ✍️ Authors \u003ca name = \"authors\"\u003e\u003c/a\u003e\r\n\r\n- [@shivkun](https://github.com/shivkun) — Design \u0026 Implementation\r\n\r\nSee also the list of [contributors](https://github.com/shivkun/corstream/contributors).\r\n\r\n## 🎉 Acknowledgements \u003ca name = \"acknowledgement\"\u003e\u003c/a\u003e\r\n\r\n- Inspiration from functional programming and reactive streams\r\n- Thanks to the maintainers of `asyncio`, `RxPy`, and `toolz`\r\n","funding_links":["https://github.com/sponsors/shivkun"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshivkun%2Fcorstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshivkun%2Fcorstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshivkun%2Fcorstream/lists"}