{"id":22940577,"url":"https://github.com/maxfischer2781/chainlet","last_synced_at":"2025-04-01T20:33:04.504Z","repository":{"id":62561647,"uuid":"81252576","full_name":"maxfischer2781/chainlet","owner":"maxfischer2781","description":"Python module for linking generators/iterators to processing chains","archived":false,"fork":false,"pushed_at":"2018-06-12T08:14:43.000Z","size":540,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-07T17:18:53.467Z","etag":null,"topics":["chain","iterator","pipeline","processing-chain","python2","python3"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maxfischer2781.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-07T20:47:35.000Z","updated_at":"2018-06-12T08:14:44.000Z","dependencies_parsed_at":"2022-11-03T15:15:44.780Z","dependency_job_id":null,"html_url":"https://github.com/maxfischer2781/chainlet","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxfischer2781%2Fchainlet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxfischer2781%2Fchainlet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxfischer2781%2Fchainlet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxfischer2781%2Fchainlet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maxfischer2781","download_url":"https://codeload.github.com/maxfischer2781/chainlet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246709923,"owners_count":20821297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chain","iterator","pipeline","processing-chain","python2","python3"],"created_at":"2024-12-14T13:23:46.982Z","updated_at":"2025-04-01T20:33:04.478Z","avatar_url":"https://github.com/maxfischer2781.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Chainlet\n\n[![Documentation Status](https://readthedocs.org/projects/chainlet/badge/?version=latest)](http://chainlet.readthedocs.io/en/latest/?badge=latest)\n[![Build Status](https://travis-ci.org/maxfischer2781/chainlet.svg?branch=master)](https://travis-ci.org/maxfischer2781/chainlet)\n[![Code Health](https://landscape.io/github/maxfischer2781/chainlet/master/landscape.svg?style=flat)](https://landscape.io/github/maxfischer2781/chainlet/master)\n[![codecov](https://codecov.io/gh/maxfischer2781/chainlet/branch/master/graph/badge.svg)](https://codecov.io/gh/maxfischer2781/chainlet)\n\nFramework for linking generator/iterators to create processing chains and pipelines.\nWith its operator based syntax, it is easy to create complex sequences from simple building blocks.\nChainlets are suitable for incremental, iterative and stream processing and beyond.\n\n## Simplistic Chains with Chainlets\n\nConsider the following use case:\nRead data from an XML, flatten it, then write to a csv.\nThis can be expressed with a chain of generators:\n\n    csv_writer(flatten(xml_reader(path='data.xml'), join='.'.join), path='data.csv')\n\nWhen written using chainlets, generator sequence and arguments are much easier to read.\nThe chainlets are joined using the `\u003e\u003e` operator:\n\n    xml_reader(path='data.xml') \u003e\u003e flatten(join='.'.join) \u003e\u003e csv_writer(path='data.csv')\n\nIn addition, chainlets can be composed much more freely.\nInstead of deeply nested call structures, chainlets have simple, flat call sequences.\n\n## Custom Chainlets for Pipelines\n\nWriting new chainlets does not require any special techniques or conventions.\nYou can directly convert existing coroutines, functions and objects.\nImplementing a moving average requires exactly one line specific to the ``chainlet`` library:\n\n    @chainlet.genlet\n    def moving_average(window_size=8):\n        buffer = collections.deque([(yield)], maxlen=window_size)\n        while True:\n            new_value = yield(sum(buffer)/len(buffer))\n            buffer.append(new_value)\n\nAll the gluing and binding is done automatically for you.\nInstead of bloating existing code, it is often easier to create and bind another simple chainlet.\n\n## Extended Pipelines with Chainlets\n\nChainlets are not limited to 1-to-1 relations, but actually allow n-to-n links.\nEach link can have multiple parents and children.\nThe following example reads XML messages via UDP, and logs them in two different verbosity levels. \n\n    udp_digest(port=31137) \u003e\u003e xml_converter()  \u003e\u003e (\n            json_writer(path='raw.json'),\n            moving_average(window_size=60) \u003e\u003e json_writer(path='avg1m.json'),\n        )\n\n## Quick Overview\n\n* The *smallest* building blocks is a `ChainLink`, ready to be subclassed and made _**big**_\n* Generate awesome *generator* pipelines, and let `GeneratorLink` put that awesome into use\n* State is for wusses, real programmers use functions; real programmers use `FunctionLink`\n* Don't wrap your head with wrappers, wrap with decorators - `linklet` and let link\n\n## Tell me more!\n\nChainlets are simple at their core, and quick to understand.\nIf you want to know more, just read the fabulous manual:\n[![Documentation Status](https://readthedocs.org/projects/chainlet/badge/?version=latest)](http://chainlet.readthedocs.io/en/latest/?badge=latest)\n\nThe module is hosted on [github](https://github.com/maxfischer2781/chainlet).\nIf you have issues or want to propose changes, check out the [issue tracker](https://github.com/maxfischer2781/chainlet/issues).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxfischer2781%2Fchainlet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaxfischer2781%2Fchainlet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxfischer2781%2Fchainlet/lists"}