{"id":15835017,"url":"https://github.com/kaelzhang/python-compton","last_synced_at":"2025-04-01T12:28:49.551Z","repository":{"id":62564226,"uuid":"248931841","full_name":"kaelzhang/python-compton","owner":"kaelzhang","description":"An abstract data flow framework for quantitative trading","archived":false,"fork":false,"pushed_at":"2021-02-03T14:38:45.000Z","size":123,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T13:16:18.916Z","etag":null,"topics":["data-flow","quant","quantitative-finance"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaelzhang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-21T08:00:59.000Z","updated_at":"2021-02-03T14:38:47.000Z","dependencies_parsed_at":"2022-11-03T16:32:14.728Z","dependency_job_id":null,"html_url":"https://github.com/kaelzhang/python-compton","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaelzhang%2Fpython-compton","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaelzhang%2Fpython-compton/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaelzhang%2Fpython-compton/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaelzhang%2Fpython-compton/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaelzhang","download_url":"https://codeload.github.com/kaelzhang/python-compton/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246404011,"owners_count":20771526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-flow","quant","quantitative-finance"],"created_at":"2024-10-05T14:04:42.962Z","updated_at":"2025-04-01T12:28:49.516Z","avatar_url":"https://github.com/kaelzhang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/kaelzhang/python-compton.svg?branch=master)](https://travis-ci.org/kaelzhang/python-compton)\n[![Coverage](https://codecov.io/gh/kaelzhang/python-compton/branch/master/graph/badge.svg)](https://codecov.io/gh/kaelzhang/python-compton)\n\u003c!-- optional appveyor tst\n[![Windows Build Status](https://ci.appveyor.com/api/projects/status/github/kaelzhang/python-compton?branch=master\u0026svg=true)](https://ci.appveyor.com/project/kaelzhang/python-compton)\n--\u003e\n\u003c!-- optional npm version\n[![NPM version](https://badge.fury.io/js/python-compton.svg)](http://badge.fury.io/js/python-compton)\n--\u003e\n\u003c!-- optional npm downloads\n[![npm module downloads per month](http://img.shields.io/npm/dm/python-compton.svg)](https://www.npmjs.org/package/python-compton)\n--\u003e\n\u003c!-- optional dependency status\n[![Dependency Status](https://david-dm.org/kaelzhang/python-compton.svg)](https://david-dm.org/kaelzhang/python-compton)\n--\u003e\n\n# python-compton\n\nAn abstract data-flow framework for quantitative trading, which decouples data initialization, data composition and data processing.\n\n## Install\n\n```sh\npip install compton\n```\n\n## Usage\n\n```py\nfrom compton import (\n  Orchestrator,\n  Provider,\n  Reducer,\n  Consumer\n)\n```\n\n## Vector\n\nWe call a tuple of hashable parameters as a vector which is used to identify a certain kind of data.\n\n```py\nfrom enum import Enum\n\nclass DataType(Enum):\n    KLINE = 1\n    ORDER_BOOK = 2\n\nclass TimeSpan(Enum):\n    DAY = 1\n    WEEK = 2\n\nvector = (DataType.KLINE, TimeSpan.DAY)\n```\n\n## Orchestrator(reducers, loop=None)\n\n- **reducers** `List[Reducer]` reducers to compose data\n- **loop?** `Optional[EventLoop]` The event loop object to use. In most cases, you should **NOT** pass this argument, unless you exact know what you are doing.\n\n```py\nOrchestrator(\n    reducers\n).connect(\n    provider\n).subscribe(\n    consumer\n).add('TSLA')\n```\n\n### orchestrator.connect(provider: Provider) -\u003e self\n\nConnects to a data provider\n\n### orchestrator.subscribe(consumer: Consumer) -\u003e self\n\nSubscribes the consumer to orchestrator.\n\n### orchestrator.add(symbol: str) -\u003e self\n\nAdds a new symbol to orchestrator, and start the data flow for `symbol`\n\n## Provider\n\n`Provider` is an abstract class which provides initial data and data updates.\n\nA provider should be implemented to support many symbols\n\nWe must inherit class `Provider` and implement some abstract method before use.\n\n- `@property vector` returns an `Vector`\n- `async def init()` method returns the initial data\n- There is an protected method `self.dispatch(symbol, payload)` to set the payload updated, which should only be called in a coroutine, or a `RuntimeError` is raised.\n\n```py\nclass MyProvider(Provider):\n    @property\n    def vector(self):\n        return (DataType.KLINE, TimeSpan.DAY)\n\n    async def init(self, symbol):\n        return {}\n```\n\n## Reducer\n\nAnother abstract class which handles data composition.\n\nThe `reducer.vector` could be a generic vector which applies partial match to other vectors\n\n```py\nclass MyReducer(Reducer):\n    @property\n    def vector(self):\n        # So, MyReducer support both\n        # - (DataType.KLINE, TimeSpan.DAY)\n        # - and (DataType.KLINE, TimeSpan.WEEK)\n        return (DataType.KLINE,)\n\n    def merge(self, old, new):\n        # `old` might be `None`, if `new` is the initial data\n        if old is None:\n            # We could clean the initial data\n            return clean(new)\n\n        return {**old, **new}\n```\n\n## Consumer\n\nA consumer could subscribes to more than one kind of data types\n\n```py\nclass MyConsumer(Consumer):\n    @property\n    def vectors(self):\n        # Subscribe to two kinds of data types\n        return [\n            (DataType.KLINE, TimeSpan.DAY),\n            (DataType.KLINE, TimeSpan.WEEK)\n        ]\n\n    @property\n    def all(self) -\u003e bool:\n        \"\"\"\n        `True` indicates that the consumer will only go processing\n        if both of the data corresponds with the two vectors have changes\n\n        And by default, `Consumer::all` is False\n        \"\"\"\n        return True\n\n    @property\n    def concurrency(self) -\u003e int:\n        \"\"\"\n        Concurrency limit for method `process()`\n\n        By default, `Consumer::concurrency` is `0` which means no limit\n        \"\"\"\n        return 1\n\n    def should_process(self, *payloads) -\u003e bool:\n        \"\"\"\n        If this method returns `False`, then the data update will not be processed\n        \"\"\"\n        return True\n\n    # Then there will be\n    # both `kline_day` and `kline_week` passed into method `process`\n    async def process(self, symbol, kline_day, kline_week):\n        await doSomething(symbol, kline_day, kline_week)\n```\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaelzhang%2Fpython-compton","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaelzhang%2Fpython-compton","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaelzhang%2Fpython-compton/lists"}