{"id":13844446,"url":"https://github.com/BitcoinExchangeFH/BitcoinExchangeFH","last_synced_at":"2025-07-11T22:32:06.616Z","repository":{"id":37759254,"uuid":"71792047","full_name":"BitcoinExchangeFH/BitcoinExchangeFH","owner":"BitcoinExchangeFH","description":"Cryptocurrency exchange market data feed handler","archived":false,"fork":false,"pushed_at":"2023-11-02T21:24:31.000Z","size":6097,"stargazers_count":936,"open_issues_count":14,"forks_count":296,"subscribers_count":88,"default_branch":"develop","last_synced_at":"2024-11-20T00:09:56.317Z","etag":null,"topics":["bitcoin","bitfinex","bitmex","bitstamp","btcc","exchange-data","gatecoin","gdax","huobi","kraken","market-data","okcoin","quoine"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BitcoinExchangeFH.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-10-24T13:30:31.000Z","updated_at":"2024-11-10T00:52:47.000Z","dependencies_parsed_at":"2023-10-20T18:27:17.872Z","dependency_job_id":null,"html_url":"https://github.com/BitcoinExchangeFH/BitcoinExchangeFH","commit_stats":{"total_commits":28,"total_committers":6,"mean_commits":4.666666666666667,"dds":0.7142857142857143,"last_synced_commit":"e1fd5e6412d413c29e944adad9407afc5b560272"},"previous_names":["gavincyi/bitcoinexchangefh"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitcoinExchangeFH%2FBitcoinExchangeFH","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitcoinExchangeFH%2FBitcoinExchangeFH/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitcoinExchangeFH%2FBitcoinExchangeFH/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitcoinExchangeFH%2FBitcoinExchangeFH/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BitcoinExchangeFH","download_url":"https://codeload.github.com/BitcoinExchangeFH/BitcoinExchangeFH/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225763419,"owners_count":17520456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcoin","bitfinex","bitmex","bitstamp","btcc","exchange-data","gatecoin","gdax","huobi","kraken","market-data","okcoin","quoine"],"created_at":"2024-08-04T17:02:42.522Z","updated_at":"2024-11-21T16:31:31.458Z","avatar_url":"https://github.com/BitcoinExchangeFH.png","language":"Python","funding_links":[],"categories":["Python (1887)","Python"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/icon.jpg\"\u003e\n\u003c/p\u003e\n\n# BitcoinExchangeFH - Cryptocurrency exchange market data feed handler\n\nBitcoinExchangeFH is a slim application to record the price depth and trades in various exchanges. You can set it up quickly and record the all the exchange data in a few minutes!\n\nUsers can\n\n1. Streaming market data to a target application (via ZeroMQ)\n2. Recording market data for backtesting and analysis.\n3. Recording market data to a in-memory database and other applications can quickly access to it.\n4. Customize the project for trading use.\n\n### MySQL\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/sample.jpg\"\u003e\n\u003c/p\u003e\n\n### Kdb+\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/sample2.jpg\"\u003e\n\u003c/p\u003e\n\n## Supported exchanges\n\nAll exchanges supported by [ccxt](https://github.com/ccxt/ccxt). Currently more than 130 exchanges are supported.\n\nWebsocket feeds of the following exchanges are supported by [cryptofeed](https://github.com/bmoscon/cryptofeed)\n\n- Bitfinex\n\n- Coinbase\n\n- Poloniex\n\n- Gemini\n\n- HitBTC\n\n- Bitstamp\n\n- BitMEX\n\n- Kraken\n\n- Binance\n\n- EXX\n\n- Huobi\n\n- OKCoin\n\n- OKEx\n\n\nIf the exchange is not supported with websocket API feed, it will automatically fall into using its REST API feed.\n\n\n## Supported database/channel\n\n- RDMBS (e.g. sqlite, MySQL, PostgreSQL)\n\n- ZeroMQ\n\n- Kdb+ (Coming soon)\n\n## Getting started\n\n```\npip install bitcoinexchangefh\nbitcoinexchangefh --configuration example/configuration.yaml\n```\n\n## Configuration\n\nThe configuration follows [YAML](https://pyyaml.org/wiki/PyYAMLDocumentation) syntax and contains two sections\n\n- subscriptions\n\n- handlers\n\n\n### Subscriptions\n\nSubscription section specifies the exchange and instruments to subscribe. \n\nThe first key is the exchange name and then follows the exchange details, \n    \n- instruments \n\n- number of depth (default is 5 if not specified)\n\n\nFor example, \n\n```\nsubscription:\n    Binance:\n        instruments:\n            - XRP/BTC\n            - BCH/BTC\n        depth\n    Poloniex:\n        instruments:\n            - ETH/BTC\n        depth: 10\n```\n\n### Handlers\n\nAfter receiving the order book or trade update, each handler is updated. For example, for SQL database handler, it is updated with the corresponding SQl statements.\n\nFor example,\n\n```\nhandlers:\n    sql: \n        connection: \"sqlite://\"\n    \n```\n\n#### SQL handler\n\nThe following settings can be customized\n\n|Parameter|Description|\n|---|---|\n|connection|Database connection string required by [SQLAlchemy](https://docs.sqlalchemy.org/en/latest/core/engines.html)|\n|is_rotate|Boolean indicating whether to rotate to record the table.|\n|rotate_frequency|String in [format](https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior) same as `strftime` and `strptime`|\n\n#### ZeroMQ handler\n\nThe feed handler acts as a [publisher](https://learning-0mq-with-pyzmq.readthedocs.io/en/latest/pyzmq/patterns/pubsub.html) in ZeroMQ. To receive the feed, please follow ZeroMQ instructions to start a [subscriber](tests/zmq/zmq_subscriber.py).\n\nThe following settings can be customized\n\n|Parameter|Description|\n|---|---|\n|connection|Connection [format](http://api.zeromq.org/3-2:zmq-connect) in ZeroMQ. For example, \"tcp://127.0.0.1:3456\"|\n\n\n## Examples\n\nYou can first create a directory `.data` and run the command\n\n```\n$ bitcoinexchangefh --configuration example/configuration.yaml \n2020-08-07 23:33:32,110 INFO Loading runner\n2020-08-07 23:33:32,110 INFO Creating handler sql\n2020-08-07 23:33:32,205 INFO Loading handler SqlHandler\n2020-08-07 23:33:32,710 INFO Loading exchange Binance\n2020-08-07 23:33:34,062 INFO Loading exchange Bitmex\n2020-08-07 23:33:40,203 INFO Start running the feed handler\n2020-08-07 23:33:40,203 INFO Running handler sql\n2020-08-07 23:33:40,210 INFO Running exchange Binance\n2020-08-07 23:33:40,215 INFO Running SqlHandler\n2020-08-07 23:33:40,219 INFO Running exchange Bitmex\n2020-08-07 23:33:40,231 INFO Joining all the processes\n2020-08-07 23:33:40,232 INFO Creating table binance_ethbtc_order\n2020-08-07 23:33:40,263 INFO Created table binance_ethbtc_order\n2020-08-07 23:33:40,265 INFO Creating table binance_xrpbtc_order\n2020-08-07 23:33:40,294 INFO Created table binance_xrpbtc_order\n2020-08-07 23:33:40,297 INFO Creating table bitmex_ethusd_order\n2020-08-07 23:33:40,322 INFO Created table bitmex_ethusd_order\n```\n\nAfter a while, the data is streamed into the file `.data/order_book.db`. You can terminate the process and the database contains the following tables.\n\n```\n$ python -c \"import sqlite3; print(sqlite3.connect('.data/order_book.db').execute('SELECT name from sqlite_master where type= \\\"table\\\"').fetchall())\"\n[('binance_ethbtc_order',), ('binance_xrpbtc_order',), ('bitmex_ethusd_order',)]\n\n```\n\n\n## Inquiries\n\nYou can first look up to the page [FAQ](https://github.com/gavincyi/BitcoinExchangeFH/wiki/FAQ). For more inquiries, you can either leave it in issues or drop me an email. I will get you back as soon as possible.\n\n## Compatibility\nThe application is compatible with version higher or equal to python 3.0.\n\n## Contributions\nAlways welcome for any contribution. Please fork the project, make the changes, and submit the merge request. :)\n\nFor any questions and comment, please feel free to contact me through email (gavincyi at gmail)\n\nYour comment will be a huge contribution to the project!\n\n## Continuity\nIf you are not satisified with python performance, you can contact me to discuss migrating the project into other languages, e.g. C++.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBitcoinExchangeFH%2FBitcoinExchangeFH","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBitcoinExchangeFH%2FBitcoinExchangeFH","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBitcoinExchangeFH%2FBitcoinExchangeFH/lists"}