{"id":33074355,"url":"https://github.com/sadighian/crypto-rl","last_synced_at":"2025-11-19T04:02:24.021Z","repository":{"id":37251719,"uuid":"138102346","full_name":"sadighian/crypto-rl","owner":"sadighian","description":"Deep Reinforcement Learning toolkit: record and replay cryptocurrency limit order book data \u0026 train a DDQN agent","archived":false,"fork":false,"pushed_at":"2022-01-12T11:56:35.000Z","size":169192,"stargazers_count":777,"open_issues_count":5,"forks_count":236,"subscribers_count":54,"default_branch":"arctic-streaming-ticks-full","last_synced_at":"2024-01-30T12:07:38.652Z","etag":null,"topics":["arctic","bitcoin","bitfinex","coinbase","coinbase-pro","cryptocurrencies","deep-reinforcement-learning","dqn","hft","limit-order-book","mongodb","multiprocessing","multithreading","order-book","orderbook","recorder","tick-data","tickstore","trading","trading-algorithms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sadighian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-21T01:06:01.000Z","updated_at":"2024-01-26T13:29:40.000Z","dependencies_parsed_at":"2022-07-12T16:14:50.354Z","dependency_job_id":null,"html_url":"https://github.com/sadighian/crypto-rl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sadighian/crypto-rl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sadighian%2Fcrypto-rl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sadighian%2Fcrypto-rl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sadighian%2Fcrypto-rl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sadighian%2Fcrypto-rl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sadighian","download_url":"https://codeload.github.com/sadighian/crypto-rl/tar.gz/refs/heads/arctic-streaming-ticks-full","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sadighian%2Fcrypto-rl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":285181688,"owners_count":27128334,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-19T02:00:05.673Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arctic","bitcoin","bitfinex","coinbase","coinbase-pro","cryptocurrencies","deep-reinforcement-learning","dqn","hft","limit-order-book","mongodb","multiprocessing","multithreading","order-book","orderbook","recorder","tick-data","tickstore","trading","trading-algorithms"],"created_at":"2025-11-14T10:00:31.747Z","updated_at":"2025-11-19T04:02:24.014Z","avatar_url":"https://github.com/sadighian.png","language":"Python","funding_links":[],"categories":["Reinforcement Learning Tools"],"sub_categories":["Books"],"readme":"# Deep Reinforcement Learning Toolkit for Cryptocurrencies\n\n\n**Table of contents:**\n\n1. Purpose\n2. Scope\n3. Dependencies\n4. Project structure\n5. Design patterns\n6. Getting started\n7. Citing this project\n8. Appendix \n\n## 1. Purpose\nThe purpose of this application is to provide a toolkit to:\n - **Record** full limit order book and trade tick data from two \n exchanges (**Coinbase Pro** and **Bitfinex**) into an [Arctic](https://github.com/manahl/arctic) \n Tickstore database (i.e., MongoDB), \n - **Replay** recorded historical data to derive feature sets for training\n - **Train** an agent to trade cryptocurrencies using the DQN algorithm (note: this agent\n  implementation is intended to be an example for users to reference)\n\n![High_Level_Overview](./design_patterns/design-pattern-high-level.PNG)\n\n\n## 2. Scope\n- **Research only:** there is no capability for live-trading at exchanges.\n- **Reproducing RL paper results:** the dataset used for [this](https://arxiv.org/abs/2004.06985) article is available on [Kaggle Datasets](https://www.kaggle.com/jsadighian/cryptorl).\n\n\n## 3. Dependencies\nSee `requirements.txt`\n\n*Note*: to run and train the DQN Agent (`./agent/dqn.py`) tensorflow and Keras-RL\nneed to be installed manually and are not listed in the `requirements.txt` \nin order to keep this project compatible with other open \nsourced reinforcement learning platforms \n(e.g., [OpenAI Baselines](https://github.com/openai/baselines)).\n\nPip install the following:\n\n```\ngit+https://github.com/manahl/arctic.git\n\nKeras==2.2.4\nKeras-Applications==1.0.7\nKeras-Preprocessing==1.0.9\nkeras-rl==0.4.2\n\ntensorboard==1.13.1\ntensorflow-estimator==1.13.0\ntensorflow-gpu==1.13.1\n```\n\n\n## 4. Project Structure\nThe key elements in this project and brief descriptions.\n```\ncrypto-rl/\n\tagent/\n\t\t\t\t...reinforcement learning algorithm implementations\n\tdata_recorder/\n\t\t\t\t...tools to connect, download, and retrieve limit order book data\n\tgym_trading/\n\t\t\t\t...extended openai.gym environment to observe limit order book data\n\tindicators/\n\t\t\t\t...technical indicators implemented to be O(1) time complexity\n\tdesign-patterns/\n\t\t\t\t...visual diagrams module architecture\n\tvenv/\n\t\t\t\t...virtual environment for local deployments\n\texperiment.py          # Entry point for running reinforcement learning experiments\n\trecorder.py            # Entry point to start recording limit order book data\n\tconfigurations.py      # Constants used throughout this project\n\trequirements.txt       # List of project dependencies\n\tsetup.py               # Run the command `python3 setup.py install` to \n\t                       #    install the extended gym environment i.e., gym_trading.py\n```\n\n\n## 5. Design Patterns\nRefer to each individual module for design pattern specifications:\n\n- [Limit Order Book, Data Recorder, and Database](./data_recorder/README.md)\n- [Stationary LOB Features](https://arxiv.org/abs/1810.09965v1)\n- [POMDP Environment](./gym_trading/README.md)\n- [Learning Algorithms and Neural Networks](./agent/README.md)\n\nSample snapshot of Limit Order Book levels:\n![plot_lob_levels](./design_patterns/plot_lob_levels.png)\n\nSample snapshot of Order Arrival flow metrics:\n![plot_order_arrivals](./design_patterns/plot_order_arrivals.png)\n\n\n## 6. Getting Started\n\nInstall the project on your machine:\n```\n# Clone the project from github\ngit clone https://github.com/sadighian/crypto-rl.git\ncd crypto-rl\n\n# Install a virtual environment for the project's dependencies\npython3 -m venv ./venv\n\n# Turn on the virtual environment\nsource venv/bin/activate\n\n# Install keras-rl dependencies\npip3 install Keras==2.2.4 Keras-Applications==1.0.7 Keras-Preprocessing==1.0.9 keras-rl==0.4.2\n tensorboard==1.13.1 tensorflow-estimator==1.13.0 tensorflow-gpu==1.13.1\n \n# Install database\npip3 install git+https://github.com/manahl/arctic.git\n\n# Install the project\npip3 install -e .\n```\n\n### 6.1 Record limit order book data from exchanges\n\n**Step 1:**\nGo to the `configurations.py` and define the crypto currencies which\nyou would like to subscribe and record. \n\nNote: basket list format is as follows `[(Coinbase_Instrument_Name, Bitfinex_Instrument_Name), ...]`\n```\nSNAPSHOT_RATE = 5  # I.e., every 5 seconds\nBASKET = [('BTC-USD', 'tBTCUSD'),\n         ('ETH-USD', 'tETHUSD'),\n         ('LTC-USD', 'tLTCUSD'),\n         ('BCH-USD', 'tBCHUSD'),\n         ('ETC-USD', 'tETCUSD')]\nRECORD_DATA = True\n```\n\n**Step 2:**\nOpen a CLI/terminal and execute the command to start recording \nfull limit order book and trade data.\n ```\n python3 recorder.py\n ```\n\n### 6.2 Replay recorded data to export stationary feature set\n\n**Step 1:**\nEnsure that you have data in your database. \n\nCheck with MongoDB shell or [Compass](https://www.mongodb.com/products/compass). \nIf you do not have data, see refer to the section above \n**6.1 Record limit order book data from exchanges**.\n\n**Step 2:**\nRun a historial data simulation to take snapshots of the\nlimit order book(s) and export their stationary features\nto a compressed csv.\n\nTo do this, you can leverage the test cases in `data_recorder/tests/`\nor write your own logic. When using the test case methods, make sure\nto change the query parameters to match what you've actually recorded and\nis in your database.\n\nExample to export features to a compressed csv:\n```\npython3 data_recorder/tests/test_extract_features.py\n```\n\n### 6.3 Train an agent\n\n**Step 1:**\nEnsure you have data in the `data_recorder/database/data_exports/` folder.\nThis is where the agent loads data from. If you do not have data exported\ninto that folder, see refer to the section above \n**6.2 Replay recorded data to export stationary feature set**.\n\n**Step 2:**\nOpen a CLI/terminal and start learning/training the agent. \n```\npython3 experiment.py --window_size=50 --weights=False --fitting_file=...\n```\nRefer to `experiment.py` to see all the keyword arguments.\n\n\n## 7. Citing this project\n\nPlease remember to cite this repository if used in your research:\n```\n    @misc{Crypto-RL,\n        author = {Jonathan Sadighian},\n        title = {Deep Reinforcement Learning Toolkit for Cryptocurrencies},\n        year = {2019},\n        publisher = {GitHub},\n        journal = {GitHub repository},\n        howpublished = {\\url{https://github.com/sadighian/crypto-rl}},\n    }\n```\n\n\n## 8. Appendix\n### 8.1 Branches\nThere are multiple branches of this project, each with a different implementation pattern \nfor persisting data:\n - **FULL** branch is intended to be the foundation for a fully automated trading system \n (i.e., implementation of design patterns that are ideal for a trading system that requires \n parallel processing) and persists streaming tick data into an **Arctic Tick Store**\n \n **Note:** the branches below (i.e., lightweight, order book snapshot, mongo integration) \n are no longer actively maintained as of October 2018, and are here for reference.\n \n - **LIGHT WEIGHT** branch is intended to record streaming data more efficiently than \n the __full__ branch (i.e., all websocket connections are made from a single process \n __and__ the limit order book is not maintained) and persists streaming tick data into \n an **Arctic tick store**\n - **ORDER BOOK SNAPSHOT** branch has the same design pattern as the __full__ branch, \n but instead of recording streaming ticks, snapshots of the limit order book are taken \n every **N** seconds and persisted into an **Arctic tick store**\n - **MONGO INTEGRATION** branch is the same implementation as **ORDER BOOK SNAPSHOT**, \n with the difference being a standard MongoDB is used, rather than Arctic. \n This branch was originally used to benchmark Arctic's performance and is not up to \n date with the **FULL** branch.\n\n### 8.2 Assumptions\n- You have installed a virtual environment and installed the project to that venv \n(e.g., `pip3 install -e .`)\n- You have mongoDB already installed\n- You know how to use a cli to start python scripts\n- You are running an ubuntu 18+ os\n\n### 8.3 Change Log\n- 2021-09-25: Updated `requirements.txt`: going forward the database requires a manual \n  installation via `pip install git+https://github.com/manahl/arctic.git`\n- 2019-12-12: Added docstrings and refactored many classes to improve code readability\n- 2019-09-18: Refactored `env`s and `broker`s for simplification and\n  added different `reward` approaches.\n- 2019-09-13: Created and implemented 'order arrival' flow metrics,\n  inspired by\n  [Multi-Level Order-Flow Imbalance in a Limit Order Book](https://arxiv.org/abs/1907.06230v1)\n  by Xu, Ke; Gould, Martin D.; Howison, Sam D.\n- 2019-09-06: Created and implemented `Indicator.py` base class\n- 2019-04-28: Reorganized project structure for simplicity\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsadighian%2Fcrypto-rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsadighian%2Fcrypto-rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsadighian%2Fcrypto-rl/lists"}