{"id":13449730,"url":"https://github.com/pskrunner14/trading-bot","last_synced_at":"2025-03-22T22:33:50.458Z","repository":{"id":42038251,"uuid":"144566152","full_name":"pskrunner14/trading-bot","owner":"pskrunner14","description":"Stock Trading Bot using Deep Q-Learning","archived":false,"fork":false,"pushed_at":"2023-12-03T13:06:25.000Z","size":9882,"stargazers_count":923,"open_issues_count":16,"forks_count":332,"subscribers_count":50,"default_branch":"master","last_synced_at":"2024-08-01T06:22:11.307Z","etag":null,"topics":["ai-agents","deep-q-learning","q-learning","reinforcement-learning","stock-price-prediction","stock-trading","stock-trading-bot","trading-algorithms","trading-bot"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pskrunner14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-13T10:44:08.000Z","updated_at":"2024-08-01T06:22:11.308Z","dependencies_parsed_at":"2022-07-13T21:44:21.728Z","dependency_job_id":null,"html_url":"https://github.com/pskrunner14/trading-bot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pskrunner14%2Ftrading-bot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pskrunner14%2Ftrading-bot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pskrunner14%2Ftrading-bot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pskrunner14%2Ftrading-bot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pskrunner14","download_url":"https://codeload.github.com/pskrunner14/trading-bot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221840721,"owners_count":16889852,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","deep-q-learning","q-learning","reinforcement-learning","stock-price-prediction","stock-trading","stock-trading-bot","trading-algorithms","trading-bot"],"created_at":"2024-07-31T06:00:52.645Z","updated_at":"2024-10-28T14:31:52.919Z","avatar_url":"https://github.com/pskrunner14.png","language":"Jupyter Notebook","funding_links":[],"categories":["Machine Learning","Backtest + live trading"],"sub_categories":["Cryptocurrencies","Machine Learning / Reinforcement Learning Focused"],"readme":"# Overview\n\nThis project implements a Stock Trading Bot, trained using Deep Reinforcement Learning, specifically Deep Q-learning. Implementation is kept simple and as close as possible to the algorithm discussed in the paper, for learning purposes.\n\n## Introduction\n\nGenerally, Reinforcement Learning is a family of machine learning techniques that allow us to create intelligent agents that learn from the environment by interacting with it, as they learn an optimal policy by trial and error. This is especially useful in many real world tasks where supervised learning might not be the best approach due to various reasons like nature of task itself, lack of appropriate labelled data, etc.\n\nThe important idea here is that this technique can be applied to any real world task that can be described loosely as a Markovian process.\n\n## Approach\n\nThis work uses a Model-free Reinforcement Learning technique called Deep Q-Learning (neural variant of Q-Learning).\nAt any given time (episode), an agent abserves it's current state (n-day window stock price representation), selects and performs an action (buy/sell/hold), observes a subsequent state, receives some reward signal (difference in portfolio position) and lastly adjusts it's parameters based on the gradient of the loss computed.\n\nThere have been several improvements to the Q-learning algorithm over the years, and a few have been implemented in this project:\n\n- [x] Vanilla DQN\n- [x] DQN with fixed target distribution\n- [x] Double DQN\n- [ ] Prioritized Experience Replay\n- [ ] Dueling Network Architectures\n\n## Results\n\nTrained on `GOOG` 2010-17 stock data, tested on 2019 with a profit of $1141.45 (validated on 2018 with profit of $863.41):\n\n![Google Stock Trading episode](./extra/visualization.png)\n\nYou can obtain similar visualizations of your model evaluations using the [notebook](./visualize.ipynb) provided.\n\n## Some Caveats\n\n- At any given state, the agent can only decide to buy/sell one stock at a time. This is done to keep things as simple as possible as the problem of deciding how much stock to buy/sell is one of portfolio redistribution.\n- The n-day window feature representation is a vector of subsequent differences in Adjusted Closing price of the stock we're trading followed by a sigmoid operation, done in order to normalize the values to the range [0, 1].\n- Training is prefferably done on CPU due to it's sequential manner, after each episode of trading we replay the experience (1 epoch over a small minibatch) and update model parameters.\n\n## Data\n\nYou can download Historical Financial data from [Yahoo! Finance](https://ca.finance.yahoo.com/) for training, or even use some sample datasets already present under `data/`.\n\n## Getting Started\n\nIn order to use this project, you'll need to install the required python packages:\n\n```bash\npip3 install -r requirements.txt\n```\n\nNow you can open up a terminal and start training the agent:\n\n```bash\npython3 train.py data/GOOG.csv data/GOOG_2018.csv --strategy t-dqn\n```\n\nOnce you're done training, run the evaluation script and let the agent make trading decisions:\n\n```bash\npython3 eval.py data/GOOG_2019.csv --model-name model_GOOG_50 --debug\n```\n\nNow you are all set up!\n\n## Acknowledgements\n\n- [@keon](https://github.com/keon) for [deep-q-learning](https://github.com/keon/deep-q-learning)\n- [@edwardhdlu](https://github.com/edwardhdlu) for [q-trader](https://github.com/edwardhdlu/q-trader)\n\n## References\n\n- [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/abs/1312.5602)\n- [Human Level Control Through Deep Reinforcement Learning](https://deepmind.com/research/publications/human-level-control-through-deep-reinforcement-learning/)\n- [Deep Reinforcement Learning with Double Q-Learning](https://arxiv.org/abs/1509.06461)\n- [Prioritized Experience Replay](https://arxiv.org/abs/1511.05952)\n- [Dueling Network Architectures for Deep Reinforcement Learning](https://arxiv.org/abs/1511.06581)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpskrunner14%2Ftrading-bot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpskrunner14%2Ftrading-bot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpskrunner14%2Ftrading-bot/lists"}