Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/iusztinpaul/yacht
Order execution in the financial markets using Deep Reinforcement Learning.
https://github.com/iusztinpaul/yacht
cryptocurrency deep-learning deep-reinforcement-learning order-execution pandas pytorch reinforcement-learning stocks trading trading-bot weights-and-biases
Last synced: 13 days ago
JSON representation
Order execution in the financial markets using Deep Reinforcement Learning.
- Host: GitHub
- URL: https://github.com/iusztinpaul/yacht
- Owner: iusztinpaul
- License: apache-2.0
- Created: 2021-02-20T14:31:38.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-12-10T08:58:34.000Z (about 2 years ago)
- Last Synced: 2024-12-02T07:47:27.853Z (24 days ago)
- Topics: cryptocurrency, deep-learning, deep-reinforcement-learning, order-execution, pandas, pytorch, reinforcement-learning, stocks, trading, trading-bot, weights-and-biases
- Language: Jupyter Notebook
- Homepage:
- Size: 72.1 MB
- Stars: 14
- Watchers: 4
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Yacht: Yet Another Comprehensive Trading Framework Using Deep Reinforcement Learning
A Deep Reinforcement Learning framework for **trading** & **order execution** in financial markets. The goal of the project
is to speed up the development & research of financial agents by building a modular and scalable codebase. The framework supports the following main features:
* Loading & preprocessing data directly from different APIs
* Training & evaluating deep reinforcement learning agents
* Use specific financial metrics or quickly implement your own
* Visualizing the performance of the agent with some intuitive graphs
The nice part is that everything is `configurable` within a config file.
The code is using popular packages like:
* pytorch
* pandas
* stable-baselines3
* gym
* wandb
* mplfinance
# Project Architecture
The architecture is split into 4 main categories:
* Data
* Environment
* Reinforcement Learning Agents
* Specific Task Layer
The `Specific Task Layer` is a glue code module that is used for training & backtesting.
It can be further be extended into the `applicaton layer`.
# Visual Representations
Visual representations of the actions taken by the agent & the current status of the agent:
----
# Install
### Requirements
* Code tested under `Python 3.8`, `pytorch 1.13.0`, and `cuda 11.6` on `Ubuntu 20.04`
1. Create a conda environment and activate it:
```shell
conda create --name yacht python=3.8
conda env activate yacht
```
2. Install torch and cuda separately with `conda`:
```shell
conda install pytorch=1.13.0 torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia
```
3. Ultimately, install other requirements with `pip`:
```shell
pip install -r requirements.txt
```
### Config Compiler
* The configuration system is built with `google protobuf`. If you want to recompile / change the protobuf files,
you should install the `protoc` compiler on your system:
```shell
sudo apt install protobuf-compiler libprotobuf-dev
```
* Run the `compilation` command from the root folder:
```shell
protoc -I=. --python_out=. yacht/config/proto/*.proto
```
### Add Secret Keys
* Create a file called `.env` at the root directory level. If you want to fully use the market APIs and
experiment trackers you should add the secret keys.
* Look at `.env.default` for the supported env vars.
* Not all `env vars` are mandatory. For example the free version of `Yahoo Finance` does not require any credentials.
----------
# Data APIs
* Currently, we have support for:
* `Binance`
* `Yahoo Finance`.
* You should set the `api keys` in the `.env` file for full support.
# Datasets & Download Data
* S&P 500
* Dow 30
* Nasdaq 100
* Russell 2000
You can set `tickers: ['NASDAQ100']` in the configuration file and all the tickers from the index will be expanded.
You can also set something like `['NASDAQ100', 'S&P500', 'AAPL']` or any combination you like.
#### Download
The data is stored in `h5` files.
```shell
python main.py download --config-file-name download_4years.config.txt --storage-dir ./storage/download_4_years --market-storage-dir ./storage
```
* The `--market-storage-dir` CLI argument is optional. If you add it the market will be placed
in a different location than your `storage-dir`. This is helpful because it can be accessed by multiple
experiments in parallel `during training` (the `h5` file will be set in a read only mode). Otherwise, while training,
only one experiment can access a specific file.
* `--market-storage-dir` should be used also during `training` & `backtesting`
* You can use the `market_mixins: [...]` from the config file to preprocess the data before it is stored.
-------
# Usage
All the supported configs can be found at `./yacht/config/configs`.
You should only add the config path relative to the root directory.
### Train
```shell
python main.py train --config-file order_execution/all/single_asset_all_universal_silent.config.txt --storage-dir ./storage/yacht --market-storage-dir ./storage
```
### Backtest
```shell
python main.py backtest --config-file order_execution/all/single_asset_all_universal_silent.config.txt --storage-dir ./storage/yacht --market-storage-dir ./storage
```
### Download Pretrained Weights 👇🏻
You can download the pretrained weights from [here](https://drive.google.com/drive/folders/1ldDB_zcE-9nNvsIsDbn5W1-5-UA_8C-k?usp=sharing).
```shell
cd /root/directory
mkdir storage
--> place the downloaded weights in ./storage
```
Suppose you downloaded and placed the pretrained weights and data correctly as showed above. In that case, you can run the following command to resume the agent:
```shell
python main.py train --config-file order_execution/all/single_asset_all_universal_silent.config.txt --storage-dir ./storage/yacht --resume-from best-train --market-storage-dir ./storage
```
For the parameter `--resume-from` we support the following combinations:
* Absolute path to the checkpoint.
* `latest-train` = resume the agent from the latest checkpoint saved during training
* `best-train` = resume the agent from the best checkpoint saved during training
**NOTE:** For the `best-train` parameter, you can choose a metric on which the agent was monitored. You
can do that with the `meta.metrics_to_load_best_on` parameter from the configuration file. For example, `metrics_to_load_best_on: ['PA', 'GLR']`
will load two agents: The agent monitored on the metric `PA` & the one who performed the best on `GLR`.
# Tutorials
* [Get started](https://medium.com/mlearning-ai/how-to-automate-your-orders-as-a-buy-and-hold-investor-84f6885015aa)
------
# Experiment Tracking
### Weights & Biases
* For now, we support `wandb` for experiment tracking and logging.
* Just add the api key in the `.env` file. Also, in the configuration file you should add:
```shell
meta: {
experiment_tracker: 'wandb'
}
```
If you want to add a specific `project_entity` add:
```shell
meta: {
project_entity: 'your_project_entity'
}
```
**NOTE:** Be aware that this name is unique between all the users that use wandb. For
example, I use `project_entity=yacht`. If you try to use it will through an unauthorized error
because you do not have access to my entity.
Here is an example of how it looks:
* If you don't want to log a specific experiment on the experiment tracker just remove the config
field or replace it with the empty string `''`.
# Hyperparameter Optimization
### Weights & Biases
* We support yyperparameter optimization with weights & biases sweeps.
* Weights & biases should work as a simple experiment tracker before using this.
* You can use any other config from `tools/tuning/configs` or generate your own.
```shell
wandb sweep tools/tuning/configs/single_asset_order_execution_crypto.yaml
wandb agent id-given-by-generated-sweep
```
--------
# More Resources
For further reading go to:
* [Available Components](docs/components.md)
* [Config Explanation](docs/config.md)
* [Trading](docs/trading.md)
* [Order Execution](docs/order_execution.md)
* [Metrics](docs/metrics.md)
* [Teacher-Student](docs/teacher.md)