{"id":28317752,"url":"https://github.com/pablodieaco/quant-drl-core","last_synced_at":"2026-03-02T04:03:31.862Z","repository":{"id":286564015,"uuid":"959910981","full_name":"pablodieaco/quant-drl-core","owner":"pablodieaco","description":"quant-drl-core is a modular and extensible framework for applying Deep Reinforcement Learning (DRL) techniques to portfolio management and financial decision-making. Built with flexibility and research in mind, it provides tools for data processing, environment simulation, agent training, and performance evaluation.","archived":false,"fork":false,"pushed_at":"2025-06-10T22:00:59.000Z","size":24261,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-10T22:19:38.615Z","etag":null,"topics":["deep-reinforcement-learning","pytorch","quantitative-finance","quantitative-trading","stable-baselines3","yfinance"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pablodieaco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-03T14:50:29.000Z","updated_at":"2025-06-10T22:00:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"ffda9aba-b200-44ba-82a1-17465bee9120","html_url":"https://github.com/pablodieaco/quant-drl-core","commit_stats":null,"previous_names":["pablodieaco/quant-drl-core"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/pablodieaco/quant-drl-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pablodieaco%2Fquant-drl-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pablodieaco%2Fquant-drl-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pablodieaco%2Fquant-drl-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pablodieaco%2Fquant-drl-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pablodieaco","download_url":"https://codeload.github.com/pablodieaco/quant-drl-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pablodieaco%2Fquant-drl-core/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259370903,"owners_count":22847469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-reinforcement-learning","pytorch","quantitative-finance","quantitative-trading","stable-baselines3","yfinance"],"created_at":"2025-05-25T06:12:38.553Z","updated_at":"2026-03-02T04:03:31.853Z","avatar_url":"https://github.com/pablodieaco.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📈 quant-drl-core\n\n**quant-drl-core** is a research-focused framework for applying Deep Reinforcement Learning (DRL) to portfolio management and financial decision-making.  \nIt is designed to be **modular**, **extensible**, and easy to integrate into experimentation pipelines or production systems.\n\n\u003e 🎓 This project was developed as part of my **Bachelor's Thesis (TFG)** for the **Double Degree in Mathematics and Computer Engineering** at the **University of Seville**.\n\n\n---\n\n## 🧠 Key Features\n\n- ✅ Modular training pipeline for multiple DRL algorithms: `PPO`, `SAC`, `DDPG`, `TD3`\n- ✅ Custom `Gymnasium`-compatible environments for realistic financial simulations\n- ✅ Support for `CNN`, `LSTM`, `Transformer` and hybrid feature extractors\n- ✅ Integrated technical indicators and raw market data handling\n- ✅ Easy portfolio configuration (by sector and number of companies)\n- ✅ Clean logging with `loguru`\n- ✅ Compatible with interactive dashboards for visualization *(e.g., Streamlit webapp)*\n\n---\n\n## 🛠️ Getting Started\n\n### 🔧 Installation\n\n```bash\ngit clone https://github.com/your-username/quant-drl-core.git\ncd quant-drl-core\npython -m venv venv\nsource venv/bin/activate         # Windows: venv\\Scripts\\activate\npip install -e .\n```\n\n---\n\n## 🧪 Training models\nUse the `train_models.py` script to train models.\n\nBasic command (default config)\n\n```bash\npython train_models.py\n```\n\nIf you want to update the `hierarchy.json` automatically. Add the `--update_hierarchy` flag:\n\n```bash\npython train_models.py --update_hierarchy\n```\n\nCustom training with options:\n\n```bash\npython train_models.py \\\n  --algorithms PPO SAC \\\n  --features LSTM CNNLSTM \\\n  --learning_rates 0.0003 0.0001 \\\n  --n_companies 10 18 \\\n  --length_train_data 8 \\\n  --length_eval_data 2 \\\n  --end_date_year 2021 \\\n  --total_timesteps 1_000_000 \\\n  --checkpoint_freq 100_000 \\\n  --update_hierarchy\n```\n\nModels and logs will be saved in:\n\n- 📁 `models/` – trained model files\n- 📁 `logs/` – experiment logs\n- 📄 `models/metadata/hierarchy.json` – metadata for evaluation\n\nOptional (Open Tensorboard)\n\n```bash\ntensorboard --logdir=logs/\n```\nTensorboard framework will be available at http://localhost:16006\n\n---\n\n## 📊 Evaluating models\nUse the `evaluate_models.py` script to evaluate trained models.\n\n- Option 1: Evaluate final models\n```bash\npython evaluate_models.py \\\n  --use_final_model\n```\n\n- Option 2: Evaluate by specific checkpoint steps\n```bash\npython evaluate_models.py \\\n  --steps_override PPO=500000 SAC=300000\n```\n\n- Option 3: Full customized evaluation\n```bash\npython evaluate_models.py \\\n  --hierarchy_path models/metadata/hierarchy.json \\\n  --results_path results/eval_sac_vs_ppo.csv \\\n  --models_dir models \\\n  --out_dir models/metadata/out \\\n  --steps_override PPO=500000 SAC=300000 \\\n  --no_move\n```\n\n---\n\n## 📚 Notebooks\n\nInside `notebooks/` you’ll find:\n\n- 📈  Data exploration\n- 📊 Testing models\n- 📝 Exporting results to LaTeX\n\n---\n\n## 🔮 Planned Features\n\n- [ ] Risk-adjusted rewards (Sharpe, Sortino, etc.)\n- [ ] Hyperparameter optimization with Optuna\n- [ ] Integration with MLFlow\n- [ ] Multi-agent extensions\n- [X] Integration with a live dashboard (Streamlit) ([quant-drl-web](https://github.com/pablodieaco/quant-drl-web))\n\n## 📝 License\n\nThis project is licensed under the **MIT License**.  \nSee the [LICENSE](./LICENSE) file for details.\n\n---\n\n## 🙋‍♂️ Author\n\nMade with ❤️ by **Pablo Diego Acosta**\n\n- 💼 LinkedIn: [linkedin.com/in/pablodiegoacosta](https://www.linkedin.com/in/pablodiegoacosta)\n\n---\n\n## 🤝 Contributing\n\nWhether you're a researcher, developer, or enthusiast — feel free to contribute!  \nYou're welcome to suggest improvements, propose new architectures, experiment with custom reward functions, or integrate new DRL algorithms.\n\n✨ Open a pull request, start a discussion, or simply share your ideas.\n\nTogether, we can make this framework even more powerful for the DRL + Finance community 🚀\n\n## 📚 References\n\nThis project is based on a wide range of research articles, theses, technical reports, and open-source implementations in the field of reinforcement learning and portfolio management.\n\nSome of the most relevant references include:\n\n- Filos, A. (2019). *Reinforcement Learning for Portfolio Management*. [arXiv:1909.09571](https://arxiv.org/abs/1909.09571)\n- Jiang, Z. et al. (2017). *A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem*. [arXiv:1706.10059](https://arxiv.org/abs/1706.10059)\n- Sutton, R. S., \u0026 Barto, A. G. (2018). *Reinforcement Learning: An Introduction*. MIT Press.\n- Haarnoja, T. et al. (2018). *Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning*. [arXiv:1801.01290](https://arxiv.org/abs/1801.01290)\n- Raffin, A. et al. (2021). *Stable-Baselines3: Reliable RL Implementations*. [JMLR](http://jmlr.org/papers/v22/20-1364.html)\n- Markowitz, H. (1959). *Portfolio Selection: Efficient Diversification of Investments*. Yale University Press.\n- Open-source: [Zenlii – Deep RL Portfolio Management](https://github.com/Zenlii/Deep-Reinforcement-Learning-for-Portfolio-Management)\n\nA complete list of all references (BibTeX format) used in this project is available [here](references.bib).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpablodieaco%2Fquant-drl-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpablodieaco%2Fquant-drl-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpablodieaco%2Fquant-drl-core/lists"}