{"id":49950685,"url":"https://github.com/pavelml-dev/ml-trading-systems","last_synced_at":"2026-05-17T19:04:04.209Z","repository":{"id":323116846,"uuid":"1091579561","full_name":"PavelML-Dev/ML-Trading-Systems","owner":"PavelML-Dev","description":"ML SUPERTREND ULTIMATE -ML- QLearning +Per + LSTM + CNN ","archived":false,"fork":false,"pushed_at":"2026-04-26T14:51:51.000Z","size":430,"stargazers_count":16,"open_issues_count":0,"forks_count":7,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-26T16:26:17.407Z","etag":null,"topics":["algorithmic-trading","backpropagation-learning-algorithm","machine-learning","open-source","pine-script","quantitative-trading","reinforcement-learning","trading"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PavelML-Dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-07T08:04:20.000Z","updated_at":"2026-04-26T14:51:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/PavelML-Dev/ML-Trading-Systems","commit_stats":null,"previous_names":["pavelml-dev/ml-trading-systems"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/PavelML-Dev/ML-Trading-Systems","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PavelML-Dev%2FML-Trading-Systems","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PavelML-Dev%2FML-Trading-Systems/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PavelML-Dev%2FML-Trading-Systems/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PavelML-Dev%2FML-Trading-Systems/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PavelML-Dev","download_url":"https://codeload.github.com/PavelML-Dev/ML-Trading-Systems/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PavelML-Dev%2FML-Trading-Systems/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33151625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-17T09:28:26.183Z","status":"ssl_error","status_checked_at":"2026-05-17T09:27:52.702Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithmic-trading","backpropagation-learning-algorithm","machine-learning","open-source","pine-script","quantitative-trading","reinforcement-learning","trading"],"created_at":"2026-05-17T19:03:59.794Z","updated_at":"2026-05-17T19:04:04.203Z","avatar_url":"https://github.com/PavelML-Dev.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 ML SuperTrend Ultimate: Deep Q-Learning + LSTM + PER\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Pine Script](https://img.shields.io/badge/Pine%20Script-v6-blue.svg)](https://www.tradingview.com/pine-script-docs/)\n[![TradingView](https://img.shields.io/badge/TradingView-Compatible-green.svg)](https://www.tradingview.com/)\n[![Made in Russia](https://img.shields.io/badge/Made%20in-Russia%20🇷🇺-blue.svg)](https://en.wikipedia.org/wiki/Russia)\n\n\u003e **First fully-working LSTM + Deep Q-Network trading system implemented in Pine Script!**\n\nA self-learning trading agent that uses cutting-edge machine learning techniques to adapt to market conditions in real-time — no external libraries, no Python, just pure Pine Script.\n\n---\n\n## 🔥 What Makes This Unique?\n\nThis is **NOT just another indicator**. This is a complete **reinforcement learning system** that:\n\n- ✅ **Learns from experience** using Deep Q-Learning\n- ✅ **Remembers patterns** with LSTM neural networks\n- ✅ **Adapts in real-time** without retraining\n- ✅ **Prioritizes important data** with PER (Prioritized Experience Replay)\n- ✅ **Works in your browser** — no GPU, no Python, no servers\n\n### Why It's Special\n\n| Traditional Indicators | ML SuperTrend Ultimate |\n|----------------------|------------------------|\n| Static parameters | **Learns optimal parameters** |\n| Same for all markets | **Adapts to each market** |\n| Looks at 1-2 bars | **Analyzes 8-20 bars history** |\n| Simple rules | **Deep neural networks** |\n| No learning | **Continuous learning** |\n\n---\n\n## ⚠️ DISCLAIMER\n\n**This is an experimental research project for educational purposes.**\n\n- **NOT financial advice**\n- **NO profit guarantees** \n- Use at **your own risk**\n- Author bears **NO responsibility** for any losses\n\nThis is a learning tool, not a production trading system. Always backtest thoroughly and use proper risk management.\n\n---\n\n## 📖 What's Inside?\n\n### 🧠 Deep Q-Network (DQN)\nThe \"brain\" that makes trading decisions.\n\n- **8 possible actions** (ATR multipliers: 0.3 → 1.5)\n- **4-layer MLP** (Multi-Layer Perceptron): 24 → 16 → 8 → 4 neurons\n- **Q-values** predict expected reward for each action\n- **Epsilon-greedy** exploration (10% → 2% decay)\n\n### 🔮 LSTM Neural Network\nUnderstands temporal patterns and market context.\n\n- **24 hidden units** (configurable)\n- **Dynamic timesteps** (8-20 bars, adapts to volatility)\n- **4 gates**: Forget, Input, Cell, Output\n- **Backpropagation Through Time (BPTT)**\n\n### 💾 Prioritized Experience Replay (PER)\nSmart memory that focuses on important lessons.\n\n- **70,000 state buffer** (replay memory)\n- **Prioritized sampling** based on TD-error\n- **Importance sampling** for bias correction\n- **Beta annealing** (0.4 → 1.0)\n\n### 🎯 Adam Optimizer\nState-of-the-art optimization for neural networks.\n\n- **Adaptive learning rate** (starts at 0.01)\n- **Momentum** + **RMSprop** combined\n- **Gradient clipping** for stability\n- **Per-parameter learning rates**\n\n### 📊 Rich Feature Set\n20+ features extracted from market data:\n\n- Technical: RSI, MACD, ATR, Stochastic\n- Volume: OBV, Volume Rate of Change\n- Advanced: Ichimoku, VWAP, Hurst proxy\n- Volatility: Heidelberg index, ATR ratios\n- Custom: NN confidence, entropy\n\n---\n\n## 🏗 Architecture Overview\n\n```\nMarket Data\n    ↓\n[Feature Extraction] → 20 features\n    ↓\n[LSTM Layer] → Temporal patterns (8-20 timesteps)\n    ↓\n[MLP Network] → 24→16→8→4 neurons\n    ↓\n[Q-Values] → 8 actions (ATR multipliers)\n    ↓\n[Action Selection] → Epsilon-greedy\n    ↓\n[SuperTrend] → Adaptive coefficient\n    ↓\nTrading Signals\n    ↓\n[Reward] → (close - entry) / episode_length\n    ↓\n[Experience Replay] → Store in buffer (70k states)\n    ↓\n[PER Sampling] → Prioritize high TD-error\n    ↓\n[Backpropagation] → Update Q-network\n    ↓\n[LSTM BPTT] → Update LSTM weights\n```\n\n---\n\n## ⚡ Key Features\n\n### 1. **Real-Time Learning**\n- No pre-training needed\n- Learns continuously as market evolves\n- TD-Error-driven updates\n\n### 2. **Adaptive Parameters**\n- ATR multiplier: 0.3 - 1.5 (agent selects)\n- LSTM timesteps: 8-20 (volatility-based)\n- Learning rate: adaptive (0.001 - 0.05)\n\n### 3. **Advanced Techniques**\n- Priority Experience Replay (PER)\n- Backpropagation Through Time (BPTT)\n- Gradient clipping\n- Adaptive Hinge Loss with L2 penalty\n- Dual-kernel CNN filter\n\n### 4. **Robust Design**\n- Dropout (0.3) prevents overfitting\n- L2 regularization (0.0008 MLP, 0.0003 LSTM)\n- Leaky ReLU activation (no vanishing gradients)\n- Epsilon decay (0.10 → 0.02)\n\n---\n\n## 🔬 Technical Specifications\n\n### Reinforcement Learning Parameters\n\n```yaml\nState Space: 20-dimensional vector (5 features × 4 timesteps)\nAction Space: 8 discrete actions [0.3, 0.4, 0.5, 0.7, 0.9, 1.0, 1.2, 1.5]\nReward Function: (close - entry_price) / episode_length\nDiscount Factor (γ): 0.99\nEpsilon: 0.10 → 0.02 (decay: 0.999)\nTraining Frequency: Every 10 bars\n```\n\n### Network Architecture\n\n```yaml\nLSTM:\n  Hidden Size: 8 (default, configurable)\n  Timesteps: 8-20 (dynamic)\n  Gates: Forget, Input, Cell, Output\n  Activation: tanh (gates), sigmoid (cell)\n\nMLP (DQN):\n  Input: 20 features\n  Layer 1: 24 neurons (Leaky ReLU)\n  Layer 2: 16 neurons (Leaky ReLU)\n  Layer 3: 8 neurons (Leaky ReLU)\n  Layer 4: 4 neurons (Leaky ReLU)\n  Output: 8 Q-values (linear)\n\nDropout: 0.3\nL2 Lambda: 0.0008 (MLP), 0.0003 (LSTM)\n```\n\n### Experience Replay\n\n```yaml\nBuffer Size: 70,000 transitions\nBatch Size: 6 samples\nPriority Alpha (α): 0.6\nPriority Beta (β): 0.4 → 1.0 (annealing)\nPriority Epsilon: 1e-5\n```\n\n### Optimizer\n\n```yaml\nType: Adam\nLearning Rate: 0.01 (adaptive: 0.001 - 0.05)\nBeta1: 0.9 (momentum)\nBeta2: 0.999 (RMSprop)\nEpsilon: 1e-8\nGradient Clip: 1.0\n```\n\n---\n\n## 🚀 Quick Start\n\n### Installation (TradingView)\n\n1. Open [TradingView](https://www.tradingview.com)\n2. Navigate to Pine Editor (bottom panel)\n3. Create new indicator\n4. Copy-paste code from `ml_supertrend_ultimate.pine`\n5. Click \"Add to Chart\"\n\n### First Run\n\n1. **Initial training**: Wait for 200-500 updates\n2. **Monitor EMA Error**: Should decrease over time\n3. **Watch TD-Error**: Convergence indicator\n4. **Enable debug panel**: See learning metrics\n\n### Recommended Settings\n\n```yaml\nTimeframe: H1 (1 hour) or H4 (4 hours)\nAsset: BTC, ETH, major forex pairs\nHistory: At least 1000 bars for initial training\nAuto Optimize: Enabled\nShow Debug Panel: Enabled (while learning)\n```\n\n---\n\n## 📊 Performance Metrics\n\nThe system tracks several metrics to show learning progress:\n\n### Training Metrics\n\n- **TD-Error**: Should decrease from ~0.5 to \u003c0.1\n- **EMA Error**: Smoothed error, should converge\n- **Update Count**: Number of gradient updates\n- **Epsilon**: Exploration rate (10% → 2%)\n\n### Q-Value Metrics\n\n- **Avg Max Q**: Average of maximum Q-values\n- **Avg Old Q**: Average of current Q-predictions\n- **Avg Target Q**: Average of target Q-values\n- **Zero TD Count**: How many samples have TD-error ≈ 0\n\n### Example Learning Curve\n\n```\nUpdates 0-500:\n  TD-Error: 0.5 → 0.3 (high, exploring)\n  EMA Error: 0.7 → 0.5 (decreasing)\n  Epsilon: 0.10 → 0.08 (still exploring)\n\nUpdates 500-2000:\n  TD-Error: 0.3 → 0.15 (converging)\n  EMA Error: 0.5 → 0.2 (good convergence)\n  Epsilon: 0.08 → 0.04 (exploitation phase)\n\nUpdates 2000+:\n  TD-Error: 0.15 → 0.05 (converged!)\n  EMA Error: 0.2 → 0.1 (stable)\n  Epsilon: 0.04 → 0.02 (minimal exploration)\n```\n\n---\n\n## 🎓 Educational Value\n\nPerfect for learning:\n\n- How **LSTM** networks work\n- **Deep Q-Learning** implementation from scratch\n- **Reinforcement Learning** for trading\n- **Neural network training** (Adam, BPTT)\n- **Experience Replay** and prioritization\n- Advanced ML techniques in constrained environment\n\n### Code Structure\n\n```\n📁 Project Root\n├── 📄 ml_supertrend_ultimate.pine  (Main indicator)\n├── 📄 README.md                     (This file)\n├── 📄 LICENSE                       (MIT)\n├── 📄 CHANGELOG.md                  (Version history)\n├── 📁 docs/\n│   ├── 📄 ARCHITECTURE.md          (Detailed architecture)\n│   ├── 📄 TRAINING.md              (Training guide)\n│   ├── 📄 FAQ.md                   (Common questions)\n│   └── 📄 RESEARCH.md              (Research notes)\n└── 📁 images/\n    ├── 🖼️ screenshot_1.png         (Trading signals)\n    ├── 🖼️ screenshot_2.png         (Debug panel)\n    └── 🖼️ architecture.png         (System diagram)\n```\n\n---\n\n## 🤝 Contributing\n\nContributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Ways to Contribute\n\n- 🐛 **Bug reports** - Found an issue? Open an issue!\n- 💡 **Feature requests** - Have an idea? Share it!\n- 📝 **Documentation** - Improve README, add examples\n- 🔧 **Code** - Submit pull requests\n- ⭐ **Star the repo** - Show your support!\n\n### Development\n\n```bash\ngit clone https://github.com/YOUR_USERNAME/ml-supertrend-ultimate.git\ncd ml-supertrend-ultimate\n# Edit ml_supertrend_ultimate.pine\n# Test on TradingView\n# Submit pull request\n```\n\n---\n\n## 📚 References\n\nThis project implements techniques from cutting-edge research:\n\n1. **Deep Q-Learning**\n   - [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/abs/1312.5602) (Mnih et al., 2013)\n\n2. **Prioritized Experience Replay**\n   - [Prioritized Experience Replay](https://arxiv.org/abs/1511.05952) (Schaul et al., 2015)\n\n3. **LSTM Networks**\n   - [Long Short-Term Memory](https://www.bioinf.jku.at/publications/older/2604.pdf) (Hochreiter \u0026 Schmidhuber, 1997)\n\n4. **Adam Optimizer**\n   - [Adam: A Method for Stochastic Optimization](https://arxiv.org/abs/1412.6980) (Kingma \u0026 Ba, 2014)\n\n---\n\n## 📞 Contact \u0026 Support\n\n- **GitHub Issues**: [Report bugs or request features](https://github.com/YOUR_USERNAME/ml-supertrend-ultimate/issues)\n- **GitHub Discussions**: [Ask questions, share ideas](https://github.com/YOUR_USERNAME/ml-supertrend-ultimate/discussions)\n- **Email**: sail-com@mail.ru\n\n---\n\n## ⭐ Show Your Support\n\nIf you find this project useful:\n\n- ⭐ **Star the repository**\n- 🔄 **Share with others**\n- 📝 **Write about it**\n- 🤝 **Contribute**\n\n---\n\n## 📝 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n```\nMIT License\n\nCopyright (c) 2025 [Diogenov Pavel]\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\n[Full MIT License text in LICENSE file]\n```\n\n---\n\n## 🙏 Acknowledgments\n\n- **Created with**: Claude Sonnet 4.5 by Anthropic 🤖\n- **Inspired by**: DeepMind's DQN research\n- **Built in**: Altai Krai, Barnaul, Russia 🇷🇺\n- **For**: The trading \u0026 ML community 🌍\n\n---\n\n## 📈 Roadmap\n\n### v1.0 (Current)\n- ✅ LSTM + DQN implementation\n- ✅ Prioritized Experience Replay\n- ✅ Adam optimizer\n- ✅ Real-time training\n\n### v1.1 (Planned)\n- [ ] Multi-asset support\n- [ ] Improved reward shaping\n- [ ] Advanced visualization\n- [ ] Performance analytics\n\n### v2.0 (Future)\n- [ ] Dueling DQN architecture\n- [ ] Double Q-Learning\n- [ ] Rainbow DQN\n- [ ] Attention mechanisms\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Made with ❤️ in Russia 🇷🇺**\n\n**Star ⭐ this repo if you found it useful!**\n\n[![Star History Chart](https://api.star-history.com/svg?repos=YOUR_USERNAME/ml-supertrend-ultimate\u0026type=Date)](https://star-history.com/#YOUR_USERNAME/ml-supertrend-ultimate\u0026Date)\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpavelml-dev%2Fml-trading-systems","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpavelml-dev%2Fml-trading-systems","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpavelml-dev%2Fml-trading-systems/lists"}