{"id":30269021,"url":"https://github.com/donpushme/price-predictor-updated","last_synced_at":"2025-10-17T02:00:08.722Z","repository":{"id":309498784,"uuid":"1032612703","full_name":"donpushme/price-predictor-updated","owner":"donpushme","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-13T02:31:00.000Z","size":79,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-06T21:40:58.533Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/donpushme.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-05T14:59:37.000Z","updated_at":"2025-08-13T02:31:03.000Z","dependencies_parsed_at":"2025-08-12T08:55:43.132Z","dependency_job_id":"a90cbeec-7248-4b29-809d-0ccdabdcb9a4","html_url":"https://github.com/donpushme/price-predictor-updated","commit_stats":null,"previous_names":["donpushme/price-predictor-updated"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/donpushme/price-predictor-updated","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donpushme%2Fprice-predictor-updated","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donpushme%2Fprice-predictor-updated/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donpushme%2Fprice-predictor-updated/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donpushme%2Fprice-predictor-updated/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/donpushme","download_url":"https://codeload.github.com/donpushme/price-predictor-updated/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donpushme%2Fprice-predictor-updated/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279270317,"owners_count":26137726,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-17T02:00:07.504Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-16T01:38:39.159Z","updated_at":"2025-10-17T02:00:08.691Z","avatar_url":"https://github.com/donpushme.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cryptocurrency and Commodity Volatility Prediction System\n\nA comprehensive AI-powered system for predicting volatility, skewness, and kurtosis of cryptocurrency and commodity price changes using PyTorch neural networks. Designed for Monte Carlo simulation with 288-step forecasting (24 hours in 5-minute intervals).\n\n## Features\n\n- **Multi-Asset Support**: Bitcoin, Ethereum, Solana, and XAU (Gold)\n- **288-Step Forecasting**: Predicts 24 hours ahead in 5-minute intervals\n- **Statistical Moments**: Predicts volatility, skewness, and kurtosis\n- **Real-Time Prediction**: Single input generates full 24-hour forecast\n- **Monte Carlo Ready**: Outputs designed for Monte Carlo simulation\n- **Database Integration**: MongoDB for scalable data and prediction storage\n- **Confidence Intervals**: Monte Carlo dropout for uncertainty estimation\n- **Time-Aware Models**: Captures US trading hour volatility patterns\n\n## Project Structure\n\n```\nprice-predict/\n├── model.py                 # Neural network architecture\n├── trainer.py               # Model training functionality\n├── predictor.py             # Real-time prediction\n├── data_processor.py        # Data preprocessing and feature engineering\n├── database_manager.py      # MongoDB operations\n├── multi_trainer.py         # Multi-asset training script\n├── multi_predictor.py       # Multi-asset prediction script\n├── config.py                # Configuration settings\n├── requirements.txt         # Python dependencies\n├── README.md               # This file\n├── training_data/          # Training data directory (create and add CSV files)\n│   ├── bitcoin_5min.csv\n│   ├── ethereum_5min.csv\n│   ├── solana_5min.csv\n│   └── xau_5min.csv\n├── models/                 # Saved models directory (auto-created)\n└── predictions/            # Prediction outputs directory (auto-created)\n```\n\n## Installation\n\n1. **Clone or download the project files**\n\n2. **Install dependencies**:\n```bash\npip install -r requirements.txt\n```\n\n3. **Set up MongoDB**:\n```bash\n# Automatic setup (Ubuntu/macOS/CentOS)\npython3 mongodb_setup.py\n\n# Or install manually:\n# Ubuntu: sudo apt-get install mongodb-org\n# macOS: brew install mongodb-community\n# Windows: Download from mongodb.com\n```\n\n4. **Prepare training data**:\n   - Create `training_data/` directory\n   - Add CSV files with the following format:\n\n```csv\ntimestamp,open,high,low,close\n2022-02-03 14:50:00,36551.864,36618.271,36500.254,36500.254\n2022-02-03 14:55:00,36500.254,36565.123,36480.567,36520.890\n...\n```\n\n## Usage\n\n### 1. Training Models\n\n#### Train all assets:\n```bash\npython multi_trainer.py --epochs 100 --batch-size 32\n```\n\n#### Train specific asset:\n```bash\npython multi_trainer.py --asset bitcoin --epochs 100\n```\n\n#### Training with custom parameters:\n```bash\npython multi_trainer.py \\\n    --data-dir training_data \\\n    --models-dir models \\\n    --epochs 150 \\\n    --batch-size 64 \\\n    --learning-rate 0.001 \\\n    --patience 25 \\\n    --device cuda\n```\n\n### 2. Making Predictions\n\n#### Single asset prediction:\n```bash\npython multi_predictor.py --asset bitcoin --price 45000\n```\n\n#### Multi-asset prediction:\n```bash\npython multi_predictor.py --prices '{\"bitcoin\": 45000, \"ethereum\": 3000, \"solana\": 100, \"xau\": 2000}'\n```\n\n#### Prediction with confidence intervals:\n```bash\npython multi_predictor.py --asset bitcoin --price 45000 --confidence\n```\n\n#### Generate Monte Carlo scenarios:\n```bash\npython multi_predictor.py --prices '{\"bitcoin\": 45000}' --scenarios 1000\n```\n\n#### Save predictions to file:\n```bash\npython multi_predictor.py --asset bitcoin --price 45000 --output predictions/bitcoin_forecast.json\n```\n\n### 3. Using the API\n\n```python\nfrom multi_predictor import MultiAssetPredictor\nfrom datetime import datetime\n\n# Initialize predictor\npredictor = MultiAssetPredictor(models_dir=\"models\")\n\n# Make prediction\ncurrent_time = datetime.now()\nprediction = predictor.predict_single_asset(\n    asset=\"bitcoin\",\n    current_price=45000.0,\n    current_time=current_time\n)\n\n# Access predictions\nvolatility_forecast = prediction['volatility']  # 288 values\nskewness_forecast = prediction['skewness']      # 288 values\nkurtosis_forecast = prediction['kurtosis']      # 288 values\ntimestamps = prediction['timestamps']           # 288 timestamps\n\nprint(f\"Next 5-min volatility: {volatility_forecast[0]:.6f}\")\nprint(f\"24-hour max volatility: {max(volatility_forecast):.6f}\")\n```\n\n### 4. Monte Carlo Simulation Example\n\n```python\nimport numpy as np\nfrom multi_predictor import MultiAssetPredictor\nfrom datetime import datetime\n\npredictor = MultiAssetPredictor()\n\n# Generate scenarios for simulation\nscenarios = predictor.generate_monte_carlo_scenarios(\n    assets=[\"bitcoin\", \"ethereum\"],\n    current_prices={\"bitcoin\": 45000, \"ethereum\": 3000},\n    current_time=datetime.now(),\n    n_scenarios=1000\n)\n\n# Use scenarios in Monte Carlo simulation\nfor scenario in scenarios[\"bitcoin\"]:\n    volatility_path = scenario['volatility']  # 288 values\n    skewness_path = scenario['skewness']      # 288 values\n    kurtosis_path = scenario['kurtosis']      # 288 values\n    \n    # Your simulation logic here\n    # Generate price path using these statistical moments\n```\n\n## Model Architecture\n\nThe system uses a sophisticated neural network architecture:\n\n- **Time Embedding**: Captures cyclical patterns (hour, day of week, US trading hours)\n- **Price Embedding**: Processes OHLC data and technical indicators\n- **LSTM Layers**: Models temporal dependencies\n- **Multi-Head Attention**: Focuses on relevant patterns\n- **Multiple Output Heads**: Separate predictors for volatility, skewness, and kurtosis\n\n### Key Features:\n- **Real-life Volatility Patterns**: Models higher volatility during US trading hours\n- **Technical Indicators**: RSI, moving averages, Parkinson volatility estimator\n- **Statistical Moments**: Rolling calculations of volatility, skewness, and kurtosis\n- **Gradient Clipping**: Prevents exploding gradients\n- **Early Stopping**: Prevents overfitting\n- **Learning Rate Scheduling**: Adaptive learning rate\n\n## Data Requirements\n\n### Input Format\nCSV files with 5-minute OHLC data:\n- `timestamp`: ISO format datetime\n- `open`: Opening price\n- `high`: Highest price\n- `low`: Lowest price  \n- `close`: Closing price\n\n### Data Processing\nThe system automatically:\n- Calculates returns and log returns\n- Computes technical indicators\n- Generates time-based features\n- Calculates rolling statistical moments\n- Normalizes features for training\n\n## Configuration\n\nModify `config.py` to customize:\n- Model architecture parameters\n- Training hyperparameters\n- Data processing settings\n- Asset-specific configurations\n- Loss function weights\n- Prediction bounds\n\n## Database\n\nThe system uses MongoDB for:\n- **Price Data Storage**: Historical OHLC data with calculated features in time-series collections\n- **Prediction Storage**: All forecasts with metadata and arrays stored as documents\n- **Model Metadata**: Training configurations and performance metrics\n- **Training History**: Loss curves and training statistics\n- **Scalability**: Better performance for large datasets and real-time predictions\n- **Indexing**: Optimized time-based queries and symbol lookups\n\n## Performance Monitoring\n\nTrack model performance through:\n- Training/validation loss curves\n- Individual loss components (volatility, skewness, kurtosis)\n- Learning rate schedules\n- Early stopping triggers\n- Model comparison metrics\n\n## Troubleshooting\n\n### Common Issues:\n\n1. **CUDA out of memory**: Reduce batch size or model hidden dimension\n2. **Training data not found**: Check file paths in `training_data/` directory\n3. **Model loading failed**: Ensure model was trained and saved properly\n4. **Prediction errors**: Verify data processor is fitted and price history is available\n5. **MongoDB connection failed**: Ensure MongoDB is running (`python3 mongodb_setup.py --check-only`)\n6. **Database errors**: Check MongoDB service status and connection settings in `config.py`\n\n### Debug Options:\n- Use smaller models for testing (`hidden_dim=64, num_layers=1`)\n- Reduce sequence length for memory constraints\n- Check data quality and preprocessing steps\n- Validate input data format and ranges\n\n## Technical Details\n\n### Neural Network\n- **Input**: Price sequences (100 timesteps) + time features\n- **Output**: 288-step forecasts for each statistical moment\n- **Architecture**: LSTM + Attention + Multiple output heads\n- **Regularization**: Dropout, weight decay, gradient clipping\n\n### Time Features\n- Hour/minute cyclical encoding\n- Day of week patterns\n- US trading hours indicator\n- Weekend/weekday classification\n\n### Statistical Moments\n- **Volatility**: Standard deviation of returns (always positive)\n- **Skewness**: Asymmetry measure (bounded by tanh)\n- **Kurtosis**: Tail heaviness (minimum 3.0 for normal distribution)\n\n### Loss Function\nWeighted combination:\n- 60% Volatility loss (most important for Monte Carlo)\n- 20% Skewness loss\n- 20% Kurtosis loss\n\n## Citation\n\nIf you use this system in your research, please cite:\n\n```\nCryptocurrency and Commodity Volatility Prediction System\nMulti-Asset Neural Network for Monte Carlo Simulation\n[Your Name/Organization], 2024\n```\n\n## License\n\nThis project is for educational and research purposes. Please ensure compliance with data usage rights and applicable regulations when using with real financial data.\n\n## Support\n\nFor issues and questions:\n1. Check the troubleshooting section\n2. Verify your data format matches requirements\n3. Test with smaller datasets first\n4. Check GPU memory usage and requirements\n\n## Future Enhancements\n\nPotential improvements:\n- Additional asset support\n- Real-time data feeds\n- Enhanced uncertainty quantification\n- Ensemble models\n- Alternative architectures (Transformers, GRU)\n- Risk management integration","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonpushme%2Fprice-predictor-updated","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdonpushme%2Fprice-predictor-updated","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonpushme%2Fprice-predictor-updated/lists"}