{"id":30592018,"url":"https://github.com/sparkience-ai/spark-insights","last_synced_at":"2025-10-10T04:05:47.067Z","repository":{"id":312075509,"uuid":"1046204878","full_name":"Sparkience-AI/spark-insights","owner":"Sparkience-AI","description":"An open-source framework for automated AI-driven data analysis and reporting.","archived":false,"fork":false,"pushed_at":"2025-08-28T11:24:34.000Z","size":23709,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-28T17:48:52.815Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sparkience-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-28T10:46:44.000Z","updated_at":"2025-08-28T11:24:37.000Z","dependencies_parsed_at":"2025-08-28T17:59:05.119Z","dependency_job_id":null,"html_url":"https://github.com/Sparkience-AI/spark-insights","commit_stats":null,"previous_names":["sparkience-ai/spark-insights"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Sparkience-AI/spark-insights","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sparkience-AI%2Fspark-insights","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sparkience-AI%2Fspark-insights/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sparkience-AI%2Fspark-insights/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sparkience-AI%2Fspark-insights/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sparkience-AI","download_url":"https://codeload.github.com/Sparkience-AI/spark-insights/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sparkience-AI%2Fspark-insights/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002684,"owners_count":26083441,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-29T17:51:34.409Z","updated_at":"2025-10-10T04:05:47.046Z","avatar_url":"https://github.com/Sparkience-AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚀 Spark Insights\r\n\r\n**Autonomous AI Data Scientist Assistant - Complete EDA, Visualization \u0026 ML Pipeline**\r\n\r\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\r\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\r\n[![SmolagentsAI](https://img.shields.io/badge/powered%20by-SmolagentsAI-blue.svg)](https://github.com/huggingface/smolagents)\r\n\r\n## 📋 Overview\r\n\r\nSpark Insights is an autonomous AI-powered data scientist that performs comprehensive Exploratory Data Analysis (EDA), generates intelligent visualizations, builds machine learning models, and creates professional PowerPoint presentations automatically. Built with SmolagentsAI framework and powered by OpenAI's advanced language models.\r\n\r\n## ✨ Key Features\r\n\r\n- **🤖 Autonomous Analysis**: Complete end-to-end data analysis without human intervention\r\n- **📊 Smart Data Processing**: Automatic data loading, cleaning, and preprocessing\r\n- **📈 Intelligent Visualizations**: Context-aware chart generation based on data characteristics\r\n- **🧠 ML Model Building**: Automatic model selection, training, and evaluation\r\n- **📄 PowerPoint Generation**: Professional presentation creation with embedded visualizations\r\n- **� Comprehensive EDA**: Statistical analysis, correlation discovery, and pattern recognition\r\n- **💡 Actionable Insights**: AI-generated recommendations and business insights\r\n- **🎯 Interactive Interface**: Simple command-line interface for data analysis\r\n\r\n## 🏗️ Project Structure\r\n\r\n```\r\nspark-insights/\r\n├── agent.py                   # Main AI agent configuration with SmolagentsAI\r\n├── main.py                    # Entry point and user interface\r\n├── config.py                  # Configuration and system prompts\r\n├── requirements.txt           # Python dependencies\r\n├── .env                       # Environment variables (API keys)\r\n├── tools/                     # Modular analysis tools\r\n│   ├── __init__.py           \r\n│   ├── file_handler.py       # Data loading and file processing\r\n│   ├── data_analysis.py      # Statistical analysis and EDA\r\n│   ├── visualization.py      # Chart generation with matplotlib/seaborn\r\n│   ├── ml_model.py          # Machine learning model building\r\n│   ├── report_generator.py   # PowerPoint presentation creation\r\n│   └── conversation_manager.py # Chat and conversation handling\r\n├── examples/                  # Sample datasets\r\n│   ├── sales.csv             # Sample sales dataset\r\n│   ├── amazon.csv            # Sample Amazon dataset\r\n│   └── README.txt            # Dataset descriptions\r\n├── plots/                     # Generated visualizations (auto-created)\r\n├── artifacts/                 # ML model artifacts and outputs\r\n├── env/                       # Python virtual environment\r\n└── __pycache__/              # Python cache files\r\n```\r\n\r\n## 🧠 AI Agent Architecture\r\n\r\nThe system uses **SmolagentsAI** framework with the following tools:\r\n\r\n- **FileHandlerTool**: Intelligent data loading for CSV/Excel files\r\n- **DataAnalysisTool**: Comprehensive statistical analysis and EDA\r\n- **VisualizationTool**: Context-aware chart generation\r\n- **MLModelTool**: Automated machine learning pipeline\r\n- **ReportGeneratorTool**: Professional PowerPoint creation\r\n- **ConversationManagerTool**: Interactive chat capabilities\r\n\r\n## 🚀 Quick Start\r\n\r\n### Prerequisites\r\n\r\n- Python 3.8 or higher\r\n- OpenAI API key\r\n- Git (for cloning)\r\n\r\n### Installation\r\n\r\n```bash\r\n# Clone the repository\r\ngit clone https://github.com/your-repo/spark-insights.git\r\ncd spark-insights\r\n\r\n# Create virtual environment\r\npython -m venv env\r\n# On Windows:\r\nenv\\Scripts\\activate\r\n# On macOS/Linux:\r\nsource env/bin/activate\r\n\r\n# Install dependencies\r\npip install -r requirements.txt\r\n```\r\n\r\n### Configuration\r\n\r\nCreate a `.env` file in the project root:\r\n\r\n```env\r\n# OpenAI Configuration\r\nOPENAI_API_KEY=your_openai_api_key_here\r\n```\r\n\r\n### Usage\r\n\r\n```bash\r\n# Run the AI Data Scientist\r\npython main.py\r\n```\r\n\r\nThe system will prompt you for a data file path and then autonomously:\r\n\r\n1. **Load and explore** your dataset\r\n2. **Perform comprehensive EDA** with statistical analysis\r\n3. **Generate intelligent visualizations** based on your data\r\n4. **Build relevant ML models** (classification/regression/clustering)\r\n5. **Extract actionable insights** and patterns\r\n6. **Create a PowerPoint presentation** with all findings\r\n\r\n### Example Analysis Flow\r\n\r\n```\r\n🚀 Welcome to Spark Insights! Your Autonomous AI Data Scientist.\r\n\r\n📊 This system will automatically:\r\n   • Perform comprehensive Exploratory Data Analysis (EDA)\r\n   • Generate dataset-specific visualizations\r\n   • Build relevant ML models\r\n   • Extract meaningful insights\r\n   • Create a custom PowerPoint presentation\r\n\r\n📁 Please provide your data file path (CSV/Excel):\r\n\u003e examples/sales.csv\r\n\r\n🔄 Starting autonomous data analysis...\r\n✅ Analysis Complete!\r\n🎯 PowerPoint Report Created!\r\n📄 File: analysis_report.pptx\r\n```\r\n\r\n## � What Spark Insights Delivers\r\n\r\n### Automated Analysis Pipeline\r\n\r\n1. **Data Quality Assessment**\r\n   - Missing value analysis\r\n   - Data type detection\r\n   - Statistical summaries\r\n   - Outlier identification\r\n\r\n2. **Exploratory Data Analysis**\r\n   - Correlation analysis\r\n   - Distribution analysis\r\n   - Categorical variable exploration\r\n   - Time series analysis (if applicable)\r\n\r\n3. **Intelligent Visualizations**\r\n   - Histograms and distribution plots\r\n   - Correlation heatmaps\r\n   - Box plots and violin plots\r\n   - Scatter plots and trend analysis\r\n   - Bar charts for categorical data\r\n\r\n4. **Machine Learning Pipeline**\r\n   - Automatic problem type detection (classification/regression)\r\n   - Feature engineering and selection\r\n   - Model training and evaluation\r\n   - Feature importance analysis\r\n   - Performance metrics and confusion matrices\r\n\r\n5. **Professional Reporting**\r\n   - Executive summary\r\n   - Dataset overview\r\n   - Visualization slides with explanations\r\n   - ML model results\r\n   - Key insights and recommendations\r\n   - Actionable conclusions\r\n\r\n### Sample Output\r\n\r\nYour analysis will generate:\r\n- **Multiple PNG visualizations** in `plots/` directory\r\n- **analysis_report.pptx** - Professional PowerPoint presentation\r\n- **Model artifacts** in `artifacts/` directory\r\n- **Console output** with step-by-step progress\r\n\r\n## 🛠️ Technical Stack\r\n\r\n- **AI Framework**: SmolagentsAI (Hugging Face)\r\n- **LLM**: OpenAI GPT models\r\n- **Data Processing**: Pandas, NumPy\r\n- **Visualization**: Matplotlib, Seaborn, Plotly\r\n- **Machine Learning**: Scikit-learn\r\n- **Reporting**: python-pptx\r\n- **Configuration**: python-dotenv, PyYAML\r\n\r\n## 📖 Documentation\r\n\r\nFor detailed documentation on each component:\r\n\r\n- [Tools Documentation](tools/README.md) - Individual tool descriptions\r\n- [Configuration Guide](config.py) - System prompts and settings\r\n- [Example Datasets](examples/README.txt) - Sample data descriptions\r\n\r\n## 🤝 Contributing\r\n\r\n1. Fork the repository\r\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\r\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\r\n4. Push to the branch (`git push origin feature/amazing-feature`)\r\n5. Open a Pull Request\r\n\r\n### Development Guidelines\r\n\r\n- Follow Python PEP 8 style guidelines\r\n- Add docstrings to all functions and classes\r\n- Test your changes with sample datasets\r\n- Update documentation for new features\r\n\r\n## 🚨 Requirements\r\n\r\n```python\r\n# Core AI and ML\r\nsmolagents\u003e=0.1.0\r\npandas\u003e=2.0.0\r\nnumpy\u003e=1.24.0\r\nscikit-learn\u003e=1.3.0\r\nmatplotlib\u003e=3.7.0\r\nseaborn\u003e=0.12.0\r\n\r\n# LLM Integration\r\nopenai\u003e=1.0.0\r\npython-dotenv\u003e=1.0.0\r\n\r\n# Reporting\r\npython-pptx\u003e=0.6.21\r\n\r\n# Additional utilities\r\nrequests\u003e=2.31.0\r\nclick\u003e=8.1.0\r\nrich\u003e=13.0.0\r\n```\r\n\r\n## � Troubleshooting\r\n\r\n### Common Issues\r\n\r\n1. **API Key Error**\r\n   ```\r\n   Error: OpenAI API key not found\r\n   ```\r\n   Solution: Ensure your `.env` file contains `OPENAI_API_KEY=your_key_here`\r\n\r\n2. **File Not Found**\r\n   ```\r\n   ❌ File not found: data.csv\r\n   ```\r\n   Solution: Provide the full path to your data file\r\n\r\n3. **Memory Issues with Large Files**\r\n   ```\r\n   MemoryError: Unable to allocate array\r\n   ```\r\n   Solution: Use smaller datasets or increase system memory\r\n\r\n## �📄 License\r\n\r\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\r\n\r\n## 🙏 Acknowledgments\r\n\r\n- **SmolagentsAI** by Hugging Face for the agent framework\r\n- **OpenAI** for advanced language model capabilities\r\n- **Python Data Science Community** for excellent libraries\r\n- Built with modern Python best practices for autonomous data analysis \r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsparkience-ai%2Fspark-insights","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsparkience-ai%2Fspark-insights","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsparkience-ai%2Fspark-insights/lists"}