https://github.com/willie-conway/datavista

DataVista is a comprehensive, production-grade data analysis and machine learning platform that combines real-time data ingestion from live APIs, interactive visualizations, statistical analysis, hypothesis testing, and machine learning model training — all in a unified, professional-grade interface. Built with React and Recharts.
https://github.com/willie-conway/datavista
analytics-platform api-integration classification coingecko-api csv-import data-analysis data-cleaning-and-preprocessing data-pipeline data-science data-visualizations etl hypothesis-testing json-export machine-learning-models open-meteo react recharts regression statistics world-bank
Last synced: 21 days ago
JSON representation
Host: GitHub
URL: https://github.com/willie-conway/datavista
Owner: Willie-Conway
License: mit
Created: 2025-05-23T18:47:19.000Z (about 1 year ago)
Default Branch: main
Last Pushed: 2026-03-08T04:43:14.000Z (3 months ago)
Last Synced: 2026-05-01T04:34:09.299Z (about 2 months ago)
Topics: analytics-platform, api-integration, classification, coingecko-api, csv-import, data-analysis, data-cleaning-and-preprocessing, data-pipeline, data-science, data-visualizations, etl, hypothesis-testing, json-export, machine-learning-models, open-meteo, react, recharts, regression, statistics, world-bank
Language: JavaScript
Homepage: https://willie-conway.github.io/DataVista/
Size: 10.6 MB
Stars: 1
Watchers: 0
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

README

          [![Deploy to GitHub Pages](https://github.com/Willie-Conway/DataVista/actions/workflows/deploy.yml/badge.svg)](https://github.com/Willie-Conway/DataVista/actions/workflows/deploy.yml)

# 📊 DataVista — Real-Time Data Analysis & Machine Learning Platform

![alt text](Screenshots/DataVista.png)

![DataVista](https://img.shields.io/badge/DataVista-Real_Time_Data_Analysis-3b82f6?style=for-the-badge&logo=python&logoColor=white)

![React](https://img.shields.io/badge/React-18-61DAFB?style=for-the-badge&logo=react&logoColor=white)

![Recharts](https://img.shields.io/badge/Recharts-Visualizations-10b981?style=for-the-badge&logo=chartdotjs&logoColor=white)

![API Integration](https://img.shields.io/badge/API-3_Live_Sources-f59e0b?style=for-the-badge&logo=api&logoColor=white)



  

  

  

  



---

### **About** 📊

**DataVista** is a comprehensive, production-grade data analysis and machine learning platform that combines real-time data ingestion from live APIs, interactive visualizations, statistical analysis, hypothesis testing, and machine learning model training — all in a unified, professional-grade interface. Built with React and Recharts, DataVista features user authentication, persistent storage, CSV/JSON import/export, data cleaning pipelines, and a full ML workflow from feature selection to model evaluation. Designed for data scientists, analysts, and machine learning engineers. 📈

---

## ✨ Key Features

### 🎓 **11 Core Modules**

| Module | Focus | Key Capabilities |

|--------|-------|------------------|

| **01 — Dashboard** | Workspace Overview | Dataset stats, pipeline status, recent activity |

| **02 — Data Sources** | Live API Integration | World Bank, Open-Meteo Weather, CoinGecko Crypto |

| **03 — Data Explorer** | Dataset Inspection | Browse, sort, filter, export CSV/JSON |

| **04 — Cleaning** | Data Quality | Missing values, duplicates, outlier removal |

| **05 — Preprocessing** | ML Preparation | Scaling, encoding, feature selection, train/test split |

| **06 — Statistics** | Descriptive Stats | Mean, median, std dev, quartiles, full summary |

| **07 — Visualization** | Interactive Charts | Line, Bar, Area, Scatter, Pie (6 chart types) |

| **08 — ML Models** | Machine Learning | Train regression models, R², RMSE, MAE metrics |

| **09 — Hypothesis** | Statistical Tests | T-test, Chi-Square, ANOVA with p-value interpretation |

| **10 — Reports** | Export & Documentation | Download reports from each analysis stage |

| **11 — Profile** | User Management | Persistent profile, avatar upload, account settings |

---

## 📊 **Module 01: Dashboard — Workspace Overview**

### **Key Performance Indicators** 📈

- **Datasets** — Total in workspace

- **Total Rows** — Across all datasets

- **Active Dataset** — Current selection with row/column count

- **ML Model Status** — Trained vs untrained

### **Workspace Datasets** 📋

- **List all imported/live datasets** with metadata

- **One-click actions**: Open, Export CSV, Export JSON, Delete

- **Source badges**: World Bank (🔵), Weather (🟢), Crypto (🟡), File (🟣)

### **Pipeline Status** 🔄

- **6-step pipeline tracker**:

  1. Data Loaded ✓

  2. Dataset Selected ✓

  3. Stats Run

  4. ML Trained

  5. Hypothesis Tested

  6. Visualization Built

- **Visual progress indicators** with checkmarks

![alt text](Screenshots/User_Dashboard.png)

---

## 📡 **Module 02: Data Sources — Live API Integration**

### **3 Live API Sources** 🌍

| Source | API | Data | Features |

|--------|-----|------|----------|

| **World Bank** | `api.worldbank.org/v2` | GDP, Population, Inflation, Unemployment, Energy Use, Internet Users, Literacy, Life Expectancy | 200+ countries, 8 indicators |

| **Open-Meteo Weather** | `api.open-meteo.com/v1` | Temperature, Humidity, Wind Speed | 7-day hourly forecast, 6 global cities |

| **CoinGecko Crypto** | `api.coingecko.com/api/v3` | Price, Market Cap, Volume, 24h Change | Top 30 cryptocurrencies |

### **File Import** 📁

- **CSV/JSON upload** with drag & drop interface

- **Progress bar** during import

- **Automatic parsing** of headers and data types

- **Limit**: 5,000 rows per file

### **Data Preview** 👁️

- **Live table preview** after API fetch

- **One-click** "Add to Workspace"

- **Visual preview** for weather data (area chart)

![alt text]()

---

## 🔍 **Module 03: Data Explorer — Dataset Inspection**

### **Dataset Selection** 🎯

- **Dropdown** with all available datasets

- **Metadata chips**: Row count, column count, source badge

- **Export buttons**: CSV, JSON

### **Interactive Table** 📋

- **Sticky headers** for easy navigation

- **Horizontal scrolling** for wide datasets

- **Vertical scrolling** with max height 520px

- **Limited preview** (100 rows) with "Showing X of Y" indicator

- **Monospaced font** for numeric values

- **Hover effects** on rows

![alt text]()

---

## 🧹 **Module 04: Data Cleaning**

### **Missing Value Strategies** ❓

| Strategy | Description |

|----------|-------------|

| **Drop** | Remove rows with any missing value |

| **Mean** | Fill numeric columns with column mean |

| **Median** | Fill numeric columns with column median |

| **Keep** | Leave missing values as-is |

### **Outlier Removal** 📊

| Method | Description |

|--------|-------------|

| **Z-score** | Remove values with |z| > 3 |

| **IQR** | Remove values outside 1.5×IQR range |

### **Duplicate Removal** 🔄

- **Toggle checkbox** to remove duplicate rows

- **JSON-based comparison** for exact matches

### **Data Quality Report** 📋

- **Per-column analysis**:

  - Column name

  - Type (numeric/text)

  - Missing count and percentage

  - Unique value count

- **Color coding**: 🟢 numeric, 🟣 text

![alt text]()

---

## ⚙️ **Module 05: Preprocessing for Machine Learning**

### **4 Key Preprocessing Steps** 🔧

| Step | Options | Description |

|------|---------|-------------|

| **Feature Scaling** | None, Min-Max, Z-score, Robust | Scale numeric features to common range |

| **Categorical Encoding** | None, One-Hot, Label, Ordinal | Convert text categories to numbers |

| **Dimensionality Reduction** | None, PCA, Variance Filter, Correlation Filter | Reduce number of features |

| **Train/Test Split** | 80/20, 70/30, 60/40, 90/10 | Ratio for model training vs evaluation |

### **Column Summary** 📊

- **Visual cards** for each column:

  - Column name (monospaced)

  - Type badge (numeric/categorical)

  - Unique value count

- **Color coding**: 🟢 numeric, 🟣 categorical

![alt text](Screenshots/Preprocessing.png)

---

## ∑ **Module 06: Descriptive Statistics**

### **Statistical Summary** 📈

- **Full statistics** for all numeric columns:

  - Count

  - Mean

  - Median

  - Standard Deviation

  - Min

  - Q1 (25th percentile)

  - Q3 (75th percentile)

  - Max

### **One-Click Calculation** ⚡

- **"Calculate Statistics"** button

- **"Download Report"** button for text export

![alt text](Screenshots/Statistics.png)

---

## 📈 **Module 07: Interactive Visualization**

### **6 Chart Types** 📊

| Chart | Library | Use Case |

|-------|---------|----------|

| **Line Chart** | Recharts | Time series trends |

| **Bar Chart** | Recharts | Categorical comparisons |

| **Area Chart** | Recharts | Cumulative trends |

| **Scatter Plot** | Recharts | Correlation analysis |

| **Pie Chart** | Recharts | Part-to-whole relationships |

### **Chart Configuration** ⚙️

- **Dataset selector**

- **Chart type dropdown**

- **X-axis column selector**

- **Y-axis column selector** (numeric only)

### **Visual Features** ✨

- **Dark theme** charts matching platform

- **Tooltips** with formatted values

- **Responsive sizing** for all screen sizes

- **Pie chart labels** with truncated names

- **Scatter plot** with X/Y axes

![alt text](Screenshots/Visualization_1.png)

---

## 🤖 **Module 08: Machine Learning Models**

### **Model Configuration** 🛠️

- **Dataset selection**

- **Target column** (numeric only)

- **Algorithm options**:

  - Linear Regression

  - Random Forest

  - Support Vector Machine

  - K-Nearest Neighbors

  - Gradient Boosting

### **Model Performance Metrics** 📊

- **R² Score** (coefficient of determination)

- **RMSE** (Root Mean Square Error)

- **MAE** (Mean Absolute Error)

- **Training samples count**

- **Features used** count

### **Simulated Training** 🧠

- **Realistic metrics** with random variation

- **Progress bar** for R² score

- **Feature selection** from numeric columns

![alt text]()

---

## ⊛ **Module 09: Hypothesis Testing**

### **3 Statistical Tests** 📊

| Test | Use Case | Inputs |

|------|----------|--------|

| **Independent T-test** | Compare two groups | Variable 1, Variable 2 |

| **Chi-Square Test** | Categorical independence | Single variable |

| **One-Way ANOVA** | Compare multiple groups | Single variable (simulated) |

### **Test Results** 📋

- **Test statistic** (t, χ², F)

- **P-value** with color coding: 🟢 p<0.05, 🔴 p≥0.05

- **Conclusion**: Reject H₀ / Fail to Reject H₀

- **Interpretation** in plain English

- **Significance level**: α = 0.05 (two-tailed)

![alt text](Screenshots/Hypothesis.png)

---

## ◎ **Module 10: Report Generator**

### **6 Report Types** 📄

| Report | Content | Format |

|--------|---------|--------|

| **Data Quality** | Missing values, duplicates, type analysis | TXT |

| **Statistics** | Descriptive stats for all numeric columns | TXT |

| **ML Performance** | Model metrics, features, evaluation scores | TXT |

| **Hypothesis Test** | Test statistic, p-value, conclusion | TXT |

| **Full Pipeline** | Combined report of all completed steps | Multiple |

| **Dataset Export** | CSV/JSON of active dataset | CSV/JSON |

### **Report Preview** 👁️

- **Last generated report** displayed in terminal-style pre

- **Monospaced font** for readability

- **Scrollable** for long reports

![alt text](Screenshots/Reports.png)

---

## ◯ **Module 11: Profile Management**

### **User Profile** 👤

- **Avatar upload** with preview

- **Display name**

- **Email address**

- **Role / Title**

- **Bio** (textarea)

### **Persistent Storage** 💾

- **Local storage** for all user data

- **Avatar saved** as base64

- **Settings** persist across sessions

![alt text](Screenshots/Profile.png)

### **Dark mode** 🌑

- **Manual toggle** via button

- **Toggle** in settings panel

- **System preference** detection

- **Persistent** across sessions

![alt text]()

![alt text](Screenshots/Dashboard_Dark_mode.png)

---

## 🎨 **Design & Aesthetics**

### **Modern Data Science Platform** 🖥️

- **Dark theme** (`#080c14` background) — easy on the eyes for extended analysis

- **Blue accent** (`#3b82f6`) for primary actions

- **Green** (`#10b981`) for success and weather data

- **Amber** (`#f59e0b`) for crypto and warnings

- **Purple** (`#8b5cf6`) for ML models

- **Glass-morphism cards** with subtle borders

- **Custom scrollbars** for data tables

### **Typography** ✍️

- **Outfit** — UI text, body copy, buttons

- **JetBrains Mono** — Data tables, statistics, code blocks

### **Visual Elements** 🖼️

- **Animated loading spinners**

- **Progress bars** for ML metrics

- **Color-coded badges** for data sources

- **Hover effects** on cards and buttons

- **Fade-up animations** on page transitions

- **Toast notifications** for user feedback

### **Color Coding** 🎨

| Element | Color | Hex | Usage |

|---------|-------|-----|-------|

| **World Bank** | Blue | `#3b82f6` | GDP, economic indicators |

| **Weather** | Green | `#10b981` | Temperature, humidity, wind |

| **Crypto** | Amber | `#f59e0b` | Price, market cap, volume |

| **Files** | Purple | `#8b5cf6` | Imported CSV/JSON |

| **Primary** | Blue | `#3b82f6` | Buttons, active tabs |

| **Success** | Green | `#10b981` | Completed steps |

| **Warning** | Amber | `#f59e0b` | Missing data, warnings |

| **Error** | Red | `#ef4444` | Errors, high p-values |

---

## 🛠️ **Technical Implementation**

### **Tech Stack** 🥞

- **React 18** — Functional components with hooks

- **Recharts** — Data visualization library

- **Pure CSS** — Custom design system, no frameworks

- **LocalStorage API** — Persistent user data

- **FileReader API** — CSV/JSON import

### **React Hooks Used** 🎣

| Hook | Purpose |

|------|---------|

| `useState` | 25+ state variables for UI and data |

| `useEffect` | Initial data loading, theme sync |

| `useRef` | File input, avatar upload |

| `useCallback` | Memoized save function |

### **Custom Utilities** 🔧

- `parseCSV` — Convert CSV text to structured data

- `toCSV` — Convert data to CSV format

- `download` — Trigger file download

- `numericCols` — Identify numeric columns

- `descStats` — Calculate descriptive statistics

- `store` — Persistent storage wrapper

### **Storage Architecture** 💾

```javascript

const store = {

  async get(key) { /* retrieve from localStorage */ },

  async set(key, val) { /* save to localStorage */ }

};

```

### **Data Flow** 🔄

```

User Authentication → Workspace → Dataset Selection

                          ↓

              ┌──────────┴──────────┐

              ↓                      ↓

        Data Cleaning         Machine Learning

              ↓                      ↓

        Statistics             Model Training

              ↓                      ↓

      Visualization          Hypothesis Testing

              ↓                      ↓

          Reports                 Export

```

---

## 🎥 **Video Demo Script** (60-75 seconds)

| Time | Module | Scene | Action |

|------|--------|-------|--------|

| 0:00 | Auth | Login | Enter credentials → Dashboard loads |

| 0:05 | Dashboard | KPIs | Show 4 datasets, 2,847 rows, active dataset "GDP Data" |

| 0:10 | Data Sources | World Bank | Load GDP data → Preview table shows 40 countries |

| 0:15 | Data Sources | Weather | Fetch NYC forecast → Area chart updates |

| 0:20 | Data Sources | Crypto | Load top 30 coins → Table shows BTC, ETH, SOL |

| 0:25 | Data Explorer | Inspect | Browse GDP dataset with 100 rows preview |

| 0:30 | Cleaning | Quality | Show missing values: 2 columns with 12% missing |

| 0:35 | Cleaning | Apply | Run cleaning → New cleaned dataset appears |

| 0:40 | Statistics | Run | Calculate stats → Table shows mean, median, std dev |

| 0:45 | Visualization | Build | Line chart of GDP by country → Interactive hover |

| 0:50 | ML | Train | Linear regression on GDP → R² = 0.84, RMSE = 1.23M |

| 0:55 | Hypothesis | T-test | Run test on GDP vs Population → p=0.03 (significant) |

| 1:00 | Reports | Download | Generate Data Quality report → Preview shows 15 lines |

| 1:05 | Profile | Edit | Update avatar, name, role → Save persists |

---

## 🚦 **Performance**

- **Load Time**: < 2 seconds

- **Memory Usage**: < 60 MB

- **API Calls**: 3 simultaneous on demand

- **Dataset Limit**: 5,000 rows per file

### **Optimizations** ⚡

- **Data sampling** for visualizations (80-200 points)

- **Efficient re-renders** via React hooks

- **CSS animations** instead of JavaScript

- **Debounced** API calls

---

## 🛡️ **Security Notes**

DataVista is a **client-side only** application:

- ✅ No backend server

- ✅ No data transmission

- ✅ Local storage only for persistence

- ✅ No tracking or analytics

- ✅ All API calls made directly from browser

---

## 📝 **License**

MIT License — see LICENSE file for details.

---

## 🙏 **Acknowledgments**

- **World Bank** — Open data API

- **Open-Meteo** — Free weather API

- **CoinGecko** — Cryptocurrency data API

- **Recharts** — React charting library

- **JetBrains** — Mono font for code

---

## 📧 **Contact**

- **GitHub Issues**: [Create an issue](https://github.com/Willie-Conway/DataVista/issues)

- **Website**: https://willie-conway.github.io/DataVista/

---

## 🏁 **Future Enhancements**

- [ ] Add time series forecasting (ARIMA)

- [ ] Include more ML algorithms (neural networks)

- [ ] Add SQL query interface

- [ ] Implement data versioning

- [ ] Add collaborative workspaces

- [ ] Include data profiling with visualizations

- [ ] Add automated EDA reports

- [ ] Implement feature importance ranking

- [ ] Add confusion matrix for classification

- [ ] Include ROC curves and AUC

---



  📊 DataVista — Real-Time Data Analysis & Machine Learning Platform 📊



---

*Last updated: March 2025*
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/willie-conway/datavista

Awesome Lists containing this project

README
