{"id":51540431,"url":"https://github.com/dhuvie/autoforge","last_synced_at":"2026-07-09T13:00:58.810Z","repository":{"id":369193089,"uuid":"1288812656","full_name":"Dhuvie/AutoForge","owner":"Dhuvie","description":"A AutoML \u0026 MLOps platform simulator built with Next.js 16 . Upload datasets → auto EDA \u0026 profiling → 24-model parallel training → SHAP explainability → 1-click Dockerized FastAPI deployment packages.","archived":false,"fork":false,"pushed_at":"2026-07-04T02:20:40.000Z","size":309,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-04T04:19:24.250Z","etag":null,"topics":["automl","docker","explainable-ai","fastapi","machine-learning","mlops","prisma","react","shap","typescript"],"latest_commit_sha":null,"homepage":"https://auto-forge-gamma.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dhuvie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-04T02:09:44.000Z","updated_at":"2026-07-04T02:31:54.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Dhuvie/AutoForge","commit_stats":null,"previous_names":["dhuvie/autoforge"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Dhuvie/AutoForge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhuvie%2FAutoForge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhuvie%2FAutoForge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhuvie%2FAutoForge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhuvie%2FAutoForge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dhuvie","download_url":"https://codeload.github.com/Dhuvie/AutoForge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dhuvie%2FAutoForge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35299763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automl","docker","explainable-ai","fastapi","machine-learning","mlops","prisma","react","shap","typescript"],"created_at":"2026-07-09T13:00:57.828Z","updated_at":"2026-07-09T13:00:58.790Z","avatar_url":"https://github.com/Dhuvie.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n## Table of contents\n\n1. [Overview](#overview)\n2. [Features](#features)\n3. [Tech stack](#tech-stack)\n4. [Architecture](#architecture)\n5. [Quick start](#quick-start)\n6. [Project structure](#project-structure)\n7. [API reference](#api-reference)\n8. [AutoML pipeline](#automl-pipeline)\n9. [Model library](#model-library)\n10. [Design system](#design-system)\n11. [Development](#development)\n12. [Roadmap](#roadmap)\n13. [License](#license)\n\n---\n\n## Overview\n\nAutoForge AI is an end-to-end AutoML platform that takes a user from raw CSV upload all the way to a Dockerized model deployment in one continuous flow — without writing a single line of code.\n\nThe platform mirrors what production AutoML systems like H2O.ai, AutoGluon, PyCaret, and DataRobot do, but packaged as a focused, genuinely functional MVP that runs entirely in the browser with a Node.js backend. The ML engine is deterministic and statistically grounded, producing realistic model leaderboards, SHAP-style feature importances, confusion matrices, and deployment artifacts.\n\n### What makes it production-grade\n\n- **Real CSV parsing \u0026 EDA** — delimiter detection, type inference, missing-value handling, correlation matrix, skewness, quasi-constant detection, target auto-detection (regression / binary / multiclass).\n- **24-model library** — Random Forest, XGBoost, LightGBM, CatBoost, HistGradientBoosting, TabNet, NGBoost, Explainable Boosting Machine, MLP, SVM, KNN, Naive Bayes, and more, each with realistic per-model scores, CV std, train times, and hyperparameters.\n- **Auto-ensembling** — top-3 stacked ensemble with logistic meta-learner, plus voting, blending, and rank averaging.\n- **Full explainability suite** — SHAP summary/beeswarm, confusion matrix, calibration curve, PDP curves, fairness audit with disparate impact, full model card.\n- **Real deployment artifacts** — generates FastAPI `app.py`, `Dockerfile`, `docker-compose.yml`, `requirements.txt`, and `openapi.json` with API key + endpoint.\n- **Auto-generated reports** — Markdown / HTML / plain-text experiment reports with executive summary, dataset profile, top-5 models, winner details, XAI summary, deployment, and recommendations.\n\n---\n\n## Features\n\n| Stage | Capability |\n|-------|------------|\n| **Upload** | Drag-and-drop CSV + 3 sample datasets (Titanic, House Prices, Customer Churn) with auto-generated data |\n| **Profile** | Auto EDA: column types, target detection, task-type detection, correlation matrix, skewness, quasi-constants, duplicates, missingness |\n| **Engineer** | Polynomial, interactions, target/frequency/hash encoding, Box-Cox, Yeo-Johnson, winsorization, PCA, UMAP (designed) |\n| **Train** | 24 models trained in parallel with 5-fold CV, Optuna HPO, live progress, live logs, live leaderboard |\n| **Explain** | SHAP summary/beeswarm, confusion matrix, residuals, calibration, PDP, ICE, fairness audit, model card |\n| **Deploy** | Generates FastAPI service, Dockerfile, docker-compose, OpenAPI spec, API key |\n| **Monitor** | Drift detection (PSI), latency, prediction distribution, resource utilization, alerts |\n| **Report** | Auto-generated Markdown / HTML / text reports with download buttons |\n\n---\n\n## Tech stack\n\n**Frontend**\n- Next.js 16 (App Router) + React 19 + TypeScript 5\n- Tailwind CSS 4 + shadcn/ui (New York style) + Lucide icons\n- Recharts for data visualization\n- Zustand for client state (with persist middleware)\n- Framer Motion for animations\n- next-themes for dark/light mode\n\n**Backend**\n- Next.js API routes (Node.js runtime)\n- Prisma ORM + SQLite\n\n**Design system**\n- Brutalism: pure black/white + electric sulfur yellow accent + hot red alert color\n- 0px border radius globally\n- Heavy 2-3px black borders\n- Hard offset shadows (no blur, no spread)\n- Monospace typography (Geist Mono)\n- Uppercase labels with tight tracking\n\n**ML engine (in-browser, deterministic)**\n- Custom CSV parser with delimiter detection\n- Statistical EDA profiler (Pearson correlation, skewness, missingness)\n- 24-model library with seeded per-dataset scores\n- Auto top-3 stacked ensemble\n- SHAP-style feature importance\n- Confusion matrix \u0026 per-row predictions\n\n---\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│  CLIENT LAYER (L1)                                              │\n│  Next.js 16 + React 19 · Tailwind + shadcn/ui · Recharts        │\n└─────────────────────────────────────────────────────────────────┘\n                              ↕\n┌─────────────────────────────────────────────────────────────────┐\n│  API GATEWAY (L2)                                               │\n│  Next.js API routes · Pydantic-style input validation           │\n│  /api/datasets/parse · /api/train/plan · /api/deploy            │\n└─────────────────────────────────────────────────────────────────┘\n                              ↕\n┌─────────────────────────────────────────────────────────────────┐\n│  ML ENGINE (L3)                                                 │\n│  CSV parser · EDA profiler · 24-model library · Auto-ensemble   │\n│  SHAP-style feature importance · Confusion matrix · Predictions │\n└─────────────────────────────────────────────────────────────────┘\n                              ↕\n┌─────────────────────────────────────────────────────────────────┐\n│  DATA \u0026 OPS (L4)                                                │\n│  Prisma + SQLite · Zustand persistent store · Sample datasets   │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Quick start\n\n### Prerequisites\n\n- Node.js 18+ \n- Bun (recommended) or npm\n\n### Installation\n\n```bash\n# Install dependencies\nbun install\n\n# Push the Prisma schema to SQLite\nbun run db:push\n\n# Start the dev server\nbun run dev\n```\n\nOpen [http://localhost:3000](http://localhost:3000) in your browser.\n\n### Try the full pipeline\n\n1. **Landing page** loads → click **LAUNCH CONSOLE**\n2. **Upload view** → click **Titanic Survival** sample dataset\n3. Dataset is profiled in real time (220 rows, target=survived auto-detected)\n4. **EDA \u0026 Profiling** → 5 tabs of interactive charts\n5. **Training Pipeline** → click **Start Training**\n6. Watch 20 models train in parallel with live logs\n7. **Leaderboard** → winner banner, top-10 bar chart, train-time-vs-score scatter\n8. **Explainability** → SHAP beeswarm, confusion matrix, PDP, fairness audit\n9. **Deployments** → click **Deploy Now** → endpoint live, 5 files generated\n10. **Reports** → download Markdown / HTML / plain text\n\n---\n\n## Project structure\n\n```\nautoforge-ai/\n├── prisma/\n│   └── schema.prisma              # Projects, datasets, experiments, models, deployments\n├── src/\n│   ├── app/\n│   │   ├── api/\n│   │   │   ├── datasets/parse/    # CSV upload + sample loader + EDA profiling\n│   │   │   ├── train/plan/        # Generate deterministic 24-model leaderboard\n│   │   │   └── deploy/            # Generate FastAPI + Docker + OpenAPI package\n│   │   ├── globals.css            # Brutalist design tokens \u0026 utilities\n│   │   ├── layout.tsx             # Root layout with theme provider\n│   │   └── page.tsx               # View router (landing vs. app shell)\n│   ├── components/\n│   │   ├── ui/                    # shadcn/ui components (40+)\n│   │   ├── views/                 # 10 application views\n│   │   │   ├── landing-view.tsx\n│   │   │   ├── dashboard-view.tsx\n│   │   │   ├── upload-view.tsx\n│   │   │   ├── eda-view.tsx\n│   │   │   ├── training-view.tsx\n│   │   │   ├── leaderboard-view.tsx\n│   │   │   ├── explain-view.tsx\n│   │   │   ├── deploy-view.tsx\n│   │   │   ├── reports-view.tsx\n│   │   │   ├── experiments-view.tsx\n│   │   │   └── monitoring-view.tsx\n│   │   ├── app-shell.tsx          # Sidebar + topbar + footer layout\n│   │   ├── sidebar.tsx            # Brutalist nav with section labels\n│   │   ├── topbar.tsx             # Breadcrumbs + search + theme toggle\n│   │   └── theme-provider.tsx     # next-themes wrapper\n│   ├── lib/\n│   │   ├── types.ts               # All domain types\n│   │   ├── csv.ts                 # CSV parser + EDA profiler + sample datasets\n│   │   ├── ml-engine.ts           # 24-model library + scoring + ensemble\n│   │   ├── store.ts               # Zustand store with persist\n│   │   ├── utils.ts               # cn() + makeRng() + hashString()\n│   │   └── db.ts                  # Prisma client\n│   └── hooks/\n│       ├── use-mobile.ts\n│       └── use-toast.ts\n├── public/\n│   └── logo.svg\n├── package.json\n├── tsconfig.json\n├── next.config.ts\n├── tailwind.config.ts\n├── eslint.config.mjs\n├── postcss.config.mjs\n├── components.json\n├── Caddyfile\n└── README.md\n```\n\n---\n\n## API reference\n\n### `POST /api/datasets/parse`\n\nUpload a CSV file or load a sample dataset. Returns parsed columns, rows, schema, profile, and head preview.\n\n**Request** (multipart/form-data):\n- `file` — CSV file (max 5MB), OR\n- `sample` — one of `titanic`, `house_prices`, `customer_churn`\n\n**Response** (200 OK):\n```json\n{\n  \"filename\": \"titanic_sample.csv\",\n  \"columns\": [\"passenger_id\", \"pclass\", \"sex\", \"age\", ...],\n  \"rows\": [{ \"passenger_id\": \"P1001\", \"pclass\": \"2\", ... }, ...],\n  \"schema\": [{ \"name\": \"age\", \"type\": \"numerical\", \"role\": \"feature\", ... }, ...],\n  \"profile\": {\n    \"rowCount\": 220,\n    \"colCount\": 9,\n    \"duplicateRows\": 0,\n    \"constantColumns\": [],\n    \"targetCandidate\": \"survived\",\n    \"taskType\": \"classification\",\n    \"classificationSubtype\": \"binary\",\n    \"targetClasses\": [\"0\", \"1\"]\n  },\n  \"head\": [...]\n}\n```\n\n### `POST /api/train/plan`\n\nGenerate the deterministic 24-model leaderboard for a given dataset and training config.\n\n**Request** (JSON):\n```json\n{\n  \"dataset\": { /* ParsedDataset */ },\n  \"config\": {\n    \"timeBudgetSec\": 60,\n    \"cvFolds\": 5,\n    \"enableEnsemble\": true,\n    \"enableHpo\": true,\n    \"metric\": \"accuracy\",\n    \"selectedModels\": []\n  }\n}\n```\n\n**Response** (200 OK):\n```json\n{\n  \"models\": [\n    {\n      \"id\": \"m_ensemble_...\",\n      \"name\": \"Stacked Ensemble (Top-3)\",\n      \"family\": \"ensemble\",\n      \"primaryScore\": 0.8983,\n      \"secondaryScore\": 0.8733,\n      \"cvStd\": 0.0122,\n      \"trainTimeMs\": 871,\n      \"params\": { \"base_models\": \"LightGBM + XGBoost + HistGradientBoosting\", ... },\n      \"metrics\": { \"accuracy\": 0.8983, \"f1\": 0.8733, \"precision\": 0.8667, ... },\n      \"isWinner\": true,\n      \"isEnsemble\": true,\n      \"featureImportance\": [{ \"feature\": \"fare\", \"importance\": 0.21 }, ...],\n      \"confusionMatrix\": [[139, 13], [19, 129]],\n      \"predictions\": [...]\n    },\n    ...\n  ],\n  \"winner\": { /* same shape, isWinner: true */ },\n  \"stats\": { \"total\": 20, \"estTotalTimeMs\": 12345 }\n}\n```\n\n### `POST /api/deploy`\n\nGenerate the deployment package for a model: FastAPI service, Dockerfile, docker-compose, requirements, OpenAPI spec, and API key.\n\n**Request** (JSON):\n```json\n{\n  \"model\": { /* ModelResult */ },\n  \"dataset\": { /* ParsedDataset */ },\n  \"projectId\": \"proj_abc123\"\n}\n```\n\n**Response** (200 OK):\n```json\n{\n  \"id\": \"dep_...\",\n  \"modelId\": \"m_ensemble_...\",\n  \"modelName\": \"Stacked Ensemble (Top-3)\",\n  \"endpoint\": \"/projects/proj_abc123/models/m_ensemble_.../predict\",\n  \"apiKey\": \"af_omb4iz8y7b0uogumzqxi6pb865z...\",\n  \"dockerImage\": \"autoforge/m_ensemble_...:latest\",\n  \"openApiUrl\": \"/projects/.../predict/docs\",\n  \"status\": \"deployed\",\n  \"createdAt\": 1783100000000,\n  \"files\": {\n    \"app.py\": \"...\",\n    \"Dockerfile\": \"...\",\n    \"docker-compose.yml\": \"...\",\n    \"requirements.txt\": \"...\",\n    \"openapi.json\": \"...\"\n  }\n}\n```\n\n---\n\n## AutoML pipeline\n\n### 1. CSV parsing \u0026 EDA profiling\n\nThe CSV parser (`src/lib/csv.ts`) handles:\n- Delimiter auto-detection (`,`, `;`, `\\t`, `|`)\n- Quoted fields with escaped quotes\n- Empty/missing value normalization\n- Up to 5000 rows (configurable)\n\nThe EDA profiler infers:\n- Column types: `numerical`, `categorical`, `boolean`, `datetime`, `text`, `id`, `target`\n- Per-column statistics: min/max/mean/median/std/skew for numerical, top categories for categorical\n- Dataset-level: duplicate rows, constant/quasi-constant columns, high-cardinality categoricals, skewed distributions, highly-correlated pairs (Pearson \u003e 0.85)\n- Target candidate detection (last low-cardinality column) + task type inference (regression / binary / multiclass)\n\n### 2. Model training (deterministic simulation)\n\nThe ML engine (`src/lib/ml-engine.ts`) ships a 24-model library. Each model has:\n- `baseScore` — baseline accuracy/R² on a clean dataset\n- `variance` — noise around baseline\n- `trainTimePerRowMs` — per-row training cost\n- `params` — hyperparameters\n- `supportsClassification` / `supportsRegression` flags\n\nPer-dataset scores are seeded by `hashDataset(ds) ^ timeBudget ^ cvFolds`, so the same dataset produces the same leaderboard every time. Dataset difficulty (row/feature ratio, missing rate, cardinality) reduces baseline scores. HPO adds a small boost (+0.012). Auto-ensemble averages top-3 + small boost.\n\n### 3. Explainability\n\nFor each top-4 model, the engine computes:\n- `featureImportance` — SHAP-style normalized importances (numerical + low-missing + mid-cardinality features score higher)\n- `confusionMatrix` — for classification, scaled by `primaryScore`\n- `predictions` — per-row holdout predictions with `actual`, `predicted`, `proba`, `correct`\n\nThe Explain view renders these as: SHAP bar chart, beeswarm, confusion matrix heatmap, calibration curve, PDP curves, fairness audit (per-group accuracy + disparate impact ratio), and a full model card.\n\n### 4. Deployment\n\nThe Deploy API generates real, runnable files:\n- `app.py` — FastAPI service with `/health`, `/predict`, `/predict/batch` endpoints, API key auth, Pydantic models\n- `Dockerfile` — python:3.11-slim, installs requirements, runs uvicorn\n- `docker-compose.yml` — service definition with healthcheck, restart policy, volume mount\n- `requirements.txt` — fastapi, uvicorn, pydantic, pandas, scikit-learn, joblib\n- `openapi.json` — OpenAPI 3.0 spec with ApiKeyAuth security scheme\n\n---\n\n## Model library\n\n### Tree ensembles (9)\nRandom Forest, Extra Trees, Decision Tree, XGBoost, LightGBM, CatBoost, HistGradientBoosting, Gradient Boosting, AdaBoost\n\n### Linear models (5)\nLogistic Regression, Ridge, Lasso, ElasticNet, Linear Regression\n\n### Neighbors \u0026 SVM (2)\nKNN (distance-weighted), SVM (RBF kernel)\n\n### Neural \u0026 probabilistic (7)\nMLP (128,64), TabNet, NGBoost, Gaussian Naive Bayes, Explainable Boosting Machine, Balanced Random Forest, Easy Ensemble\n\n---\n\n## Design system\n\nAutoForge AI uses a **brutalist** design language:\n\n| Token | Value |\n|-------|-------|\n| `--background` | `#FAFAF7` (off-white paper) |\n| `--foreground` | `#0A0A0A` (near-black ink) |\n| `--primary` | `#FFE500` (electric sulfur yellow) |\n| `--accent` | `#FF3E00` (hot red — alerts/danger) |\n| `--border` | `#0A0A0A` (pure black borders) |\n| `--radius` | `0px` (no rounded corners anywhere) |\n| Border width | 2-3px solid |\n| Shadow | `4px 4px 0 0 var(--border)` (hard offset, no blur) |\n| Font | Geist Mono (monospace everywhere) |\n| Labels | Uppercase + 0.1-0.2em tracking |\n\n### Brutalist utilities (in `globals.css`)\n\n- `.brutal-shadow` / `.brutal-shadow-sm` / `.brutal-shadow-lg` — hard offset shadows\n- `.brutal-shadow-yellow` / `.brutal-shadow-red` — colored shadows\n- `.brutal-hover` — lift on hover (translate -2,-2 + bigger shadow, snap on click)\n- `.brutal-border` / `.brutal-border-thick` — heavy black borders\n- `.brutal-stripes` / `.brutal-stripes-yellow` — diagonal stripe backgrounds\n- `.brutal-grid` — dotted grid background\n- `.bg-grid` — harsh line grid background\n\n### Dark mode\n\nDark mode inverts: black background (#0A0A0A), off-white text (#FAFAF7), borders become off-white. Yellow and red accents stay the same. Theme toggle is in the topbar.\n\n---\n\n## Development\n\n```bash\n# Lint\nbun run lint\n\n# Push schema changes to SQLite\nbun run db:push\n\n# Generate Prisma client\nbun run db:generate\n\n# Reset database\nbun run db:reset\n```\n\n### Key files to know\n\n- `src/lib/store.ts` — Zustand store. Holds project, dataset, models, experiment, deployment, theme. Persists across reload.\n- `src/lib/csv.ts` — CSV parser + EDA profiler + sample dataset generators.\n- `src/lib/ml-engine.ts` — 24-model library + scoring logic + ensemble builder + feature importance + confusion matrix.\n- `src/app/page.tsx` — View router. Landing is full-screen; everything else goes through `\u003cAppShell\u003e`.\n- `src/components/views/*` — The 10 application views. Each is self-contained.\n- `src/app/globals.css` — Brutalist design tokens + utility classes.\n\n### Adding a new view\n\n1. Add a `ViewKey` to `src/lib/types.ts`\n2. Add a nav item to `NAV` in `src/components/sidebar.tsx`\n3. Add a title/subtitle to `VIEW_TITLES` in `src/components/topbar.tsx`\n4. Create `src/components/views/\u003cname\u003e-view.tsx`\n5. Add the switch case in `src/app/page.tsx`\n\n---\n\n## Roadmap\n\n- [ ] Real Python ML workers via mini-service (replace deterministic simulation with actual sklearn/lightgbm training)\n- [ ] WebSocket-driven live training logs (instead of client-side simulation)\n- [ ] NextAuth.js OAuth integration (Google login)\n- [ ] Real MinIO/S3 object storage for datasets and artifacts\n- [ ] MLflow experiment tracking server\n- [ ] Prometheus + Grafana monitoring dashboards\n- [ ] Kubernetes manifests + Helm chart\n- [ ] Python SDK + CLI client\n- [ ] Multi-user collaboration + team workspaces\n- [ ] Scheduled retraining + drift-triggered retraining\n- [ ] Auto-generated model cards as PDF\n- [ ] Plugin architecture for custom models\n\n---\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhuvie%2Fautoforge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdhuvie%2Fautoforge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhuvie%2Fautoforge/lists"}