{"id":42060827,"url":"https://github.com/kchia/component-forge","last_synced_at":"2026-01-26T07:39:49.415Z","repository":{"id":317745482,"uuid":"1065367706","full_name":"kchia/component-forge","owner":"kchia","description":"Transforming screenshots and Figma designs (coming) into production React components using multi-agent RAG","archived":false,"fork":false,"pushed_at":"2025-12-02T20:19:21.000Z","size":16328,"stargazers_count":0,"open_issues_count":15,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-05T20:28:21.870Z","etag":null,"topics":["ai-agents","ai-engineering","fullstack-ai","multi-agent-systems","openai","rag"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kchia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-27T15:28:09.000Z","updated_at":"2025-12-02T20:19:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"f6a1a61b-ac92-4cf7-ac7c-c350b4d24433","html_url":"https://github.com/kchia/component-forge","commit_stats":null,"previous_names":["kchia/component-forge"],"tags_count":0,"template":false,"template_full_name":"kchia/ai-engineering-starter","purl":"pkg:github/kchia/component-forge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kchia%2Fcomponent-forge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kchia%2Fcomponent-forge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kchia%2Fcomponent-forge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kchia%2Fcomponent-forge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kchia","download_url":"https://codeload.github.com/kchia/component-forge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kchia%2Fcomponent-forge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28769853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-26T06:37:25.426Z","status":"ssl_error","status_checked_at":"2026-01-26T06:37:23.039Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","ai-engineering","fullstack-ai","multi-agent-systems","openai","rag"],"created_at":"2026-01-26T07:39:49.361Z","updated_at":"2026-01-26T07:39:49.408Z","avatar_url":"https://github.com/kchia.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧩 ComponentForge\n\n[AI MAKERSPACE CERTIFICATION CHALLENGE WALKTHROUGH](./docs/coursework/AI_ENGINEERING_TASKS.md)\n\n[LIVE DEMO](https://www.loom.com/share/4ea3f116df1945d1b54f3cb96eb85d87?sid=fc647a7d-592a-45bb-b8fc-9652449554f7)\n\n**AI-powered design-to-code component generation** that transforms Figma designs and screenshots into production-ready, accessible React components using shadcn/ui patterns.\n\nTransform design assets into high-quality TypeScript components in seconds, not hours.\n\n[![Next.js](https://img.shields.io/badge/Next.js-15.5.4-black?style=flat-square\u0026logo=next.js)](https://nextjs.org/)\n[![shadcn/ui](https://img.shields.io/badge/shadcn%2Fui-Latest-black?style=flat-square)](https://ui.shadcn.com/)\n[![OpenAI](https://img.shields.io/badge/OpenAI-AsyncOpenAI-blue?style=flat-square\u0026logo=openai)](https://platform.openai.com/)\n[![TypeScript](https://img.shields.io/badge/TypeScript-5.9.3-blue?style=flat-square\u0026logo=typescript)](https://www.typescriptlang.org/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](https://opensource.org/licenses/MIT)\n\n## ✨ Features\n\n### 🎨 **AI-Powered Design-to-Code**\n\n- **📷 Screenshot Processing**: Extract design tokens from any UI screenshot using GPT-4V\n- **🎯 Figma Integration**: Direct token extraction from Figma files (colors, typography, spacing)\n- **🤖 Multi-Agent Pipeline**: Custom 6-agent system with parallel execution via asyncio\n- **📐 Pattern Matching**: Intelligent retrieval of shadcn/ui component patterns (hybrid BM25 + semantic search)\n- **✨ Code Generation**: Production-ready TypeScript + Storybook components\n\n### 🛠️ **Production-Ready Stack**\n\n- **⚡ Modern Frontend**: Next.js 15.5.4 + React 19 + shadcn/ui + Tailwind CSS v4\n- **🚀 Powerful Backend**: FastAPI + OpenAI SDK + Custom Multi-Agent System + LangSmith observability\n- **♿ Accessibility First**: Built-in axe-core testing for WCAG compliance\n- **📊 State Management**: Zustand (client) + TanStack Query (server state)\n- **🗄️ Vector Database**: Qdrant for semantic search and pattern retrieval\n- **🐳 Containerized**: PostgreSQL + Redis + Qdrant via Docker Compose\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- **Node.js** 18+\n- **Python** 3.11+\n- **Docker Desktop** (for services)\n- **OpenAI API Key** (for AI features)\n\n### 1. Install Dependencies\n\n```bash\nmake install\n```\n\nThis will:\n\n- Install npm packages (shadcn/ui, TanStack Query, Zustand, axe-core)\n- Install Playwright browsers for E2E testing\n- Create Python virtual environment\n- Install AI dependencies (OpenAI SDK, LangSmith for tracing, Pillow)\n- Copy environment templates (`.env` and `.env.local.example`)\n\n### 2. Start Development Environment\n\n```bash\nmake dev\n```\n\nOr manually in separate terminals:\n\n```bash\n# Terminal 1: Start Docker services\ndocker-compose up -d\n\n# Terminal 2: Start backend\ncd backend \u0026\u0026 source venv/bin/activate \u0026\u0026 uvicorn src.main:app --reload\n\n# Terminal 3: Start frontend\ncd app \u0026\u0026 npm run dev\n```\n\n### 2.5. Seed Qdrant Vector Database\n\n**⚠️ CRITICAL: Required for hybrid retrieval (BM25 + semantic search)**\n\nAfter starting Docker services, seed the Qdrant vector database with component pattern embeddings:\n\n```bash\nmake seed-patterns\n```\n\nOr manually:\n\n```bash\ncd backend\nsource venv/bin/activate\npython scripts/seed_patterns.py\n```\n\n**Expected output:**\n\n```\nINFO: Loading pattern library...\nINFO: Loaded 10 patterns from library\nINFO: Creating Qdrant collection 'patterns'...\nINFO: Generating embeddings for 10 patterns...\nINFO: Pattern seeding complete! (10 vectors)\n```\n\n**Why this is required:**\n\n- Enables semantic search (70% of retrieval accuracy)\n- Without seeding, system falls back to BM25-only mode (keyword search)\n\n**Verify seeding succeeded:**\n\n```bash\ncurl http://localhost:6333/collections/patterns | jq '.result.vectors_count'\n# Should return: 10\n```\n\n### 3. Configure Environment\n\nCopy and configure your environment files:\n\n```bash\n# Backend configuration\ncp backend/.env.example backend/.env\n# Add your OPENAI_API_KEY and other secrets\n\n# Frontend configuration\ncp app/.env.local.example app/.env.local\n# Add your AUTH_SECRET and API URLs\n```\n\n### 4. Access Your Application\n\n- **ComponentForge UI**: http://localhost:3000\n- **Backend API Docs**: http://localhost:8000/docs\n- **Health Check**: http://localhost:8000/health\n- **Qdrant Dashboard**: http://localhost:6333/dashboard\n- **Storybook**: http://localhost:6006 (see below for setup)\n\n### 5. Verify Hybrid Retrieval is Active\n\n**Check that semantic search is working (not just BM25 fallback):**\n\n```bash\n# Test retrieval endpoint\ncurl -X POST http://localhost:8000/api/v1/retrieval/search \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"requirements\": {\"component_type\": \"Button\"}}' \\\n  | jq '.retrieval_metadata'\n```\n\n**Expected output (SUCCESS):**\n\n```json\n{\n  \"methods_used\": [\"bm25\", \"semantic\"],\n  \"weights\": { \"bm25\": 0.3, \"semantic\": 0.7 }\n}\n```\n\n**Failure output (degraded mode):**\n\n```json\n{\n  \"methods_used\": [\"bm25\"],\n  \"weights\": { \"bm25\": 1.0, \"semantic\": 0.0 }\n}\n```\n\n**If you see BM25-only mode:**\n\n1. Verify Qdrant is running: `curl http://localhost:6333/health`\n2. Check pattern collection exists: `curl http://localhost:6333/collections/patterns`\n3. Re-run seeding: `make seed-patterns`\n4. Restart backend: Kill and restart `uvicorn src.main:app --reload`\n\n## 📚 Documentation\n\nComprehensive documentation is available in the [`docs/`](./docs) directory:\n\n- **[Getting Started](./docs/getting-started/README.md)** - Installation, FAQ, and contributing guide\n- **[Architecture](./docs/architecture/overview.md)** - System design and technical decisions\n- **[API Reference](./docs/api/overview.md)** - Endpoints, authentication, and error codes\n- **[Features](./docs/features/README.md)** - Token extraction, Figma integration, observability\n- **[Testing](./docs/testing/README.md)** - Integration tests, manual testing, and test reference\n- **[Deployment](./docs/deployment/README.md)** - Production deployment and security guidelines\n- **[Development](./docs/development/README.md)** - Setup guides and best practices\n- **[Backend Docs](./backend/docs/README.md)** - Backend-specific documentation\n- **[Backend Analysis](./docs/backend/)** - Caching strategy, guardrails assessment, and technical analysis\n\n## 🏗️ AI Pipeline Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────────────────┐\n│                              ComponentForge AI Pipeline                                     │\n├─────────────────┬──────────────────────────────────┬─────────────────┬─────────────────────────┤\n│  📷 Input       │  🤖 Multi-Agent System (6 Agents)│  📐 Retrieval   │  ✨ Generation         │\n│                 │                                  │                 │                         │\n│ • Screenshots   │ 1. Token Extractor (GPT-4V)      │ • BM25 Keyword  │ • TypeScript Component  │\n│ • Figma Files   │ 2. Component Classifier          │   Search        │ • Storybook Stories     │\n│ • Design Specs  │ ──────────────────────────────   │ • Semantic      │ • Accessibility Tests   │\n│                 │ Orchestrator → Parallel (4):     │   Similarity    │ • Design Tokens JSON   │\n│                 │ 3. Props Proposer     ┐          │ • Weighted      │                         │\n│                 │ 4. Events Proposer    │ Async    │   Fusion        │                         │\n│                 │ 5. States Proposer    │ Parallel │ • Explainability│                         │\n│                 │ 6. A11y Proposer      ┘          │                 │                         │\n└─────────────────┴──────────────────────────────────┴─────────────────┴─────────────────────────┘\n\n┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐\n│   Frontend      │    │   Backend       │    │   Services      │\n│   (Next.js)     │◄──►│   (FastAPI)     │◄──►│   (Docker)      │\n│                 │    │                 │    │                 │\n│ • Next.js 15    │    │ • OpenAI SDK    │    │ • PostgreSQL    │\n│ • shadcn/ui     │    │ • Custom Agents │    │ • Qdrant Vector │\n│ • Zustand       │    │ • LangSmith     │    │ • Redis Cache   │\n│ • TanStack      │    │ • GPT-4V        │    │                 │\n└─────────────────┘    └─────────────────┘    └─────────────────┘\n```\n\n### Tech Stack\n\n**Frontend (`/app`)**\n\n- **Next.js 15.5.4** with App Router and React 19\n- **shadcn/ui + Radix UI** for accessible component library\n- **Tailwind CSS v4** with CSS variables for theming\n- **Zustand** for client state management\n- **TanStack Query** for server state and caching\n- **TypeScript 5.9.3** for strict type safety\n- **axe-core** for accessibility testing\n- **Playwright** for E2E testing\n\n**Backend (`/backend`)**\n\n- **FastAPI** for high-performance async API\n- **OpenAI SDK** (`AsyncOpenAI`) for direct GPT-4V and GPT-4o integration\n- **Custom Multi-Agent System** with 6 specialized agents (token extraction, classification, requirements)\n- **LangSmith** for optional AI observability and tracing (gracefully degrades if unavailable)\n- **GPT-4V** for vision and screenshot processing\n- **Pillow** for image preprocessing\n- **SQLAlchemy** with async PostgreSQL\n- **Pydantic** for data validation\n- **asyncio** for parallel agent orchestration\n\n**Services (`docker-compose.yml`)**\n\n- **PostgreSQL 16** - Primary database (Port 5432)\n- **Qdrant** - Vector database for AI (Ports 6333/6334)\n- **Redis 7** - Cache and sessions (Port 6379)\n\n## 🛠️ Development Commands\n\n```bash\n# Install all dependencies\nmake install\n\n# Start development environment\nmake dev\n\n# Run all tests\nmake test\n\n# Prepare demo environment\nmake demo\n\n# Clean up containers and dependencies\nmake clean\n\n# Show help\nmake help\n```\n\n### Component Development with Storybook\n\n```bash\n# Start Storybook development server\ncd app \u0026\u0026 npm run storybook\n\n# Build static Storybook for deployment\ncd app \u0026\u0026 npm run build-storybook\n```\n\nStorybook runs on http://localhost:6006 and provides:\n\n- **Interactive component development** - Build and test components in isolation\n- **Visual documentation** - Auto-generated docs for all component variants\n- **Accessibility testing** - Built-in a11y addon for WCAG compliance checks\n- **Component testing** - Integrated Vitest for component unit tests\n\n## 📁 Project Structure\n\n```\ncomponent-forge/\n├── app/                                    # Next.js 15 Frontend (React 19)\n│   ├── src/\n│   │   ├── app/                           # App Router pages and routes\n│   │   │   ├── demo/                      # Demo page for testing\n│   │   │   ├── extract/                   # Token extraction flow\n│   │   │   ├── patterns/                  # Pattern library browsing\n│   │   │   ├── preview/                   # Component preview page\n│   │   │   ├── requirements/              # Requirements management\n│   │   │   ├── layout.tsx                 # Root layout with providers\n│   │   │   ├── page.tsx                   # Home page\n│   │   │   ├── error.tsx                  # Error boundary\n│   │   │   ├── providers.tsx              # React Query, Zustand providers\n│   │   │   └── globals.css                # Global styles and CSS variables\n│   │   ├── components/\n│   │   │   ├── ui/                        # shadcn/ui base components (Button, Card, etc.)\n│   │   │   ├── composite/                 # Composed business components\n│   │   │   ├── extract/                   # Token extraction components\n│   │   │   ├── patterns/                  # Pattern display components\n│   │   │   ├── preview/                   # Code preview and editor\n│   │   │   ├── requirements/              # Requirements form components\n│   │   │   ├── tokens/                    # Design token components\n│   │   │   ├── layout/                    # Layout components (Header, Footer)\n│   │   │   └── onboarding/                # User onboarding flow\n│   │   ├── hooks/                         # Custom React hooks\n│   │   ├── lib/                           # Utilities and helpers\n│   │   ├── services/                      # API client services\n│   │   ├── store/                         # Zustand store (global state)\n│   │   ├── stores/                        # Individual feature stores\n│   │   ├── stories/                       # Storybook stories for components\n│   │   └── types/                         # TypeScript type definitions\n│   ├── e2e/                               # Playwright E2E tests\n│   ├── public/                            # Static assets (images, fonts)\n│   ├── components.json                    # shadcn/ui configuration\n│   ├── eslint.config.mjs                  # ESLint configuration\n│   ├── next.config.ts                     # Next.js configuration\n│   ├── playwright.config.ts               # Playwright test configuration\n│   ├── postcss.config.mjs                 # PostCSS configuration\n│   ├── tsconfig.json                      # TypeScript configuration\n│   ├── vitest.config.ts                   # Vitest test configuration\n│   ├── .env.local.example                 # Frontend environment template\n│   ├── package.json                       # Dependencies (React 19, Next.js 15.5.4)\n│   └── README.md                          # Frontend documentation\n│\n├── backend/                                # FastAPI Backend\n│   ├── src/\n│   │   ├── agents/                        # 6 AI agents (custom system)\n│   │   │   ├── token_extractor.py         # GPT-4V token extraction\n│   │   │   ├── component_classifier.py    # Component type classification\n│   │   │   ├── props_proposer.py          # Props inference\n│   │   │   ├── events_proposer.py         # Event handlers inference\n│   │   │   ├── states_proposer.py         # State management inference\n│   │   │   ├── a11y_proposer.py           # Accessibility requirements\n│   │   │   └── requirement_orchestrator.py # asyncio-based parallel orchestration\n│   │   ├── api/                           # API routes and endpoints\n│   │   ├── cache/                         # Redis caching layer\n│   │   ├── core/                          # Core utilities and database\n│   │   ├── generation/                    # Code generation and validation\n│   │   │   ├── generator_service.py       # TypeScript generation\n│   │   │   ├── code_validator.py          # ESLint, TypeScript validation\n│   │   │   └── storybook_generator.py     # Storybook story generation\n│   │   ├── monitoring/                    # LangSmith observability and metrics\n│   │   ├── prompts/                       # AI prompt templates\n│   │   ├── retrieval/                     # Pattern retrieval system\n│   │   │   ├── bm25_retriever.py          # Keyword-based search\n│   │   │   ├── semantic_retriever.py      # Vector similarity search\n│   │   │   ├── weighted_fusion.py         # Hybrid retrieval (0.3/0.7)\n│   │   │   └── explainer.py               # Confidence scoring\n│   │   ├── services/                      # Business logic services\n│   │   ├── types/                         # Pydantic models and schemas\n│   │   ├── validation/                    # Input validation and sanitization\n│   │   └── main.py                        # FastAPI application entry point\n│   ├── docs/                              # Backend technical documentation\n│   ├── tests/                             # Unit and integration tests\n│   │   ├── unit/                          # Unit tests for individual modules\n│   │   └── integration/                   # Integration tests for workflows\n│   ├── scripts/                           # Utility scripts (seed data, etc.)\n│   ├── alembic/                           # Database migrations\n│   ├── .env.example                       # Backend environment template\n│   ├── requirements.txt                   # Python dependencies (LangGraph, etc.)\n│   ├── pyproject.toml                     # Python project configuration\n│   └── venv/                              # Python virtual environment\n│\n├── docs/                                   # Comprehensive Documentation\n│   ├── getting-started/                   # Installation, setup, FAQ\n│   ├── architecture/                      # System design and architecture decisions\n│   ├── api/                               # API reference and examples\n│   ├── features/                          # Feature documentation\n│   ├── testing/                           # Testing guides and strategies\n│   ├── deployment/                        # Production deployment guides\n│   ├── development/                       # Development workflow and guides\n│   ├── project-history/                   # Epic completion reports\n│   ├── coursework/                        # Academic coursework documentation\n│   ├── adr/                               # Architecture Decision Records\n│   ├── backend/                           # Backend-specific documentation\n│   ├── screenshots/                       # Documentation screenshots\n│   ├── slides/                            # Presentation materials\n│   └── README.md                          # Documentation index\n│\n├── scripts/                                # Utility Scripts\n│   ├── seed_patterns.py                   # Seed pattern library to Qdrant\n│   └── setup_dev.sh                       # Development environment setup\n│\n├── notebooks/                              # Jupyter Notebooks\n│   └── (research and experimentation)\n│\n├── .claude/                                # Claude Code Configuration\n│   └── BASE-COMPONENTS.md                 # Component library specification\n│\n├── docker-compose.yml                      # Services (PostgreSQL, Qdrant, Redis)\n├── Makefile                                # Development commands (install, dev, test)\n├── CLAUDE.md                               # Claude Code project instructions\n├── LICENSE                                 # MIT License\n├── RAG_Fusion.ipynb                        # RAG-Fusion evaluation notebook\n├── pyproject.toml                          # Python project metadata\n└── README.md                               # This file\n```\n\n## 🔧 Configuration\n\n### Environment Variables\n\n**Frontend (`.env.local`)**\n\n```bash\n# Authentication (Auth.js v5)\nAUTH_SECRET=your-32-char-secret-key\nNEXTAUTH_URL=http://localhost:3000\n\n# API Connection\nNEXT_PUBLIC_API_URL=http://localhost:8000\nAPI_URL=http://localhost:8000\n\n# AI Configuration\nNEXT_PUBLIC_OPENAI_MODEL=gpt-4o\nNEXT_PUBLIC_VISION_MODEL=gpt-4o\n\n# Feature Flags\nNEXT_PUBLIC_ENABLE_FIGMA_INTEGRATION=true\nNEXT_PUBLIC_ENABLE_SCREENSHOT_UPLOAD=true\nNEXT_PUBLIC_ENABLE_ACCESSIBILITY_TESTING=true\n```\n\n**Backend (`.env`)**\n\n```bash\n# Database Configuration\nDATABASE_URL=postgresql+asyncpg://demo_user:demo_pass@localhost:5432/demo_db\n\n# Vector Database (Qdrant)\nQDRANT_URL=http://localhost:6333\nQDRANT_API_KEY=your-qdrant-api-key\n\n# Cache (Redis)\nREDIS_URL=redis://localhost:6379\n\n# AI Services - Required for ComponentForge\nOPENAI_API_KEY=your-openai-api-key\n\n# Optional: LangSmith Tracing (for AI observability)\nLANGCHAIN_API_KEY=your-langsmith-api-key  # Optional: Only if using LangSmith\nLANGCHAIN_TRACING_V2=true                   # Optional: Enable LangSmith tracing\nLANGCHAIN_ENDPOINT=https://api.smith.langchain.com\nLANGCHAIN_PROJECT=component-forge\n\n# Authentication\nAUTH_SECRET=your-auth-secret-key-here\n```\n\n## 🧪 Testing\n\n```bash\n# Backend tests (AI agents, API endpoints)\ncd backend \u0026\u0026 source venv/bin/activate \u0026\u0026 pytest tests/ -v\n\n# Frontend unit tests (components, utilities)\ncd app \u0026\u0026 npm test\n\n# Component tests with Storybook + Vitest\ncd app \u0026\u0026 npx vitest\n\n# Accessibility testing (axe-core)\ncd app \u0026\u0026 npm run test:a11y\n\n# E2E tests with Playwright (full component generation flow)\ncd app \u0026\u0026 npm run test:e2e\n```\n\n## 📊 Evaluation Framework\n\nComponentForge includes a comprehensive end-to-end evaluation system that validates the complete screenshot-to-code pipeline with quantified metrics.\n\n### Golden Dataset\n\n15 component screenshots with ground truth data:\n\n- 8 component types: Button (3), Card (2), Badge (3), Input (2), Checkbox, Alert (2), Select, Switch\n- Expected tokens, pattern IDs, and code properties\n- Located in `backend/data/golden_dataset/`\n\n### Run Evaluation\n\n#### CLI Script (Terminal Output)\n\n```bash\ncd backend\nexport OPENAI_API_KEY='your-key-here'\npython scripts/run_e2e_evaluation.py\n```\n\nDisplays formatted metrics and saves JSON report to `backend/logs/`.\n\n#### Automated Tests (CI/CD)\n\n```bash\ncd backend\npytest tests/evaluation/test_e2e_pipeline.py -v\n```\n\nValidates metrics against thresholds. Fails if pipeline success \u003c 80%.\n\n#### API Endpoint (Programmatic Access)\n\n```bash\ncurl http://localhost:8000/api/v1/evaluation/metrics\n```\n\nReturns comprehensive JSON with E2E and retrieval-only metrics.\n\n#### Dashboard (Visual UI)\n\nNavigate to: **http://localhost:3000/evaluation**\n\nFeatures:\n\n- Overall pipeline metrics (success rate, latency)\n- Stage-by-stage performance breakdown\n- Retrieval comparison (E2E vs isolated)\n- Per-screenshot results with detailed logs\n- Visual log viewer for debugging pipeline failures\n- Export JSON functionality\n\n### Metrics \u0026 Targets\n\n| Metric                | Target | Description                               |\n| --------------------- | ------ | ----------------------------------------- |\n| Pipeline Success Rate | ≥ 80%  | % producing valid code end-to-end         |\n| Token Extraction      | ≥ 85%  | % of tokens correctly extracted           |\n| Retrieval MRR         | ≥ 0.90 | Context precision (mean reciprocal rank)  |\n| Retrieval Hit@3       | ≥ 90%  | Context recall (correct pattern in top-3) |\n| Code Compilation      | ≥ 90%  | % of generated code that compiles         |\n| Quality Score         | ≥ 0.85 | Average code quality from validator       |\n| E2E Latency           | \u003c 20s  | Time from screenshot to valid code        |\n\nAll metrics align with industry-standard RAGAS framework.\n\n### Documentation\n\n- Full docs: `backend/src/evaluation/README.md`\n- Demo materials: `docs/coursework/DEMO_METRICS.md`\n- Dataset format: `backend/data/golden_dataset/README.md`\n- Evaluation fixes: `backend/src/evaluation/EVALUATION_FIXES.md`\n- Pipeline gaps analysis: `backend/src/evaluation/EVALUATION_GAPS.md`\n\n## 📊 AI Pipeline Monitoring\n\n### Health Checks \u0026 APIs\n\n- **ComponentForge Health**: http://localhost:8000/health\n- **API Documentation**: http://localhost:8000/docs (FastAPI Swagger)\n- **Metrics**: http://localhost:8000/metrics (Prometheus format)\n- **Storybook**: http://localhost:6006 (Component library \u0026 testing)\n\n### AI Observability\n\n- **LangSmith Traces**: Monitor agent performance and costs\n- **Token Extraction Confidence**: Track vision model accuracy\n- **Pattern Retrieval Scores**: Semantic search effectiveness\n- **Generation Quality**: TypeScript compilation and accessibility scores\n\n### Infrastructure\n\n- **Qdrant Dashboard**: http://localhost:6333/dashboard (Vector operations)\n- **PostgreSQL**: Database performance and query logs\n- **Redis**: Cache hit rates and performance\n\n## 🐳 Docker Services\n\nThe project includes three essential services:\n\n```yaml\n# PostgreSQL Database\npostgres:5432\n  - User: demo_user\n  - Password: demo_pass\n  - Database: demo_db\n\n# Qdrant Vector Database\nqdrant:6333/6334\n  - Dashboard: :6333/dashboard\n  - API: :6333\n\n# Redis Cache\nredis:6379\n  - Memory limit: 256MB\n  - Policy: allkeys-lru\n```\n\n## 🚨 Troubleshooting\n\n### Common Issues\n\n**Docker not starting:**\n\n```bash\n# Ensure Docker Desktop is running\nopen -a Docker\n\n# Check Docker status\ndocker --version\ndocker-compose ps\n```\n\n**Python environment issues:**\n\n```bash\n# Recreate virtual environment\nrm -rf backend/venv\ncd backend \u0026\u0026 python -m venv venv\nsource venv/bin/activate \u0026\u0026 pip install -r requirements.txt\n```\n\n**Node.js dependency issues:**\n\n```bash\n# Clean install\ncd app \u0026\u0026 rm -rf node_modules package-lock.json\nnpm install\n```\n\n**Port conflicts:**\n\n- Frontend (3000), Backend (8000), PostgreSQL (5432), Qdrant (6333), Redis (6379)\n- Check for other services using these ports: `lsof -i :3000`\n\n**Qdrant/Semantic Search Issues:**\n\n**Symptom: \"Semantic retriever unavailable\" in backend logs**\n\nThis means the system is running in BM25-only fallback mode (degraded accuracy).\n\n**Solution:**\n\n```bash\n# 1. Verify Qdrant is running\ncurl http://localhost:6333/health\n\n# 2. Check if patterns collection exists\ncurl http://localhost:6333/collections/patterns\n\n# 3. If collection missing, seed it\ncd backend\nsource venv/bin/activate\npython scripts/seed_patterns.py\n\n# 4. Restart backend to reinitialize semantic retriever\n# (Kill uvicorn and restart)\n```\n\n**Symptom: \"Architecture mismatch (arm64 vs x86_64)\" when seeding**\n\nYour Python venv was created with wrong architecture.\n\n**Solution:**\n\n```bash\n# Recreate venv with correct architecture\ncd backend\ndeactivate\nrm -rf venv\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n\n# Retry seeding\npython scripts/seed_patterns.py\n```\n\n**Symptom: OpenAI API errors during seeding**\n\nSeeding requires OpenAI API to generate embeddings.\n\n**Solution:**\n\n```bash\n# Check API key is set\necho $OPENAI_API_KEY\n\n# If empty, add to backend/.env\necho \"OPENAI_API_KEY=your-key-here\" \u003e\u003e backend/.env\n\n# Export it\nexport OPENAI_API_KEY=\"your-key-here\"\n\n# Retry seeding\ncd backend \u0026\u0026 source venv/bin/activate\npython scripts/seed_patterns.py\n```\n\n## 🎯 ComponentForge Workflow\n\n### 1. Design Input\n\n- **Screenshot Upload**: Drag \u0026 drop any UI design screenshot\n- **Figma Integration**: Connect with Personal Access Token\n- **Design Analysis**: GPT-4V extracts visual design patterns\n\n### 2. AI Processing Pipeline\n\n- **Token Extraction**: Colors, typography, spacing with confidence scores\n- **Requirement Proposal**: Inferred props, states, behaviors, accessibility needs\n- **Pattern Retrieval**: Semantic search through shadcn/ui component patterns\n- **Quality Validation**: TypeScript, ESLint, axe-core accessibility checks\n\n### 3. Generated Output\n\n- **TypeScript Component**: Production-ready React component with proper types\n- **Storybook Stories**: Interactive documentation and testing\n- **Accessibility**: WCAG-compliant with proper ARIA attributes\n- **Design Tokens**: JSON file with extracted design system values\n\n## 📝 Development Notes\n\n- **AI-First Architecture**: Direct OpenAI SDK integration with optional LangSmith tracing\n- **Custom Agent System**: 6 specialized agents with manual asyncio orchestration\n- **Hot Reloading**: Both frontend and backend support instant updates\n- **Type Safety**: Strict TypeScript across the entire stack\n- **Accessibility**: Built-in axe-core testing prevents WCAG violations\n- **Production Ready**: Docker containerization for easy deployment\n\n## Future Enhancements\n\nComponentForge is designed to scale from atomic components to complex compositions and full page layouts. Current release focuses on establishing quality foundations; future versions will tackle increasing complexity.\n\n### Phase 1: Atomic Components (Current - v1.0)\n\n**Focus:** Single-component generation with quality validation\n\n- Basic shadcn/ui components (Button, Card, Input, Badge, etc.)\n- Design token extraction with 85%+ accuracy\n- Pattern retrieval from vector database\n- Validation pipeline (TypeScript, ESLint, axe-core)\n- **Success metrics:** 90% compilation rate, 80% pipeline success\n\n### Phase 2: Complex Compositions (v1.5)\n\n**Focus:** Multi-component generation with compositional reasoning\n\n- **Composite components:** ProductCard, DataTable, MultiStepForm, NavigationMenu\n- **Multi-pattern retrieval:** AI selects and composes 5-10 sub-components\n- **Prop threading:** Automatically connects parent/child component data\n- **State scaffolding:** Generates Zustand stores for complex state\n- **Examples:**\n  - ProductCard = Card + AspectRatio + Image + Badge + Typography + Button\n  - DataTable = Table + Select (filters) + Pagination + Dropdown (actions)\n  - MultiStepForm = Tabs + FormFields[] + ValidationLogic + ProgressIndicator\n\n### Phase 3: Full Page Layouts (v2.0)\n\n**Focus:** Complete page generation with architectural decisions\n\n- **Full dashboard pages:** Sidebar + Header + MetricCards + DataTables + Charts\n- **Landing pages:** Hero + Features + Testimonials + CTA sections\n- **Application pages:** Settings (Tabs + Forms), Profile (Layout + Cards)\n- **Architectural AI:**\n  - Layout detection (Grid, Flex, responsive breakpoints)\n  - Routing decisions (Next.js App Router patterns)\n  - State management strategy (global vs local state)\n  - API mocking (fake endpoints for development)\n- **Multi-file generation:**\n  - `app/page.tsx` - Main page component\n  - `components/*.tsx` - Extracted components\n  - `stores/*.ts` - State management\n  - `api/*.ts` - Mock API routes\n\n### Near-Term Enhancements (All Phases)\n\n1. **Design System Import** - Upload Figma Design System → Auto-populate vector DB\n2. **Learning System (Phase 4)** - AI learns from user edits and preferences\n3. **Custom Component Libraries** - Support Material-UI, Chakra UI, Ant Design\n4. **Multi-framework Support** - Vue, Svelte, Angular code generation\n5. **Advanced Testing** - Auto-generate unit, integration, and E2E tests\n6. **Real-time Collaboration** - WebSocket support for team editing\n7. **Version Control** - Component versioning and diff tracking\n8. **Performance Optimization** - Bundle analysis and optimization suggestions\n\n### Why the Phased Approach?\n\n- **Quality first:** Atomic components establish 90% quality baseline before scaling\n- **Infrastructure reuse:** Multi-agent system, vector DB, validation pipeline built for all phases\n- **Learning foundation:** Phase 1 builds pattern library that Phase 2 and 3 depend on\n- **Risk management:** If complex generation fails, atomic generation still provides value\n- **Technical scaling:** Compositional reasoning requires validated atomic patterns first\n\n**Current state:** Phase 1 complete with production-ready infrastructure.\n**Vision:** Full design-to-code automation from screenshots to complete applications.\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n**Transform designs into code with AI!** 🧩✨\n\nBuilt with ❤️ for developers who want to focus on logic, not repetitive UI coding.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkchia%2Fcomponent-forge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkchia%2Fcomponent-forge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkchia%2Fcomponent-forge/lists"}