An open API service indexing awesome lists of open source software.

https://github.com/kchia/component-forge

Transforming screenshots and Figma designs (coming) into production React components using multi-agent RAG
https://github.com/kchia/component-forge

ai-agents ai-engineering fullstack-ai multi-agent-systems openai rag

Last synced: 5 months ago
JSON representation

Transforming screenshots and Figma designs (coming) into production React components using multi-agent RAG

Awesome Lists containing this project

README

          

# 🧩 ComponentForge

[AI MAKERSPACE CERTIFICATION CHALLENGE WALKTHROUGH](./docs/coursework/AI_ENGINEERING_TASKS.md)

[LIVE DEMO](https://www.loom.com/share/4ea3f116df1945d1b54f3cb96eb85d87?sid=fc647a7d-592a-45bb-b8fc-9652449554f7)

**AI-powered design-to-code component generation** that transforms Figma designs and screenshots into production-ready, accessible React components using shadcn/ui patterns.

Transform design assets into high-quality TypeScript components in seconds, not hours.

[![Next.js](https://img.shields.io/badge/Next.js-15.5.4-black?style=flat-square&logo=next.js)](https://nextjs.org/)
[![shadcn/ui](https://img.shields.io/badge/shadcn%2Fui-Latest-black?style=flat-square)](https://ui.shadcn.com/)
[![OpenAI](https://img.shields.io/badge/OpenAI-AsyncOpenAI-blue?style=flat-square&logo=openai)](https://platform.openai.com/)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.9.3-blue?style=flat-square&logo=typescript)](https://www.typescriptlang.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](https://opensource.org/licenses/MIT)

## ✨ Features

### 🎨 **AI-Powered Design-to-Code**

- **πŸ“· Screenshot Processing**: Extract design tokens from any UI screenshot using GPT-4V
- **🎯 Figma Integration**: Direct token extraction from Figma files (colors, typography, spacing)
- **πŸ€– Multi-Agent Pipeline**: Custom 6-agent system with parallel execution via asyncio
- **πŸ“ Pattern Matching**: Intelligent retrieval of shadcn/ui component patterns (hybrid BM25 + semantic search)
- **✨ Code Generation**: Production-ready TypeScript + Storybook components

### πŸ› οΈ **Production-Ready Stack**

- **⚑ Modern Frontend**: Next.js 15.5.4 + React 19 + shadcn/ui + Tailwind CSS v4
- **πŸš€ Powerful Backend**: FastAPI + OpenAI SDK + Custom Multi-Agent System + LangSmith observability
- **β™Ώ Accessibility First**: Built-in axe-core testing for WCAG compliance
- **πŸ“Š State Management**: Zustand (client) + TanStack Query (server state)
- **πŸ—„οΈ Vector Database**: Qdrant for semantic search and pattern retrieval
- **🐳 Containerized**: PostgreSQL + Redis + Qdrant via Docker Compose

## πŸš€ Quick Start

### Prerequisites

- **Node.js** 18+
- **Python** 3.11+
- **Docker Desktop** (for services)
- **OpenAI API Key** (for AI features)

### 1. Install Dependencies

```bash
make install
```

This will:

- Install npm packages (shadcn/ui, TanStack Query, Zustand, axe-core)
- Install Playwright browsers for E2E testing
- Create Python virtual environment
- Install AI dependencies (OpenAI SDK, LangSmith for tracing, Pillow)
- Copy environment templates (`.env` and `.env.local.example`)

### 2. Start Development Environment

```bash
make dev
```

Or manually in separate terminals:

```bash
# Terminal 1: Start Docker services
docker-compose up -d

# Terminal 2: Start backend
cd backend && source venv/bin/activate && uvicorn src.main:app --reload

# Terminal 3: Start frontend
cd app && npm run dev
```

### 2.5. Seed Qdrant Vector Database

**⚠️ CRITICAL: Required for hybrid retrieval (BM25 + semantic search)**

After starting Docker services, seed the Qdrant vector database with component pattern embeddings:

```bash
make seed-patterns
```

Or manually:

```bash
cd backend
source venv/bin/activate
python scripts/seed_patterns.py
```

**Expected output:**

```
INFO: Loading pattern library...
INFO: Loaded 10 patterns from library
INFO: Creating Qdrant collection 'patterns'...
INFO: Generating embeddings for 10 patterns...
INFO: Pattern seeding complete! (10 vectors)
```

**Why this is required:**

- Enables semantic search (70% of retrieval accuracy)
- Without seeding, system falls back to BM25-only mode (keyword search)

**Verify seeding succeeded:**

```bash
curl http://localhost:6333/collections/patterns | jq '.result.vectors_count'
# Should return: 10
```

### 3. Configure Environment

Copy and configure your environment files:

```bash
# Backend configuration
cp backend/.env.example backend/.env
# Add your OPENAI_API_KEY and other secrets

# Frontend configuration
cp app/.env.local.example app/.env.local
# Add your AUTH_SECRET and API URLs
```

### 4. Access Your Application

- **ComponentForge UI**: http://localhost:3000
- **Backend API Docs**: http://localhost:8000/docs
- **Health Check**: http://localhost:8000/health
- **Qdrant Dashboard**: http://localhost:6333/dashboard
- **Storybook**: http://localhost:6006 (see below for setup)

### 5. Verify Hybrid Retrieval is Active

**Check that semantic search is working (not just BM25 fallback):**

```bash
# Test retrieval endpoint
curl -X POST http://localhost:8000/api/v1/retrieval/search \
-H "Content-Type: application/json" \
-d '{"requirements": {"component_type": "Button"}}' \
| jq '.retrieval_metadata'
```

**Expected output (SUCCESS):**

```json
{
"methods_used": ["bm25", "semantic"],
"weights": { "bm25": 0.3, "semantic": 0.7 }
}
```

**Failure output (degraded mode):**

```json
{
"methods_used": ["bm25"],
"weights": { "bm25": 1.0, "semantic": 0.0 }
}
```

**If you see BM25-only mode:**

1. Verify Qdrant is running: `curl http://localhost:6333/health`
2. Check pattern collection exists: `curl http://localhost:6333/collections/patterns`
3. Re-run seeding: `make seed-patterns`
4. Restart backend: Kill and restart `uvicorn src.main:app --reload`

## πŸ“š Documentation

Comprehensive documentation is available in the [`docs/`](./docs) directory:

- **[Getting Started](./docs/getting-started/README.md)** - Installation, FAQ, and contributing guide
- **[Architecture](./docs/architecture/overview.md)** - System design and technical decisions
- **[API Reference](./docs/api/overview.md)** - Endpoints, authentication, and error codes
- **[Features](./docs/features/README.md)** - Token extraction, Figma integration, observability
- **[Testing](./docs/testing/README.md)** - Integration tests, manual testing, and test reference
- **[Deployment](./docs/deployment/README.md)** - Production deployment and security guidelines
- **[Development](./docs/development/README.md)** - Setup guides and best practices
- **[Backend Docs](./backend/docs/README.md)** - Backend-specific documentation
- **[Backend Analysis](./docs/backend/)** - Caching strategy, guardrails assessment, and technical analysis

## πŸ—οΈ AI Pipeline Architecture

```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ ComponentForge AI Pipeline β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ πŸ“· Input β”‚ πŸ€– Multi-Agent System (6 Agents)β”‚ πŸ“ Retrieval β”‚ ✨ Generation β”‚
β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β€’ Screenshots β”‚ 1. Token Extractor (GPT-4V) β”‚ β€’ BM25 Keyword β”‚ β€’ TypeScript Component β”‚
β”‚ β€’ Figma Files β”‚ 2. Component Classifier β”‚ Search β”‚ β€’ Storybook Stories β”‚
β”‚ β€’ Design Specs β”‚ ────────────────────────────── β”‚ β€’ Semantic β”‚ β€’ Accessibility Tests β”‚
β”‚ β”‚ Orchestrator β†’ Parallel (4): β”‚ Similarity β”‚ β€’ Design Tokens JSON β”‚
β”‚ β”‚ 3. Props Proposer ┐ β”‚ β€’ Weighted β”‚ β”‚
β”‚ β”‚ 4. Events Proposer β”‚ Async β”‚ Fusion β”‚ β”‚
β”‚ β”‚ 5. States Proposer β”‚ Parallel β”‚ β€’ Explainabilityβ”‚ β”‚
β”‚ β”‚ 6. A11y Proposer β”˜ β”‚ β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Frontend β”‚ β”‚ Backend β”‚ β”‚ Services β”‚
β”‚ (Next.js) │◄──►│ (FastAPI) │◄──►│ (Docker) β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β€’ Next.js 15 β”‚ β”‚ β€’ OpenAI SDK β”‚ β”‚ β€’ PostgreSQL β”‚
β”‚ β€’ shadcn/ui β”‚ β”‚ β€’ Custom Agents β”‚ β”‚ β€’ Qdrant Vector β”‚
β”‚ β€’ Zustand β”‚ β”‚ β€’ LangSmith β”‚ β”‚ β€’ Redis Cache β”‚
β”‚ β€’ TanStack β”‚ β”‚ β€’ GPT-4V β”‚ β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```

### Tech Stack

**Frontend (`/app`)**

- **Next.js 15.5.4** with App Router and React 19
- **shadcn/ui + Radix UI** for accessible component library
- **Tailwind CSS v4** with CSS variables for theming
- **Zustand** for client state management
- **TanStack Query** for server state and caching
- **TypeScript 5.9.3** for strict type safety
- **axe-core** for accessibility testing
- **Playwright** for E2E testing

**Backend (`/backend`)**

- **FastAPI** for high-performance async API
- **OpenAI SDK** (`AsyncOpenAI`) for direct GPT-4V and GPT-4o integration
- **Custom Multi-Agent System** with 6 specialized agents (token extraction, classification, requirements)
- **LangSmith** for optional AI observability and tracing (gracefully degrades if unavailable)
- **GPT-4V** for vision and screenshot processing
- **Pillow** for image preprocessing
- **SQLAlchemy** with async PostgreSQL
- **Pydantic** for data validation
- **asyncio** for parallel agent orchestration

**Services (`docker-compose.yml`)**

- **PostgreSQL 16** - Primary database (Port 5432)
- **Qdrant** - Vector database for AI (Ports 6333/6334)
- **Redis 7** - Cache and sessions (Port 6379)

## πŸ› οΈ Development Commands

```bash
# Install all dependencies
make install

# Start development environment
make dev

# Run all tests
make test

# Prepare demo environment
make demo

# Clean up containers and dependencies
make clean

# Show help
make help
```

### Component Development with Storybook

```bash
# Start Storybook development server
cd app && npm run storybook

# Build static Storybook for deployment
cd app && npm run build-storybook
```

Storybook runs on http://localhost:6006 and provides:

- **Interactive component development** - Build and test components in isolation
- **Visual documentation** - Auto-generated docs for all component variants
- **Accessibility testing** - Built-in a11y addon for WCAG compliance checks
- **Component testing** - Integrated Vitest for component unit tests

## πŸ“ Project Structure

```
component-forge/
β”œβ”€β”€ app/ # Next.js 15 Frontend (React 19)
β”‚ β”œβ”€β”€ src/
β”‚ β”‚ β”œβ”€β”€ app/ # App Router pages and routes
β”‚ β”‚ β”‚ β”œβ”€β”€ demo/ # Demo page for testing
β”‚ β”‚ β”‚ β”œβ”€β”€ extract/ # Token extraction flow
β”‚ β”‚ β”‚ β”œβ”€β”€ patterns/ # Pattern library browsing
β”‚ β”‚ β”‚ β”œβ”€β”€ preview/ # Component preview page
β”‚ β”‚ β”‚ β”œβ”€β”€ requirements/ # Requirements management
β”‚ β”‚ β”‚ β”œβ”€β”€ layout.tsx # Root layout with providers
β”‚ β”‚ β”‚ β”œβ”€β”€ page.tsx # Home page
β”‚ β”‚ β”‚ β”œβ”€β”€ error.tsx # Error boundary
β”‚ β”‚ β”‚ β”œβ”€β”€ providers.tsx # React Query, Zustand providers
β”‚ β”‚ β”‚ └── globals.css # Global styles and CSS variables
β”‚ β”‚ β”œβ”€β”€ components/
β”‚ β”‚ β”‚ β”œβ”€β”€ ui/ # shadcn/ui base components (Button, Card, etc.)
β”‚ β”‚ β”‚ β”œβ”€β”€ composite/ # Composed business components
β”‚ β”‚ β”‚ β”œβ”€β”€ extract/ # Token extraction components
β”‚ β”‚ β”‚ β”œβ”€β”€ patterns/ # Pattern display components
β”‚ β”‚ β”‚ β”œβ”€β”€ preview/ # Code preview and editor
β”‚ β”‚ β”‚ β”œβ”€β”€ requirements/ # Requirements form components
β”‚ β”‚ β”‚ β”œβ”€β”€ tokens/ # Design token components
β”‚ β”‚ β”‚ β”œβ”€β”€ layout/ # Layout components (Header, Footer)
β”‚ β”‚ β”‚ └── onboarding/ # User onboarding flow
β”‚ β”‚ β”œβ”€β”€ hooks/ # Custom React hooks
β”‚ β”‚ β”œβ”€β”€ lib/ # Utilities and helpers
β”‚ β”‚ β”œβ”€β”€ services/ # API client services
β”‚ β”‚ β”œβ”€β”€ store/ # Zustand store (global state)
β”‚ β”‚ β”œβ”€β”€ stores/ # Individual feature stores
β”‚ β”‚ β”œβ”€β”€ stories/ # Storybook stories for components
β”‚ β”‚ └── types/ # TypeScript type definitions
β”‚ β”œβ”€β”€ e2e/ # Playwright E2E tests
β”‚ β”œβ”€β”€ public/ # Static assets (images, fonts)
β”‚ β”œβ”€β”€ components.json # shadcn/ui configuration
β”‚ β”œβ”€β”€ eslint.config.mjs # ESLint configuration
β”‚ β”œβ”€β”€ next.config.ts # Next.js configuration
β”‚ β”œβ”€β”€ playwright.config.ts # Playwright test configuration
β”‚ β”œβ”€β”€ postcss.config.mjs # PostCSS configuration
β”‚ β”œβ”€β”€ tsconfig.json # TypeScript configuration
β”‚ β”œβ”€β”€ vitest.config.ts # Vitest test configuration
β”‚ β”œβ”€β”€ .env.local.example # Frontend environment template
β”‚ β”œβ”€β”€ package.json # Dependencies (React 19, Next.js 15.5.4)
β”‚ └── README.md # Frontend documentation
β”‚
β”œβ”€β”€ backend/ # FastAPI Backend
β”‚ β”œβ”€β”€ src/
β”‚ β”‚ β”œβ”€β”€ agents/ # 6 AI agents (custom system)
β”‚ β”‚ β”‚ β”œβ”€β”€ token_extractor.py # GPT-4V token extraction
β”‚ β”‚ β”‚ β”œβ”€β”€ component_classifier.py # Component type classification
β”‚ β”‚ β”‚ β”œβ”€β”€ props_proposer.py # Props inference
β”‚ β”‚ β”‚ β”œβ”€β”€ events_proposer.py # Event handlers inference
β”‚ β”‚ β”‚ β”œβ”€β”€ states_proposer.py # State management inference
β”‚ β”‚ β”‚ β”œβ”€β”€ a11y_proposer.py # Accessibility requirements
β”‚ β”‚ β”‚ └── requirement_orchestrator.py # asyncio-based parallel orchestration
β”‚ β”‚ β”œβ”€β”€ api/ # API routes and endpoints
β”‚ β”‚ β”œβ”€β”€ cache/ # Redis caching layer
β”‚ β”‚ β”œβ”€β”€ core/ # Core utilities and database
β”‚ β”‚ β”œβ”€β”€ generation/ # Code generation and validation
β”‚ β”‚ β”‚ β”œβ”€β”€ generator_service.py # TypeScript generation
β”‚ β”‚ β”‚ β”œβ”€β”€ code_validator.py # ESLint, TypeScript validation
β”‚ β”‚ β”‚ └── storybook_generator.py # Storybook story generation
β”‚ β”‚ β”œβ”€β”€ monitoring/ # LangSmith observability and metrics
β”‚ β”‚ β”œβ”€β”€ prompts/ # AI prompt templates
β”‚ β”‚ β”œβ”€β”€ retrieval/ # Pattern retrieval system
β”‚ β”‚ β”‚ β”œβ”€β”€ bm25_retriever.py # Keyword-based search
β”‚ β”‚ β”‚ β”œβ”€β”€ semantic_retriever.py # Vector similarity search
β”‚ β”‚ β”‚ β”œβ”€β”€ weighted_fusion.py # Hybrid retrieval (0.3/0.7)
β”‚ β”‚ β”‚ └── explainer.py # Confidence scoring
β”‚ β”‚ β”œβ”€β”€ services/ # Business logic services
β”‚ β”‚ β”œβ”€β”€ types/ # Pydantic models and schemas
β”‚ β”‚ β”œβ”€β”€ validation/ # Input validation and sanitization
β”‚ β”‚ └── main.py # FastAPI application entry point
β”‚ β”œβ”€β”€ docs/ # Backend technical documentation
β”‚ β”œβ”€β”€ tests/ # Unit and integration tests
β”‚ β”‚ β”œβ”€β”€ unit/ # Unit tests for individual modules
β”‚ β”‚ └── integration/ # Integration tests for workflows
β”‚ β”œβ”€β”€ scripts/ # Utility scripts (seed data, etc.)
β”‚ β”œβ”€β”€ alembic/ # Database migrations
β”‚ β”œβ”€β”€ .env.example # Backend environment template
β”‚ β”œβ”€β”€ requirements.txt # Python dependencies (LangGraph, etc.)
β”‚ β”œβ”€β”€ pyproject.toml # Python project configuration
β”‚ └── venv/ # Python virtual environment
β”‚
β”œβ”€β”€ docs/ # Comprehensive Documentation
β”‚ β”œβ”€β”€ getting-started/ # Installation, setup, FAQ
β”‚ β”œβ”€β”€ architecture/ # System design and architecture decisions
β”‚ β”œβ”€β”€ api/ # API reference and examples
β”‚ β”œβ”€β”€ features/ # Feature documentation
β”‚ β”œβ”€β”€ testing/ # Testing guides and strategies
β”‚ β”œβ”€β”€ deployment/ # Production deployment guides
β”‚ β”œβ”€β”€ development/ # Development workflow and guides
β”‚ β”œβ”€β”€ project-history/ # Epic completion reports
β”‚ β”œβ”€β”€ coursework/ # Academic coursework documentation
β”‚ β”œβ”€β”€ adr/ # Architecture Decision Records
β”‚ β”œβ”€β”€ backend/ # Backend-specific documentation
β”‚ β”œβ”€β”€ screenshots/ # Documentation screenshots
β”‚ β”œβ”€β”€ slides/ # Presentation materials
β”‚ └── README.md # Documentation index
β”‚
β”œβ”€β”€ scripts/ # Utility Scripts
β”‚ β”œβ”€β”€ seed_patterns.py # Seed pattern library to Qdrant
β”‚ └── setup_dev.sh # Development environment setup
β”‚
β”œβ”€β”€ notebooks/ # Jupyter Notebooks
β”‚ └── (research and experimentation)
β”‚
β”œβ”€β”€ .claude/ # Claude Code Configuration
β”‚ └── BASE-COMPONENTS.md # Component library specification
β”‚
β”œβ”€β”€ docker-compose.yml # Services (PostgreSQL, Qdrant, Redis)
β”œβ”€β”€ Makefile # Development commands (install, dev, test)
β”œβ”€β”€ CLAUDE.md # Claude Code project instructions
β”œβ”€β”€ LICENSE # MIT License
β”œβ”€β”€ RAG_Fusion.ipynb # RAG-Fusion evaluation notebook
β”œβ”€β”€ pyproject.toml # Python project metadata
└── README.md # This file
```

## πŸ”§ Configuration

### Environment Variables

**Frontend (`.env.local`)**

```bash
# Authentication (Auth.js v5)
AUTH_SECRET=your-32-char-secret-key
NEXTAUTH_URL=http://localhost:3000

# API Connection
NEXT_PUBLIC_API_URL=http://localhost:8000
API_URL=http://localhost:8000

# AI Configuration
NEXT_PUBLIC_OPENAI_MODEL=gpt-4o
NEXT_PUBLIC_VISION_MODEL=gpt-4o

# Feature Flags
NEXT_PUBLIC_ENABLE_FIGMA_INTEGRATION=true
NEXT_PUBLIC_ENABLE_SCREENSHOT_UPLOAD=true
NEXT_PUBLIC_ENABLE_ACCESSIBILITY_TESTING=true
```

**Backend (`.env`)**

```bash
# Database Configuration
DATABASE_URL=postgresql+asyncpg://demo_user:demo_pass@localhost:5432/demo_db

# Vector Database (Qdrant)
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=your-qdrant-api-key

# Cache (Redis)
REDIS_URL=redis://localhost:6379

# AI Services - Required for ComponentForge
OPENAI_API_KEY=your-openai-api-key

# Optional: LangSmith Tracing (for AI observability)
LANGCHAIN_API_KEY=your-langsmith-api-key # Optional: Only if using LangSmith
LANGCHAIN_TRACING_V2=true # Optional: Enable LangSmith tracing
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
LANGCHAIN_PROJECT=component-forge

# Authentication
AUTH_SECRET=your-auth-secret-key-here
```

## πŸ§ͺ Testing

```bash
# Backend tests (AI agents, API endpoints)
cd backend && source venv/bin/activate && pytest tests/ -v

# Frontend unit tests (components, utilities)
cd app && npm test

# Component tests with Storybook + Vitest
cd app && npx vitest

# Accessibility testing (axe-core)
cd app && npm run test:a11y

# E2E tests with Playwright (full component generation flow)
cd app && npm run test:e2e
```

## πŸ“Š Evaluation Framework

ComponentForge includes a comprehensive end-to-end evaluation system that validates the complete screenshot-to-code pipeline with quantified metrics.

### Golden Dataset

15 component screenshots with ground truth data:

- 8 component types: Button (3), Card (2), Badge (3), Input (2), Checkbox, Alert (2), Select, Switch
- Expected tokens, pattern IDs, and code properties
- Located in `backend/data/golden_dataset/`

### Run Evaluation

#### CLI Script (Terminal Output)

```bash
cd backend
export OPENAI_API_KEY='your-key-here'
python scripts/run_e2e_evaluation.py
```

Displays formatted metrics and saves JSON report to `backend/logs/`.

#### Automated Tests (CI/CD)

```bash
cd backend
pytest tests/evaluation/test_e2e_pipeline.py -v
```

Validates metrics against thresholds. Fails if pipeline success < 80%.

#### API Endpoint (Programmatic Access)

```bash
curl http://localhost:8000/api/v1/evaluation/metrics
```

Returns comprehensive JSON with E2E and retrieval-only metrics.

#### Dashboard (Visual UI)

Navigate to: **http://localhost:3000/evaluation**

Features:

- Overall pipeline metrics (success rate, latency)
- Stage-by-stage performance breakdown
- Retrieval comparison (E2E vs isolated)
- Per-screenshot results with detailed logs
- Visual log viewer for debugging pipeline failures
- Export JSON functionality

### Metrics & Targets

| Metric | Target | Description |
| --------------------- | ------ | ----------------------------------------- |
| Pipeline Success Rate | β‰₯ 80% | % producing valid code end-to-end |
| Token Extraction | β‰₯ 85% | % of tokens correctly extracted |
| Retrieval MRR | β‰₯ 0.90 | Context precision (mean reciprocal rank) |
| Retrieval Hit@3 | β‰₯ 90% | Context recall (correct pattern in top-3) |
| Code Compilation | β‰₯ 90% | % of generated code that compiles |
| Quality Score | β‰₯ 0.85 | Average code quality from validator |
| E2E Latency | < 20s | Time from screenshot to valid code |

All metrics align with industry-standard RAGAS framework.

### Documentation

- Full docs: `backend/src/evaluation/README.md`
- Demo materials: `docs/coursework/DEMO_METRICS.md`
- Dataset format: `backend/data/golden_dataset/README.md`
- Evaluation fixes: `backend/src/evaluation/EVALUATION_FIXES.md`
- Pipeline gaps analysis: `backend/src/evaluation/EVALUATION_GAPS.md`

## πŸ“Š AI Pipeline Monitoring

### Health Checks & APIs

- **ComponentForge Health**: http://localhost:8000/health
- **API Documentation**: http://localhost:8000/docs (FastAPI Swagger)
- **Metrics**: http://localhost:8000/metrics (Prometheus format)
- **Storybook**: http://localhost:6006 (Component library & testing)

### AI Observability

- **LangSmith Traces**: Monitor agent performance and costs
- **Token Extraction Confidence**: Track vision model accuracy
- **Pattern Retrieval Scores**: Semantic search effectiveness
- **Generation Quality**: TypeScript compilation and accessibility scores

### Infrastructure

- **Qdrant Dashboard**: http://localhost:6333/dashboard (Vector operations)
- **PostgreSQL**: Database performance and query logs
- **Redis**: Cache hit rates and performance

## 🐳 Docker Services

The project includes three essential services:

```yaml
# PostgreSQL Database
postgres:5432
- User: demo_user
- Password: demo_pass
- Database: demo_db

# Qdrant Vector Database
qdrant:6333/6334
- Dashboard: :6333/dashboard
- API: :6333

# Redis Cache
redis:6379
- Memory limit: 256MB
- Policy: allkeys-lru
```

## 🚨 Troubleshooting

### Common Issues

**Docker not starting:**

```bash
# Ensure Docker Desktop is running
open -a Docker

# Check Docker status
docker --version
docker-compose ps
```

**Python environment issues:**

```bash
# Recreate virtual environment
rm -rf backend/venv
cd backend && python -m venv venv
source venv/bin/activate && pip install -r requirements.txt
```

**Node.js dependency issues:**

```bash
# Clean install
cd app && rm -rf node_modules package-lock.json
npm install
```

**Port conflicts:**

- Frontend (3000), Backend (8000), PostgreSQL (5432), Qdrant (6333), Redis (6379)
- Check for other services using these ports: `lsof -i :3000`

**Qdrant/Semantic Search Issues:**

**Symptom: "Semantic retriever unavailable" in backend logs**

This means the system is running in BM25-only fallback mode (degraded accuracy).

**Solution:**

```bash
# 1. Verify Qdrant is running
curl http://localhost:6333/health

# 2. Check if patterns collection exists
curl http://localhost:6333/collections/patterns

# 3. If collection missing, seed it
cd backend
source venv/bin/activate
python scripts/seed_patterns.py

# 4. Restart backend to reinitialize semantic retriever
# (Kill uvicorn and restart)
```

**Symptom: "Architecture mismatch (arm64 vs x86_64)" when seeding**

Your Python venv was created with wrong architecture.

**Solution:**

```bash
# Recreate venv with correct architecture
cd backend
deactivate
rm -rf venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Retry seeding
python scripts/seed_patterns.py
```

**Symptom: OpenAI API errors during seeding**

Seeding requires OpenAI API to generate embeddings.

**Solution:**

```bash
# Check API key is set
echo $OPENAI_API_KEY

# If empty, add to backend/.env
echo "OPENAI_API_KEY=your-key-here" >> backend/.env

# Export it
export OPENAI_API_KEY="your-key-here"

# Retry seeding
cd backend && source venv/bin/activate
python scripts/seed_patterns.py
```

## 🎯 ComponentForge Workflow

### 1. Design Input

- **Screenshot Upload**: Drag & drop any UI design screenshot
- **Figma Integration**: Connect with Personal Access Token
- **Design Analysis**: GPT-4V extracts visual design patterns

### 2. AI Processing Pipeline

- **Token Extraction**: Colors, typography, spacing with confidence scores
- **Requirement Proposal**: Inferred props, states, behaviors, accessibility needs
- **Pattern Retrieval**: Semantic search through shadcn/ui component patterns
- **Quality Validation**: TypeScript, ESLint, axe-core accessibility checks

### 3. Generated Output

- **TypeScript Component**: Production-ready React component with proper types
- **Storybook Stories**: Interactive documentation and testing
- **Accessibility**: WCAG-compliant with proper ARIA attributes
- **Design Tokens**: JSON file with extracted design system values

## πŸ“ Development Notes

- **AI-First Architecture**: Direct OpenAI SDK integration with optional LangSmith tracing
- **Custom Agent System**: 6 specialized agents with manual asyncio orchestration
- **Hot Reloading**: Both frontend and backend support instant updates
- **Type Safety**: Strict TypeScript across the entire stack
- **Accessibility**: Built-in axe-core testing prevents WCAG violations
- **Production Ready**: Docker containerization for easy deployment

## Future Enhancements

ComponentForge is designed to scale from atomic components to complex compositions and full page layouts. Current release focuses on establishing quality foundations; future versions will tackle increasing complexity.

### Phase 1: Atomic Components (Current - v1.0)

**Focus:** Single-component generation with quality validation

- Basic shadcn/ui components (Button, Card, Input, Badge, etc.)
- Design token extraction with 85%+ accuracy
- Pattern retrieval from vector database
- Validation pipeline (TypeScript, ESLint, axe-core)
- **Success metrics:** 90% compilation rate, 80% pipeline success

### Phase 2: Complex Compositions (v1.5)

**Focus:** Multi-component generation with compositional reasoning

- **Composite components:** ProductCard, DataTable, MultiStepForm, NavigationMenu
- **Multi-pattern retrieval:** AI selects and composes 5-10 sub-components
- **Prop threading:** Automatically connects parent/child component data
- **State scaffolding:** Generates Zustand stores for complex state
- **Examples:**
- ProductCard = Card + AspectRatio + Image + Badge + Typography + Button
- DataTable = Table + Select (filters) + Pagination + Dropdown (actions)
- MultiStepForm = Tabs + FormFields[] + ValidationLogic + ProgressIndicator

### Phase 3: Full Page Layouts (v2.0)

**Focus:** Complete page generation with architectural decisions

- **Full dashboard pages:** Sidebar + Header + MetricCards + DataTables + Charts
- **Landing pages:** Hero + Features + Testimonials + CTA sections
- **Application pages:** Settings (Tabs + Forms), Profile (Layout + Cards)
- **Architectural AI:**
- Layout detection (Grid, Flex, responsive breakpoints)
- Routing decisions (Next.js App Router patterns)
- State management strategy (global vs local state)
- API mocking (fake endpoints for development)
- **Multi-file generation:**
- `app/page.tsx` - Main page component
- `components/*.tsx` - Extracted components
- `stores/*.ts` - State management
- `api/*.ts` - Mock API routes

### Near-Term Enhancements (All Phases)

1. **Design System Import** - Upload Figma Design System β†’ Auto-populate vector DB
2. **Learning System (Phase 4)** - AI learns from user edits and preferences
3. **Custom Component Libraries** - Support Material-UI, Chakra UI, Ant Design
4. **Multi-framework Support** - Vue, Svelte, Angular code generation
5. **Advanced Testing** - Auto-generate unit, integration, and E2E tests
6. **Real-time Collaboration** - WebSocket support for team editing
7. **Version Control** - Component versioning and diff tracking
8. **Performance Optimization** - Bundle analysis and optimization suggestions

### Why the Phased Approach?

- **Quality first:** Atomic components establish 90% quality baseline before scaling
- **Infrastructure reuse:** Multi-agent system, vector DB, validation pipeline built for all phases
- **Learning foundation:** Phase 1 builds pattern library that Phase 2 and 3 depend on
- **Risk management:** If complex generation fails, atomic generation still provides value
- **Technical scaling:** Compositional reasoning requires validated atomic patterns first

**Current state:** Phase 1 complete with production-ready infrastructure.
**Vision:** Full design-to-code automation from screenshots to complete applications.

## 🀝 Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## πŸ“„ License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

**Transform designs into code with AI!** 🧩✨

Built with ❀️ for developers who want to focus on logic, not repetitive UI coding.