{"id":48477898,"url":"https://github.com/janardannn/ai-eval-lab","last_synced_at":"2026-04-07T07:52:21.313Z","repository":{"id":339603887,"uuid":"1162625973","full_name":"janardannn/ai-eval-lab","owner":"janardannn","description":"AI-proctored practical sessions for hardware engineers. Streams environments to the browser, captures real-time workflow telemetry, and evaluates the process.","archived":false,"fork":false,"pushed_at":"2026-03-02T18:44:13.000Z","size":344,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-02T18:54:36.108Z","etag":null,"topics":["docker","elevenlabs-api","freecad","gemini-api","kicad","next-auth","nextjs","novnc","postgres","prisma","redis","typescript","websockify","x11vnc"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/janardannn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-20T13:53:47.000Z","updated_at":"2026-03-02T18:39:09.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/janardannn/ai-eval-lab","commit_stats":null,"previous_names":["janardannn/ai-eval-lab"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/janardannn/ai-eval-lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janardannn%2Fai-eval-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janardannn%2Fai-eval-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janardannn%2Fai-eval-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janardannn%2Fai-eval-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/janardannn","download_url":"https://codeload.github.com/janardannn/ai-eval-lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janardannn%2Fai-eval-lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31504897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T03:10:19.677Z","status":"ssl_error","status_checked_at":"2026-04-07T03:10:13.982Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","elevenlabs-api","freecad","gemini-api","kicad","next-auth","nextjs","novnc","postgres","prisma","redis","typescript","websockify","x11vnc"],"created_at":"2026-04-07T07:52:21.205Z","updated_at":"2026-04-07T07:52:21.286Z","avatar_url":"https://github.com/janardannn.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ai-eval-lab\n\nAI-proctored practical exam platform for engineering tools. Students complete hands-on tasks in browser-streamed desktop applications (KiCad, FreeCAD) while an AI conducts interviews, monitors progress, and evaluates performance against rubrics.\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────┐\n│  Browser                                                │\n│  ┌──────────┐  ┌──────────┐  ┌────────────────────────┐ │\n│  │ AI Panel │  │  noVNC   │  │  Timer + Controls      │ │\n│  │ TTS/STT  │  │ (iframe) │  │                        │ │\n│  └────┬─────┘  └─────┬────┘  └────────────────────────┘ │\n└───────┼──────────────┼──────────────────────────────────┘\n        │              │\n        ▼              ▼\n┌──────────────┐  ┌──────────────┐\n│  Next.js     │  │  Environment │\n│  API Routes  │  │  Container   │\n│              │  │              │\n│  - AI Q\u0026A    │  │  Xvfb        │\n│  - Grading   │  │  x11vnc      │\n│  - Sessions  │  │  websockify  │\n│  - Admin     │  │  pcbnew      │\n│              │  │  poller      │──► POST /api/poller\n└──────┬───────┘  └──────────────┘\n       │\n  ┌────┴───────┐\n  ▼            ▼\n┌────────┐  ┌─────────┐\n│Postgres│  │Redis    │\n│        │  │         │\n│Users   │  │Queue    │\n│Sessions│  │State    │\n│Grades  │  │Heartbeat│\n└────────┘  └─────────┘\n```\n\n## Tech Stack\n\n- **Frontend + Backend:** Next.js 16, React, TypeScript, Tailwind CSS\n- **Database:** PostgreSQL 16 (Prisma ORM)\n- **Cache/Queue:** Redis 7 (ioredis)\n- **AI:** Google Gemini (grading, Q\u0026A, nudges, STT)\n- **TTS:** ElevenLabs\n- **Containers:** Docker + dockerode (dynamic provisioning)\n- **VNC:** Xvfb + x11vnc + websockify + noVNC\n- **Auth:** NextAuth v5 (GitHub, Google OAuth)\n- **Email:** Resend\n\n## Features\n\n- **Three-phase exams:** Intro interview → Domain viva → Hands-on lab\n- **Live KiCad streaming:** Browser-based PCB design via VNC\n- **AI proctor:** Adaptive questioning, audio recording, TTS responses\n- **Board poller:** 3-second snapshots of PCB state (footprints, tracks, zones)\n- **AI grading:** LLM evaluates timeline + Q\u0026A against rubric checkpoints\n- **Nudge engine:** Detects stagnation, sends encouraging prompts\n- **Admin panel:** Dashboard, assessment wizard, session browser, user management\n- **Queue system:** Capacity-limited container pool with waiting room\n- **Heartbeat monitoring:** Auto-cleanup of dead sessions\n\n## Getting Started\n\n### Prerequisites\n\n- Node.js 22+\n- Docker\n- PostgreSQL 16\n- Redis 7\n\n### Local Development\n\n```bash\n# Clone and install\ngit clone git@github.com:janardannn/ai-eval-lab.git\ncd ai-eval-lab/apps/web\nnpm install\n\n# Setup environment\ncp ../../.env.example .env\n# Edit .env with your API keys\n\n# Database\nnpx prisma migrate dev\nnpx prisma db seed\n\n# Build KiCad image\ncd ../../docker/kicad\ndocker build -t ai-eval-lab-kicad .\n\n# Run\ncd ../../apps/web\nnpm run dev\n```\n\n### Docker Compose (Development)\n\n```bash\ncd docker\ndocker compose up --build\n```\n\n### Docker Compose (Production)\n\n```bash\ncd docker\nDOMAIN=yourdomain.com POSTGRES_PASSWORD=securepwd docker compose -f docker-compose.prod.yml up -d\n```\n\nCaddy automatically provisions HTTPS via Let's Encrypt.\n\n## Environment Variables\n\n| Variable | Description |\n|---|---|\n| `DATABASE_URL` | PostgreSQL connection string |\n| `REDIS_URL` | Redis connection string |\n| `GEMINI_API_KEY` | Google AI API key (Gemini) |\n| `ELEVENLABS_API_KEY` | ElevenLabs TTS key |\n| `ELEVENLABS_VOICE_ID` | ElevenLabs voice ID |\n| `AUTH_SECRET` | NextAuth secret (`npx auth secret`) |\n| `GITHUB_CLIENT_ID/SECRET` | GitHub OAuth app |\n| `GOOGLE_CLIENT_ID/SECRET` | Google OAuth app |\n| `RESEND_API_KEY` | Resend email API key |\n| `EMAIL_FROM` | Sender email address |\n| `NEXT_PUBLIC_APP_URL` | Public URL (e.g., `https://yourdomain.com`) |\n| `KICAD_IMAGE` | Docker image name for KiCad container |\n| `MAX_CONTAINERS` | Max concurrent exam containers |\n| `CRON_SECRET` | Secret for cleanup cron endpoint |\n\n## Exam Flow\n\n1. Student browses assessments at `/lab/kicad`\n2. Starts exam → enters queue if at capacity\n3. Container provisioned → VNC iframe loads KiCad\n4. **Intro phase:** AI asks background questions (TTS/STT)\n5. **Domain phase:** Adaptive technical questions\n6. **Lab phase:** Student designs PCB, poller captures snapshots\n7. Student submits → file extracted → container torn down\n8. AI grades against rubric → verdict page with detailed report\n9. Email notification sent with results\n\n## Admin\n\nSet `isAdmin: true` on a user record in the database, then access `/admin`:\n\n- **Dashboard:** Stats, verdict distribution, completion trends\n- **Assessments:** Create/edit with 5-step wizard, upload reference files\n- **Sessions:** Filter, regrade, manual verdict override\n- **Users:** View history, scores\n\n## Project Structure\n\n```\napps/web/src/\n├── app/\n│   ├── api/\n│   │   ├── admin/          # Dashboard, CRUD, regrade, override\n│   │   ├── ai/             # Question generation, answer eval, nudges\n│   │   ├── grader/         # LLM grading + email notification\n│   │   ├── session/        # Start, status, heartbeat, end\n│   │   └── poller/          # Snapshot ingestion from KiCad poller\n│   ├── admin/              # Admin panel pages\n│   ├── lab/                # Assessment browsing + start\n│   ├── queue/              # Waiting room\n│   └── session/            # Exam workspace + verdict\n├── components/             # AIProctor, VNCViewer, Timer\n├── hooks/                  # useHeartbeat, useNudge\n└── lib/                    # Redis, Docker, AI, Grader, Email, TTS/STT\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanardannn%2Fai-eval-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjanardannn%2Fai-eval-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanardannn%2Fai-eval-lab/lists"}