{"id":30249043,"url":"https://github.com/jefferson1104/let-me-ask","last_synced_at":"2026-04-09T02:02:16.604Z","repository":{"id":303534924,"uuid":"1015813986","full_name":"jefferson1104/let-me-ask","owner":"jefferson1104","description":"A project built with Node.js, React.js, and artificial intelligence designed to assist streamers during Q\u0026A sessions with their audience. The app helps organize and prioritize incoming questions, enabling a more efficient and dynamic interaction with viewers.","archived":false,"fork":false,"pushed_at":"2025-08-05T02:43:03.000Z","size":245,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-15T08:13:19.783Z","etag":null,"topics":["ai","docker","gemini-ai","nodejs","postgresql","reactjs"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jefferson1104.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-08T04:45:57.000Z","updated_at":"2025-08-05T02:43:07.000Z","dependencies_parsed_at":"2025-07-08T06:47:37.283Z","dependency_job_id":"626489b9-4f1c-42f5-a490-89a0d889d221","html_url":"https://github.com/jefferson1104/let-me-ask","commit_stats":null,"previous_names":["jefferson1104/let-me-ask"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jefferson1104/let-me-ask","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jefferson1104%2Flet-me-ask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jefferson1104%2Flet-me-ask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jefferson1104%2Flet-me-ask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jefferson1104%2Flet-me-ask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jefferson1104","download_url":"https://codeload.github.com/jefferson1104/let-me-ask/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jefferson1104%2Flet-me-ask/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31581864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"online","status_checked_at":"2026-04-09T02:00:06.848Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","docker","gemini-ai","nodejs","postgresql","reactjs"],"created_at":"2025-08-15T08:07:28.071Z","updated_at":"2026-04-09T02:02:16.568Z","avatar_url":"https://github.com/jefferson1104.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Let Me Ask - AI-Powered Streaming Assistant\n\n\u003cdiv align=\"center\"\u003e\n\n![Let Me Ask Logo](https://img.shields.io/badge/Let%20Me%20Ask-AI%20Assistant-blue?style=for-the-badge\u0026logo=google)\n![TypeScript](https://img.shields.io/badge/TypeScript-007ACC?style=for-the-badge\u0026logo=typescript\u0026logoColor=white)\n![React](https://img.shields.io/badge/React-20232A?style=for-the-badge\u0026logo=react\u0026logoColor=61DAFB)\n![Fastify](https://img.shields.io/badge/Fastify-000000?style=for-the-badge\u0026logo=fastify\u0026logoColor=white)\n![PostgreSQL](https://img.shields.io/badge/PostgreSQL-316192?style=for-the-badge\u0026logo=postgresql\u0026logoColor=white)\n\n\u003c/div\u003e\n\n## 📖 Overview\n\n**Let Me Ask** is an innovative AI-powered streaming assistant designed to help content creators and streamers efficiently respond to viewer questions. The application leverages Google's Gemini AI to transcribe live audio content, create semantic embeddings, and generate intelligent responses based on the streamer's actual spoken content.\n\n### 🎯 Key Features\n\n- **Real-time Audio Recording**: Capture live stream audio with high-quality recording capabilities\n- **AI-Powered Transcription**: Automatic speech-to-text conversion using Google Gemini AI\n- **Semantic Search**: Advanced embedding-based search to find relevant content from stream recordings\n- **Intelligent Q\u0026A**: Generate contextual answers based on the streamer's actual spoken content\n- **Room-based Organization**: Organize content by streaming sessions/rooms\n- **Vector Database**: PostgreSQL with pgvector extension for efficient similarity search\n\n## 🏗️ Architecture\n\nThe project follows a modern full-stack architecture with clear separation of concerns:\n\n```\nlet-me-ask/\n├── client/          # React frontend application\n├── server/          # Fastify backend API\n└── README.md        # This file\n```\n\n### Frontend (React + TypeScript)\n\n- **Framework**: React 19 with TypeScript\n- **Styling**: Tailwind CSS with Radix UI components\n- **State Management**: TanStack Query for server state\n- **Forms**: React Hook Form with Zod validation\n- **Routing**: React Router DOM\n- **Build Tool**: Vite\n\n### Backend (Fastify + TypeScript)\n\n- **Framework**: Fastify with TypeScript\n- **Database**: PostgreSQL with pgvector extension\n- **ORM**: Drizzle ORM\n- **AI Integration**: Google Gemini AI API\n- **Validation**: Zod schema validation\n- **File Upload**: Multipart file handling\n\n## 🚀 Getting Started\n\n### Prerequisites\n\n- **Node.js** (v18 or higher)\n- **Docker** and **Docker Compose**\n- **Google Gemini API Key** ([Get one here](https://makersuite.google.com/app/apikey))\n\n### Environment Setup\n\n1. **Clone the repository**\n\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd let-me-ask\n   ```\n\n2. **Set up environment variables**\n\n   Create a `.env` file in the `server/` directory:\n\n   ```env\n   PORT=3333\n   DATABASE_URL=postgresql://lma:lma@localhost:5432/let_me_ask_agents\n   GEMINI_API_KEY=your_gemini_api_key_here\n   ```\n\n### Backend Setup\n\n1. **Navigate to the server directory**\n\n   ```bash\n   cd server\n   ```\n\n2. **Install dependencies**\n\n   ```bash\n   npm install\n   ```\n\n3. **Start the database**\n\n   ```bash\n   docker-compose up -d\n   ```\n\n4. **Run database migrations**\n\n   ```bash\n   npm run db:generate\n   npm run db:migrate\n   ```\n\n5. **Seed the database (optional)**\n\n   ```bash\n   npm run db:seed\n   ```\n\n6. **Start the development server**\n\n   ```bash\n   npm run dev\n   ```\n\n   The backend will be available at `http://localhost:3333`\n\n### Frontend Setup\n\n1. **Navigate to the client directory**\n\n   ```bash\n   cd client\n   ```\n\n2. **Install dependencies**\n\n   ```bash\n   npm install\n   ```\n\n3. **Start the development server**\n\n   ```bash\n   npm run dev\n   ```\n\n   The frontend will be available at `http://localhost:5173`\n\n## 🔧 Available Scripts\n\n### Backend Scripts\n\n```bash\nnpm run dev          # Start development server with hot reload\nnpm run start        # Start production server\nnpm run db:generate  # Generate database migrations\nnpm run db:migrate   # Run database migrations\nnpm run db:seed      # Seed database with sample data\nnpm run db:studio    # Open Drizzle Studio for database management\n```\n\n### Frontend Scripts\n\n```bash\nnpm run dev          # Start development server\nnpm run build        # Build for production\nnpm run preview      # Preview production build\n```\n\n## 🎮 How It Works\n\n### 1. Room Creation\n\n- Create a new streaming room with a name and description\n- Each room represents a streaming session\n\n### 2. Audio Recording\n\n- Use the recording interface to capture live stream audio\n- Audio is automatically chunked into 5-second segments\n- Each chunk is sent to the backend for processing\n\n### 3. AI Processing Pipeline\n\n```\nAudio Recording → Transcription → Embedding Generation → Database Storage\n```\n\n- **Transcription**: Audio is converted to text using Gemini AI\n- **Embedding Generation**: Text is converted to 768-dimensional vectors\n- **Storage**: Transcriptions and embeddings are stored in PostgreSQL with pgvector\n\n### 4. Question Answering\n\n```\nUser Question → Embedding Search → Context Retrieval → AI Answer Generation\n```\n\n- **Semantic Search**: User questions are converted to embeddings and compared with stored content\n- **Context Retrieval**: Most similar audio chunks are retrieved (similarity \u003e 0.7)\n- **Answer Generation**: Gemini AI generates answers based on the retrieved context\n\n## 🗄️ Database Schema\n\n### Rooms\n\n- `id`: Unique identifier\n- `name`: Room name\n- `description`: Room description\n- `createdAt`: Creation timestamp\n\n### Audio Chunks\n\n- `id`: Unique identifier\n- `roomId`: Reference to room\n- `transcription`: Transcribed text content\n- `embeddings`: 768-dimensional vector representation\n- `createdAt`: Creation timestamp\n\n### Questions\n\n- `id`: Unique identifier\n- `roomId`: Reference to room\n- `question`: User's question\n- `answer`: AI-generated answer\n- `createdAt`: Creation timestamp\n\n## 🤖 AI Integration\n\n### Google Gemini AI Features\n\n1. **Audio Transcription**\n\n   - Model: `gemini-2.5-flash`\n   - Multi-language support with automatic translation to US English\n   - High-accuracy speech-to-text conversion\n\n2. **Embedding Generation**\n\n   - Model: `text-embedding-004`\n   - 768-dimensional vectors for semantic search\n   - Optimized for retrieval tasks\n\n3. **Answer Generation**\n   - Context-aware responses based on streamer's content\n   - Professional and educational tone\n   - Citation of relevant content when appropriate\n\n### Vector Similarity Search\n\nThe application uses PostgreSQL's pgvector extension to perform efficient similarity searches:\n\n```sql\n-- Find most similar audio chunks\nSELECT * FROM audio_chunks\nWHERE room_id = ?\n  AND 1 - (embeddings \u003c=\u003e ?::vector) \u003e 0.7\nORDER BY embeddings \u003c=\u003e ?::vector\nLIMIT 3;\n```\n\n## 🔒 Security \u0026 Best Practices\n\n- **Input Validation**: All inputs are validated using Zod schemas\n- **CORS Configuration**: Proper CORS setup for frontend-backend communication\n- **Environment Variables**: Sensitive data stored in environment variables\n- **Type Safety**: Full TypeScript implementation for type safety\n- **Error Handling**: Comprehensive error handling throughout the application\n\n## 🛠️ Development\n\n### Code Quality\n\n- **Biome**: Code formatting and linting\n- **TypeScript**: Static type checking\n- **ESLint**: Code quality rules\n\n### Database Management\n\n- **Drizzle ORM**: Type-safe database operations\n- **Drizzle Studio**: Visual database management interface\n- **Migration System**: Version-controlled database schema changes\n\n## 📝 API Endpoints\n\n### Rooms\n\n- `GET /rooms` - List all rooms\n- `POST /rooms` - Create a new room\n\n### Audio\n\n- `POST /rooms/:roomId/audio` - Upload audio chunk\n\n### Questions\n\n- `GET /rooms/:roomId/questions` - Get room questions\n- `POST /rooms/:roomId/questions` - Create a new question\n\n### Health\n\n- `GET /health` - Health check endpoint\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## 📄 License\n\nThis project is licensed under the ISC License.\n\n## 🙏 Acknowledgments\n\n- **Google Gemini AI** for providing powerful AI capabilities\n- **Fastify** for the high-performance web framework\n- **Drizzle ORM** for type-safe database operations\n- **React** and **Vite** for the modern frontend development experience\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Built with ❤️ for the streaming community**\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjefferson1104%2Flet-me-ask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjefferson1104%2Flet-me-ask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjefferson1104%2Flet-me-ask/lists"}