{"id":31040432,"url":"https://github.com/rahulrmcoder/holohelp","last_synced_at":"2025-09-14T08:40:33.154Z","repository":{"id":304001648,"uuid":"1017456243","full_name":"RahulRmCoder/holohelp","owner":"RahulRmCoder","description":"HoloHelp is an innovative AR (Augmented Reality) application that combines **computer vision**, **3D modeling**, and **AI-powered assistance** to provide step-by-step troubleshooting for mobile phones and general assembly/repair guidance for any appliance or furniture.","archived":false,"fork":false,"pushed_at":"2025-07-10T15:14:22.000Z","size":172,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-10T21:50:26.621Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RahulRmCoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-10T15:00:17.000Z","updated_at":"2025-07-10T15:14:26.000Z","dependencies_parsed_at":"2025-07-10T21:50:35.715Z","dependency_job_id":"a1407f3d-c871-4397-9b51-4bce2afd391f","html_url":"https://github.com/RahulRmCoder/holohelp","commit_stats":null,"previous_names":["rahulrmcoder/holohelp"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/RahulRmCoder/holohelp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RahulRmCoder%2Fholohelp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RahulRmCoder%2Fholohelp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RahulRmCoder%2Fholohelp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RahulRmCoder%2Fholohelp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RahulRmCoder","download_url":"https://codeload.github.com/RahulRmCoder/holohelp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RahulRmCoder%2Fholohelp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275081662,"owners_count":25402336,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-14T02:00:10.474Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-14T08:40:31.784Z","updated_at":"2025-09-14T08:40:33.145Z","avatar_url":"https://github.com/RahulRmCoder.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HoloHelp - Enhanced AR Mobile Troubleshooting Assistant\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Node.js](https://img.shields.io/badge/Node.js-18+-green.svg)](https://nodejs.org/)\n[![React](https://img.shields.io/badge/React-18+-blue.svg)](https://reactjs.org/)\n[![Three.js](https://img.shields.io/badge/Three.js-AR-red.svg)](https://threejs.org/)\n\n\u003e **Revolutionary AR-powered mobile phone diagnostics with voice guidance and 3D component highlighting**\n\n## 🚀 Overview\n\nHoloHelp is an innovative AR (Augmented Reality) application that combines **computer vision**, **3D modeling**, and **AI-powered assistance** to provide step-by-step troubleshooting for mobile phones and general assembly/repair guidance for any appliance or furniture.\n\n### ✨ Key Features\n\n**📱 AR Mobile Phone Diagnostics**\n- Real-time phone detection using computer vision\n- **3D AR phone models** with realistic components\n- **Component highlighting** for specific parts (battery, power button, etc.)\n- **Voice-guided instructions** for each troubleshooting step\n- **Particle effects** and progress visualization\n- **Step-by-step AR overlays** with smooth animations\n\n**🤖 AI-Powered Assistant**\n- **Generic troubleshooting** for any furniture, appliance, or device\n- **Voice interaction** - ask questions through speech\n- **Assembly instructions** for ANY item (IKEA furniture, Samsung TV, etc.)\n- **Gemini AI integration** for unlimited possibilities\n- **Context-aware responses** based on detected devices\n\n**🎯 Enhanced AR Features**\n- **3D phone models** with detailed components\n- **Holographic highlighting** of specific parts\n- **Voice-guided step-by-step instructions**\n- **Particle effects** for celebrations\n- **Progress visualization** with completion tracking\n- **Smooth animations** and realistic rendering\n\n## 🛠 Technology Stack\n\n### Frontend\n- **React 18** - Modern UI framework\n- **Vite** - Lightning-fast build tool\n- **Three.js** - 3D graphics and AR rendering\n- **Tailwind CSS** - Utility-first styling\n- **Lucide React** - Beautiful icons\n- **Web Speech API** - Voice recognition and synthesis\n\n### Backend\n- **Node.js** - Server runtime\n- **Express.js** - Web framework\n- **Multer** - File upload handling\n- **CORS** - Cross-origin resource sharing\n\n### AI Integration\n- **Gemini AI** - Advanced AI assistance\n- **Web Speech API** - Voice interaction\n- **Computer Vision** - Object detection\n\n## 🚀 Quick Start\n\n### Prerequisites\n- **Node.js 18+** \n- **npm** or **yarn**\n- **Modern browser** with WebGL support\n- **Camera access** for AR features\n\n### Installation\n\n1. **Clone the repository**\n```bash\ngit clone https://github.com/RahulRmCoder/holohelp.git\ncd holohelp\n```\n\n2. **Install frontend dependencies**\n```bash\nnpm install\n```\n\n3. **Install backend dependencies**\n```bash\ncd backend\nnpm install\n```\n\n4. **Start the backend server**\n```bash\ncd backend\nnpm start\n```\nBackend will run on `http://localhost:3001`\n\n5. **Start the frontend development server**\n```bash\ncd .. # Back to root directory\nnpm run dev\n```\nFrontend will run on `http://localhost:5173`\n\n### 🔧 Configuration\n\n#### Environment Variables\nCreate a `.env` file in the root directory:\n```env\n# Gemini AI (Optional - for enhanced AI features)\nVITE_GEMINI_API_KEY=your_gemini_api_key_here\n\n# Backend URL (Default: http://localhost:3001)\nVITE_BACKEND_URL=http://localhost:3001\n```\n\n#### Gemini AI Setup (Optional)\nFor enhanced AI features:\n1. Visit [Google AI Studio](https://makersuite.google.com/app/apikey)\n2. Create a new API key\n3. Add it to your `.env` file as `VITE_GEMINI_API_KEY`\n\n## 📱 Usage\n\n### AR Mobile Phone Troubleshooting\n\n1. **Start Detection**\n   - Click \"🚀 Start Enhanced Phone AR Scanning\"\n   - Allow camera permissions\n   - Point camera at your mobile phone\n\n2. **Problem Selection**\n   - Choose from available problems:\n     - **Battery Drain** - Optimize battery usage\n     - **Slow Performance** - Speed up your phone\n     - **Overheating** - Cool down your device\n     - **WiFi Issues** - Fix connectivity problems\n     - **Storage Full** - Free up space\n\n3. **AR Guidance**\n   - Follow 3D AR instructions\n   - Listen to voice guidance\n   - Watch component highlighting\n   - Complete each step with visual feedback\n\n### AI Assistant\n\n1. **Voice Interaction**\n   - Click the microphone button\n   - Ask questions like:\n     - \"How to assemble Samsung TV stand?\"\n     - \"Fix my WiFi router\"\n     - \"Build IKEA desk\"\n\n2. **Text Chat**\n   - Type your question\n   - Get detailed step-by-step instructions\n   - Works for ANY item or appliance\n\n## 🎯 Supported Features\n\n### Mobile Phone Problems\n| Problem | Description | AR Features |\n|---------|-------------|-------------|\n| **Battery Drain** | Optimize battery usage | Settings highlighting, app closure guidance |\n| **Slow Performance** | Speed up device | Restart guidance, cache clearing |\n| **Overheating** | Cool down device | Component cooling, app management |\n| **WiFi Issues** | Fix connectivity | Network settings, signal visualization |\n| **Storage Full** | Free up space | Storage visualization, cleanup guidance |\n\n### AI Assistant Capabilities\n- **Assembly instructions** for ANY item\n- **Troubleshooting** for any device\n- **Voice interaction** support\n- **Context-aware** responses\n- **Step-by-step guidance**\n\n## 🏗 Architecture\n\n### Frontend Structure\n```\nsrc/\n├── components/\n│   ├── ARCanvas.jsx          # Main AR rendering component\n│   ├── HelpAssistant.jsx     # AI chat assistant\n│   ├── ObjectDetector.jsx    # Computer vision detection\n│   ├── MobileProblemSelector.jsx # Problem selection UI\n│   └── ui/                   # Reusable UI components\n├── services/\n│   ├── api.js               # Backend API calls\n│   └── geminiService.js     # AI integration\n├── utils/\n│   ├── voiceGuidance.js     # Voice features\n│   └── arEffects.js         # AR effects and animations\n└── App.jsx                  # Main application component\n```\n\n### Backend Structure\n```\nbackend/\n├── server.js                # Main Express server\n├── routes/\n│   ├── devices.js          # Device recognition\n│   ├── instructions.js     # AR instructions\n│   └── chat.js             # AI chat endpoints\n└── data/\n    └── instructions.json   # Troubleshooting database\n```\n\n## 🎨 AR Features Deep Dive\n\n### 3D Phone Models\n- **Realistic rendering** with PBR materials\n- **Component separation** (screen, buttons, camera)\n- **Dynamic screen content** showing actual phone UI\n- **Smooth animations** and rotations\n\n### Component Highlighting\n- **Targeted highlights** for specific parts\n- **Pulsing animations** to draw attention\n- **Progress rings** showing completion\n- **Floating arrows** for guidance\n\n### Voice Guidance\n- **Step-by-step narration** for each instruction\n- **Problem-specific introductions**\n- **Completion celebrations**\n- **Multi-language support** (planned)\n\n### Visual Effects\n- **Particle systems** for celebrations\n- **Smooth transitions** between steps\n- **Progress visualization** with rings and bars\n- **Component animations** (phone floating, rotating)\n\n## 🔧 Development\n\n### Available Scripts\n\n```bash\n# Frontend\nnpm run dev          # Start development server\nnpm run build        # Build for production\nnpm run preview      # Preview production build\nnpm run lint         # Run ESLint\n\n# Backend\nnpm start            # Start backend server\nnpm run dev          # Start with nodemon\nnpm test             # Run tests\n```\n\n### Adding New Problems\n\n1. **Add to backend instructions** in `server.js`:\n```javascript\n\"new_problem\": {\n  device_model: \"Mobile_Phone_New_Fix\",\n  device_type: \"mobile_phone\",\n  instruction_set_id: \"phone-new-001\",\n  ar_enhanced: true,\n  steps: [\n    {\n      step: 1,\n      title: \"Problem Step\",\n      description: \"Step description\",\n      voiceover: \"Voice guidance text\",\n      highlight: \"ComponentName\",\n      position: [0, 0, 0],\n      component_target: \"target_component\",\n      ar_effects: [\"highlight_component\"],\n      duration: 10000\n    }\n  ]\n}\n```\n\n2. **Update problem selector** in `MobileProblemSelector.jsx`\n\n3. **Add AR effects** in `ARCanvas.jsx`\n\n### Testing\n\n#### Manual Testing\n1. **Camera functionality** - Test object detection\n2. **AR rendering** - Verify 3D models load\n3. **Voice guidance** - Test speech synthesis\n4. **Step completion** - Verify progress tracking\n\n#### Browser Support\n- **Chrome/Edge** - Full support\n- **Firefox** - WebGL support required\n- **Safari** - iOS 12+ for AR features\n- **Mobile browsers** - Camera access required\n\n## 🌟 Contributing\n\nWe welcome contributions! Please follow these steps:\n\n1. **Fork the repository**\n2. **Create a feature branch**\n   ```bash\n   git checkout -b feature/amazing-feature\n   ```\n3. **Commit your changes**\n   ```bash\n   git commit -m 'Add amazing feature'\n   ```\n4. **Push to the branch**\n   ```bash\n   git push origin feature/amazing-feature\n   ```\n5. **Open a Pull Request**\n\n### Development Guidelines\n- Follow **React best practices**\n- Use **TypeScript** for new features (planned)\n- Add **comments** for complex AR logic\n- Test on **multiple devices**\n- Ensure **voice guidance** works properly\n\n## 📚 API Documentation\n\n### Backend Endpoints\n\n#### Health Check\n```\nGET /health\n```\nReturns server status and AR capabilities.\n\n#### Device Recognition\n```\nPOST /api/recognize-device\nContent-Type: multipart/form-data\n\n{\n  \"detectedObjects\": [\"cell phone\", \"smartphone\"]\n}\n```\n\n#### Problem Selection\n```\nPOST /api/phone-problem\nContent-Type: application/json\n\n{\n  \"problem\": \"battery_drain\",\n  \"ar_enhanced\": true\n}\n```\n\n#### Chat Assistant\n```\nPOST /api/chat\nContent-Type: application/json\n\n{\n  \"message\": \"How to fix WiFi?\",\n  \"context\": \"AR_MODE\",\n  \"deviceType\": \"mobile_phone\"\n}\n```\n\n## 📱 Mobile Support\n\n### iOS\n- **iOS 12+** for AR features\n- **Safari** or **Chrome** browser\n- **Camera permissions** required\n\n### Android\n- **Android 7+** for full functionality\n- **Chrome** browser recommended\n- **WebGL** support required\n\n### PWA Features (Planned)\n- **Offline mode** for instructions\n- **Push notifications**\n- **App-like experience**\n\n## 🚀 Deployment\n\n### Production Build\n```bash\n# Build frontend\nnpm run build\n\n# Build backend\ncd backend\nnpm run build\n```\n\n### Docker Deployment\n```bash\n# Build and run with Docker\ndocker-compose up --build\n```\n\n### Environment Variables for Production\n```env\nNODE_ENV=production\nVITE_BACKEND_URL=https://your-backend-url.com\nVITE_GEMINI_API_KEY=your_production_key\n```\n\n## 📊 Performance\n\n### Optimization Features\n- **Efficient AR rendering** with 60fps target\n- **Optimized 3D models** with LOD\n- **Smooth animations** with requestAnimationFrame\n- **Memory management** for long sessions\n\n### Browser Requirements\n- **WebGL 2.0** support\n- **Camera API** access\n- **Web Speech API** for voice features\n- **Modern ES6+** JavaScript support\n\n## 🔐 Security\n\n### Privacy\n- **No data collection** without consent\n- **Local processing** for sensitive operations\n- **Secure API endpoints**\n- **CORS protection**\n\n### Camera Access\n- **Permission-based** camera access\n- **No image storage** on servers\n- **Local processing** only\n\n## 🗺 Roadmap\n\n### Version 2.1 (Planned)\n- [ ] **Multi-language support**\n- [ ] **More device types** (tablets, smartwatches)\n- [ ] **Cloud sync** for progress\n- [ ] **Offline mode**\n\n### Version 2.2 (Planned)\n- [ ] **Advanced AR features** (hand tracking)\n- [ ] **Video tutorials** integration\n- [ ] **Community sharing** of fixes\n- [ ] **Analytics dashboard**\n\n### Version 3.0 (Future)\n- [ ] **VR support** for immersive experience\n- [ ] **AI-powered** automatic problem detection\n- [ ] **IoT integration** for smart home devices\n- [ ] **Enterprise features**\n\n## 📄 License\n\nThis project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.\n\n## 🙏 Acknowledgments\n\n- **Three.js** - Amazing 3D graphics library\n- **React** - Powerful UI framework\n- **Gemini AI** - Advanced AI assistance\n- **OpenCV.js** - Computer vision capabilities\n- **Web Speech API** - Voice interaction\n\n## 📞 Support\n\n### Get Help\n- **GitHub Issues** - Report bugs and request features\n- **Documentation** - Check this README and code comments\n- **Community** - Join discussions in Issues\n\n### Contact\n\n- **GitHub**: [@RahulRmCoder](https://github.com/RahulRmCoder)\n\n---\n\n**Made with ❤️ by the HoloHelp Team**\n\n*Revolutionizing device troubleshooting through AR technology* 🚀📱✨\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulrmcoder%2Fholohelp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frahulrmcoder%2Fholohelp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulrmcoder%2Fholohelp/lists"}