{"id":28633246,"url":"https://github.com/rishn/smart-scan-whatsapp-ocr","last_synced_at":"2025-06-12T15:07:44.546Z","repository":{"id":289588838,"uuid":"971755906","full_name":"rishn/Smart-Scan-WhatsApp-OCR","owner":"rishn","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-24T03:18:55.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-24T03:30:14.651Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rishn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-24T02:34:41.000Z","updated_at":"2025-04-24T03:18:58.000Z","dependencies_parsed_at":"2025-04-24T03:40:21.485Z","dependency_job_id":null,"html_url":"https://github.com/rishn/Smart-Scan-WhatsApp-OCR","commit_stats":null,"previous_names":["rishn/smart-scan-whatsapp-ocr"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rishn/Smart-Scan-WhatsApp-OCR","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishn%2FSmart-Scan-WhatsApp-OCR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishn%2FSmart-Scan-WhatsApp-OCR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishn%2FSmart-Scan-WhatsApp-OCR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishn%2FSmart-Scan-WhatsApp-OCR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rishn","download_url":"https://codeload.github.com/rishn/Smart-Scan-WhatsApp-OCR/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishn%2FSmart-Scan-WhatsApp-OCR/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259490543,"owners_count":22865766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-12T15:07:43.695Z","updated_at":"2025-06-12T15:07:44.538Z","avatar_url":"https://github.com/rishn.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Smart Scan - OCR for WhatsApp\n\n**Empowering WhatsApp users with in-chat OCR capabilities.**\n\n---\n\n## 📖 Overview\n\n**Smart Scan** enables WhatsApp Web users to instantly extract text and QR data from images — with privacy-first, on-device OCR.\n\n- 🧾 Extract text from invoices, notes, Wi-Fi passwords\n- 💸 Detect QR codes and initiate UPI payments instantly\n- 🔒 Fully on-device — no cloud, no leaks\n\n---\n\n## 📹 Demo\n\nWatch Smart Scan in action: \n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://drive.google.com/file/d/1w3yzi3O1wO7HWPpdN8QJA0Xh7XT0H9DL/view\"\u003e\n    \u003cimg src=\"https://github.com/user-attachments/assets/0bbc3f9b-7999-48b4-a290-f9cf7c930dc2\" alt=\"Watch Windows Opera Demo\" width=\"600\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## 🚀 Try the Prototype Live\n\n👉 **Check out the prototype here:** [Smart Scan Demo](https://smart-scan-whatsapp.onrender.com)  \n\u003e _Test it with images of invoices, QR codes, or handwritten notes!_\n\n---\n\n## 🚨 Problem Overview\n\n### **User Pain Point**\nWhatsApp users often receive important information as images, such as:\n- QR codes for payments\n- Contact information\n- Addresses or handwritten notes\n\nCurrently, WhatsApp lacks native OCR functionality, forcing users to:\n1. Save or screenshot the image.\n2. Open a third-party app (e.g., Google Lens).\n3. Extract the information manually.\n\nThis process introduces **latency**, **workflow friction**, and **user experience breakdown**, especially in fast-paced scenarios.\n\n### **Impact**\n- Delayed QR-based payments.\n- Inefficient access to text-based data (e.g., invoice numbers, OTPs).\n- Reduced productivity and reliance on third-party apps, risking data privacy.\n\n### **Case Study**\nRefer the case study here 👉 [Case Study](https://drive.google.com/file/d/1_Lr699S9o0geR2beNWAjHq1b7l3Y1TMe/view)  \n\n---\n\n## 💡 Solution: Smart Scan\n\nSmart Scan integrates OCR capabilities directly into WhatsApp Web, enabling users to extract actionable content from images without leaving the app.\n\n### **Key Features**\n- **Extraction**: Extracts clean, copyable text from any image.\n- **UPI QR Code Detection**: Detect UPI IDs and QR codes for payments.\n- **Contextual Actions**:\n  - Copy text or UPI links.\n  - Initiate payments via UPI apps (e.g., GPay, PhonePe, Paytm, WhatsApp Pay).\n  - Translate text (future enhancement).\n- **Responsive UI**:\n  - Close modals by clicking outside the modal or on the sidebar.\n\n---\n\n## 🖼️ Screenshots\n\n### **1. Image Preview Modal**\nThe `ImagePreviewModal` allows users to view and analyze images. Extracted text and QR codes are displayed with actionable buttons.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/assets/ImagePreviewModal.png?raw=true\" alt=\"Image Preview Modal\" /\u003e\n\u003c/p\u003e\n\n---\n\n### **2. UPI Payment Modal**\nThe `UPIPaymentModal` detects UPI QR codes and provides options to copy the UPI link or initiate payments.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/assets/UPIPaymentModal.png?raw=true\" alt=\"UPI Payment Modal\" /\u003e\n\u003c/p\u003e\n\n---\n\n## 🛠️ Technical Details\n\n### **Architecture**\n- **OCR Engine**: Uses lightweight libraries like `Tesseract` and `jsQR` for text and QR code detection.\n- **On-Device Processing**: Ensures user privacy and low latency.\n\n### Feature Architecture Flow\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/assets/FeatureArchitectureFlow.png?raw=true\" alt=\"Feature Architecture Flow\" /\u003e\n\u003c/p\u003e\n\n### **Key Components**\n1. **ImagePreviewModal**:\n   - Displays images with extracted text and QR codes.\n   - Allows users to copy text or analyze images.\n2. **UPIPaymentModal**:\n   - Detects UPI QR codes and provides actionable options.\n3. **Toast Notifications**:\n   - Provides feedback (e.g., \"Text copied!\") using a light-themed WhatsApp-style UI.\n  \n### **Product Spec**\nRefer the product spec here 👉 [Product Spec](https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/product-spec.md)\n\n---\n\n## 🚀 How It Works\n\n1. **Upload an Image**:\n   - Drag and drop or attach an image in the chat.\n2. **Analyze the Image**:\n   - Tap ‘Analyze’ to extract content\n3. **Take Action**:\n   - Copy extracted text or UPI links.\n   - Initiate payments via UPI apps.\n\n---\n\n## 📊 Benefits\n\n### **For Users**\n- **Time Efficiency**: Reduces OCR processing time from ~30 seconds (third-party apps) to ~3 seconds.\n- **Privacy**: On-device processing ensures secure data handling.\n- **Convenience**: No switching apps. No copy-pasting. Just scan and act.\n\n### Time Saved\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/assets/TimeSavings.png?raw=true\" alt=\"Time Saved\" /\u003e\n\u003c/p\u003e\n\n### **For Businesses**\n- **Faster Payments**: Accelerates QR-based checkouts and payments.\n- **Improved Productivity**: Streamlines workflows for SMBs and micro-entrepreneurs.\n- **Enhanced Engagement**: Increases time spent within WhatsApp.\n\n---\n\n## 🏆 Competitive Advantage\n\n| Platform               | Built-in OCR   | Contextual Actions             | Needs External App |\n|------------------------|----------------|--------------------------------|---------------------|\n| **Google Lens**        | ✅ Yes          | ✅ Rich                         | ❌ No               |\n| **iMessage (iOS)**     | ✅ Live Text    | ✅ Copy, Translate, Call        | ❌ No               |\n| **Telegram**           | ❌ No           | ❌ Limited                      | ✅ Yes              |\n| **WhatsApp (Now)**     | ❌ Partial      | ❌ Very Limited                 | ✅ Yes              |\n| **WhatsApp (Smart Scan)** | ✅ Yes       | ✅ Copy, Pay, Translate         | ❌ No               |\n\n### UX Comparison\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/rishn/Smart-Scan-WhatsApp-OCR/blob/main/assets/UXComparison.png?raw=true\" alt=\"UX Comparison\" /\u003e\n\u003c/p\u003e\n\n---\n\n## 📈 Future Enhancements\n\n- **Multilingual OCR**: Auto-detect languages and provide translation suggestions.\n- **Barcode Scanning**: Extend support to barcodes and event tickets.\n- **Advanced Contextual Actions**: Add options like \"Add to Contacts\" or \"Open URLs.\"\n\n---\n\n## 🧑‍💻 Installation \u0026 Setup\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/rishn/Smart-Scan-Whatsapp-OCR.git\n   cd Smart-Scan-Whatsapp-OCR\n   ```\n2. Install dependencies:\n    ```bash\n    npm install\n    ```\n3. Start the development server:\n    ```bash\n    npm start\n    ```\n4. Open the app in your browser:\n    ```bash\n    http://localhost:3000\n    ```\n---\n\n## 🙌 Acknowledgments\n- **Libraries Used**:\n  - [Tesseract.js](https://github.com/naptha/tesseract.js) - For text recognition.\n  - [jsQR](https://github.com/cozmo/jsQR) - For QR code detection.\n\n- **Inspiration**:\n  - WhatsApp's existing UPI QR detection.\n  - Google Lens for its seamless OCR capabilities.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishn%2Fsmart-scan-whatsapp-ocr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frishn%2Fsmart-scan-whatsapp-ocr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishn%2Fsmart-scan-whatsapp-ocr/lists"}