{"id":28864037,"url":"https://github.com/yosef-alsabbah/cloud-based-document-analytics-service-2","last_synced_at":"2026-04-29T23:31:12.593Z","repository":{"id":290924353,"uuid":"976005347","full_name":"Yosef-AlSabbah/Cloud-Based-Document-Analytics-Service-2","owner":"Yosef-AlSabbah","description":"Cloud-based service for uploading, scraping, and managing PDF/DOCX documents. Features include title sorting, content search with highlights, rule-based classification, and storage stats. Integrated with cloud platforms for scalable document analytics.","archived":false,"fork":false,"pushed_at":"2025-06-05T12:24:56.000Z","size":275,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-20T08:43:12.252Z","etag":null,"topics":["cloud-computing","document-analysis","document-classification","document-management","full-stack","machine-learning","react","serverless","supabase","text-analysis","typescript","vite"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Yosef-AlSabbah.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-01T09:59:23.000Z","updated_at":"2025-06-20T07:34:42.000Z","dependencies_parsed_at":"2025-05-01T11:39:26.373Z","dependency_job_id":null,"html_url":"https://github.com/Yosef-AlSabbah/Cloud-Based-Document-Analytics-Service-2","commit_stats":null,"previous_names":["yosef-alsabbah/cloud-based-document-analytics-service"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Yosef-AlSabbah/Cloud-Based-Document-Analytics-Service-2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yosef-AlSabbah%2FCloud-Based-Document-Analytics-Service-2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yosef-AlSabbah%2FCloud-Based-Document-Analytics-Service-2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yosef-AlSabbah%2FCloud-Based-Document-Analytics-Service-2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yosef-AlSabbah%2FCloud-Based-Document-Analytics-Service-2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Yosef-AlSabbah","download_url":"https://codeload.github.com/Yosef-AlSabbah/Cloud-Based-Document-Analytics-Service-2/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yosef-AlSabbah%2FCloud-Based-Document-Analytics-Service-2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32448333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T22:27:22.272Z","status":"ssl_error","status_checked_at":"2026-04-29T22:10:49.234Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-computing","document-analysis","document-classification","document-management","full-stack","machine-learning","react","serverless","supabase","text-analysis","typescript","vite"],"created_at":"2025-06-20T08:33:02.833Z","updated_at":"2026-04-29T23:31:12.587Z","avatar_url":"https://github.com/Yosef-AlSabbah.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud Document Analytics Platform\n\n\u003e Advanced cloud-based document analytics system for searching, sorting, and classifying documents using machine learning algorithms\n\n![Platform Preview](https://via.placeholder.com/800x400/3B82F6/FFFFFF?text=Cloud+Document+Analytics)\n\n## 📑 Abstract\n\nThe Cloud Document Analytics Platform is a comprehensive solution designed for efficient document management, analysis, and classification in cloud environments. Built with modern web technologies and leveraging cloud-native services, this platform enables users to extract meaningful insights from various document formats through advanced search capabilities, intelligent classification, and detailed analytics.\n\nThis project demonstrates the practical application of cloud computing concepts by implementing a serverless architecture with Supabase as the backend service provider, coupled with a responsive React frontend. The system showcases how distributed document processing can be achieved efficiently in the cloud while maintaining security, scalability, and performance.\n\n## 1. Introduction\n\nThe Cloud Document Analytics Platform addresses the growing need for intelligent document management systems in educational, corporate, and research environments. As digital content continues to grow exponentially, traditional document management systems struggle with organizing, searching, and extracting insights from large document collections.\n\nThis platform adopts a cloud-first development methodology, leveraging serverless architecture patterns to minimize operational overhead while maximizing scalability. By utilizing Supabase's Backend-as-a-Service (BaaS) capabilities, the system achieves a separation of concerns between frontend and backend components while maintaining robust data security through Row Level Security policies. The development process followed an iterative approach with continuous integration and deployment practices, enabling rapid feature development and refinement.\n\n## 2. Cloud Software Program/Service Requirements\n\n### User Stories\n\n- As a researcher, I want to upload multiple document formats so that I can analyze documents regardless of their source.\n- As a student, I want to search within document content so that I can quickly find relevant information.\n- As a teacher, I want to categorize documents automatically so that I can maintain an organized collection.\n- As an analyst, I want to visualize document metrics so that I can understand the composition of my document collection.\n- As a content collector, I want to scrape web pages for content so that I can build my document repository efficiently.\n- As a mobile user, I want a responsive interface so that I can access my documents from any device.\n\n### Use Cases\n\n1. **Document Management**:\n   - Upload documents (PDF, DOC, DOCX)\n   - View document metadata\n   - Delete documents\n   - Download documents\n\n2. **Content Analysis**:\n   - Full-text search with relevance scoring\n   - Metadata extraction\n   - Content summarization\n   - Classification by content type\n\n3. **Web Scraping**:\n   - URL input for content extraction\n   - Automatic document generation from web content\n   - Scheduled scraping of specified sources\n\n4. **Analytics**:\n   - Document type distribution\n   - Upload frequency trends\n   - Search term analytics\n   - Classification accuracy metrics\n\n## 3. Software Architecture and Design\n\n### Architecture Diagram\n\n```\n┌─────────────────┐     ┌───────────────────────┐     ┌────────────────────┐\n│                 │     │                       │     │                    │\n│  React Frontend ├─────┤ Supabase BaaS Layer   ├─────┤ Edge Functions    │\n│  (Vite + TS)    │     │ (Auth, DB, Storage)   │     │ (Web Scraper)     │\n│                 │     │                       │     │                    │\n└────────┬────────┘     └───────────┬───────────┘     └────────────────────┘\n         │                          │\n         │                          │\n┌────────▼──────────┐     ┌─────────▼───────────┐\n│                   │     │                     │\n│  UI Components    │     │  PostgreSQL DB      │\n│  (Shadcn/UI)      │     │  (Document Store)   │\n│                   │     │                     │\n└───────────────────┘     └─────────────────────┘\n```\n\n### Component Design\n\nThe system is designed with several key functional components:\n\n1. **Document Processing Pipeline**:\n   - File upload handler with type validation\n   - Content extraction module for different document formats\n   - Metadata parser for capturing document properties\n\n2. **Search Algorithm**:\n   - Tokenization and normalization of document content\n   - Inverted index for efficient term matching\n   - TF-IDF based relevance scoring system\n   - Fuzzy matching capabilities for handling typos\n\n3. **Classification System**:\n   - Feature extraction from document content and metadata\n   - Rule-based classification for common document types\n   - Category matching based on content analysis\n   - User feedback loop for improving classification accuracy\n\n4. **Web Scraping Service**:\n   - URL validation and sanitization\n   - HTML parsing and content extraction\n   - Document conversion from web content\n   - Rate limiting to prevent abuse\n\n### Database Design\n\nThe system uses a PostgreSQL database (provided by Supabase) with the following schema:\n\n```sql\n-- Documents table\nCREATE TABLE public.documents (\n  id UUID NOT NULL DEFAULT gen_random_uuid() PRIMARY KEY,\n  user_id UUID REFERENCES auth.users NOT NULL,\n  name TEXT NOT NULL,\n  title TEXT,\n  content TEXT,\n  file_path TEXT,\n  size BIGINT,\n  type TEXT,\n  classification TEXT,\n  relevance_score DECIMAL,\n  upload_time TIMESTAMP WITH TIME ZONE DEFAULT now(),\n  created_at TIMESTAMP WITH TIME ZONE DEFAULT now(),\n  updated_at TIMESTAMP WITH TIME ZONE DEFAULT now()\n);\n```\n\n### User Interface Design\n\nThe UI is designed with a focus on usability and aesthetics:\n\n- **Dashboard Layout**: Card-based interface showing document statistics and recent uploads\n- **Search Interface**: Prominent search bar with filtering options and result highlighting\n- **Upload Component**: Drag-and-drop interface with progress indication\n- **Document List**: Sortable and filterable list with action buttons\n- **Classification Panel**: Visual representation of document categories with distribution charts\n\n## 4. Used Cloud Services and Interfaces\n\nThe platform leverages the following cloud services:\n\n1. **Supabase**:\n   - **Authentication**: User registration and login management\n   - **Database**: PostgreSQL database for document metadata storage\n   - **Storage**: Object storage for document files\n   - **Edge Functions**: Serverless functions for web scraping and processing\n\n2. **Vercel**:\n   - **Hosting**: Frontend application deployment\n   - **CI/CD Pipeline**: Automated build and deployment\n   - **CDN**: Global content delivery network\n   - **Analytics**: Usage and performance monitoring\n\n3. **Integration Services**:\n   - **PDF Processing**: PDF-lib for document manipulation\n   - **Word Processing**: Mammoth for DOCX/DOC conversion\n\n## 5. Implementation\n\n### Frontend Implementation\n\nThe frontend is built using React with TypeScript, utilizing the Vite build tool for optimal developer experience and build performance. Key implementation details include:\n\n```tsx\n// Document Upload Component\nconst DocumentUpload: React.FC = () =\u003e {\n  const [files, setFiles] = useState\u003cFile[]\u003e([]);\n  const [uploading, setUploading] = useState(false);\n  \n  const handleUpload = async () =\u003e {\n    setUploading(true);\n    try {\n      // Process each file\n      for (const file of files) {\n        // Extract metadata\n        const metadata = await extractMetadata(file);\n        \n        // Upload to Supabase storage\n        const { data, error } = await supabase.storage\n          .from('documents')\n          .upload(`${uuidv4()}-${file.name}`, file);\n          \n        if (error) throw error;\n        \n        // Store document record\n        await supabase.from('documents').insert({\n          name: file.name,\n          title: metadata.title,\n          size: file.size,\n          type: file.type,\n          file_path: data.path,\n        });\n      }\n    } catch (error) {\n      console.error('Error uploading:', error);\n    } finally {\n      setUploading(false);\n      setFiles([]);\n    }\n  };\n  \n  // Render upload interface\n  return (\n    // Upload interface implementation\n  );\n};\n```\n\n### Document Processing Implementation\n\n```typescript\n// Document Processing Utility\nexport const processDocument = async (file: File): Promise\u003cDocumentData\u003e =\u003e {\n  const fileType = file.name.split('.').pop()?.toLowerCase();\n  let content = '';\n  \n  switch (fileType) {\n    case 'pdf':\n      content = await extractPdfContent(file);\n      break;\n    case 'docx':\n    case 'doc':\n      content = await extractWordContent(file);\n      break;\n    default:\n      throw new Error('Unsupported file type');\n  }\n  \n  // Extract metadata\n  const metadata = extractMetadata(content);\n  \n  // Classify document\n  const classification = classifyDocument(content, metadata);\n  \n  return {\n    content,\n    metadata,\n    classification,\n  };\n};\n```\n\n### Search Implementation\n\n```typescript\n// Search functionality\nexport const searchDocuments = async (\n  query: string,\n  filters: SearchFilters\n): Promise\u003cSearchResult[]\u003e =\u003e {\n  // Tokenize search query\n  const tokens = tokenizeQuery(query);\n  \n  // Build SQL search conditions\n  let searchCondition = '';\n  if (tokens.length \u003e 0) {\n    searchCondition = tokens.map(token =\u003e \n      `content ILIKE '%${token}%' OR title ILIKE '%${token}%'`\n    ).join(' OR ');\n  }\n  \n  // Apply filters\n  let filterConditions = [];\n  if (filters.type) filterConditions.push(`type = '${filters.type}'`);\n  if (filters.classification) filterConditions.push(`classification = '${filters.classification}'`);\n  \n  // Execute search\n  const { data, error } = await supabase\n    .from('documents')\n    .select('*')\n    .or(searchCondition)\n    .and(filterConditions.join(' AND '));\n    \n  if (error) throw error;\n  \n  // Calculate relevance scores\n  return data.map(doc =\u003e ({\n    ...doc,\n    relevance_score: calculateRelevanceScore(doc, query)\n  }));\n};\n```\n\n## 6. Data\n\nThe platform utilizes Supabase's PostgreSQL database for structured data storage with the following model:\n\n### Document Data Model\n\n- **documents**: Stores metadata and content for uploaded documents\n  - Primary identification (UUID)\n  - User ownership reference\n  - Document attributes (name, title, size, type)\n  - Content and classification data\n  - Temporal metadata (upload time, update time)\n\n### Storage Model\n\nDocument files are stored in Supabase Storage buckets with the following structure:\n\n- **documents/**: Root bucket for all document files\n  - **{user_id}/**: Segregated by user for security\n    - **{document_id}-{filename}**: Individual document files\n\n### Security Implementation\n\n- Row Level Security (RLS) policies ensure users can only access their own documents\n- Storage bucket policies restrict file access based on user authentication\n- JWT-based authentication for secure API access\n\n## 7. The Used Cloud Platform\n\n### Supabase Platform Architecture\n\nSupabase provides a comprehensive Backend-as-a-Service platform with the following components:\n\n1. **PostgreSQL Database**:\n   - High-performance relational database\n   - Full-text search capabilities\n   - Real-time subscriptions\n   - Row-level security policies\n\n2. **Authentication Service**:\n   - User management\n   - Multiple auth providers\n   - JWT token handling\n   - Secure password storage\n\n3. **Storage Service**:\n   - S3-compatible object storage\n   - Public and private buckets\n   - Access control policies\n   - Image transformations\n\n4. **Edge Functions**:\n   - Deno-based serverless functions\n   - Globally distributed execution\n   - Low-latency responses\n   - Secure environment variables\n\n### Vercel Deployment Platform\n\nVercel provides a seamless frontend deployment platform with:\n\n1. **Build System**:\n   - Optimized for modern JavaScript frameworks\n   - Automatic dependency installation\n   - Environment variable management\n\n2. **Edge Network**:\n   - Global CDN distribution\n   - Automatic SSL/TLS\n   - High-performance edge caching\n   - Instant cache invalidation\n\n## 8. Deployment on the Platform\n\n### Deployment Process\n\nThe application deployment follows a streamlined process:\n\n1. **Code Repository Setup**:\n   - GitHub repository for version control\n   - Branch protection rules for main branch\n   - Pre-commit hooks for code quality\n\n2. **Supabase Configuration**:\n   - Database schema initialization\n   - RLS policy setup\n   - Storage bucket creation\n   - Edge function deployment\n\n3. **Vercel Deployment**:\n   - Connection to GitHub repository\n   - Build configuration:\n     ```\n     Framework Preset: Vite\n     Build Command: npm run build\n     Output Directory: dist\n     Install Command: npm install\n     ```\n   - Environment variable setup\n   - Domain configuration\n\n4. **Continuous Integration/Deployment**:\n   - Automatic builds on push to main branch\n   - Preview deployments for pull requests\n   - Rollback capability for failed deployments\n\n## 9. User Support\n\n### User Documentation\n\n#### Getting Started\n\n1. **Account Creation**:\n   - Navigate to the application URL\n   - Click \"Sign Up\" and enter your details\n   - Verify your email address\n\n2. **Document Upload**:\n   - Click \"Upload\" button on the dashboard\n   - Select files or drag and drop documents\n   - Wait for processing to complete\n\n3. **Searching Documents**:\n   - Use the search bar at the top of the interface\n   - Enter keywords related to your document\n   - Apply filters to narrow results\n\n4. **Document Classification**:\n   - Navigate to the Classification panel\n   - View automatic document categorization\n   - Manually adjust categories if needed\n\n#### Troubleshooting\n\nCommon issues and their solutions are documented in the [Troubleshooting](#troubleshooting) section.\n\n### Source Code and Live Application\n\n- **Source Code**: [GitHub Repository](https://github.com/yourusername/cloud-docu-analyzer-nexus)\n- **Live Application**: [Cloud Document Analytics Platform](https://cloud-docu-analyzer-nexus.vercel.app)\n\n## 10. Conclusion\n\nThe Cloud Document Analytics Platform demonstrates the power of modern cloud-native development for creating efficient document management and analysis systems. By leveraging serverless architecture and BaaS platforms like Supabase, the application achieves high performance, scalability, and security without requiring extensive backend infrastructure management.\n\n### Current Limitations\n\n- Document processing is limited to specific formats (PDF, DOC, DOCX)\n- Classification accuracy depends on document content quality\n- Web scraper may not handle all website structures efficiently\n- Storage limitations based on free-tier constraints\n\n### Future Enhancements\n\n1. **Enhanced AI Classification**:\n   - Integration with machine learning models for improved categorization\n   - Document similarity detection\n   - Content summarization\n\n2. **Advanced Analytics**:\n   - Sentiment analysis of document content\n   - Topic modeling and clustering\n   - Trend identification across document collections\n\n3. **Collaboration Features**:\n   - Document sharing capabilities\n   - Comment and annotation tools\n   - Version control for documents\n\n4. **Performance Optimizations**:\n   - Parallel processing for large document batches\n   - Caching strategies for frequently accessed content\n   - Progressive loading for large documents\n\n## References\n\n1. Supabase Documentation. (2023). *Authentication*. Retrieved from https://supabase.com/docs/guides/auth\n2. Vercel Documentation. (2023). *Deployment*. Retrieved from https://vercel.com/docs/concepts/deployments/overview\n3. React Documentation. (2023). *React Hooks*. Retrieved from https://reactjs.org/docs/hooks-intro.html\n4. Vite Documentation. (2023). *Features*. Retrieved from https://vitejs.dev/guide/features.html\n5. Mozilla Developer Network. (2023). *Using the Fetch API*. Retrieved from https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch\n6. PostgreSQL Documentation. (2023). *Full Text Search*. Retrieved from https://www.postgresql.org/docs/current/textsearch.html\n\n---\n\n**Made with ❤️ by Yousef M. Y. Al Sabbah**\n\n*Islamic University of Gaza - Faculty of Information Technology*\n\n---\n\n*Last updated: June 5, 2025*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyosef-alsabbah%2Fcloud-based-document-analytics-service-2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyosef-alsabbah%2Fcloud-based-document-analytics-service-2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyosef-alsabbah%2Fcloud-based-document-analytics-service-2/lists"}