{"id":50766152,"url":"https://github.com/kenken64/examtopics-data-labeler","last_synced_at":"2026-06-11T14:01:24.508Z","repository":{"id":302783388,"uuid":"1013589591","full_name":"kenken64/examtopics-data-labeler","owner":"kenken64","description":"📋 ExamTopics Data Labeler Project Summary 🎯 Project Overview The ExamTopics Data Labeler is a comprehensive web application designed for IT certification preparation and management. It combines PDF processing, AI-powered question generation, secure authentication, and sophisticated data management into a unified platform.","archived":false,"fork":false,"pushed_at":"2025-10-08T12:05:23.000Z","size":12096,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-08T13:24:02.690Z","etag":null,"topics":["data-labeling","nextjs","nodejs","ocr","openai","pytho","react"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kenken64.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-04T06:34:57.000Z","updated_at":"2025-10-08T11:18:08.000Z","dependencies_parsed_at":"2025-07-04T08:23:46.011Z","dependency_job_id":"9ce273b9-2e8d-440a-9163-04ff480e016d","html_url":"https://github.com/kenken64/examtopics-data-labeler","commit_stats":null,"previous_names":["kenken64/examtopics-data-labeler"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kenken64/examtopics-data-labeler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kenken64%2Fexamtopics-data-labeler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kenken64%2Fexamtopics-data-labeler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kenken64%2Fexamtopics-data-labeler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kenken64%2Fexamtopics-data-labeler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kenken64","download_url":"https://codeload.github.com/kenken64/examtopics-data-labeler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kenken64%2Fexamtopics-data-labeler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34201842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-labeling","nextjs","nodejs","ocr","openai","pytho","react"],"created_at":"2026-06-11T14:01:12.591Z","updated_at":"2026-06-11T14:01:24.492Z","avatar_url":"https://github.com/kenken64.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ExamBot Data Labeler Web Application\n\n## Screenshots\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"screen1.png\" alt=\"ExamBot Web Application Interface\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eWeb Application - PDF Data Labeler and Management Interface\u003c/em\u003e\u003c/p\u003e\n  \n  \u003cimg src=\"screen2.png\" alt=\"ExamBot Telegram Bot Interface\" width=\"400\"\u003e\n  \u003cp\u003e\u003cem\u003eTelegram Bot - Interactive Quiz Session\u003c/em\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n## Overview\n\nThis project has evolved into a comprehensive AWS certification preparation and management ecosystem. It features a web-based PDF data labeler, robust user authentication, various management interfaces, and an interactive Telegram bot for quiz practice.\n\n## Features\n\n### Core Functionality\n\n- **PDF Data Labeler**: Upload PDF documents, view them page by page, convert content to Markdown, and label specific sections (questions, answers, explanations) for quiz creation.\n- **Analytics Dashboard**: Comprehensive dashboard with interactive charts and visual analytics showing certificate statistics, quiz performance trends over the last 30 days, user engagement metrics, and access code distribution with real-time data from completed Telegram bot quiz sessions.\n- **User Authentication**: Secure user registration and login using Passkeys (WebAuthn) with JWT for session management.\n- **Role-Based Access**: Protected routes ensure only authenticated users can access certain parts of the application.\n- **Telegram Quiz Bot**: Interactive bot for AWS certification practice with real-time quiz sessions, progress tracking, and detailed explanations.\n\n### Management Interfaces\n\n- **Certificate Management**: CRUD operations for AWS certification types (e.g., SAA-C03, DVA-C02) with support for logo URLs and PDF certificate uploads via Cloudinary integration.\n- **Payee Management**: Manage customer payment records, including credit card details (masked), payment status, and associated certificates.\n- **Access Code Management**: Generate unique access codes for paid customers, linking them to specific certificates.\n- **Question Assignment Management**: Assign and reorder specific questions from quizzes to generated access codes, allowing for customized quiz experiences per customer. Questions can be enabled/disabled for an access code.\n- **Saved Questions Viewer**: Browse questions by certificate code or search by access code (original or generated) to view question details, options, and explanations.\n\n### Telegram Bot Features\n\n- **Interactive Quiz Sessions**: Real-time multiple-choice questions with A, B, C, D options\n- **Certificate Selection**: Choose from available AWS certification types\n- **Access Code Integration**: Validate generated access codes to access personalized quizzes\n- **Instant Feedback**: Immediate results with detailed explanations for wrong answers\n- **Progress Tracking**: Complete quiz statistics and performance monitoring\n- **Bookmark System**: Save questions for later review with `/bookmark` command\n- **Revision Mode**: Review previously answered questions to reinforce learning\n- **Help System**: Comprehensive `/help` command with usage instructions\n\n## Technologies Used\n\n### Backend (Python - PDF Conversion Service)\n\n- **Flask**: A micro web framework for Python.\n- **docling**: A library for document processing and conversion.\n- **pypdf**: A pure-Python PDF library.\n- **Flask-Cors**: For handling Cross Origin Resource Sharing (CORS).\n\n### Frontend (Next.js - Main Application)\n\n- **Next.js**: A React framework for building production-ready web applications.\n- **React**: A JavaScript library for building user interfaces.\n- **MongoDB / Mongoose**: For database interactions and schema modeling.\n- **jsonwebtoken**: For creating and verifying JSON Web Tokens.\n- **@simplewebauthn/server \u0026 @simplewebauthn/browser**: For implementing FIDO2 (Passkey) authentication.\n- **shadcn/ui**: A collection of re-usable components built using Radix UI and Tailwind CSS for a modern UI.\n- **Tailwind CSS**: A utility-first CSS framework.\n- **pdfjs-dist**: A PDF rendering library from Mozilla.\n- **Chart.js \u0026 react-chartjs-2**: Modern charting library for creating interactive data visualizations and analytics dashboard.\n\n### Telegram Bot (Node.js - Quiz Bot Service)\n\n- **Grammy**: Modern Telegram bot framework for Node.js.\n- **MongoDB**: Database integration for quiz data and progress tracking.\n- **dotenv**: Environment configuration management.\n\n## Setup and Installation\n\nFollow these steps to set up and run the project locally.\n\n### Prerequisites\n\n- Node.js (LTS version recommended)\n- pnpm (Package manager for Node.js - `npm install -g pnpm`)\n- Python 3.8+ (or compatible version)\n- pip (Python package installer)\n- MongoDB instance (local or cloud-hosted)\n\n### 1. Backend Setup (PDF Conversion Service)\n\n1.  Navigate to the `backend` directory:\n    ```bash\n    cd backend\n    ```\n\n2.  Create a Python virtual environment:\n    ```bash\n    python3 -m venv venv\n    ```\n\n3.  Activate the virtual environment:\n    - On macOS/Linux:\n      ```bash\n      source venv/bin/activate\n      ```\n    - On Windows:\n      ```bash\n      .\\venv\\Scripts\\activate\n      ```\n\n4.  Install the required Python packages:\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n5.  Run the Flask application:\n    ```bash\n    python app.py\n    ```\n    The backend server will start on `http://0.0.0.0:5000`.\n\n### 2. Frontend Setup (Next.js Application)\n\n1.  Navigate to the `frontend` directory:\n    ```bash\n    cd frontend\n    ```\n\n2.  **Environment Variables**: Create a `.env.local` file in the `frontend` directory and add the following:\n\n    ```\n    MONGODB_URI=your_mongodb_connection_string\n    RP_ID=localhost # Or your domain (e.g., your-app.com)\n    RP_NAME=\"AWS Cert Web\" # Your application name\n    ORIGIN=http://localhost:3000 # Your application's origin\n    JWT_SECRET=your_super_secret_jwt_key # Generate a strong, random key\n    NEXT_PUBLIC_PDF_CONVERSION_API_URL=http://localhost:5000 # URL of your Python backend\n    ```\n    *Replace placeholders with your actual values.*\n\n3.  **Important: Copy PDF.js Worker File**\n    Before installing dependencies, ensure the `pdf.worker.min.js` file is in your `public` directory. If it's not there, copy it manually:\n    ```bash\n    cp node_modules/pdfjs-dist/build/pdf.worker.min.js public/pdf.worker.min.js\n    ```\n    (Note: If `cp` fails, you might need to adjust the source path based on your `node_modules` structure or manually locate and copy the file.)\n\n4.  Install Node.js dependencies:\n    ```bash\n    pnpm install\n    ```\n\n5.  **Initialize shadcn/ui**: Run the shadcn/ui initialization command. Follow the prompts, selecting `Default` style and `Slate` as the base color.\n    ```bash\n    npx shadcn@latest init\n    ```\n\n6.  **Add shadcn/ui Components**: Add the necessary UI components.\n    ```bash\n    npx shadcn@latest add button input label avatar badge card select separator sheet\n    ```\n\n7.  **Seed Database (Optional but Recommended)**: Populate your MongoDB with sample data for certificates, payees, and question assignments.\n    ```bash\n    node seed-certificates.js\n    node seed-payees.js\n    node seed-access-code-questions.js\n    ```\n\n8.  Run the Next.js development server:\n    ```bash\n    pnpm dev\n    ```\n    The frontend application will be accessible at `http://localhost:3000`.\n\n### 3. Telegram Bot Setup (Quiz Bot Service)\n\n1.  Navigate to the `telegram-bot` directory:\n    ```bash\n    cd telegram-bot\n    ```\n\n2.  **Environment Variables**: Create a `.env` file in the `telegram-bot` directory and add the following:\n    ```\n    BOT_TOKEN=your_telegram_bot_token_from_botfather\n    MONGODB_URI=your_mongodb_connection_string\n    ```\n    *Get your bot token from [@BotFather](https://t.me/botfather) on Telegram.*\n\n3.  Install Node.js dependencies:\n    ```bash\n    npm install\n    ```\n\n4.  Run the Telegram bot:\n    ```bash\n    node bot.js\n    ```\n    The bot will start and be available on Telegram.\n\n## Usage\n\n### Web Application\n\n1.  Ensure both the backend (Flask) and frontend (Next.js) servers are running.\n2.  Open your web browser and go to `http://localhost:3000`.\n3.  **Login/Register**: You will be greeted by the login page. You can register a new user using a passkey.\n4.  **Navigation**: After successful login, you will be redirected to the analytics dashboard. Use the sliding menu (accessible from the top-left corner) to navigate to different sections:\n    -   **Home**: Analytics dashboard with interactive charts and key metrics.\n    -   **Exam Q Labeler**: The PDF Data Labeler interface for processing documents.\n    -   **Certificates**: Manage certification types.\n    -   **Payees**: Manage customer payment records.\n    -   **Access Codes**: Generate and view access codes for paid customers.\n    -   **Manage Questions**: Assign and reorder questions for specific generated access codes.\n    -   **Saved Questions**: Search and view questions by access code or certificate.\n\n### Telegram Bot\n\n1.  Ensure the bot is running (`node bot.js` in the `telegram-bot` directory).\n2.  Find your bot on Telegram using the username you set with BotFather.\n3.  **Start a Quiz Session**: Send `/start` to begin\n4.  **Available Commands**:\n    -   `/start` - Start new quiz session\n    -   `/help` - Show comprehensive help guide\n    -   `/bookmark \u003cnumber\u003e` - Save question for later review\n    -   `/bookmarks` - View saved bookmarks\n    -   `/revision` - Review previously answered questions\n\n## Important Notes\n\n-   **Passkey Challenge Storage**: The current implementation uses a global variable for storing WebAuthn challenges (`currentChallenge`). **This is for demonstration purposes only and is highly insecure.** In a production environment, you must implement a secure server-side session management system (e.g., using a database, Redis, or a dedicated session library) to store and retrieve challenges.\n-   **PDF Conversion API**: The `NEXT_PUBLIC_PDF_CONVERSION_API_URL` environment variable in the frontend points to your Python Flask backend. Ensure this URL is correct.\n-   **Data Transformation**: The application includes utilities (`frontend/app/utils/questionTransform.ts`) to handle transformations between different question data formats from the database to the frontend.\n\n## Documentation\n\nThis project includes comprehensive technical documentation organized in the `frontend/specs/` folder. Below is a complete reference to all available documentation:\n\n| Document | Description | Category |\n|----------|-------------|----------|\n| [ACCESS_CODE_QUESTIONS_LINKING.md](./frontend/specs/ACCESS_CODE_QUESTIONS_LINKING.md) | **Complete access code questions linking system documentation** | **Features** |\n| [ACCESS_CODE_QUESTIONS_README.md](./frontend/specs/ACCESS_CODE_QUESTIONS_README.md) | Documentation for access code questions functionality | Features |\n| [HELP_COMMAND_IMPLEMENTATION.md](./telegram-bot/HELP_COMMAND_IMPLEMENTATION.md) | Telegram bot help command implementation guide | Telegram Bot |\n| [OCR_FEATURE_README.md](./frontend/specs/OCR_FEATURE_README.md) | **PDF-to-Markdown OCR conversion feature using OpenAI's vision API** | **Features** |\n| [QUESTION_STRUCTURE_ANALYSIS.md](./telegram-bot/QUESTION_STRUCTURE_ANALYSIS.md) | Analysis of question data structure and bot compatibility issues | Telegram Bot |\n| [TELEGRAM_BOT_README.md](./telegram-bot/README.md) | **Telegram bot setup, features, and implementation guide** | **Telegram Bot** |\n| [AI_CACHING_COMPLETE.md](./frontend/specs/AI_CACHING_COMPLETE.md) | AI response caching implementation completion guide | AI Implementation |\n| [AI_CACHING_IMPLEMENTATION_SUMMARY.md](./frontend/specs/AI_CACHING_IMPLEMENTATION_SUMMARY.md) | Summary of AI caching implementation details | AI Implementation |\n| [AI_EXPLANATION_SETUP.md](./frontend/specs/AI_EXPLANATION_SETUP.md) | Setup guide for AI explanation features | AI Implementation |\n| [AI_IMPLEMENTATION_COMPLETE.md](./frontend/specs/AI_IMPLEMENTATION_COMPLETE.md) | Complete AI second opinion feature implementation guide | AI Implementation |\n| [AI_SETUP_COMPLETE.md](./frontend/specs/AI_SETUP_COMPLETE.md) | AI setup completion documentation | AI Implementation |\n| [APPLICATION_FLOW_DIAGRAM.md](./frontend/specs/APPLICATION_FLOW_DIAGRAM.md) | **Comprehensive application architecture and flow diagram** | **Architecture** |\n| [DATA_TRANSFORMATION_IMPLEMENTATION.md](./frontend/specs/DATA_TRANSFORMATION_IMPLEMENTATION.md) | Data transformation implementation details | Backend |\n| [IMPLEMENTATION_COMPLETE.md](./frontend/specs/IMPLEMENTATION_COMPLETE.md) | Overall project implementation completion status | Project Status |\n| [IMPLEMENTATION_SUCCESS.md](./frontend/specs/IMPLEMENTATION_SUCCESS.md) | Implementation success documentation | Project Status |\n| [JWT_AUTHENTICATION_COMPLETE.md](./frontend/specs/JWT_AUTHENTICATION_COMPLETE.md) | JWT authentication implementation guide | Authentication |\n| [JWT_AUTHENTICATION_FIX_SUMMARY.md](./frontend/specs/JWT_AUTHENTICATION_FIX_SUMMARY.md) | JWT authentication fixes and solutions | Authentication |\n| [LOCAL_DEVELOPMENT_FIX.md](./frontend/specs/LOCAL_DEVELOPMENT_FIX.md) | Local development environment fixes | Development |\n| [LOGIN_DEBUG_ANALYSIS.md](./frontend/specs/LOGIN_DEBUG_ANALYSIS.md) | Login functionality debug analysis | Authentication |\n| [LOGIN_NAVIGATION_FIX.md](./frontend/specs/LOGIN_NAVIGATION_FIX.md) | Login navigation fixes and improvements | Authentication |\n| [MARKDOWN_VISUAL_GUIDE.md](./frontend/specs/MARKDOWN_VISUAL_GUIDE.md) | Visual guide for markdown rendering | UI/UX |\n| [NAVIGATION_FIX_COMPLETE.md](./frontend/specs/NAVIGATION_FIX_COMPLETE.md) | Navigation system fixes completion | UI/UX |\n| [REACT_MARKDOWN_COMPLETE.md](./frontend/specs/REACT_MARKDOWN_COMPLETE.md) | React markdown implementation completion | UI/UX |\n| [REACT_MARKDOWN_IMPLEMENTATION.md](./frontend/specs/REACT_MARKDOWN_IMPLEMENTATION.md) | React markdown implementation details | UI/UX |\n| [SLIDING_MENU_LOGOUT_FIX.md](./frontend/specs/SLIDING_MENU_LOGOUT_FIX.md) | Sliding menu logout functionality fixes | UI/UX |\n| [TITLE_CONFIGURATION_FIX.md](./frontend/specs/TITLE_CONFIGURATION_FIX.md) | Title configuration fixes and updates | Configuration |\n| [VERCEL_AI_COMPLETE_SETUP.md](./frontend/specs/VERCEL_AI_COMPLETE_SETUP.md) | Complete Vercel AI setup guide | AI Implementation |\n| [VERCEL_AI_SETUP.md](./frontend/specs/VERCEL_AI_SETUP.md) | Vercel AI initial setup documentation | AI Implementation |\n| [VERCEL_API_KEY_SETUP.md](./frontend/specs/VERCEL_API_KEY_SETUP.md) | Vercel API key setup instructions | Configuration |\n\n### Documentation Categories\n\n- **Architecture**: Complete system architecture, flow diagrams, and technical specifications\n- **AI Implementation**: Documentation related to AI features, caching, and Vercel AI integration\n- **Authentication**: JWT authentication, login functionality, and security implementations\n- **UI/UX**: User interface components, navigation, and markdown rendering\n- **Configuration**: Environment setup, API keys, and configuration management\n- **Features**: Specific application features and functionality\n- **Backend**: Data transformation and backend implementation details\n- **Development**: Local development setup and debugging guides\n- **Project Status**: Overall implementation status and completion documentation\n- **Telegram Bot**: Telegram bot implementation, commands, and data structure analysis\n\n### Quick Reference\n\nFor developers new to the project, we recommend starting with these key documents:\n\n1. **[APPLICATION_FLOW_DIAGRAM.md](./frontend/specs/APPLICATION_FLOW_DIAGRAM.md)** - Complete system architecture overview\n2. **[TELEGRAM_BOT_README.md](./telegram-bot/README.md)** - Telegram bot setup and features guide\n3. **[JWT_AUTHENTICATION_FIX_SUMMARY.md](./frontend/specs/JWT_AUTHENTICATION_FIX_SUMMARY.md)** - Authentication implementation details\n4. **[AI_IMPLEMENTATION_COMPLETE.md](./frontend/specs/AI_IMPLEMENTATION_COMPLETE.md)** - AI features and setup guide\n5. **[OCR_FEATURE_README.md](./frontend/specs/OCR_FEATURE_README.md)** - PDF-to-Markdown OCR conversion using OpenAI\n6. **[LOCAL_DEVELOPMENT_FIX.md](./frontend/specs/LOCAL_DEVELOPMENT_FIX.md)** - Development environment troubleshooting\n\n## Contributing\n\nFeel free to fork this repository, open issues, or submit pull requests.\n\n## License\n\nThis project is open-sourced under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkenken64%2Fexamtopics-data-labeler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkenken64%2Fexamtopics-data-labeler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkenken64%2Fexamtopics-data-labeler/lists"}