{"id":22092953,"url":"https://github.com/dibyendubiswas1998/document-tagging","last_synced_at":"2026-04-11T00:16:56.193Z","repository":{"id":204657395,"uuid":"707502494","full_name":"dibyendubiswas1998/Document-Tagging","owner":"dibyendubiswas1998","description":"Creare CI-CD pipeline that helps to automate the process of assigning relevant tags or categories to large volumes of unstructured textual data.","archived":false,"fork":false,"pushed_at":"2023-12-26T14:12:02.000Z","size":40053,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-29T07:14:10.928Z","etag":null,"topics":["aws-ec2","docker","dvc","git","mlflow","python3","pytorch","s3-bucket"],"latest_commit_sha":null,"homepage":"http://65.0.74.77:8080/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dibyendubiswas1998.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-20T03:29:18.000Z","updated_at":"2023-12-05T14:17:18.000Z","dependencies_parsed_at":"2023-12-26T16:21:51.171Z","dependency_job_id":"929323c5-9732-40c9-adaa-b901f3891273","html_url":"https://github.com/dibyendubiswas1998/Document-Tagging","commit_stats":null,"previous_names":["dibyendubiswas1998/document-tagging"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dibyendubiswas1998%2FDocument-Tagging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dibyendubiswas1998%2FDocument-Tagging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dibyendubiswas1998%2FDocument-Tagging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dibyendubiswas1998%2FDocument-Tagging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dibyendubiswas1998","download_url":"https://codeload.github.com/dibyendubiswas1998/Document-Tagging/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245188429,"owners_count":20574767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-ec2","docker","dvc","git","mlflow","python3","pytorch","s3-bucket"],"created_at":"2024-12-01T03:11:57.967Z","updated_at":"2025-12-30T23:28:59.122Z","avatar_url":"https://github.com/dibyendubiswas1998.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Document Tagging\n\n## Problem Statement:\nDocument-Tagging system that leverages advanced natural language processing (NLP) techniques and transformer-based pre-trained models for efficient document categorization. This system aims to automatically generate the relevant tags or categories based on large volumes of unstructured textual data. Users can upload documents, which undergo preprocessing and are then classified into multiple relevant categories using NLP models. [Download Problem Statement PDF](./documents/problem%20statement.pdf)\n\nHere I build CI-CD pipeline for generating relevant tags based on given textual data.\u003cbr\u003e\n[DagsHub](https://dagshub.com/dibyendubiswas1998/Document-Tagging.mlflow)\u003cbr\u003e\n[WebApp](http://65.0.74.77:8080/)\n\u003cbr\u003e\u003cbr\u003e\n\n## Project Documentation:\n[High Level Design Dcocument (HLD)](./documents/High%20Level%20Design%20(HLD).pdf)\n\n[Architecture Document](./documents/Architecture.pdf)\n\n[Wireframe Document](./documents/Wireframe.pdf)\n\n[Low Level Design Document (LLD)](./documents/Low%20Level%20Design%20(LLD).pdf)\n\n[Detailed Project Report (DPR)](./documents/Detailed%20Project%20Report%20(DPR).docx.pdf)\n\u003cbr\u003e\u003cbr\u003e\n\n\n## Project Workflow:\n* **Step-01:** Load the raw or custom data from **AWS S3 Bucket**, provided by user. And save the data into particular directory.\n\n* **Step-02:** Preprocessed the raw data, like handle the missing values, duplicate values, text-preprocessing, vectorization, separate the X and Y, create tensor dataset and split them into train, test and validation sets.\n\n* **Step-03:** Create the model (default: bert-base-uncased), and train the model. After that save the pre-trained model \u0026 tokenizer in a particular directory.\n\n* **Step-04:** Evaluate the model baed on test datasets and save the inflormation on [DagsHub](https://dagshub.com/dibyendubiswas1998/Document-Tagging.mlflow) using mlflow. \n\n* **Step-05:** Create a Web Application for generating the tags and host the entier application on AWS.\n\u003cbr\u003e\u003cbr\u003e\n\n\n\n## Tech Stack:\n![Tech Stack](./documents/tech%20stack.png)\n\u003cbr\u003e\u003cbr\u003e\n\n\n## How to Run the Application:\n```bash\n    # For Windows OS:\n    docker pull dibyendubiswas1998/document_tagging\n    docker run -p 8080:8080 dibyendubiswas1998/document_tagging\n\n    # For Ubuntu OS:\n    sudo docker pull dibyendubiswas1998/document_tagging\n    sudo docker run -p 8080:8080 dibyendubiswas1998/document_tagging\n\n```\n\u003cbr\u003e\u003cbr\u003e\n\n\n## Web Interface:\n![Web Interface](./documents/web%20interface.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdibyendubiswas1998%2Fdocument-tagging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdibyendubiswas1998%2Fdocument-tagging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdibyendubiswas1998%2Fdocument-tagging/lists"}