{"id":19056622,"url":"https://github.com/aishwaryahastak/mtl_distillbert","last_synced_at":"2026-05-11T17:30:19.643Z","repository":{"id":243935311,"uuid":"812040344","full_name":"AishwaryaHastak/MTL_DistillBERT","owner":"AishwaryaHastak","description":"Mutli-Task Transformer Based Model ","archived":false,"fork":false,"pushed_at":"2024-09-09T18:30:30.000Z","size":6241,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-02T11:43:26.246Z","etag":null,"topics":["classification","fine-tuning","large-language-models","multitask-learning","regression","sentence-transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AishwaryaHastak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-07T20:38:40.000Z","updated_at":"2024-11-22T07:22:38.000Z","dependencies_parsed_at":"2024-09-09T22:49:48.671Z","dependency_job_id":"99696b08-17fd-4a6b-92e2-73fccfe4936f","html_url":"https://github.com/AishwaryaHastak/MTL_DistillBERT","commit_stats":null,"previous_names":["aishwaryahastak/sentencetransformer","aishwaryahastak/mtl_distillbert"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AishwaryaHastak%2FMTL_DistillBERT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AishwaryaHastak%2FMTL_DistillBERT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AishwaryaHastak%2FMTL_DistillBERT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AishwaryaHastak%2FMTL_DistillBERT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AishwaryaHastak","download_url":"https://codeload.github.com/AishwaryaHastak/MTL_DistillBERT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240112648,"owners_count":19749665,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","fine-tuning","large-language-models","multitask-learning","regression","sentence-transformers"],"created_at":"2024-11-08T23:50:39.161Z","updated_at":"2026-05-11T17:30:19.613Z","avatar_url":"https://github.com/AishwaryaHastak.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌟 Multi-Task Learning on Amazon Clothes Review Dataset using DistilBERT\n\n## Overview\n\nThis project focuses on building a multi-task learning model to analyze women's clothing reviews using the DistilBERT transformer. The goal is to predict both the product rating as a classification task and the user's age as a regression task based on textual reviews. This project leverages the Hugging Face library, PyTorch, and Python for model development and data processing, with Flask for deployment.\n\n## Dataset\n\nThe [Women's Clothing Review Analysis](https://www.kaggle.com/datasets/nicapotato/womens-ecommerce-clothing-reviews) project aims to develop a sophisticated model capable of extracting meaningful insights from clothing reviews. By employing a multi-task learning approach, we address two related tasks simultaneously:\n1. **Classification**: Predicting the product rating from the review text. 🏷️\n2. **Regression**: Predicting the user's age from the review text. 🎂\n\n## Technical Tools Used\n\n- **Data Processing**: Pandas for data cleaning and manipulation. 🧹\n- **Modeling Framework**: Hugging Face Transformers and PyTorch are used to develop and train the multi-task learning model. 🤖\n- **Model Architecture**: DistilBERT transformer with additional classification and regression heads. 🔄\n- **Deployment**: Flask for creating a web application to deploy the model and provide an interface for predictions. 🌐\n\n## Multi-Task Learning (MTL)\n\nMulti-task learning (MTL) is a machine learning approach where a single model is trained on multiple tasks simultaneously. This method leverages shared representations and features to improve performance across tasks, particularly when they are related. \n\n### Key Concepts\n\n- **Shared Representation**: DistilBERT generates embeddings from the text that serve as input for both regression and classification tasks. 🔍\n- **Regression Head**: A linear layer predicting continuous values, such as user age. 📉\n- **Classification Head**: A linear layer predicting discrete labels, such as product ratings. ⭐\n- **Weighted Loss**: The loss over all tasks is combined by assigning tunable weights (importance) to each task. ⚖️\n\n### Benefits\n\n- **Improved Generalization**: Sharing representations between tasks helps the model generalize better and avoid overfitting. 🌟\n- **Efficient Use of Data**: Related tasks can use shared representations to leverage data more effectively, especially when labeled data is scarce. 📊\n- **Better Performance**: Multi-task learning often enhances performance on individual tasks compared to training separate models for each. 🚀\n\n\n### Installation\n\nInstall the required dependencies by running the following command:\n\n```bash\npip install -r requirements.txt\n```\n\n\n### Running the Application\n\nTo run the application, execute the `main.py` file using Python. This command can be executed in your terminal or command prompt:\n\n```bash\npython main.py\n```\n\nRunning this command will start the application and execute the main script, allowing you to interact with the functionality provided by the program.\n\n\nAlternatively, if you prefer to run the application using the Flask web framework, you can use the following command:\n```bash\npython app.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faishwaryahastak%2Fmtl_distillbert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faishwaryahastak%2Fmtl_distillbert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faishwaryahastak%2Fmtl_distillbert/lists"}