{"id":25525237,"url":"https://github.com/lakshmi-bashyam/analysis_of_transfer_learning_methods","last_synced_at":"2026-01-24T06:40:19.681Z","repository":{"id":211131270,"uuid":"323131234","full_name":"Lakshmi-bashyam/Analysis_of_transfer_learning_methods","owner":"Lakshmi-bashyam","description":"Report on study compares pre-training objectives, architectures, unlabeled data sets, transfer approaches, and other factors on of language understanding tasks. Analysing the inner workings of T5 and related transformer model architecture on multiple standard NLP datasets.","archived":false,"fork":false,"pushed_at":"2020-12-20T18:16:25.000Z","size":8451,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-21T19:53:30.348Z","etag":null,"topics":["bert","nlp","t5","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lakshmi-bashyam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-12-20T17:50:04.000Z","updated_at":"2020-12-20T18:27:35.000Z","dependencies_parsed_at":"2023-12-06T17:45:58.383Z","dependency_job_id":null,"html_url":"https://github.com/Lakshmi-bashyam/Analysis_of_transfer_learning_methods","commit_stats":null,"previous_names":["lakshmi-bashyam/analysis_of_transfer_learning_methods"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Lakshmi-bashyam/Analysis_of_transfer_learning_methods","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakshmi-bashyam%2FAnalysis_of_transfer_learning_methods","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakshmi-bashyam%2FAnalysis_of_transfer_learning_methods/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakshmi-bashyam%2FAnalysis_of_transfer_learning_methods/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakshmi-bashyam%2FAnalysis_of_transfer_learning_methods/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lakshmi-bashyam","download_url":"https://codeload.github.com/Lakshmi-bashyam/Analysis_of_transfer_learning_methods/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakshmi-bashyam%2FAnalysis_of_transfer_learning_methods/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28716845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T05:53:42.649Z","status":"ssl_error","status_checked_at":"2026-01-24T05:53:41.698Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","nlp","t5","transfer-learning"],"created_at":"2025-02-19T20:18:46.057Z","updated_at":"2026-01-24T06:40:19.668Z","avatar_url":"https://github.com/Lakshmi-bashyam.png","language":"TeX","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Exploring the limits of transfer learning with text-to-text transformer\nReport for Machine learning for NLP seminar offered by Language technology department @ Uds\n\n## Abstract\nTransfer learning has become widely popular learning technique in many applications specifically in a natural language processing setting. There can be many ways to transfer the knowledge from one setting to another. In this report, the popular technique of building pre-trained transfer models and its performance of\ndownstream task is discussed in detail.\n\nThis type of transfer learning involves two steps, First is to train a machine learning model on a data rich task. Here, data rich task refers to the task for which\ndata is easily available such as language modelling where the data can be simply scraped from the web pages which are abundantly available. This model called\nthe pretrained model captures the low-level details of the task such as semantics or grammar of the language. Second step involves reusing this general-purpose\nknowledge gained by fine tuning the pretrained model on specific NLP tasks such as named entity recognition or sentiment classification task.\n\nThe paper “Exploring the limits of transfer learning with text-to-text transformer” extends this idea into a text-in, text out model that can be used across many NLP tasks. It also presents a detailed comparative study of different transfer learning techniques and incorporates the best performing methods to present the final model T5 along with the C4 dataset that was used to train it. Together it achieves state of the art performance at the time of its release on many NLP tasks.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flakshmi-bashyam%2Fanalysis_of_transfer_learning_methods","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flakshmi-bashyam%2Fanalysis_of_transfer_learning_methods","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flakshmi-bashyam%2Fanalysis_of_transfer_learning_methods/lists"}