{"id":19696430,"url":"https://github.com/jayyip/m3tl","last_synced_at":"2026-04-10T05:45:01.242Z","repository":{"id":38373967,"uuid":"159620189","full_name":"JayYip/m3tl","owner":"JayYip","description":"BERT for Multitask Learning","archived":false,"fork":false,"pushed_at":"2023-04-12T06:03:03.000Z","size":30553,"stargazers_count":547,"open_issues_count":27,"forks_count":125,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-04-03T22:08:03.346Z","etag":null,"topics":["bert","cws","encoder-decoder","multi-task-learning","multitask-learning","named-entity-recognition","ner","nlp","part-of-speech","pretrained-models","text-classification","transformer","word-segmentation"],"latest_commit_sha":null,"homepage":"https://jayyip.github.io/m3tl/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JayYip.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-11-29T06:43:15.000Z","updated_at":"2025-04-01T06:41:39.000Z","dependencies_parsed_at":"2023-02-16T14:15:59.175Z","dependency_job_id":"5117ccc2-4411-4f6e-8004-25e6e37faef6","html_url":"https://github.com/JayYip/m3tl","commit_stats":{"total_commits":487,"total_committers":9,"mean_commits":"54.111111111111114","dds":0.03696098562628336,"last_synced_commit":"63c44c37d4b2d695d5074dd76e28a0273d00064f"},"previous_names":["jayyip/bert-multitask-learning","jayyip/bert-multiple-gpu"],"tags_count":55,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JayYip%2Fm3tl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JayYip%2Fm3tl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JayYip%2Fm3tl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JayYip%2Fm3tl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JayYip","download_url":"https://codeload.github.com/JayYip/m3tl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248625443,"owners_count":21135512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","cws","encoder-decoder","multi-task-learning","multitask-learning","named-entity-recognition","ner","nlp","part-of-speech","pretrained-models","text-classification","transformer","word-segmentation"],"created_at":"2024-11-11T19:35:04.136Z","updated_at":"2025-12-13T19:32:11.113Z","avatar_url":"https://github.com/JayYip.png","language":"Jupyter Notebook","readme":"# M3TL\n\n\n\n**M**ulti-**M**odal **M**ulti-**T**ask **L**earning\n\n## Install\n\n```\npip install m3tl\n```\n\n## What is it\n\nThis is a project that uses transformers(based on huggingface transformers) as base model to do **multi-modal multi-task learning**.   \n\n## Why do I need this\n\nMulti-task learning(MTL) is gaining more and more attention, especially in deep learning era. It is widely used in NLP, CV, recommendation, etc. However, MTL usually involves complicated data preprocessing, task managing and task interaction. Other open-source projects, like TencentNLP and PyText, supports MTL but in a naive way and it's not straightforward to implement complicated MTL algorithm. In this project, we try to make writing MTL model as easy as single task learning model and further extend MTL to multi-modal multi-task learning. To do so, we expose following MTL related programable module to user:\n\n- problem sampling strategy\n- loss combination strategy\n- gradient surgery\n- model after base model(transformers)\n\nApart from programable modules, we also provide various built-in SOTA MTL algorithms.\n\nIn a word, you can use this project to:\n\n- implement complicated MTL algorithm\n- do SOTA MTL without diving into details\n- do multi-modal learning\n\nAnd since we use transformers as base model, you get all the benefits that you can get from transformers!\n\n## What type of problems are supported?\n\n```\nparams = Params()\nfor problem_type in params.list_available_problem_types():\n    print('`{problem_type}`: {desc}'.format(\n        desc=params.problem_type_desc[problem_type], problem_type=problem_type))\n\n```\n\n    `cls`: Classification\n    `multi_cls`: Multi-Label Classification\n    `seq_tag`: Sequence Labeling\n    `masklm`: Masked Language Model\n    `pretrain`: NSP+MLM(Deprecated)\n    `regression`: Regression\n    `vector_fit`: Vector Fitting\n    `premask_mlm`: Pre-masked Masked Language Model\n    `contrastive_learning`: Contrastive Learning\n\n\n\n## Get Started\n\nPlease see tutorials.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjayyip%2Fm3tl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjayyip%2Fm3tl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjayyip%2Fm3tl/lists"}