{"id":13738541,"url":"https://github.com/chrisby/torchMTL","last_synced_at":"2025-05-08T16:34:33.472Z","repository":{"id":64859606,"uuid":"305702580","full_name":"chrisby/torchMTL","owner":"chrisby","description":"A lightweight module for Multi-Task Learning in pytorch.","archived":false,"fork":false,"pushed_at":"2023-08-07T03:33:27.000Z","size":297,"stargazers_count":161,"open_issues_count":0,"forks_count":19,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-07T18:14:09.895Z","etag":null,"topics":["deep-learning","framework","machine-learning","mtl","multi-task-learning","python3","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrisby.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-20T12:36:27.000Z","updated_at":"2025-04-05T18:04:16.000Z","dependencies_parsed_at":"2024-10-31T09:02:07.315Z","dependency_job_id":"5993301f-91d7-46a4-a5d0-033b1175ec2e","html_url":"https://github.com/chrisby/torchMTL","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisby%2FtorchMTL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisby%2FtorchMTL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisby%2FtorchMTL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisby%2FtorchMTL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrisby","download_url":"https://codeload.github.com/chrisby/torchMTL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253105659,"owners_count":21855071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","framework","machine-learning","mtl","multi-task-learning","python3","pytorch"],"created_at":"2024-08-03T03:02:26.059Z","updated_at":"2025-05-08T16:34:33.163Z","avatar_url":"https://github.com/chrisby.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"![torchMTL Logo](https://github.com/chrisby/torchMTL/blob/main/images/torchmtl_logo.png \"torchMTL Logo\")    \nA lightweight module for Multi-Task Learning in pytorch.\n\n`torchmtl` tries to help you composing modular multi-task architectures with minimal effort. All you need is a list of dictionaries in which you define your layers and how they build on each other. From this, `torchmtl` constructs a meta-computation graph which is executed in each forward pass of the created `MTLModel`. To combine outputs from multiple layers, simple [wrapper functions](https://github.com/chrisby/torchMTL/blob/main/torchmtl/wrapping_layers.py) are provided.\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4362515.svg)](https://doi.org/10.5281/zenodo.4362515)\n\n\n### Installation\n`torchmtl` can be installed via `pip`:\n```\npip install torchmtl\n```\n\n### Quickstart (or find examples [here](https://github.com/chrisby/torchMTL/tree/main/examples))\nAssume you want to train a network on three tasks as shown below.  \n![example](https://github.com/chrisby/torchMTL/blob/main/images/example.png \"example\")  \n\nTo construct such an architecture with `torchmtl`, you simply have to define the following list\n\n```python\ntasks = [\n        {\n            'name': \"Embed1\",\n            'layers': Sequential(*[Linear(16, 32), Linear(32, 8)]),\n            # No anchor_layer means this layer receives input directly\n        },    \n        {\n            'name': \"Embed2\",\n            'layers': Sequential(*[Linear(16, 32), Linear(32, 8)]),\n            # No anchor_layer means this layer receives input directly\n        },\n        {\n            'name': \"CatTask\",\n            'layers': Concat(dim=1),\n            'loss_weight': 1.0,\n            'anchor_layer': ['Embed1', 'Embed2']\n        },\n        {\n            'name': \"Task1\",\n            'layers': Sequential(*[Linear(8, 32), Linear(32, 1)]),\n            'loss': MSELoss(),\n            'loss_weight': 1.0,\n            'anchor_layer': 'Embed1'            \n        },\n        {\n            'name': \"Task2\",\n            'layers': Sequential(*[Linear(8, 64), Linear(64, 1)]),\n            'loss': BCEWithLogitsLoss(),\n            'loss_weight': 1.0,\n            'anchor_layer': 'Embed2'            \n        }, \n        {\n            'name': \"FNN\",\n            'layers': Sequential(*[Linear(16, 32), Linear(32, 32)]),\n            'anchor_layer': 'CatTask'\n        },\n        {\n            'name': \"Task3\",\n            'layers': Sequential(*[Linear(32, 16), Linear(16, 1)]),\n            'anchor_layer': 'FNN',\n            'loss': MSELoss(),\n            'loss_weight': 'auto',\n            'loss_init_val': 1.0\n        }\n    ]\n```\n\nYou can build your final model with the following lines in which you specify from which layers you would like to receive the output.\n```python\nfrom torchmtl import MTLModel\nmodel = MTLModel(tasks, output_tasks=['Task1', 'Task2', 'Task3'])\n```\n\nThis constructs a **meta-computation graph** which is executed in each forward pass of your `model`. You can verify whether the graph was properly built by plotting it using the `networkx` library:\n```python\nimport networkx as nx\npos = nx.planar_layout(model.g)\nnx.draw(model.g, pos, font_size=14, node_color=\"y\", node_size=450, with_labels=True)\n```\n![graph example](https://github.com/chrisby/torchMTL/blob/main/images/torchmtl_graph.png \"graph example\")  \n\n#### The training loop\nYou can now enter the typical `pytorch` training loop and you will have access to everything you need to update your model:\n```python\nfor X, y in data_loader:\n    optimizer.zero_grad()\n\n    # Our model will return a list of predictions (from the layers specified in `output_tasks`),\n    # loss functions, and regularization parameters (as defined in the tasks variable)\n    y_hat, l_funcs, l_weights = model(X)\n    \n    loss = 0\n    # We can now iterate over the tasks and accumulate the losses\n    for i in range(len(y_hat)):\n        loss += l_weights[i] * l_funcs[i](y_hat[i], y[i])\n    \n    loss.backward()\n    optimizer.step()\n\n```\n\n### Details on the layer definition\nThere are 6 keys that can be specified (`name` and `layers` **must** always be present):  \n\n**`layers`**  \nBasically takes any `nn.Module` that you can think of. You can plug in a transformer or just a handful of fully connected layers.  \n\n**`anchor_layer`**  \nThis defines from which other layer this layer receives its input. Take care that the respective dimensions match.  \n\n**`loss`**  \nThe loss function you want to compute on the output of this layer (`l_funcs`). Can be set to `None` or omitted altogether when only access to the layer's output is needed.   \n\n**`loss_weight`**  \nThe scalar with which you want to regularize the respective loss (`l_weights`). If set to `'auto'`, a `nn.Parameter` is returned which will be updated through backpropagation. Can be set to `None` or omitted altogether when only access to the layer's output is needed.  \n\n**`loss_init_val`**  \nOnly needed if `loss_weight: 'auto'`. The initialization value of the `loss_weight` parameter.\n\n### Wrapping functions\nNodes of the **meta-computation graph** don't have to be pytorch Modules. They can be *concatenation* functions or indexing functions that return a certain element of the input. If your `X` consists of two types of input data `X=[X_1, X_2]`, you can use the `SimpleSelect` layer to select the `X_1` by setting  \n```python\nfrom torchmtl.wrapping_layers import SimpleSelect\n{ ...,\n  'layers' = SimpleSelect(selection_axis=0),\n  ...\n}\n```\nIt should be trivial to write your own wrapping layers, but I try to provide useful ones with this library. If you have any layers in mind but no time to implement them, feel free to [open an issue](https://github.com/chrisby/torchMTL/issues).\n\n#### Cite\n```\n@misc{torchMTL: A lightweight module for Multi-Task Learning in pytorch,\n  author = {Bock, Christian},\n  doi = {10.5281/zenodo.4362515},\n  url = {https://github.com/chrisby/torchMTL},\n  year = {2020}\n}\n```\n\n#### Credits\nLogo credits and license: I reused and remixed (moved the dot and rotated the resulting logo a couple times) the pytorch logo from [here](https://github.com/pytorch/pytorch/blob/master/docs/source/_static/img/pytorch-logo-dark.png) (accessed through [wikimedia commons](https://commons.wikimedia.org/wiki/File:Pytorch_logo.png)) which can be used under the [Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/deed.en) license. Hence, this logo falls under the same license. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisby%2FtorchMTL","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrisby%2FtorchMTL","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisby%2FtorchMTL/lists"}