{"id":27360766,"url":"https://github.com/osainz59/t5-encoder","last_synced_at":"2025-07-19T16:35:14.127Z","repository":{"id":118908167,"uuid":"578122839","full_name":"osainz59/t5-encoder","owner":"osainz59","description":"A extension of Transformers library to include T5ForSequenceClassification class.","archived":false,"fork":false,"pushed_at":"2023-04-17T18:11:26.000Z","size":44,"stargazers_count":38,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-13T01:24:27.369Z","etag":null,"topics":["classification","t5","t5-model","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/osainz59.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-12-14T10:06:45.000Z","updated_at":"2025-03-04T09:12:02.000Z","dependencies_parsed_at":"2023-05-05T01:18:50.490Z","dependency_job_id":null,"html_url":"https://github.com/osainz59/t5-encoder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/osainz59/t5-encoder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osainz59%2Ft5-encoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osainz59%2Ft5-encoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osainz59%2Ft5-encoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osainz59%2Ft5-encoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/osainz59","download_url":"https://codeload.github.com/osainz59/t5-encoder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osainz59%2Ft5-encoder/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265966120,"owners_count":23857056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","t5","t5-model","transformers"],"created_at":"2025-04-13T01:19:31.997Z","updated_at":"2025-07-19T16:35:14.120Z","avatar_url":"https://github.com/osainz59.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# T5 Encoder-only extension for Transformers\nThis repository contains the implementation of `T5ForSequenceClassification` and `T5ForTokenClassification` fully compatible with [Transformers](https://github.com/huggingface/transformers) library. While this could be a feature from the library itself is not implemented yet, so this repository contains the code for preliminary experiments before being actually included to the library.\n\nThis implementation is inspired by [EncT5: A Framework for Fine-tuning T5 as Non-autoregressive Models](https://arxiv.org/abs/2110.08426) and [A Universal Discriminator for Zero-Shot Generalization](https://arxiv.org/pdf/2211.08099.pdf) that made use of T5 encoder only.\n\n## Installation and use\nYou can simply install this library by running the following command:\n```bash\npython -m pip install git+https://github.com/osainz59/t5-encoder\n```\nTo use the implemented classes you have to simply import `t5_encoder` along with transformers. Example:\n```python\nimport t5_encoder\nfrom transformers import AutoTokenizer, AutoModelForSequenceClassification\n\ntokenizer = AutoTokenizer.from_pretrained(\"google/t5-v1_1-base\")\nmodel = AutoModelForSequenceClassification.from_pretrained(\"google/t5-v1_1-base\")\n\noutputs = model(**tokenizer(\"This is a sentence to classify.\", return_tensors=\"pt\"))\nprint(outputs.logits)\n\u003e\u003e\u003e tensor([[ 0.0512, -0.0594]], grad_fn=\u003cAddmmBackward0\u003e)\n```\n\n## GLUE results\n\n| Model | CoLA | SST2 | MRPC | STSb | QQP | MNLI | QNLI | RTE | WNLI |\n|:------|:------|:-----|:-----|:-----|:----|:-----|:-----|:----|:-----|\n| RoBERTa\u003csub\u003elarge\u003c/sub\u003e  | **68.0** | **96.4** | 90.9 | 92.4 | **92.2** | 90.2/90.2 | 94.7 | 86.6 | **91.3** |\n| T5\u003csub\u003elarge\u003c/sub\u003e | 61.2 | 96.3 | 92.4 | 89.9 | 89.9 | 89.9/89.6 | **94.8** | 87.2 | 85.6 | \n| T5-Enc\u003csub\u003elarge\u003c/sub\u003e | 55.0 | 96.1 | **93.3** | **92.7** | 91.4 | **90.5/90.4** | 94.7 | **88.8** | 47.9 |\n\n## NER results\n| Model | CoNLL-2003 (F1) |\n|:------|:------|\n|[RoBERTa](https://huggingface.co/Gladiator/roberta-large_ner_conll2003)\u003csub\u003elarge\u003c/sub\u003e | 96.57 |\n| T5 | - |\n| T5-Enc\u003csub\u003elarge\u003c/sub\u003e | 95.49 |\n\n**Important:** Those results are obtained by a single run, for those datasets with very few examples the performance might change drastically.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosainz59%2Ft5-encoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fosainz59%2Ft5-encoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosainz59%2Ft5-encoder/lists"}