{"id":23725977,"url":"https://github.com/voidful/tfkit","last_synced_at":"2025-09-04T02:31:19.504Z","repository":{"id":46513494,"uuid":"229414641","full_name":"voidful/TFkit","owner":"voidful","description":"🤖📇 handling multiple nlp task in one pipeline","archived":false,"fork":false,"pushed_at":"2025-06-30T01:52:17.000Z","size":16671,"stargazers_count":56,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-08-30T18:35:07.023Z","etag":null,"topics":["multi-label-classification","multi-task","nlp","tagger","tagging","text-classification","text-generation","text-processing","transformer-models","transformers"],"latest_commit_sha":null,"homepage":"https://voidful.github.io/TFkit/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voidful.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-21T10:58:39.000Z","updated_at":"2025-06-30T01:52:20.000Z","dependencies_parsed_at":"2024-06-21T18:54:57.643Z","dependency_job_id":"7f5d0c26-f59a-48c8-8e7d-e0c1c8dbedac","html_url":"https://github.com/voidful/TFkit","commit_stats":{"total_commits":511,"total_committers":2,"mean_commits":255.5,"dds":0.00195694716242667,"last_synced_commit":"6ae93b8d0b3e2ed8467a7fa25f8ce5118575c0af"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/voidful/TFkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voidful%2FTFkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voidful%2FTFkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voidful%2FTFkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voidful%2FTFkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voidful","download_url":"https://codeload.github.com/voidful/TFkit/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voidful%2FTFkit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273541897,"owners_count":25124056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multi-label-classification","multi-task","nlp","tagger","tagging","text-classification","text-generation","text-processing","transformer-models","transformers"],"created_at":"2024-12-31T00:18:03.302Z","updated_at":"2025-09-04T02:31:19.492Z","avatar_url":"https://github.com/voidful.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp  align=\"center\"\u003e\n    \u003cbr\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/voidful/TFkit/master/docs/img/tfkit.png\" width=\"300\"/\u003e\n    \u003cbr\u003e\n\u003c/p\u003e\n\u003cbr/\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://pypi.org/project/tfkit/\"\u003e\n        \u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/tfkit\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/voidful/tfkit\"\u003e\n        \u003cimg alt=\"Download\" src=\"https://img.shields.io/pypi/dm/tfkit\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/voidful/tfkit\"\u003e\n        \u003cimg alt=\"Last Commit\" src=\"https://img.shields.io/github/last-commit/voidful/tfkit\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://www.codefactor.io/repository/github/voidful/tfkit/overview/master\"\u003e\n        \u003cimg src=\"https://www.codefactor.io/repository/github/voidful/tfkit/badge/master\" alt=\"CodeFactor\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/voidful/tfkit\"\u003e\n        \u003cimg src=\"https://visitor-badge.glitch.me/badge?page_id=voidful.tfkit\" alt=\"Visitor\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://codecov.io/gh/voidful/TFkit\"\u003e\n      \u003cimg src=\"https://codecov.io/gh/voidful/TFkit/branch/master/graph/badge.svg\" /\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n## What is it\nTFKit is a tool kit mainly for language generation.  \nIt leverages the use of transformers on many tasks with different models in this all-in-one framework.   \nAll you need is a little change of config.  \n\n## Task Supported\nWith transformer models - BERT/ALBERT/T5/BART......  \n|  |  |\n|-|-|\n| Text Generation | :memo: seq2seq language model |\n| Text Generation | :pen: causal language model |\n| Text Generation | :printer: once generation model / once generation model with ctc loss |\n| Text Generation | :pencil: onebyone generation model |\n\n# Getting Started\nLearn more from the [document](https://voidful.github.io/TFkit/).  \n\n## How To Use\n\n### Step 0: Install\nSimple installation from PyPI\n```bash\npip install git+https://github.com/voidful/TFkit.git@refactor-dataset\n```\n\n### Step 1: Prepare dataset in csv format\n[Task format](https://voidful.tech/TFkit/tasks/)\n``` \ninput, target\n```\n\n### Step 2: Train model\n```bash\ntfkit-train \\\n--task clas \\\n--config xlm-roberta-base \\\n--train training_data.csv \\\n--test testing_data.csv \\\n--lr 4e-5 \\\n--maxlen 384 \\\n--epoch 10 \\\n--savedir roberta_sentiment_classifier\n```\n\n### Step 3: Evaluate\n```bash\ntfkit-eval \\\n--task roberta_sentiment_classifier/1.pt \\\n--metric clas \\\n--valid testing_data.csv\n```\n\n## Advanced features\n\u003cdetails\u003e\n  \u003csummary\u003eMulti-task training \u003c/summary\u003e\n\n  ```bash\n  tfkit-train \\\n    --task clas clas \\\n    --config xlm-roberta-base \\\n    --train training_data_taskA.csv training_data_taskB.csv \\\n    --test testing_data_taskA.csv testing_data_taskB.csv \\\n    --lr 4e-5 \\\n    --maxlen 384 \\\n    --epoch 10 \\\n    --savedir roberta_sentiment_classifier_multi_task\n  ```\n\u003c/details\u003e\n\n## Not maintained task\nDue to time constraints, the following tasks are temporarily not supported\n|  |  |\n|-|-|\n| Classification  | :label: multi-class and multi-label classification |\n| Question Answering  | :page_with_curl: extractive qa |\n| Question Answering  | :radio_button: multiple-choice qa |\n| Tagging  | :eye_speech_bubble: sequence level tagging / sequence level with crf  |\n| Self-supervise Learning | :diving_mask: mask language model |\n\n## Supplement\n- [transformers models list](https://huggingface.co/models): you can find any pretrained models here   \n- [nlprep](https://github.com/voidful/NLPrep): download and preprocessing data in one line     \n- [nlp2go](https://github.com/voidful/nlp2go): create demo api as quickly as possible.\n\n\n## Contributing\nThanks for your interest.There are many ways to contribute to this project. Get started [here](https://github.com/voidful/tfkit/blob/master/CONTRIBUTING.md).\n\n## License ![PyPI - License](https://img.shields.io/github/license/voidful/tfkit)\n\n* [License](https://github.com/voidful/tfkit/blob/master/LICENSE)\n\n## Icons reference\nIcons modify from \u003ca href=\"http://www.freepik.com/\" title=\"Freepik\"\u003eFreepik\u003c/a\u003e from \u003ca href=\"https://www.flaticon.com/\" title=\"Flaticon\"\u003ewww.flaticon.com\u003c/a\u003e      \nIcons modify from \u003ca href=\"https://www.flaticon.com/authors/nikita-golubev\" title=\"Nikita Golubev\"\u003eNikita Golubev\u003c/a\u003e from \u003ca href=\"https://www.flaticon.com/\" title=\"Flaticon\"\u003ewww.flaticon.com\u003c/a\u003e      \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoidful%2Ftfkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoidful%2Ftfkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoidful%2Ftfkit/lists"}