{"id":26120109,"url":"https://github.com/gt4sd/zero-shot-bert-adapters","last_synced_at":"2025-04-13T11:24:33.965Z","repository":{"id":54602858,"uuid":"522484460","full_name":"GT4SD/zero-shot-bert-adapters","owner":"GT4SD","description":"Implementation of Z-BERT-A: a zero-shot pipeline for unknown intent detection.","archived":false,"fork":false,"pushed_at":"2023-06-13T08:24:36.000Z","size":2052,"stargazers_count":39,"open_issues_count":3,"forks_count":10,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-27T02:38:50.818Z","etag":null,"topics":["bert","deep-learning","intent","intent-classification","intent-detection","nlp","pytorch","transformers","zero-shot","zero-shot-classification"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2208.07084","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GT4SD.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-08T09:29:08.000Z","updated_at":"2024-12-19T07:55:39.000Z","dependencies_parsed_at":"2023-02-08T12:32:08.288Z","dependency_job_id":null,"html_url":"https://github.com/GT4SD/zero-shot-bert-adapters","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT4SD%2Fzero-shot-bert-adapters","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT4SD%2Fzero-shot-bert-adapters/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT4SD%2Fzero-shot-bert-adapters/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GT4SD%2Fzero-shot-bert-adapters/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GT4SD","download_url":"https://codeload.github.com/GT4SD/zero-shot-bert-adapters/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248704324,"owners_count":21148346,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","deep-learning","intent","intent-classification","intent-detection","nlp","pytorch","transformers","zero-shot","zero-shot-classification"],"created_at":"2025-03-10T13:05:54.655Z","updated_at":"2025-04-13T11:24:33.934Z","avatar_url":"https://github.com/GT4SD.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Zero-shot-BERT-adapters\nRepository containing code for Zero-shot-BERT-adapters (Z-BERT-A), a two-stage method for multilingual intent discovery relying on a Transformer architecture, fine-tuned with Adapters, which is initially trained for Natural Language Inference (NLI) and later applied for unknown intent classification in a zero-shot setting.\n\n\u003cimg src=\"https://user-images.githubusercontent.com/6382701/195600358-24c3f136-96f0-4a8a-8eaf-220970ca0604.png\" width=523 height=466\u003e\n\n## Deployment example of an use-case of Zero-shot-BERT-adapters pipeline\n\n![image](https://i.postimg.cc/y8yNP4gp/image.png)\n\n\n\n## Important information\nThis repository makes use of the module [Adapter Transformers](https://github.com/adapter-hub/adapter-transformers).\u003cbr\u003e\nThe [wget](https://www.gnu.org/software/wget/) tool is required for this package, for Windows users please make sure to install it accordingly before running this module.\n\n## Installation\nIn order to install Zero-shot-BERT-adapters it is just needed to execute the following pip command which will make sure everything is accordingly installed.\nZero-shot-BERT-adapters uses spaCy with 'en_core_web_trf', this package will try to install it automatically itself if not present but make sure you have it installed through the suggested way of [spaCy](https://github.com/explosion/spaCy).\n\n```console\npip install git+https://github.com/GT4SD/zero-shot-bert-adapters.git\n```\n\n## Usage information\n\nIn order to reproduce the results here there is the sample code which can also be found in an example [Jupyter Notebook](https://colab.research.google.com/drive/1k-MqbwbU870wGlpcSbaRNV3sx7h-rW9_?usp=sharing).\n\n```python\nimport torch\nfrom zberta.data.data_snli import DataSNLI\nfrom zberta.data.data_banking import DataBanking\nimport zberta.model.model as model\nfrom zberta.train.trainer import Trainer\nfrom zberta.intent_discovery.unknown_intents import unknown_intents_set\nfrom zberta.intent_discovery.zberta import ZBERTA\n\nmodel_name = \"bert-base-uncased\"\ntraining = False\ntesting = False\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nif training:\n    snli = DataSNLI(model_name, device)\n    train_iterator, valid_iterator, test_iterator = snli.iterators()\n    berta = model.instantiate_model(snli.labels(), snli.output_dim(), device, model_name, snli.nli_labels())\n    trainer = Trainer(berta, train_iterator, valid_iterator, test_iterator, device)\n    trainer.start_training()\n    if testing:\n        trainer.start_testing()\nz_banking = DataBanking(model_name, device)\nz_dataset = z_banking.z_iterator()\nz_intents = unknown_intents_set(z_dataset['test']['text'])\nberta = model.instantiate_model(z_banking.labels(), z_banking.output_dim(), device, model_name,\n                                z_banking.nli_labels(), path=\"model.pt\", dict=True)\nzberta = ZBERTA(berta, model_name, z_dataset['test']['text'], z_intents, z_dataset['test']['category'])\nz_acc = zberta.zero_shot_intents()\nprint(z_acc)\n```\n\nFor simple usage of the Zero-shot-BERT-adapters pipeline instead it's just needed to load the pipeline and model through this simple code which encapsulate all the complexity behind.\n\n```python\nimport torch\nfrom zberta.intent_discovery.zberta import ZBERTA\nimport zberta.model.model as model\n\nmodel_name = \"bert-base-uncased\"\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nberta = model.instantiate_model(2, 2, device, model_name,\n                                {\"entailment\": 0, \"contradiction\": 1}, path=\"model.pt\", dict=True)\nzberta = ZBERTA(berta, model_name)\n```\n\nAs example run:\n\n```python\nzberta.find_new_intents([\"I want to buy a book but I lost all my money, where can I make a withdrawal?\"])\n```\n```\nOutput: ['make withdrawal']\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgt4sd%2Fzero-shot-bert-adapters","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgt4sd%2Fzero-shot-bert-adapters","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgt4sd%2Fzero-shot-bert-adapters/lists"}