{"id":27169182,"url":"https://github.com/smilelab-fl/fedlab-nlp","last_synced_at":"2025-04-09T06:31:43.218Z","repository":{"id":46138767,"uuid":"515099680","full_name":"SMILELab-FL/FedLab-NLP","owner":"SMILELab-FL","description":"FedLab-NLP: A Fast and highly Customizable Framework for Federated Learning in Natural Language Processing","archived":false,"fork":false,"pushed_at":"2022-08-19T03:12:47.000Z","size":100,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-05-12T00:45:21.007Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SMILELab-FL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-18T08:36:34.000Z","updated_at":"2024-03-25T09:13:45.000Z","dependencies_parsed_at":"2022-09-13T13:52:46.342Z","dependency_job_id":null,"html_url":"https://github.com/SMILELab-FL/FedLab-NLP","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SMILELab-FL%2FFedLab-NLP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SMILELab-FL%2FFedLab-NLP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SMILELab-FL%2FFedLab-NLP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SMILELab-FL%2FFedLab-NLP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SMILELab-FL","download_url":"https://codeload.github.com/SMILELab-FL/FedLab-NLP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247991694,"owners_count":21029754,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-09T06:31:42.353Z","updated_at":"2025-04-09T06:31:43.190Z","avatar_url":"https://github.com/SMILELab-FL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e  \n    \u003cp\u003e FedLab-NLP \u003c/p\u003e  \n\u003c/h1\u003e  \n \n \u003cp align=\"center\"\u003e \n\t \u003cimg alt=\"SMILELAB\" src=\"https://img.shields.io/badge/owner-SMILELAB-orange\"\u003e\n\t \u003cimg alt=\"Licence\" src=\"https://img.shields.io/badge/License-Apache%202.0-yellow\"\u003e\n\t \u003cimg alt=\"Build\" src=\"https://img.shields.io/badge/build-processing-green\"\u003e\n \u003c/p\u003e\n \n\u003ch3 align=\"center\"\u003e  \n    \u003cp\u003e 致力于联邦自然语言处理发展 \u003c/p\u003e  \n\u003c/h3\u003e  \n  \n1️⃣ 本项目来自 [SMILELab-FL](https://github.com/SMILELab-FL/FedLab-NLP) , 我们主要致力于联邦自然语言处理的发展；  \n2️⃣ `FedLab-NLP`是一款轻量级、可自由定制联邦学习过程并面向于自然语言处理的代码框架；  \n3️⃣ `FedLab-NLP`主要实现基于 [FedLab](https://github.com/SMILELab-FL/FedLab) 和 [HuggingFace](https://github.com/huggingface/transformers)，能完成自然语言预训练大模型在联邦学习场景下工作；  \n4️⃣ 任何使用问题可以联系 :e-mail: iezhuo17@gmail.com\n  \n## Installation\n\n### 目录组织  \n我们建议按照如下的文件结构来使用`FedLab-NLP`:    \n\n    data:  存放数据集，注意该数据集需要已经经过 iid/niid 划分的联邦数据集;\n    pretrained:  在nlp文件夹里存储从huggingface中下载的预训练模型，如`bert-base-uncased`;\n    code:  在该文件夹下面使用`FedLab-NLP`;\n    output: 存储模型输出的记录等。\n\n目录组织如下：\n```grapha  \n├── workspace  \n│   └── data  \n|   |   ├── fedglue  \n|   |   └── fedner  \n│   ├── pretrained  \n│   │   ├── nlp  \n│   │   └── cv  \n│   ├── output  \n│   └── code  \n│       └── FedLab-NLP  \n```  \n  \n运行路径生成：  \n```bash  \nmkdir workspace  \ncd workspace  \nmkdir data  \nmkdir code  \nmkdir pretrained  \ncd pretrained  \nmkdir nlp  \ncd ..  \ncd code  \n```  \n \n### 环境安装  \n建议运行环境的`python`版本为`3.7+`，我们建议使用`pytorch`版本为`1.10+` \n```bash  \ngit clone git@git.openi.org.cn:Trustworthy-DL/FedLab-NLP.git  \ncd FedLab-NLP  \npip install -r resquirements.txt  \n```\n\n## Usage\n支持 `fedavg` 联邦学习算法\n```bash\nbash fed_run.sh {your_file_path}/workspace {task_name} fedavg 10001 {server_gpu_id} {client1_gpu_id} {client2_gpu_id}\n```\n\n支持 `centralized` 集中训练学习算法\n```bash\nbash cen_run.sh {your_file_path}/workspace {task_name} centralized 10001 {server_gpu_id}\n```\n## Contributing\n\n## License\n[Apache License 2.0](https://github.com/SMILELab-FL/FedLab-NLP/blob/main/LICENSE)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmilelab-fl%2Ffedlab-nlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmilelab-fl%2Ffedlab-nlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmilelab-fl%2Ffedlab-nlp/lists"}