{"id":13478519,"url":"https://github.com/microsoft/BioGPT","last_synced_at":"2025-03-27T07:31:16.337Z","repository":{"id":59813719,"uuid":"524871042","full_name":"microsoft/BioGPT","owner":"microsoft","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-25T10:33:53.000Z","size":32193,"stargazers_count":4387,"open_issues_count":75,"forks_count":460,"subscribers_count":71,"default_branch":"main","last_synced_at":"2025-03-27T01:08:26.026Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-15T05:55:55.000Z","updated_at":"2025-03-25T06:52:29.000Z","dependencies_parsed_at":"2024-09-20T21:40:31.273Z","dependency_job_id":null,"html_url":"https://github.com/microsoft/BioGPT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FBioGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FBioGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FBioGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FBioGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/BioGPT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245802505,"owners_count":20674684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T16:01:58.134Z","updated_at":"2025-03-27T07:31:16.303Z","avatar_url":"https://github.com/microsoft.png","language":"Python","funding_links":[],"categories":["Python","精选开源项目合集","排行榜 [2025-03-18]","其他_生物医药","Techniques and Models","Drug Response Prediction","🧪 Scientific Pretraining, SFT, Reasoning, and Agent Datasets","🤖 Foundation Models for Science","Pharmacology \u0026 ADME \u0026 Metabolism"],"sub_categories":["专业领域机器人","网络服务_其他","GPT-2 models","LLM for biology","🧬 Life Sciences","Domain-Specific Models"],"readme":"# BioGPT\nThis repository contains the implementation of [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://academic.oup.com/bib/advance-article/doi/10.1093/bib/bbac409/6713511?guestAccessKey=a66d9b5d-4f83-4017-bb52-405815c907b9), by Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon and Tie-Yan Liu.\n\n\n# Requirements and Installation\n\n* [PyTorch](http://pytorch.org/) version == 1.12.0\n* Python version == 3.10\n* fairseq version == 0.12.0:\n\n``` bash\ngit clone https://github.com/pytorch/fairseq\ncd fairseq\ngit checkout v0.12.0\npip install .\npython setup.py build_ext --inplace\ncd ..\n```\n* Moses\n``` bash\ngit clone https://github.com/moses-smt/mosesdecoder.git\nexport MOSES=${PWD}/mosesdecoder\n```\n* fastBPE\n``` bash\ngit clone https://github.com/glample/fastBPE.git\nexport FASTBPE=${PWD}/fastBPE\ncd fastBPE\ng++ -std=c++11 -pthread -O3 fastBPE/main.cc -IfastBPE -o fast\n```\n* sacremoses\n``` bash\npip install sacremoses\n```\n* sklearn\n``` bash\npip install scikit-learn\n```\n\nRemember to set the environment variables `MOSES` and `FASTBPE` to the path of Moses and fastBPE respetively, as they will be required later.\n\n# Getting Started\n## Pre-trained models\nWe provide our pre-trained BioGPT model checkpoints along with fine-tuned checkpoints for downstream tasks, available both through URL download as well as through the Hugging Face 🤗 Hub. \n\n|Model|Description|URL|🤗 Hub|\n|----|----|---|---|\n|BioGPT|Pre-trained BioGPT model checkpoint|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/Pre-trained-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:37:35Z\u0026se=2099-12-30T23:37:35Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=3CcG1TOhqJPBhkVutvVn3PtUq0vPyLBgwggUfojypfY%3D)|[link](https://huggingface.co/microsoft/biogpt)|\n|BioGPT-Large|Pre-trained BioGPT-Large model checkpoint|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/Pre-trained-BioGPT-Large.tgz?sp=r\u0026st=2023-11-13T15:38:13Z\u0026se=2099-12-30T23:38:13Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=ib1SZut9wAwrsxGWtFtIZDhrnRg92dwPJmoY2lr3MTg%3D)|[link](https://huggingface.co/microsoft/biogpt-large)|\n|BioGPT-QA-PubMedQA-BioGPT|Fine-tuned BioGPT for question answering task on PubMedQA|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/QA-PubMedQA-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:38:43Z\u0026se=2099-12-30T23:38:43Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=A5SQae6ifsXmrsgpj4E2flhyXm4iHc%2FqO5b8HGOMyjc%3D)| |\n|BioGPT-QA-PubMedQA-BioGPT-Large|Fine-tuned BioGPT-Large for question answering task on PubMedQA|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/QA-PubMedQA-BioGPT-Large.tgz?sp=r\u0026st=2023-11-13T15:39:40Z\u0026se=2099-12-30T23:39:40Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=t%2B%2FD%2BxVoIxiuyDsD0VXv%2FjSGoS0VcrdVXycYhWZoxUc%3D)||\n|BioGPT-RE-BC5CDR|Fine-tuned BioGPT for relation extraction task on BC5CDR|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/RE-BC5CDR-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:35:14Z\u0026se=2099-12-30T23:35:14Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=uXlLIHlVeKIbS%2BVmdzAmlNCeKdoKO2lxsSmwSi%2FH8nE%3D)| |\n|BioGPT-RE-DDI|Fine-tuned BioGPT for relation extraction task on DDI|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/RE-DDI-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:35:58Z\u0026se=2099-12-30T23:35:58Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=DkaQMuM%2FXAsM2p8%2BUs45ecuqhlSRF1DUYRBJNcxD6Pk%3D)| |\n|BioGPT-RE-DTI|Fine-tuned BioGPT for relation extraction task on KD-DTI|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/RE-DTI-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:36:23Z\u0026se=2099-12-30T23:36:23Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=bRgUZyqGuwYdM%2FVFzIv6Xa0GThkXq6bVzszmTe9c%2BKM%3D)| |\n|BioGPT-DC-HoC|Fine-tuned BioGPT for document classification task on HoC|[link](https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/DC-HoC-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:37:17Z\u0026se=2099-12-30T23:37:17Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=1DxroWPt%2FBppCTy7QHs842lLy8SQRcUeUwSfMzDFvl0%3D)| |\n\nDownload them and extract them to the `checkpoints` folder of this project.\n\nFor example:\n``` bash\nmkdir checkpoints\ncd checkpoints\nwget https://msralaphilly2.blob.core.windows.net/release/BioGPT/checkpoints/Pre-trained-BioGPT.tgz?sp=r\u0026st=2023-11-13T15:37:35Z\u0026se=2099-12-30T23:37:35Z\u0026spr=https\u0026sv=2022-11-02\u0026sr=b\u0026sig=3CcG1TOhqJPBhkVutvVn3PtUq0vPyLBgwggUfojypfY%3D\ntar -zxvf Pre-trained-BioGPT.tgz\n```\n\n## Example Usage\nUse pre-trained BioGPT model in your code:\n```python\nimport torch\nfrom fairseq.models.transformer_lm import TransformerLanguageModel\nm = TransformerLanguageModel.from_pretrained(\n        \"checkpoints/Pre-trained-BioGPT\", \n        \"checkpoint.pt\", \n        \"data\",\n        tokenizer='moses', \n        bpe='fastbpe', \n        bpe_codes=\"data/bpecodes\",\n        min_len=100,\n        max_len_b=1024)\nm.cuda()\nsrc_tokens = m.encode(\"COVID-19 is\")\ngenerate = m.generate([src_tokens], beam=5)[0]\noutput = m.decode(generate[0][\"tokens\"])\nprint(output)\n```\n\nUse fine-tuned BioGPT model on KD-DTI for drug-target-interaction in your code:\n```python\nimport torch\nfrom src.transformer_lm_prompt import TransformerLanguageModelPrompt\nm = TransformerLanguageModelPrompt.from_pretrained(\n        \"checkpoints/RE-DTI-BioGPT\", \n        \"checkpoint_avg.pt\", \n        \"data/KD-DTI/relis-bin\",\n        tokenizer='moses', \n        bpe='fastbpe', \n        bpe_codes=\"data/bpecodes\",\n        max_len_b=1024,\n        beam=1)\nm.cuda()\nsrc_text=\"\" # input text, e.g., a PubMed abstract\nsrc_tokens = m.encode(src_text)\ngenerate = m.generate([src_tokens], beam=args.beam)[0]\noutput = m.decode(generate[0][\"tokens\"])\nprint(output)\n```\n\nFor more downstream tasks, please see below.\n\n## Downstream tasks\nSee corresponding folder in [examples](examples):\n### [Relation Extraction on BC5CDR](examples/RE-BC5CDR)\n### [Relation Extraction on KD-DTI](examples/RE-DTI/)\n### [Relation Extraction on DDI](examples/RE-DDI)\n### [Document Classification on HoC](examples/DC-HoC/)\n### [Question Answering on PubMedQA](examples/QA-PubMedQA/)\n### [Text Generation](examples/text-generation/)\n\n## Hugging Face 🤗 Usage\n\nBioGPT has also been integrated into the Hugging Face `transformers` library, and model checkpoints are available on the Hugging Face Hub.\n\nYou can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we set a seed for reproducibility:\n\n```python\nfrom transformers import pipeline, set_seed\nfrom transformers import BioGptTokenizer, BioGptForCausalLM\nmodel = BioGptForCausalLM.from_pretrained(\"microsoft/biogpt\")\ntokenizer = BioGptTokenizer.from_pretrained(\"microsoft/biogpt\")\ngenerator = pipeline('text-generation', model=model, tokenizer=tokenizer)\nset_seed(42)\ngenerator(\"COVID-19 is\", max_length=20, num_return_sequences=5, do_sample=True)\n```\n\nHere is how to use this model to get the features of a given text in PyTorch:\n\n```python\nfrom transformers import BioGptTokenizer, BioGptForCausalLM\ntokenizer = BioGptTokenizer.from_pretrained(\"microsoft/biogpt\")\nmodel = BioGptForCausalLM.from_pretrained(\"microsoft/biogpt\")\ntext = \"Replace me by any text you'd like.\"\nencoded_input = tokenizer(text, return_tensors='pt')\noutput = model(**encoded_input)\n```\n\nBeam-search decoding:\n\n```python\nimport torch\nfrom transformers import BioGptTokenizer, BioGptForCausalLM, set_seed\n\ntokenizer = BioGptTokenizer.from_pretrained(\"microsoft/biogpt\")\nmodel = BioGptForCausalLM.from_pretrained(\"microsoft/biogpt\")\n\nsentence = \"COVID-19 is\"\ninputs = tokenizer(sentence, return_tensors=\"pt\")\n\nset_seed(42)\n\nwith torch.no_grad():\n    beam_output = model.generate(**inputs,\n                                 min_length=100,\n                                 max_length=1024,\n                                 num_beams=5,\n                                 early_stopping=True\n                                )\ntokenizer.decode(beam_output[0], skip_special_tokens=True)\n```\n\nFor more information, please see the [documentation](https://huggingface.co/docs/transformers/main/en/model_doc/biogpt) on the Hugging Face website.\n\n## Demos\n\nCheck out these demos on Hugging Face Spaces:\n* [Text Generation with BioGPT-Large](https://huggingface.co/spaces/katielink/biogpt-large-demo)\n* [Question Answering with BioGPT-Large-PubMedQA](https://huggingface.co/spaces/katielink/biogpt-qa-demo)\n\n# License\n\nBioGPT is MIT-licensed.\nThe license applies to the pre-trained models as well.\n\n# Contributing\n\nThis project welcomes contributions and suggestions.  Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.\n\nWhen you submit a pull request, a CLA bot will automatically determine whether you need to provide\na CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions\nprovided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n# Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft \ntrademarks or logos is subject to and must follow \n[Microsoft's Trademark \u0026 Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FBioGPT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2FBioGPT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FBioGPT/lists"}