https://github.com/stanford-crfm/BioMedLM

https://github.com/stanford-crfm/BioMedLM

Last synced: about 1 month ago
JSON representation

• Host: GitHub
• URL: https://github.com/stanford-crfm/BioMedLM
• Owner: stanford-crfm
• Created: 2022-12-14T08:01:35.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2023-08-20T13:16:35.000Z (over 1 year ago)
• Last Synced: 2024-11-07T11:04:56.174Z (about 1 month ago)
• Language: Python
• Size: 74.2 KB
• Stars: 606
• Watchers: 21
• Forks: 64
• Open Issues: 23
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-ChatGPT-repositories - pubmedgpt - Composing methods for ML training efficiency (Others)
• StarryDivineSky - stanford-crfm/BioMedLM

README

        
# BioMedLM

Code used for pre-training and fine-tuning the [BioMedLM](https://huggingface.co/stanford-crfm/pubmedgpt) model.

Note: This model was previously known as PubMedGPT, but the NIH has asked us to change the name since they hold the trademark on "PubMed", so the new name is BioMedLM!

### Links

[Blog](https://crfm.stanford.edu/2022/12/15/pubmedgpt.html)

[Model](https://huggingface.co/stanford-crfm/pubmedgpt/tree/main)

[MosaicML Composer](https://github.com/mosaicml/composer)

### Example Usage

```

import torch

from transformers import GPT2LMHeadModel, GPT2Tokenizer

device = torch.device("cuda")

tokenizer = GPT2Tokenizer.from_pretrained("stanford-crfm/BioMedLM")

model = GPT2LMHeadModel.from_pretrained("stanford-crfm/BioMedLM").to(device)

input_ids = tokenizer.encode(

    "Photosynthesis is ", return_tensors="pt"

).to(device)

sample_output = model.generate(input_ids, do_sample=True, max_length=50, top_k=50)

print("Output:\n" + 100 * "-")

print(tokenizer.decode(sample_output[0], skip_special_tokens=True))

```