https://github.com/erfanium/huggingface-sentence-generator

HuggingFace text-generation model that never generates <eos> token and always continues the current sentence
https://github.com/erfanium/huggingface-sentence-generator

Last synced: 10 months ago
JSON representation

HuggingFace text-generation model that never generates <eos> token and always continues the current sentence

• Host: GitHub
• URL: https://github.com/erfanium/huggingface-sentence-generator
• Owner: erfanium
• Created: 2023-09-20T13:01:39.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2023-09-20T13:07:06.000Z (almost 3 years ago)
• Last Synced: 2025-02-05T12:45:35.627Z (over 1 year ago)
• Homepage:
• Size: 1000 Bytes
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# The Idea

```py

from transformers import GPT2LMHeadModel, AutoTokenizer

import torch

tokenizer = AutoTokenizer.from_pretrained("gpt2")

class MyGPT2LMHeadModel(GPT2LMHeadModel):

    def __init__(self, config):

        super().__init__(config)

        with torch.no_grad():

            self._masking = torch.zeros(

                1, 1, config.vocab_size, device=self.device, dtype=self.dtype

            )

            self._masking[0, 0, tokenizer.eos_token_id] = -2

    def __call__(self, *inputs, **kwargs):

        result = super().__call__(*inputs, **kwargs)

        result.logits += self._masking

        return result

```