https://github.com/vishvamporwal/ispeakshakespeare

Decoder-only implementation of the transformer architecture for Shakespearean language generation.
https://github.com/vishvamporwal/ispeakshakespeare

Last synced: about 1 month ago
JSON representation

Decoder-only implementation of the transformer architecture for Shakespearean language generation.

• Host: GitHub
• URL: https://github.com/vishvamporwal/ispeakshakespeare
• Owner: VishvamPorwal
• Created: 2024-03-24T15:44:17.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-03-24T22:47:12.000Z (10 months ago)
• Last Synced: 2024-10-28T09:12:35.588Z (3 months ago)
• Language: Python
• Homepage:
• Size: 537 KB
• Stars: 2
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# iSpeakShakespeare

Decoder-only implementation of [Attention is All You Need](https://arxiv.org/abs/1706.03762) for Shakespearean language generation.

![Image from the paper "Attention is all you need"](image.png)

For training:

```

python train.py

```

![training](image-1.png)

Example Generation:

```

So sea command net far the withems hims in

I shee exch shiment's noldandege in nat's bity he

exoneswre as your sond he come and your that, swall

whave; hor'd hearb is to fearfulticced 'twoed;

Foor fend endys breas wourse fe be loinss.

Why havole wor to to with hus on.

ETHOS:

Be not do Madarract?

GLORINUS:

I ague ho lest, his soult.

Never:

For pappaciles tshiner ade fors tham to lonstsen,

Than herre. trand it altick noner, be beit.

Ti be; ither, your conter, we is spens,

And hast his besty bri

```

Inspired by [NanoGPT](https://github.com/karpathy/nanoGPT)

Data from [tinyshakespeare](https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt)