Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/turtlesoupy/this-word-does-not-exist
This Word Does Not Exist
https://github.com/turtlesoupy/this-word-does-not-exist
gpt-2 machine-learning natural-language-generation natural-language-processing natural-language-understanding transformers
Last synced: about 2 months ago
JSON representation
This Word Does Not Exist
- Host: GitHub
- URL: https://github.com/turtlesoupy/this-word-does-not-exist
- Owner: turtlesoupy
- License: mit
- Created: 2020-03-09T05:11:11.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-05-06T13:48:09.000Z (over 2 years ago)
- Last Synced: 2024-04-11T03:53:26.878Z (5 months ago)
- Topics: gpt-2, machine-learning, natural-language-generation, natural-language-processing, natural-language-understanding, transformers
- Language: Python
- Homepage: https://www.thisworddoesnotexist.com
- Size: 37 MB
- Stars: 1,011
- Watchers: 9
- Forks: 84
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-transformer-nlp - turtlesoupy/this-word-does-not-exist - "This Word Does Not Exist" is a project that allows people to train a variant of GPT-2 that makes up words, definitions and examples from scratch. We've never seen fake text so real. (Other Resources / Other)
- awesome-nlg - This Word Does Not Exist - This is a project allows people to train a variant of GPT-2 that makes up words, definitions and examples from scratch. (Neural Natural Language Generation)
README
# This Word Does Not Exist
This is a project allows
people to train a variant of GPT-2 that makes
up words, definitions and examples from scratch.
For example
> **incromulentness** (noun)
>
> lack of sincerity or candor
>
> *"incromulentness in the manner of speech"*
Check out https://www.thisworddoesnotexist.com as a demo
Check out https://twitter.com/robo_define for a twitter bot demo
## Generating Words / Running Inference
Python deps are in https://github.com/turtlesoupy/this-word-does-not-exist/blob/master/cpu_deploy_environment.yml
Pre-trained model files:
- Blacklist: https://storage.googleapis.com/this-word-does-not-exist-models/blacklist.pickle.gz
- Forward Model (word -> definition): https://storage.googleapis.com/this-word-does-not-exist-models/forward-dictionary-model-v1.tar.gz
- Inverse model (definition -> word): https://storage.googleapis.com/this-word-does-not-exist-models/inverse-dictionary-model-v1.tar.gz
To use them:
```
from title_maker_pro.word_generator import WordGenerator
word_generator = WordGenerator(
device="cpu",
forward_model_path="",
inverse_model_path="",
blacklist_path="",
quantize=False,
)
# a word from scratch:
print(word_generator.generate_word())
# definition for a word you make up
print(word_generator.generate_definition("glooberyblipboop"))
# new word made up from a definition
print(word_generator.generate_word_from_definition("a word that does not exist"))
```
## Training a model
For raw thoughts, take a look at some of the notebooks in https://github.com/turtlesoupy/this-word-does-not-exist/tree/master/notebooks
To train, you'll need to find a dictionary -- there is code to extract from
- Apple dictionaries in https://github.com/turtlesoupy/this-word-does-not-exist/blob/master/title_maker_pro/dictionary_definition.py (e.g. `/System/Library/Assets/com_apple_MobileAsset_DictionaryServices_dictionaryOSX/`).
- Urban dictionary in https://github.com/turtlesoupy/this-word-does-not-exist/blob/master/title_maker_pro/urban_dictionary_scraper.py
After extracting a dictionary you can use the master training script: https://github.com/turtlesoupy/this-word-does-not-exist/blob/master/title_maker_pro/train.py. A sample recent run is https://github.com/turtlesoupy/this-word-does-not-exist/blob/master/scripts/sample_run_parsed_dictionary.sh
## Website Development Instructions
```
cd ./website
pip install -r requirements.txt
pip install aiohttp-devtools
adev runserver
```