https://github.com/hayatiyrtgl/nlp-keras

I used Python as programming language.
https://github.com/hayatiyrtgl/nlp-keras

keras nlp nlp-machine-learning

Last synced: about 2 months ago
JSON representation

I used Python as programming language.

• Host: GitHub
• URL: https://github.com/hayatiyrtgl/nlp-keras
• Owner: HayatiYrtgl
• Created: 2023-11-12T12:29:57.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-11-12T12:32:30.000Z (about 1 year ago)
• Last Synced: 2024-11-05T09:15:55.834Z (3 months ago)
• Topics: keras, nlp, nlp-machine-learning
• Language: Python
• Homepage:
• Size: 3.91 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# NLP-Keras

This code contains a series of classes used for training a language model on a text dataset. The functionalities are as follows:

1. **PreProcessing Class:**

   - `corpus_processing`: A function that reads the dataset and splits it into lines.

   - `tokenization`: A function that uses a Tokenizer to convert the text into numerical sequences and generates n-gram sequences.

   - `tokenizer_saver`: A function that saves the Tokenizer to a JSON file.

2. **GetTokenizer Class:**

   - `__init__`: A class used to load the Tokenizer from a JSON file.

3. **CreateModel Class:**

   - `__init__`: A class that creates, compiles, and trains the model. It supports both medium and large models.

   - `m_model`: A function that creates a medium-sized language model.

   - `l_model`: A function that creates a large-sized language model.

   - `compiler`: A function that compiles and trains the model.

These classes aim to collectively create a language model learned from a text dataset. After training, the model file (`new_model.h5`) and training history (`new_model.xlsx`) are saved.

If you encounter any issues while running this code or if you need assistance with another topic, feel free to ask.