https://github.com/zotroneneis/lstm_language_model
LSTM-based language model in TensorFlow
https://github.com/zotroneneis/lstm_language_model
language-model language-modeling lstm machine-learning nlp python recurrent-neural-networks tensorflow
Last synced: 4 months ago
JSON representation
LSTM-based language model in TensorFlow
- Host: GitHub
- URL: https://github.com/zotroneneis/lstm_language_model
- Owner: zotroneneis
- License: mit
- Created: 2018-04-11T10:34:32.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2020-04-02T13:47:06.000Z (over 5 years ago)
- Last Synced: 2025-04-30T20:47:15.489Z (5 months ago)
- Topics: language-model, language-modeling, lstm, machine-learning, nlp, python, recurrent-neural-networks, tensorflow
- Language: Python
- Size: 1.64 MB
- Stars: 8
- Watchers: 2
- Forks: 6
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
LSTM-based language model
==============================
This repository contains all code and resources related to my master thesis on the topic
"Recurrent Neural Language Modeling - Using Transfer Learning to Perform Radiological Sentence Completion"
Abstract:
Motivated by the potential benefits of a system that accelerates the process of writing radiological reports, we present a Recurrent Neural Network Language Model for modeling radiological language. We show that recurrent neural language models can be used to produce convincing radiological reports and investigate how their performance can be improved by using advanced regularization and initialization techniques. Furthermore, we study the use of transfer learning to create topic-specific language models.
Data
==============================
The original data used in the thesis is confidential. Therefore, this repository features a version of the code that runs on the Penn Treebank dataset available [here](http:/www.fit.vutbr.cz/~imikolov/rnnlm/).
To run the code you will have to preprocess the data first. Further details on this can be found in the folder *src/data/*
Testable Features
==============================
By adapting the config file, the following features can be tested:
- Variational dropout of hidden layers
- Weight tying
- Embedding dropout
- Pre-trained embeddings
Thesis and Presentation
==============================
The thesis and slides can be found in the *reports* folder
Project Organization
------------
├── LICENSE
├── README.md <- The top-level README
├── data
│ ├── raw <- Original PTB files
│ ├── training_files <- Preprocessed PTB word ids
│ └── embeddings <- word embeddings
│ │
│ ├── fasttext
│ └── word2vec
│
├── models <- Trained and serialized models
│ ├── checkpoints <- Model checkpoints
│ └── tensorboard <- Tensorboard logs
│
├── reports <- Thesis and presentation slides
│
├── requirements.txt <- The requirements file for reproducing the analysis environment
│
├── src <- Source code for use in this project.
│ │
│ ├── __init__.py <- Makes src a Python module
│ │
│ ├── main.py <- main file for training, testing, etc.
│ │
│ ├── config_ptb.yamp <- config file, specifying model params
│ │
│ ├── data <- scripts to preprocess data
│ │ │
│ │ ├── README.md
│ │ ├── preprocess_ptb.py
│ │ └── inverse_dict.py
│ │
│ ├── embeddings <- scripts to train word embeddings
│ │ │
│ │ ├── README.md
│ │ ├── fasttext.py
│ │ └── word2vec.py
│ │
│ ├── models <- Scripts to train models and then use trained models to make predictions
│ │ │
│ │ └── ptb_basic_model.py
│ │
│ └── visualization <- Scripts to create exploratory and results oriented visualizations
│ │
│ ├── README.md
│ └── visualize_embeddings.py
│
└──
--------
Project based on the cookiecutter data science project template. #cookiecutterdatascience