https://github.com/shreyansh26/sentence-vae
A re-implementation of the Sentence VAE paper, Generating Sentences from a Continuous Space
https://github.com/shreyansh26/sentence-vae
gru rnn text-generation text-generation-using-rnn vae vae-implementation vae-pytorch
Last synced: 6 months ago
JSON representation
A re-implementation of the Sentence VAE paper, Generating Sentences from a Continuous Space
- Host: GitHub
- URL: https://github.com/shreyansh26/sentence-vae
- Owner: shreyansh26
- Created: 2022-09-22T16:43:26.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2022-09-22T16:43:39.000Z (almost 3 years ago)
- Last Synced: 2024-11-14T09:40:57.242Z (8 months ago)
- Topics: gru, rnn, text-generation, text-generation-using-rnn, vae, vae-implementation, vae-pytorch
- Language: Python
- Homepage:
- Size: 35 MB
- Stars: 3
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Sentence VAE Implementation
A re-implementation of the Sentence VAE paper, [Generating Sentences from a Continuous Space](https://arxiv.org/abs/1511.06349). The paper talks about modelling sentences to latent space representations and allowing deterministic decoding of these latent vectors to produce well-formed sentences. This also allows interpolation between two vectors to produce coherent sentences.
*The inspiration for this implementation is from [Tim Baumgärtner's repository](https://github.com/timbmg/Sentence-VAE). The data processing code and some helper functions have been taken from there. The main purpose of this project was to enhance my understanding of VAEs and implement this paper for learning purposes*
## Results
### Performance
#### ELBO
#### Epoch-averaged ELBO
#### KL Divergence loss
#### Negative Log-likelihood Loss
#### KL Weight
### Samples
Sentences have been obtained after sampling from z ~ N(0, I).
```
there ' s a great difference between the yeast and the cuban drug
but the company has n't yet been able to pinpoint a license
he said the committee appointed the president ' s motion - year-old civil rights plan to meet with the bush administration ' s request
caltrans said the galileo worm had a good luck to the late 1970s
some people are facing the variations in the area that they are n't aware of their ignorance says mr . newhouse
the u . s . is the only n years of the krasnoyarsk of the krasnoyarsk regime of the krasnoyarsk regime of the krasnoyarsk regime of pravda
but we do n't think that the rest of the gun is n't entirely
william c . brown president of the prentice hall foundation and construction concern
they are inevitably inclined to lure the same circumstances that would be settled
and the difficulties are facing difficulties
```
### Interpolation
Sentences have been obtained after sampling twice from z ~ N(0, I) and then interpolating the two samples.
```
i ' m not going to be a disaster
researchers said the ordinance is slated to be introduced by the virus of the virus that seeks to transform the drug centers
i think the fed is n't committed to the disaster
humana inc . said it signed a $ n million contract for the start-up of the national aeronautics and space force
i think the fed is n't committed to the disaster
osha said the contract signed a $ n million contract with meredith and universities that are used by the drug centers
i think the fed is n't commenting on the region
osha said the contract signed a definitive agreement with the facility and a new york facility in the netherlands and a number of the facility
we do n't want to see the reduction in the next few months
osha said the contract signed a definitive agreement with the facility and a new york facility in the netherlands and a number of seats in the netherlands
```
## Training
Use the download_data.sh file to get the Penn Treebank data.
Training is as simple as executing
```
python train.py
```
To replicate the above results, I used the following config
```
python train.py --learning_rate 0.001 --num_layers 2 --word_dropout_rate 0.2 --annealing_till 2500
```
Other configs can be seen from the [train.py](train.py) file.
## Inference
To use the checkpoint provided with this repository, run
```
python inference.py -nl 2
```
The configs have to be added based on the training configs of the model checkpoint being used.