https://github.com/shoyamanishi/tfstackdecoder
A stack decoder for Tensorflow 2.x seq2seq models that generates N-Best.
https://github.com/shoyamanishi/tfstackdecoder
decoder nbest seq2seq tensorflow
Last synced: about 2 months ago
JSON representation
A stack decoder for Tensorflow 2.x seq2seq models that generates N-Best.
- Host: GitHub
- URL: https://github.com/shoyamanishi/tfstackdecoder
- Owner: ShoYamanishi
- License: mit
- Created: 2020-09-06T18:13:44.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-09-19T14:34:29.000Z (almost 6 years ago)
- Last Synced: 2025-10-20T22:54:35.154Z (9 months ago)
- Topics: decoder, nbest, seq2seq, tensorflow
- Language: Python
- Homepage:
- Size: 8.79 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TFStackDecoder
A stack decoder for Tensorflow 2.x seq2seq models that generates N-Best.
I made it to retrieve N-best predicted sentences in the descending order of probability.
The predicited sentences can be similar in terms of edit distance.
I want the same output always, given the same input, and therefore
[tfa.seq2se2.BeamSearchDecoder](https://www.tensorflow.org/addons/api_docs/python/tfa/seq2seq/BeamSearchDecoder)
is not suitable, as it uses sampling and designed to give variety of outputs.
The output will look like this:
```
Input:
Predicted translatons with probability (Nbest):
[0.1748] <マクドナルド>
[0.1044] <マクダナルド>
[0.0557] <マックドナルド>
[0.0546] <マクダノルド>
[0.0400] <マクダナード>
```
It assumes [the general RNN encoder-decoder framework](https://www.tensorflow.org/tutorials/text/nmt_with_attention).
The algorithm is based on the one given in Section 10.2 A^* ("Stack") Decoding in [JM09](https://www.pearson.com/us/higher-education/program/Jurafsky-Speech-and-Language-Processing-2nd-Edition/PGM181706.html).
# Dependencies
* [Tensorflow 2.x](https://www.tensorflow.org/)
* [RedBlackTree](https://github.com/ShoYamanishi/RedBlackTree)
# Contact
For any inquiries, please contact:
Shoichiro Yamanishi
yamanishi72@gmail.com
# Usage:
```python
from TFStackDecoder.stack_decoder import StackDecoder
from TFStackDecoder.stack_decoder import StackDecoderPath
# This is a replacement to evaluate() defined in the NMT tutorial
# It does not generate data for attention plots.
# It generates N-best sentences in the descending order of probability.
def evaluate_nbest(sentence):
sentence = preprocess_sentence(sentence)
inputs = [inp_lang.word_index[i] for i in sentence.split(' ')]
inputs = tf.keras.preprocessing.sequence.pad_sequences([inputs],
maxlen=max_length_inp,
padding='post')
inputs = tf.convert_to_tensor(inputs)
hidden = [tf.zeros((1, units))]
enc_out, enc_hidden = encoder(inputs, hidden)
dec_hidden = enc_hidden
dec_input = tf.expand_dims([targ_lang.word_index['']], 0)
BOS = [targ_lang.word_index['']
EOS = [targ_lang.word_index['']
# Here decoder is assumed to have been defined as a GRU decoder
# with BahdanauAttention.
stack_decoder = StackDecoder(decoder, BOS, EOS, use_attn=True)
BEAM_WIDTH = 20
NUM_NBEST = 5
MAX_LEN = max_length_targ + 2 # this is defined early in the NMT tutorial.
# + 2 is for BOS and EOS.
nbest_list = stack_decoder.NBest( enc_out, enc_hidden, BEAM_WIDTH, NUM_NBEST, MAX_LEN )
results = []
for r in nbest_list:
result = []
for i in r.sentence:
result.append( targ_lang.index_word[i] )
result = ' '.join(result)
results.append((r.log_prob, result))
return results, sentence
```
# References
* NMT : [Neural machine translation with attention](https://www.tensorflow.org/tutorials/text/nmt_with_attention)
* JM09 : Speech and Language Processing, 2nd., Daniel Jurafsky & James H. Martin, 2009 Pearson Education