{"id":21923093,"url":"https://github.com/26hzhang/sequencetosequence","last_synced_at":"2025-07-22T19:34:43.175Z","repository":{"id":92547898,"uuid":"131999486","full_name":"26hzhang/SequenceToSequence","owner":"26hzhang","description":"A seq2seq with attention dialogue/MT model implemented by TensorFlow.","archived":false,"fork":false,"pushed_at":"2018-07-17T12:55:06.000Z","size":36156,"stargazers_count":11,"open_issues_count":2,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-19T18:22:30.511Z","etag":null,"topics":["attention-mechanism","dialogue","machine-translation","residual","seq2seq","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/26hzhang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-05-03T13:30:45.000Z","updated_at":"2022-03-14T01:44:29.000Z","dependencies_parsed_at":"2023-03-07T02:15:50.846Z","dependency_job_id":null,"html_url":"https://github.com/26hzhang/SequenceToSequence","commit_stats":null,"previous_names":["26hzhang/sequencetosequence"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/26hzhang/SequenceToSequence","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/26hzhang%2FSequenceToSequence","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/26hzhang%2FSequenceToSequence/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/26hzhang%2FSequenceToSequence/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/26hzhang%2FSequenceToSequence/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/26hzhang","download_url":"https://codeload.github.com/26hzhang/SequenceToSequence/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/26hzhang%2FSequenceToSequence/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266561444,"owners_count":23948632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention-mechanism","dialogue","machine-translation","residual","seq2seq","tensorflow"],"created_at":"2024-11-28T21:09:05.479Z","updated_at":"2025-07-22T19:34:43.155Z","avatar_url":"https://github.com/26hzhang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sequence to Sequence Learning\n\n![](https://img.shields.io/badge/Python-3.6.5-brightgreen.svg) ![](https://img.shields.io/badge/Tensorflow-1.8.0-yellowgreen.svg)\n\nThis repository builds a sequence to sequence learning algorithm with attention mechanism, which aims to tackle some practical tasks such as simple dialogue, machine translation, pronounce to word and etc. This repo is implemented by tensorflow.\n\n### Usage\nBefore starting the experiment, you need to pull the data first (_use cornell dataset as example_):\n```bash\n$ cd dataset\n$ bash down_cornell.sh\n```\nIt will create a directory `raw/cornell`, and the downloaded raw data will be stored under this directory.  \n**Note**: other `.sh` data pullers will download and unzip data into `raw/` folder as a sub-directory with a specific name.\n\nThen go back tp the repository root, and execute the following commands to start a training or inference task:\n```bash\n$ cd ..\n$ python3 cornell_dialogue.py --mode train  # or decode if you have pretrained checkpoints\n```\nIt will cleanup the dataset, create vocabularies + train/test dataset indices and save the processed data to `dataset/data/cornell` directory (If the processed data already exists, will skip this process).  \nThen load the pre-setup configurations (you can change the configurations in the python file), and create the model and start a training session.\n```bash\nNo preprocessed dataset found, create from cornell raw data...\nRead cornell movie lines: 304713it [00:02, 128939.96it/s]\nRead cornell movie conversations: 83097it [00:01, 46060.47it/s]\nCreate cornell utterance pairs: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 83097/83097 [01:02\u003c00:00, 1319.20it/s]\nBuild vocabulary: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 158669/158669 [00:02\u003c00:00, 77018.89it/s]\nBuild dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 158669/158669 [00:01\u003c00:00, 89873.72it/s]\nLoad configurations...\nLoad dataset and create batches...\nPrepare train batches: 4711it [00:02, 2225.27it/s]\nPrepare test batches: 248it [00:00, 3951.25it/s]\nBuilding model...\nsource embedding shape: [None, None, 1024]\ntarget input embedding shape: [None, None, 1024]\nbi-directional rnn output shape: [None, None, 2048]\nencoder input projection shape: [None, None, 1024]\nencoder output shape: [None, None, 1024]\ndecoder rnn output shape: [None, None, 10004] (last dimension is vocab size)\nnumber of trainable parameters: 78197524.\nStart training...\nEpoch 1 / 60:\n   1/4711 [..............................] - ETA: 1468s - Global Step: 1 - Train Loss: 9.2197 - Perplexity: 10094.0631\n...\n```\n\n### Datasets\nList of datasets that the mode of this repository is able to handle.\n\n- [x] [Cornell Movie--Dialogs Corpus](http://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html).\n- [x] Twitter Chat, borrowed from [[marsan-ma/chat_corpus]](https://github.com/Marsan-Ma/chat_corpus/), with 700k lines tweets, where odd lines are tweets and even lines are responded tweets.\n- [x] [CMU Pronouncing Dictionary](http://svn.code.sf.net/p/cmusphinx/code/trunk/cmudict/cmudict-0.7b).\n- [ ] [IWSLT 2012 MT Track](http://hltc.cs.ust.hk/iwslt/index.php/evaluation-campaign/ted-task.html#MTtrack) dataset, _English-French translation_.\n- [x] [IWSLT Evaluation 2016 MT Track](https://sites.google.com/site/iwsltevaluation2016/mt-track) dataset, _English-French translation_.\n- [x] [Europarl](http://www.statmt.org) dataset, _English-French translation_, reference: [[How to Prepare a French-to-English Dataset for Machine Translation]](https://machinelearningmastery.com/prepare-french-english-dataset-machine-translation/).\n\n### Implementation List\n- [x] Build basic model.\n- [x] Add Bahdanau and Luong attention.\n- [x] Add dropout wrapper\n- [x] Add residual wrapper.\n- [x] Add learning rate decay.\n- [x] Add different training optimizers.\n- [x] Add bidirectional rnn for encoder.\n- [ ] Add sub-word module, ref: [[BPE]](https://github.com/rsennrich/subword-nmt).\n- [ ] Add GNMTAttentionMultiCell wrapper, ref: [[Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation]](https://arxiv.org/pdf/1609.08144.pdf). source: [[tensorflow/nmt/nmt/gnmt_model.py]](https://github.com/tensorflow/nmt/blob/master/nmt/gnmt_model.py).\n- [ ] Add BLEU measurement.\n\n### Reference\n- [tensorflow/nmt](https://github.com/tensorflow/nmt).\n- [suriyadeepan/practical_seq2seq](https://github.com/suriyadeepan/practical_seq2seq).\n- [JayParks/tf-seq2seq](https://github.com/JayParks/tf-seq2seq).\n- [google/seq2seq](https://github.com/google/seq2seq).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F26hzhang%2Fsequencetosequence","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F26hzhang%2Fsequencetosequence","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F26hzhang%2Fsequencetosequence/lists"}