{"id":15009429,"url":"https://github.com/a-jacobson/minimal-nmt","last_synced_at":"2025-04-09T17:14:27.015Z","repository":{"id":89564682,"uuid":"124620711","full_name":"A-Jacobson/minimal-nmt","owner":"A-Jacobson","description":"A minimal nmt example to serve as an seq2seq+attention reference.","archived":false,"fork":false,"pushed_at":"2019-09-23T19:07:40.000Z","size":241,"stargazers_count":36,"open_issues_count":0,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-09T17:14:17.810Z","etag":null,"topics":["attention-mechanism","beam-search","neural-machine-translation","python-3-6","pytorch","seq2seq","tutorial"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/A-Jacobson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-10T04:04:04.000Z","updated_at":"2022-01-02T13:10:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"9ccc5904-02f7-4a63-859c-7d52cd6bb37b","html_url":"https://github.com/A-Jacobson/minimal-nmt","commit_stats":{"total_commits":7,"total_committers":3,"mean_commits":"2.3333333333333335","dds":0.4285714285714286,"last_synced_commit":"dc75e83579a181586acabfa3f22ad269d1e31fbf"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/A-Jacobson%2Fminimal-nmt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/A-Jacobson%2Fminimal-nmt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/A-Jacobson%2Fminimal-nmt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/A-Jacobson%2Fminimal-nmt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/A-Jacobson","download_url":"https://codeload.github.com/A-Jacobson/minimal-nmt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248074925,"owners_count":21043490,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention-mechanism","beam-search","neural-machine-translation","python-3-6","pytorch","seq2seq","tutorial"],"created_at":"2024-09-24T19:25:13.660Z","updated_at":"2025-04-09T17:14:26.990Z","avatar_url":"https://github.com/A-Jacobson.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Minimal Neural Machine Translation\n![im](assets/encoder-decoder.png)\n\n## Resources\nNEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE\nhttps://arxiv.org/pdf/1409.0473.pdf\n\nEffective Approaches to Attention-based Neural Machine Translation\nhttps://arxiv.org/pdf/1508.04025.pdf\n\nMassive Exploration of Neural Machine Translation Architectures\nhttps://arxiv.org/pdf/1703.03906.pdf\n\n\n## Contents\n- Encoder --\u003e Attention --\u003e Decoder Architecture.\n- Luong Attention.\n- Training on Multi30k German to English translation task.\n- Attention Visualization.\n- Teacher Forcing.\n- Greedy Decoding.\n- [nmt tutorial notebook](https://github.com/A-Jacobson/minimal-nmt/blob/master/nmt_tutorial.ipynb)\n- minimal beam search decoding.\n\n## Setup\n\n1. install pytorch 0.4:\n```\nconda install pytorch -c pytorch=0.4.1 \n```\n\n2. install other requirements:\n```\npip install -r requirements.txt\n```\n## System Requirements\nTraining with a batch size of 32 takes ~3gb GPU ram.\nIf this is too much, lower the batch size or reduce network dimensionality in `hyperparams.py`.\n\n## Usage\n```\npython train.py\n```\n\nview logs in Tensorboard\ndecent alignments should be seen after 2-3 epochs.\n```\ntensorboard --logdir runs\n```\n(partially trained attention heatmap)\n\n![img](assets/attention_vis.png)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa-jacobson%2Fminimal-nmt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fa-jacobson%2Fminimal-nmt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa-jacobson%2Fminimal-nmt/lists"}