{"id":31780641,"url":"https://github.com/ashly1991/transformer-nmt-tf2","last_synced_at":"2026-05-14T20:05:53.557Z","repository":{"id":316242564,"uuid":"1062630638","full_name":"Ashly1991/transformer-nmt-tf2","owner":"Ashly1991","description":"Transformer neural machine translation in TensorFlow 2 with tensorflow-text; tutorial-based build with masks, positional encodings, and experiments.","archived":false,"fork":false,"pushed_at":"2025-09-23T14:14:01.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-23T14:42:42.583Z","etag":null,"topics":["attention","encoder-decoder","jupyter-notebook","neural-machine-translation","nmt","tensorflow","tensorflow-text","transformer"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ashly1991.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-23T14:07:12.000Z","updated_at":"2025-09-23T14:14:04.000Z","dependencies_parsed_at":"2025-09-23T14:42:45.081Z","dependency_job_id":null,"html_url":"https://github.com/Ashly1991/transformer-nmt-tf2","commit_stats":null,"previous_names":["ashly1991/transformer-nmt-tf2"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Ashly1991/transformer-nmt-tf2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ashly1991%2Ftransformer-nmt-tf2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ashly1991%2Ftransformer-nmt-tf2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ashly1991%2Ftransformer-nmt-tf2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ashly1991%2Ftransformer-nmt-tf2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ashly1991","download_url":"https://codeload.github.com/Ashly1991/transformer-nmt-tf2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ashly1991%2Ftransformer-nmt-tf2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279003304,"owners_count":26083555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","encoder-decoder","jupyter-notebook","neural-machine-translation","nmt","tensorflow","tensorflow-text","transformer"],"created_at":"2025-10-10T08:16:55.471Z","updated_at":"2025-10-10T08:16:57.725Z","avatar_url":"https://github.com/Ashly1991.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Transformer Neural Machine Translation (TensorFlow 2)\n\nSelf-contained implementation of a **Transformer** for **neural machine translation** in TensorFlow 2. The project covers tokenization, positional encodings, masking (padding \u0026 look-ahead), multi-head self-attention, encoder–decoder stacks, training, and inference/decoding.\n\n## Highlights\n- **Tokenization \u0026 vocab** (with `tensorflow-text`) and **positional encodings**.\n- **Masks**: padding mask for loss/attention; look-ahead mask for the decoder.\n- **Transformer blocks**: scaled dot-product attention, **multi-head attention**, FFN, residuals + layer norm.\n- **Training loop** with cross-entropy + accuracy; masked loss to ignore padding.\n- **Inference** (greedy by default; extendable to beam search).\n- **Reproducibility**: seeds set in the notebook; notes on deterministic decoding.\n\n## What I learned (from this build)\n- How a Transformer uses self-attention, multi-head attention, and positional encodings to model sequences.  \n- Why positional encodings are needed (attention is permutation-invariant).  \n- How masks (padding \u0026 look-ahead) affect attention and the loss.  \n- Encoder–decoder structure; **teacher forcing** at training vs **auto-regressive** decoding at inference.  \n- Practical setup (tokenization, vocabularies, training loop, decoding).\n\n## How to run\n```bash\npython -m venv .venv \u0026\u0026 source .venv/bin/activate   # Windows: .venv\\Scripts\\activate\npip install -r requirements.txt\njupyter lab transformer-nmt.ipynb\n```\n\n## Requirements\nPinned for stability/performance with this implementation:\n```\ntensorflow==2.14.0\ntensorflow-text==2.14.0\ntensorflow-datasets\nnumpy\nmatplotlib\njupyterlab\n```\n\n## Notes\n- Greedy decoding is deterministic with fixed weights and dropout disabled.  \n  To ensure repeatable translations, set seeds and avoid sampling at inference.  \n- Swap tokenizers/vocabs + final projection size to use a different language pair; core architecture stays the same.\n\n## License\nMIT — see `LICENSE`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashly1991%2Ftransformer-nmt-tf2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashly1991%2Ftransformer-nmt-tf2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashly1991%2Ftransformer-nmt-tf2/lists"}