{"id":21529579,"url":"https://github.com/sooftware/transformer","last_synced_at":"2025-10-09T20:40:26.976Z","repository":{"id":48163222,"uuid":"277098969","full_name":"sooftware/transformer","owner":"sooftware","description":"A PyTorch Implementation of \"Attention Is All You Need\"","archived":false,"fork":false,"pushed_at":"2021-10-03T08:30:53.000Z","size":6577,"stargazers_count":38,"open_issues_count":0,"forks_count":9,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-09T23:51:36.752Z","etag":null,"topics":["attention","attention-is-all-you-need","nlp","seq2seq","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sooftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-04T11:44:01.000Z","updated_at":"2024-09-05T04:26:06.000Z","dependencies_parsed_at":"2022-08-27T20:11:49.634Z","dependency_job_id":null,"html_url":"https://github.com/sooftware/transformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sooftware/transformer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Ftransformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Ftransformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Ftransformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Ftransformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sooftware","download_url":"https://codeload.github.com/sooftware/transformer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Ftransformer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001982,"owners_count":26083259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","attention-is-all-you-need","nlp","seq2seq","transformer"],"created_at":"2024-11-24T01:58:11.994Z","updated_at":"2025-10-09T20:40:26.961Z","avatar_url":"https://github.com/sooftware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# transformer\n\nA PyTorch Implementation of Transformer in [Attention Is All You Need](https://arxiv.org/abs/1706.03762).  \nThis repository focused on implementing the contents of the paper as much as possible.\n  \n## Intro \n  \n\u003cimg src=\"https://tutorials.pytorch.kr/_images/transformer_architecture.jpg\" height=500\u003e  \n  \nThis repository focused on implementing the contents of the paper as much as possible,   \nwhile at the same time striving for a readable code. To improve readability,      \nI designed the model structure to fit as much as possible to the blocks in the above Transformers figure.\n  \n## Installation\nThis project recommends Python 3.7 or higher.\nWe recommend creating a new virtual environment for this project (using virtual env or conda).\n  \n### Prerequisites\n* Numpy: `pip install numpy` (Refer [here](https://github.com/numpy/numpy) for problem installing Numpy).\n* Pytorch: Refer to [PyTorch website](http://pytorch.org/) to install the version w.r.t. your environment.  \n  \n### Install from source\nCurrently we only support installation from source code using setuptools. Checkout the source code and run the\nfollowing commands:  \n  \n```\npip install -e .\n```\n\n## Usage\n\n```python\nimport torch\nimport torch.nn as nn\nfrom transformer import Transformer\n\nBATCH_SIZE, SEQ_LENGTH, D_MODEL = 3, 10, 64\n\ncuda = torch.cuda.is_available()  \ndevice = torch.device('cuda' if cuda else 'cpu')\n\ninputs = torch.zeros(BATCH_SIZE, SEQ_LENGTH).long().to(device)\ninput_lengths = torch.LongTensor([12345, 12300, 12000])\ntargets = torch.LongTensor([[1, 3, 3, 3, 3, 3, 4, 5, 6, 2],\n                            [1, 3, 3, 3, 3, 3, 4, 5, 2, 0],\n                            [1, 3, 3, 3, 3, 3, 4, 2, 0, 0]]).to(device)\ntarget_lengths = torch.LongTensor([9, 8, 7])\n\nmodel = nn.DataParallel(Transformer(num_input_embeddings=30, num_output_embeddings=50, \n                                    d_model=64, \n                                    num_encoder_layers=3, num_decoder_layers=3)).to(device)\n\n# Forward propagate\noutputs = model(inputs, input_lengths, targets, target_lengths)\n\n# Inference\noutputs = model(inputs, input_lengths)\n```\n\n## Troubleshoots and Contributing\nIf you have any questions, bug reports, and feature requests, please [open an issue](https://github.com/sooftware/conformer/issues) on github or   \ncontacts sh951011@gmail.com please.\n  \nI appreciate any kind of feedback or contribution.  Feel free to proceed with small issues like bug fixes, documentation improvement.  For major contributions and new features, please discuss with the collaborators in corresponding issues.  \n  \n## Code Style\nI follow [PEP-8](https://www.python.org/dev/peps/pep-0008/) for code style. Especially the style of docstrings is important to generate documentation.\n\n## Author\n  \n* Soohwan Kim [@sooftware](https://github.com/sooftware)\n* Contacts: sh951011@gmail.com","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsooftware%2Ftransformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsooftware%2Ftransformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsooftware%2Ftransformer/lists"}