{"id":35286242,"url":"https://github.com/dreamgonfly/transformer-pytorch","last_synced_at":"2026-04-01T20:20:21.183Z","repository":{"id":62585176,"uuid":"149839228","full_name":"dreamgonfly/transformer-pytorch","owner":"dreamgonfly","description":"A PyTorch implementation of Transformer in \"Attention is All You Need\"","archived":false,"fork":false,"pushed_at":"2020-12-06T07:55:18.000Z","size":2898,"stargazers_count":106,"open_issues_count":3,"forks_count":30,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-01-03T04:56:10.435Z","etag":null,"topics":["deep-learning","machine-translation","natural-language-processing"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1706.03762","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dreamgonfly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-22T02:43:29.000Z","updated_at":"2025-05-27T07:41:34.000Z","dependencies_parsed_at":"2022-11-03T22:03:26.139Z","dependency_job_id":null,"html_url":"https://github.com/dreamgonfly/transformer-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dreamgonfly/transformer-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamgonfly%2Ftransformer-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamgonfly%2Ftransformer-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamgonfly%2Ftransformer-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamgonfly%2Ftransformer-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dreamgonfly","download_url":"https://codeload.github.com/dreamgonfly/transformer-pytorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamgonfly%2Ftransformer-pytorch/sbom","scorecard":{"id":355634,"data":{"date":"2025-08-11","repo":{"name":"github.com/dreamgonfly/transformer-pytorch","commit":"c31e163ed57321e405771ef7fb556d4d92fd5efb"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-18T09:30:52.764Z","repository_id":62585176,"created_at":"2025-08-18T09:30:52.764Z","updated_at":"2025-08-18T09:30:52.764Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31018657,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-27T03:51:26.850Z","status":"ssl_error","status_checked_at":"2026-03-27T03:51:09.693Z","response_time":164,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-translation","natural-language-processing"],"created_at":"2025-12-30T15:30:26.300Z","updated_at":"2026-03-27T04:25:36.686Z","avatar_url":"https://github.com/dreamgonfly.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Transformer-pytorch\nA PyTorch implementation of Transformer in \"Attention is All You Need\" (https://arxiv.org/abs/1706.03762)\n\nThis repo focuses on clean, readable, and modular implementation of the paper.\n\n\u003cimg width=\"559\" alt=\"screen shot 2018-09-27 at 1 49 14 pm\" src=\"https://user-images.githubusercontent.com/2340721/46123973-44b08900-c25c-11e8-9468-7aef9e4e3f18.png\"\u003e\n\n## Requirements\n- Python 3.6+\n- [PyTorch 4.1+](http://pytorch.org/)\n- [NumPy](http://www.numpy.org/)\n- [NLTK](https://www.nltk.org/)\n- [tqdm](https://github.com/tqdm/tqdm)\n\n## Usage\n\n### Prepare datasets\nThis repo comes with example data in `data/` directory. To begin, you will need to prepare datasets with given data as follows:\n```\n$ python prepare_datasets.py --train_source=data/example/raw/src-train.txt --train_target=data/example/raw/tgt-train.txt --val_source=data/example/raw/src-val.txt --val_target=data/example/raw/tgt-val.txt --save_data_dir=data/example/processed\n```\n\nThe example data is brought from [OpenNMT-py](https://github.com/OpenNMT/OpenNMT-py).\nThe data consists of parallel source (src) and target (tgt) data for training and validation.\nA data file contains one sentence per line with tokens separated by a space.\nBelow are the provided example data files.\n\n- `src-train.txt`\n- `tgt-train.txt`\n- `src-val.txt`\n- `tgt-val.txt`\n\n### Train model\nTo train model, provide the train script with a path to processed data and save files as follows:\n\n```\n$ python train.py --data_dir=data/example/processed --save_config=checkpoints/example_config.json --save_checkpoint=checkpoints/example_model.pth --save_log=logs/example.log \n```\n\nThis saves model config and checkpoints to given files, respectively.\nYou can play around with hyperparameters of the model with command line arguments. \nFor example, add `--epochs=300` to set the number of epochs to 300. \n\n### Translate\nTo translate a sentence in source language to target language:\n```\n$ python predict.py --source=\"There is an imbalance here .\" --config=checkpoints/example_config.json --checkpoint=checkpoints/example_model.pth\n\nCandidate 0 : Hier fehlt das Gleichgewicht .\nCandidate 1 : Hier fehlt das das Gleichgewicht .\nCandidate 2 : Hier fehlt das das das Gleichgewicht .\n```\n\nIt will give you translation candidates of the given source sentence.\nYou can adjust the number of candidates with command line argument. \n\n### Evaluate\nTo calculate BLEU score of a trained model:\n```\n$ python evaluate.py --save_result=logs/example_eval.txt --config=checkpoints/example_config.json --checkpoint=checkpoints/example_model.pth\n\nBLEU score : 0.0007947\n```\n\n## File description\n- `models.py` includes Transformer's encoder, decoder, and multi-head attention.\n- `embeddings.py` contains positional encoding.\n- `losses.py` contains label smoothing loss.\n- `optimizers.py` contains Noam optimizer.\n- `metrics.py` contains accuracy metric.\n- `beam.py` contains beam search.\n- `datasets.py` has code for loading and processing data. \n- `trainer.py` has code for training model.\n- `prepare_datasets.py` processes data.\n- `train.py` trains model.\n- `predict.py` translates given source sentence with a trained model.\n- `evaluate.py` calculates BLEU score of a trained model.\n\n## Reference\n- [OpenNMT-py](https://github.com/OpenNMT/OpenNMT-py)\n\n## Author\n[@dreamgonfly](https://github.com/dreamgonfly)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdreamgonfly%2Ftransformer-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdreamgonfly%2Ftransformer-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdreamgonfly%2Ftransformer-pytorch/lists"}