{"id":21529603,"url":"https://github.com/sooftware/attentions","last_synced_at":"2025-04-05T17:05:34.243Z","repository":{"id":37422201,"uuid":"249026882","full_name":"sooftware/attentions","owner":"sooftware","description":"PyTorch implementation of some attentions for Deep Learning Researchers. ","archived":false,"fork":false,"pushed_at":"2022-03-04T07:30:05.000Z","size":82,"stargazers_count":529,"open_issues_count":3,"forks_count":70,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T16:04:38.544Z","etag":null,"topics":["additive-attention","attention","dot-product-attention","location-aware-attention","location-sensitive-attension","multi-head-attention","pytorch","relative-multi-head-attention","relative-positional-encoding"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sooftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-21T17:31:05.000Z","updated_at":"2025-03-22T18:46:01.000Z","dependencies_parsed_at":"2022-07-14T04:20:28.938Z","dependency_job_id":null,"html_url":"https://github.com/sooftware/attentions","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Fattentions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Fattentions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Fattentions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sooftware%2Fattentions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sooftware","download_url":"https://codeload.github.com/sooftware/attentions/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247369952,"owners_count":20927928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["additive-attention","attention","dot-product-attention","location-aware-attention","location-sensitive-attension","multi-head-attention","pytorch","relative-multi-head-attention","relative-positional-encoding"],"created_at":"2024-11-24T01:58:20.103Z","updated_at":"2025-04-05T17:05:34.224Z","avatar_url":"https://github.com/sooftware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp  align=\"center\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/42150335/105606759-8dea5680-5dde-11eb-96a0-78f632a926c2.png\" height=100\u003e\n  \n\u003cp  align=\"center\"\u003eAn Apache 2.0 PyTorch implementation of some attentions for Deep Learning Researchers.\n  \n***\n  \n\u003cp  align=\"center\"\u003e \n     \u003ca href=\"https://www.codefactor.io/repository/github/sooftware/attentions\"\u003e\n          \u003cimg src=\"https://www.codefactor.io/repository/github/sooftware/attentions/badge\"\u003e \n     \u003c/a\u003e\n     \u003ca href=\"https://github.com/sooftware/KoSpeech/blob/latest/LICENSE\"\u003e\n          \u003cimg src=\"http://img.shields.io/badge/license-Apache--2.0-informational\"\u003e \n     \u003c/a\u003e\n     \u003ca href=\"https://github.com/pytorch/pytorch\"\u003e\n          \u003cimg src=\"http://img.shields.io/badge/framework-PyTorch-informational\"\u003e \n     \u003c/a\u003e\n     \u003ca href=\"https://www.python.org/dev/peps/pep-0008/\"\u003e\n          \u003cimg src=\"http://img.shields.io/badge/codestyle-PEP--8-informational\"\u003e \n     \u003c/a\u003e\n  \n  \n## Intro\n  \n`attentions` provides some attentions used in natural language processing using pytorch.   \nthese attentions can used in neural machine translation, speech recognition, image captioning etc...  \n  \n![image](https://user-images.githubusercontent.com/42150335/83331902-7bf9f780-a2d3-11ea-8f7e-172f55deef45.png)\n  \n`attention` allows to attend to different parts of the source sentence at each step of the output generation.   \nInstead of encoding the input sequence into a single fixed context vector, we let the model learn how to generate a context vector for each output time step.  \n  \n## Implementation list\n  \n \n|Name|Citation|  \n|---|---|  \n|Additive Attention|[Bahdanau et al., 2015](https://arxiv.org/pdf/1409.0473.pdf)|  \n|Dot-Product Attention|[Luong et al., 2015](https://arxiv.org/pdf/1508.04025.pdf)|  \n|Location-Aware (Location Sensitive) Attention|[Chorowski et al., 2015](http://papers.nips.cc/paper/5847-attention-based-models-for-speech-recognition.pdf)|    \n|Scaled Dot-Product Attention|[Vaswani et al., 2017](https://arxiv.org/abs/1706.03762)|  \n|Multi-Head Attention|[Vaswani et al., 2017](https://arxiv.org/abs/1706.03762)|\n|Relative Multi-Head Self Attention|[ZihangDai et al., 2019](https://arxiv.org/abs/1901.02860)|  \n  \n  \n## Troubleshoots and Contributing\nIf you have any questions, bug reports, and feature requests, please [open an issue](https://github.com/sooftware/nlp-attentions/issues) on Github.  \nor Contacts sh951011@gmail.com please.\n  \nI appreciate any kind of feedback or contribution.  Feel free to proceed with small issues like bug fixes, documentation improvement.  For major contributions and new features, please discuss with the collaborators in corresponding issues.  \n  \n## Code Style\nI follow [PEP-8](https://www.python.org/dev/peps/pep-0008/) for code style. Especially the style of docstrings is important to generate documentation.  \n  \n## Author\n  \n* Soohwan Kim [@sooftware](https://github.com/sooftware)\n* Contacts: sh951011@gmail.com\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsooftware%2Fattentions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsooftware%2Fattentions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsooftware%2Fattentions/lists"}