{"id":19669914,"url":"https://github.com/dohlee/tranception-pytorch","last_synced_at":"2025-04-29T00:31:17.404Z","repository":{"id":115192346,"uuid":"605171998","full_name":"dohlee/tranception-pytorch","owner":"dohlee","description":"Implementation of Tranception, a SOTA transformer model for protein fitness prediction, in PyTorch.","archived":false,"fork":false,"pushed_at":"2023-03-18T14:43:25.000Z","size":170,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-04T20:36:11.143Z","etag":null,"topics":["bioinformatics","biology","computational-biology","deep-learning","protein-sequences","reproduction","reproduction-code"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dohlee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-02-22T15:45:50.000Z","updated_at":"2024-08-22T07:54:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"8827cdb8-0567-48fb-a519-14a3218e87fd","html_url":"https://github.com/dohlee/tranception-pytorch","commit_stats":{"total_commits":24,"total_committers":1,"mean_commits":24.0,"dds":0.0,"last_synced_commit":"a9bb41e51caa086d0be8aaf6f40edf77a246ba4d"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dohlee%2Ftranception-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dohlee%2Ftranception-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dohlee%2Ftranception-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dohlee%2Ftranception-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dohlee","download_url":"https://codeload.github.com/dohlee/tranception-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224142852,"owners_count":17262884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","biology","computational-biology","deep-learning","protein-sequences","reproduction","reproduction-code"],"created_at":"2024-11-11T17:03:07.778Z","updated_at":"2024-11-11T17:03:08.337Z","avatar_url":"https://github.com/dohlee.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tranception-pytorch\n\n![mode](img/banner.png)\n\nImplementation of Tranception, a SOTA transformer model for protein fitness prediction, in PyTorch. The official implementation of tranception can be found [here](https://github.com/OATML-Markslab/Tranception).\n\n## Installation\n\n```bash\n$ pip install tranception-pytorch-dohlee\n```\n\n## Usage\n\n```python\nimport torch\nfrom tranception_pytorch import Tranception\n\n# Parameters for Tranception S\nnum_heads = 12\nnum_layers = 12\nembed_dim = 768\nmax_length = 1024\n\nmodel = Tranception(\n    embed_dim=embed_dim,\n    num_heads=num_heads,\n    num_layers=num_layers,\n    max_length=max_length,\n)\nmodel = model.cuda()\n\nx = torch.randn([1, 1000])  # batch, seq_len (encoded as integer in [0, 21], 21 for [MASK])\n\n# Autoregressive inference\nout = model.log_likelihood(x)\n```\n\n## Reproduction status\n\nLive training metrics for Tranception S model using UniRef50 can be found [here](https://wandb.ai/dohlee/tranception-pytorch/reports/Tranception-training--VmlldzozODIyODI0).\n\n## Citation\n```bibtex\n@inproceedings{notin2022tranception,\n  title={Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval},\n  author={Notin, Pascal and Dias, Mafalda and Frazer, Jonathan and Hurtado, Javier Marchena and Gomez, Aidan N and Marks, Debora and Gal, Yarin},\n  booktitle={International Conference on Machine Learning},\n  pages={16990--17017},\n  year={2022},\n  organization={PMLR}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdohlee%2Ftranception-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdohlee%2Ftranception-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdohlee%2Ftranception-pytorch/lists"}