{"id":30523918,"url":"https://github.com/lightonai/rita","last_synced_at":"2025-08-26T20:52:06.243Z","repository":{"id":40303049,"uuid":"489272326","full_name":"lightonai/RITA","owner":"lightonai","description":"RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.","archived":false,"fork":false,"pushed_at":"2023-01-24T10:02:41.000Z","size":224,"stargazers_count":74,"open_issues_count":1,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2023-05-14T23:05:15.427Z","etag":null,"topics":["deep-learning","generative-models","protein-design"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lightonai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-06T08:15:58.000Z","updated_at":"2023-05-11T15:50:06.000Z","dependencies_parsed_at":"2023-02-13T20:01:56.478Z","dependency_job_id":null,"html_url":"https://github.com/lightonai/RITA","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/lightonai/RITA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2FRITA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2FRITA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2FRITA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2FRITA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lightonai","download_url":"https://codeload.github.com/lightonai/RITA/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightonai%2FRITA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272254507,"owners_count":24901057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","generative-models","protein-design"],"created_at":"2025-08-26T20:51:58.516Z","updated_at":"2025-08-26T20:52:06.224Z","avatar_url":"https://github.com/lightonai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cimg src=\"_static/lighton_small.png\" width=60/\u003e RITA: a Study on Scaling Up Generative Protein Sequence Models\n\n[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)  [![Twitter](https://img.shields.io/twitter/follow/LightOnIO?style=social)](https://twitter.com/LightOnIO)\n\nRITA is a family of autoregressive protein models, developed by a collaboration of [Lighton](https://lighton.ai/), the [OATML group](https://oatml.cs.ox.ac.uk/) at Oxford, and the [Debbie Marks Lab](https://www.deboramarkslab.com/) at Harvard. \n\nModel | #Params | d_model | layers | lm loss uniref-100\n--- | --- | --- | --- | --- | \n[Small](https://huggingface.co/lightonai/RITA_s) | 85M  | 768 | 12 | 2.31\n[Medium](https://huggingface.co/lightonai/RITA_m) | 300M | 1024 | 24 | 2.01\n[Large](https://huggingface.co/lightonai/RITA_l)| 680M | 1536 | 24 | 1.82\n[XLarge](https://huggingface.co/lightonai/RITA_xl)| 1.2B | 2048 | 24 | 1.70 \n\n## Results\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"_static/perplexity.png\" width=800/\u003e\n\u003c/p\u003e\n\nFor full results see our preprint: https://arxiv.org/abs/2205.05789\n## Usage \nInstantiate a model like so:\n``` python\nfrom transformers import AutoModel, AutoModelForCausalLM\nmodel = AutoModelForCausalLM.from_pretrained(\"lightonai/RITA_s\", trust_remote_code=True)\ntokenizer = AutoTokenizer.from_pretrained(\"lightonai/RITA_s\")\n```\nfor generation we support pipelines:\n``` python\nfrom transformers import pipeline\nrita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)\nsequences = rita_gen(\"MAB\", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, \n                     num_return_sequences=2, eos_token_id=2)\nfor seq in sequences:\n    print(f\"seq: {seq['generated_text'].replace(' ', '')}\")\n```\nOr see `example.py`\n\n## How to cite    \n\n    @article{hesslow2022rita,\n      title={RITA: a Study on Scaling Up Generative Protein Sequence Models},\n      author={Hesslow, Daniel and Zanichelli, Niccol{\\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},\n      journal={arXiv preprint arXiv:2205.05789},\n      year={2022}\n    }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightonai%2Frita","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightonai%2Frita","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightonai%2Frita/lists"}