{"id":19156504,"url":"https://github.com/kyegomez/griffin","last_synced_at":"2025-05-07T07:42:54.427Z","repository":{"id":225859607,"uuid":"767073364","full_name":"kyegomez/Griffin","owner":"kyegomez","description":"Implementation of Griffin from the paper: \"Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models\"","archived":false,"fork":false,"pushed_at":"2025-04-04T12:57:51.000Z","size":37965,"stargazers_count":52,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-14T13:21:29.397Z","etag":null,"topics":["ai","feedforward","ml","mlp","neural-network","pytorch","recurrent-networks","rnns","zeta"],"latest_commit_sha":null,"homepage":"https://discord.gg/GYbXvDGevY","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kyegomez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["kyegomez"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2024-03-04T16:51:37.000Z","updated_at":"2025-03-27T06:39:52.000Z","dependencies_parsed_at":"2024-03-04T18:15:57.019Z","dependency_job_id":"ac9406ec-bd2c-4efd-9351-834e76b1798b","html_url":"https://github.com/kyegomez/Griffin","commit_stats":null,"previous_names":["kyegomez/griffin"],"tags_count":0,"template":false,"template_full_name":"kyegomez/Python-Package-Template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FGriffin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FGriffin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FGriffin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FGriffin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kyegomez","download_url":"https://codeload.github.com/kyegomez/Griffin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252837039,"owners_count":21811828,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","feedforward","ml","mlp","neural-network","pytorch","recurrent-networks","rnns","zeta"],"created_at":"2024-11-09T08:34:44.791Z","updated_at":"2025-05-07T07:42:54.405Z","avatar_url":"https://github.com/kyegomez.png","language":"Python","funding_links":["https://github.com/sponsors/kyegomez"],"categories":[],"sub_categories":[],"readme":"[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)\n\n# Griffin\nImplementation of Griffin from the paper: \"Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models\". [PAPER LINK](https://huggingface.co/papers/2402.19427)\n\n\n## install\n`$ pip install griffin-torch`\n\n\n## usage\n```python\nimport torch\nfrom griffin_torch.main import Griffin\n\n# Forward pass\nx = torch.randint(0, 100, (1, 10))\n\n# Model\nmodel = Griffin(\n    dim=512,  # Dimension of the model\n    num_tokens=100,  # Number of tokens in the input\n    seq_len=10,  # Length of the input sequence\n    depth=8,  # Number of transformer blocks\n    mlp_mult=4,  # Multiplier for the hidden dimension in the MLPs\n    dropout=0.1,  # Dropout rate\n)\n\n# Forward pass\ny = model(x)\n\nprint(y)\n\n```\n\n\n\n# License\nMIT\n\n\n# Citation\n```\n@misc{de2024griffin,\n    title={Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models}, \n    author={Soham De and Samuel L. Smith and Anushan Fernando and Aleksandar Botev and George Cristian-Muraru and Albert Gu and Ruba Haroun and Leonard Berrada and Yutian Chen and Srivatsan Srinivasan and Guillaume Desjardins and Arnaud Doucet and David Budden and Yee Whye Teh and Razvan Pascanu and Nando De Freitas and Caglar Gulcehre},\n    year={2024},\n    eprint={2402.19427},\n    archivePrefix={arXiv},\n    primaryClass={cs.LG}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Fgriffin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyegomez%2Fgriffin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Fgriffin/lists"}