{"id":13780287,"url":"https://github.com/google-deepmind/recurrentgemma","last_synced_at":"2026-02-06T14:27:26.253Z","repository":{"id":232321419,"uuid":"777181107","full_name":"google-deepmind/recurrentgemma","owner":"google-deepmind","description":"Open weights language model from Google DeepMind, based on Griffin.","archived":false,"fork":false,"pushed_at":"2026-01-22T16:37:08.000Z","size":96,"stargazers_count":662,"open_issues_count":2,"forks_count":33,"subscribers_count":18,"default_branch":"main","last_synced_at":"2026-02-04T13:59:17.470Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-03-25T11:13:36.000Z","updated_at":"2026-02-03T02:39:01.000Z","dependencies_parsed_at":"2024-04-11T15:39:44.411Z","dependency_job_id":"a5c2569f-c3df-4a5a-9e08-1cd087436668","html_url":"https://github.com/google-deepmind/recurrentgemma","commit_stats":null,"previous_names":["google-deepmind/recurrentgemma"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/recurrentgemma","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Frecurrentgemma","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Frecurrentgemma/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Frecurrentgemma/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Frecurrentgemma/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/recurrentgemma/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Frecurrentgemma/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29164854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T12:44:37.655Z","status":"ssl_error","status_checked_at":"2026-02-06T12:44:13.991Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T18:01:14.146Z","updated_at":"2026-02-06T14:27:26.231Z","avatar_url":"https://github.com/google-deepmind.png","language":"Python","readme":"# RecurrentGemma\n\nRecurrentGemma is a family of open-weights Language Models by [Google DeepMind](https://deepmind.google/), based on the novel [Griffin architecture](https://arxiv.org/abs/2402.19427). This architecture achieves fast inference when generating long sequences by replacing global attention with a mixture of local attention and linear recurrences.\n\nThis repository contains the model implementation and examples for sampling and fine-tuning. We recommend most users adopt the [Flax](https://github.com/google/flax) implementation, which is highly optimized. We also provide an un-optimized [PyTorch](https://github.com/pytorch/pytorch) implementation for reference.\n\n### Learn more about RecurrentGemma\n\n-   The [RecurrentGemma technical report](https://storage.googleapis.com/deepmind-media/gemma/recurrentgemma-report.pdf) gives specific details on the training and evaluation of RecurrentGemma.\n-   The [Griffin paper](https://arxiv.org/abs/2402.19427) describes the underlying model architecture.\n\n## Quick start\n\n### Installation\n\n#### Using Poetry\nRecurrentGemma uses [Poetry](https://python-poetry.org/docs/) for dependency\nmanagement.\n\nTo install dependencies for the full project:\n* Checkout the code.\n* `poetry install -E full` to create a virtual environment with all dependencies.\n* `poetry shell` to activate the created virtual environment.\n\nIf you only need to install a subset of dependencies use one of the alternative\nlibrary-specific commands below.\n\n#### Using pip\nIf you want to use `pip` instead of Poetry, \nthen create a virtual environment (run `python -m venv recurrentgemma-demo` and `. recurrentgemma-demo/bin/activate`) and:\n\n* Checkout the code.\n* `pip install .[full]`\n\n#### Installing library-specific packages\n\n##### JAX\nTo install dependencies only for the JAX pathway use:\n`poetry install -E jax` or (`pip install .[jax]`).\n\n##### PyTorch\nTo install dependencies only for the PyTorch pathway use:\n`poetry install -E torch` (or `pip install .[torch]`).\n\n##### Tests\nTo install dependencies required for running unit tests use:\n`poetry install -E test` (or `pip install .[test]`)\n\n### Downloading the models\n\nThe model checkpoints are available through Kaggle at\nhttp://kaggle.com/models/google/recurrentgemma.\nSelect either the **Flax** or **PyTorch** model variations, click the ⤓ button\nto download the model archive, then extract the contents to a local directory.\n\nIn both cases, the archive contains both the model weights and\nthe tokenizer.\n\n### Running the unit tests\n\nTo run the tests, install the optional `[test]` dependencies (e.g. using `pip install .[test]`) from the root of the source tree, then:\n\n```\npytest .\n```\n\n## Examples\n\nTo run the example sampling script, pass the paths to the weights directory and tokenizer:\n\n```\npython examples/sampling_jax.py \\\n  --path_checkpoint=/path/to/archive/contents/2b/ \\\n  --path_tokenizer=/path/to/archive/contents/tokenizer.model\n```\n\n### Colab notebook tutorials\n\n-   [`colabs/sampling_tutorial_jax.ipynb`](https://colab.sandbox.google.com/github/google-deepmind/recurrentgemma/blob/main/colabs/sampling_tutorial_jax.ipynb)\n    contains a [Colab](http://colab.google) notebook with a sampling example using JAX.\n\n-   [`colabs/sampling_tutorial_pytorch.ipynb`](https://colab.sandbox.google.com/github/google-deepmind/recurrentgemma/blob/main/colabs/sampling_tutorial_pytorch.ipynb)\n    contains a [Colab](http://colab.google) notebook with a sampling example using PyTorch.\n\n-   [`colabs/fine_tuning_tutorial_jax.ipynb`](https://colab.sandbox.google.com/github/google-deepmind/recurrentgemma/blob/main/colabs/fine_tuning_tutorial_jax.ipynb)\n    contains a [Colab](http://colab.google) with a basic tutorial on how to\n    fine-tune RecurrentGemma for a task, such as English to French translation, using JAX.\n\nTo run these notebooks you will need to have a Kaggle account and first read and accept\nthe Gemma license terms and conditions from the [RecurrentGemma page](http://kaggle.com/models/google/recurrentgemma).\nAfter this you can run the notebooks, which will automatically download the weights and tokenizer from there.\n\nCurrently different notebooks are supported under the following hardware:\n\n| Hardware            | T4  | P100 | V100 | A100 | TPUv2 | TPUv3+ |\n|---------------------|:---:|:----:|:----:|:----:|:-----:|:------:|\n| Sampling in Jax     | ✅  | ✅   | ✅   | ✅   | ✅    | ✅    |\n| Sampling in PyTorch | ✅  | ✅   | ✅   | ✅   | ✅    | ✅    |\n| Finetuning in Jax   | ✅  | ✅   | ✅   | ✅   | ❌    | ✅    |\n\n\n## System Requirements\n\nRecurrentGemma code can run on CPU, GPU or TPU.\nThe code has been optimized for running on TPU using the Flax implementation,\nwhich contains a low level [Pallas](https://jax.readthedocs.io/en/latest/pallas/index.html) kernel to perform the linear scan in the recurrent layers.\n\n## Contributing\n\nWe are open to bug reports and issues. Please see\n[CONTRIBUTING.md](CONTRIBUTING.md) for details on PRs.\n\n## License\n\nCopyright 2024 DeepMind Technologies Limited\n\nThis code is licensed under the Apache License, Version 2.0 (the \\\"License\\\");\nyou may not use this file except in compliance with the License. You may obtain\na copy of the License at http://www.apache.org/licenses/LICENSE-2.0.\n\nUnless required by applicable law or agreed to in writing, software distributed\nunder the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR\nCONDITIONS OF ANY KIND, either express or implied. See the License for the\nspecific language governing permissions and limitations under the License.\n\n## Disclaimer\n\nThis is not an official Google product.\n","funding_links":[],"categories":["Open LLM","Uncategorized","Models","Python"],"sub_categories":["Uncategorized","Open Models"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Frecurrentgemma","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Frecurrentgemma","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Frecurrentgemma/lists"}