{"id":13441252,"url":"https://github.com/xai-org/grok-1","last_synced_at":"2025-05-14T11:08:14.297Z","repository":{"id":228249441,"uuid":"773286980","full_name":"xai-org/grok-1","owner":"xai-org","description":"Grok open release","archived":false,"fork":false,"pushed_at":"2024-08-30T04:17:25.000Z","size":1008,"stargazers_count":50240,"open_issues_count":119,"forks_count":8355,"subscribers_count":628,"default_branch":"main","last_synced_at":"2025-05-07T10:52:32.855Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xai-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-17T08:53:38.000Z","updated_at":"2025-05-07T10:19:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"6b03935c-b8b3-447f-8b17-dca67cec2363","html_url":"https://github.com/xai-org/grok-1","commit_stats":null,"previous_names":["xai-org/grok"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xai-org%2Fgrok-1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xai-org%2Fgrok-1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xai-org%2Fgrok-1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xai-org%2Fgrok-1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xai-org","download_url":"https://codeload.github.com/xai-org/grok-1/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254129481,"owners_count":22019628,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:31.634Z","updated_at":"2025-05-14T11:08:14.242Z","avatar_url":"https://github.com/xai-org.png","language":"Python","funding_links":[],"categories":["Python","⭐ Overview of Mainstream LLMs","Miscellaneous","A01_文本生成_文本对话","English-centric","精选文章","Model","Licensing and Usage","💬 大语言模型（LLM）","Repos","Summary","开源开放的基础大模型列表","排行榜 [2025-03-18]","📦 Legacy \u0026 Inactive Projects"],"sub_categories":["大语言对话模型及数据","开源大语言模型","Media Coverage","里程碑模型"],"readme":"# Grok-1\n\nThis repository contains JAX example code for loading and running the Grok-1 open-weights model.\n\nMake sure to download the checkpoint and place the `ckpt-0` directory in `checkpoints` - see [Downloading the weights](#downloading-the-weights)\n\nThen, run\n\n```shell\npip install -r requirements.txt\npython run.py\n```\n\nto test the code.\n\nThe script loads the checkpoint and samples from the model on a test input.\n\nDue to the large size of the model (314B parameters), a machine with enough GPU memory is required to test the model with the example code.\nThe implementation of the MoE layer in this repository is not efficient. The implementation was chosen to avoid the need for custom kernels to validate the correctness of the model.\n\n# Model Specifications\n\nGrok-1 is currently designed with the following specifications:\n\n- **Parameters:** 314B\n- **Architecture:** Mixture of 8 Experts (MoE)\n- **Experts Utilization:** 2 experts used per token\n- **Layers:** 64\n- **Attention Heads:** 48 for queries, 8 for keys/values\n- **Embedding Size:** 6,144\n- **Tokenization:** SentencePiece tokenizer with 131,072 tokens\n- **Additional Features:**\n  - Rotary embeddings (RoPE)\n  - Supports activation sharding and 8-bit quantization\n- **Maximum Sequence Length (context):** 8,192 tokens\n\n# Downloading the weights\n\nYou can download the weights using a torrent client and this magnet link:\n\n```\nmagnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e\u0026tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php\u0026tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969\u0026tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce\n```\n\nor directly using [HuggingFace 🤗 Hub](https://huggingface.co/xai-org/grok-1):\n```\ngit clone https://github.com/xai-org/grok-1.git \u0026\u0026 cd grok-1\npip install huggingface_hub[hf_transfer]\nhuggingface-cli download xai-org/grok-1 --repo-type model --include ckpt-0/* --local-dir checkpoints --local-dir-use-symlinks False\n```\n\n# License\n\nThe code and associated Grok-1 weights in this release are licensed under the\nApache 2.0 license. The license only applies to the source files in this\nrepository and the model weights of Grok-1.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxai-org%2Fgrok-1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxai-org%2Fgrok-1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxai-org%2Fgrok-1/lists"}