{"id":22675596,"url":"https://github.com/stability-ai/stablecode","last_synced_at":"2025-10-10T10:15:56.860Z","repository":{"id":191324636,"uuid":"676460796","full_name":"Stability-AI/StableCode","owner":"Stability-AI","description":"Code Assistance/ Developer Productivity suite of Models ","archived":false,"fork":false,"pushed_at":"2024-02-14T10:26:22.000Z","size":761,"stargazers_count":125,"open_issues_count":2,"forks_count":13,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-12T14:24:02.366Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Stability-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-09T08:50:55.000Z","updated_at":"2025-01-28T14:53:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"a20b1750-30e8-4ab4-bcf8-ede8287b37e0","html_url":"https://github.com/Stability-AI/StableCode","commit_stats":null,"previous_names":["stability-ai/stablecode"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Stability-AI/StableCode","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Stability-AI%2FStableCode","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Stability-AI%2FStableCode/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Stability-AI%2FStableCode/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Stability-AI%2FStableCode/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Stability-AI","download_url":"https://codeload.github.com/Stability-AI/StableCode/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Stability-AI%2FStableCode/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279003498,"owners_count":26083594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-09T17:57:45.193Z","updated_at":"2025-10-10T10:15:56.821Z","avatar_url":"https://github.com/Stability-AI.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# StableCode: Stability AI Developer Productivity/Developer\n![Stochastic Parrot](assets/stablecodelogo.png)\n\u003cbr/\u003e*“ “a colorful parrot with glasses typing away at a computer, flat icon, vector” – SDXL 0.9\n” — [Stable Diffusion XL](https://clipdrop.co/stable-diffusion)*\n\n\n\n\nThis repository contains Stability AI's ongoing development of the StableCode series of code models and will be continuously updated with new checkpoints. The following provides an overview of all currently available models. More coming soon.\n\n## News\n\n*January 16, 2024*\n- Released the next version of Stable Code for developers. Catch the release blog post [here](https://stability.ai/news/stable-code-2024-llm-code-completion-release).\n\n*August 9, 2023*\n- Released an improved version of StableCode-Completion-Alpha-3B -\u003e [StableCode-Completion-Alpha-3B-v1.1](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b/tree/v1.1). This model is trained on more tokens of the top 5 languages.\n\n*August 8, 2023*\n\nReleased the initial suite of StableCode-Alphas. Catch the release blog post [here](https://stability.ai/blog/stablecode-llm-generative-ai-coding). Which includes,\n- StableCode-Completion-Alpha-3B\n- StableCode-Completion-Alpha-3B-4k\n- StableCode-Instruct-Alpha-3b\n\n*January 16, 2023*\n\nReleased StableCode-3B. Catch the release blog post [here](https://stability.ai/news/stable-code-2024-llm-code-completion-release?utm_source=x\u0026utm_medium=website\u0026utm_campaign=blog). Which includes,\n- StableCode-3B a super strong coding model competitive with code-llama-7b in 3B size.\n\n\n## Models\n\n### StableCode-Completion-Alpha suite of Models\n\n-  [StableCode-Completion-Alpha-3B](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b) - `StableCode-Completion-Alpha-3B` is a 3 billion parameter decoder-only code completion model pre-trained on a diverse set of programming languages that were the top used languages based on the 2023 stackoverflow developer survey with a context length of 16k. Trained on a special augmented version of the [starcoder-dataset](https://huggingface.co/datasets/bigcode/starcoderdata/viewer/bigcode--starcoderdata/train?row=0).\n- [StableCode-Completion-Alpha-3B-4k](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b-4k) -  StableCode-Completion-Alpha-3B-4K is a 3 billion parameter decoder-only code completion model pre-trained on diverse set of programming languages that topped the stackoverflow developer survey with a context length of 4k.\n\n\n\n#### Training Details\n\nFollowing similar work, we use a multi-stage approach to context length extension ([Nijkamp et al., 2023](https://blog.salesforceairesearch.com/xgen/)), scheduling 390 billion tokens at context length 4096 followed by 100 billion tokens at 16k tokens. We found that sequence length warmup ([Li et al., 2022](https://arxiv.org/abs/2108.06084)) helped stabilize early spikes during the first ~80 billion tokens of pre-training. However, it was not applied to the final runs due to significant throughput penalties as length shapes grew across the curriculum.\n\n#### Training Data\nThe training is done in two stages, initial pretraining with the top 12 languages, which we got inspired by Stackoverflow developer survey,\nThe initial subsets - `Java, Javascript, Python, TypeScript, PHP, SQL, Rust, C, MarkDown, Go, C++, Shell.`\n  \nThis is then followed by continued pretraining with top 6 languages to be an expert in those languages - `Java, Javascript, Python, C, C++, Go`\n#### Evaluation\n\nThe following zero-shot evaluations are performed with the awesome [BigCode Evaluation Harness](https://github.com/bigcode-project/bigcode-evaluation-harness),   \n\n\n\n| Name                                                                                                               | HuggingFace Name                              | Type              |    | Context Length | Human-Eval |\n| ------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | ----------------- | -- | -------------- | ---------- |\n|                                                                                                                    |                                               |                   |    |                | pass@1     | pass@10 |\n| [StableCode-Completion-Alpha-3B](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b)                | stabilityai/stablecode-completion-alpha-3b    | Base              | 3B | 16384          | 20.18      | 33.75 |\n| [StableCode-Completion-Alpha-3b-4k](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b-4k/)         | stabilityai/stablecode-completion-alpha-3b-4k | Base              | 3B | 4096           | 17.68      | 27.01 |\n| [Stablecode-Instruct-Alpha-3b](https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b/)                   | stabilityai/stablecode-instruct-alpha-3b      | Instruction Tuned | 3B | 4096           | 26.89      | 36.18 |\n| [StableCode-Completion-Alpha-3B v1.1](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b/tree/v1.1) | stablecode-completion-alpha-3b@v1.1      | Base              | 3B | 16384          | 22.06      | 33.37 |\n| [StableCode-3B](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b/tree/v1.1) | stable-code-3b      | Base              | 3B | 16384          | 32.400      |\n\n\n\n   \n## Quickstart\n\nAll StableCode models are hosted on [the Hugging Face hub](https://huggingface.co/StabilityAI). Check out this [notebook](https://github.com/Stability-AI/StableLM/blob/main/notebooks/stablelm-alpha.ipynb) to run inference with limited GPU capabilities.\n\nGet started on generating code with `StableCode-Completion-Alpha` by using the following code snippet:\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList\n\ntokenizer = AutoTokenizer.from_pretrained(\"stabilityai/stablecode-completion-alpha-3b\")\nmodel = AutoModelForCausalLM.from_pretrained(\"stabilityai/stablecode-completion-alpha-3b\")\nmodel.half().cuda()\n\nclass StopOnTokens(StoppingCriteria):\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -\u003e bool:\n        stop_ids = set([50278, 50279, 50277, 1, 0])\n        return input_ids[0][-1] in stop_ids\n\n\nprompt = f\"import torch\\nimport torch.nn as nn\"\n\ninputs = tokenizer(prompt, return_tensors=\"pt\").to(\"cuda\")\ntokens = model.generate(\n  **inputs,\n  max_new_tokens=64,\n  temperature=0.7,\n  do_sample=True,\n  stopping_criteria=StoppingCriteriaList([StopOnTokens()])\n)\nprint(tokenizer.decode(tokens[0], skip_special_tokens=True))\n```\n\n\n## Licenses\n\n- Base model checkpoints (`StableCode-Completion-Alpha-3B`) are licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)\n\n- Instruct-tuned checkpoints (`StableCode-Instruct-Alpha-3B`) are licensed under [StableCode Research License](https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b/blob/main/LICENSE.md) Copyright (c) Stability AI Ltd. All Rights Reserved\n\n- All code in this repository is licensed under the Apache License 2.0 license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstability-ai%2Fstablecode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstability-ai%2Fstablecode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstability-ai%2Fstablecode/lists"}