{"id":37076245,"url":"https://github.com/graphcore-research/unit-scaling","last_synced_at":"2026-01-14T08:58:17.037Z","repository":{"id":161509659,"uuid":"623425369","full_name":"graphcore-research/unit-scaling","owner":"graphcore-research","description":"A library for unit scaling in PyTorch","archived":false,"fork":false,"pushed_at":"2025-07-11T10:39:33.000Z","size":13836,"stargazers_count":130,"open_issues_count":11,"forks_count":11,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-19T00:02:30.259Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://graphcore-research.github.io/unit-scaling/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/graphcore-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-04T10:51:27.000Z","updated_at":"2025-08-29T18:15:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"df13c21f-3557-47e5-985a-fc024d269756","html_url":"https://github.com/graphcore-research/unit-scaling","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/graphcore-research/unit-scaling","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graphcore-research%2Funit-scaling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graphcore-research%2Funit-scaling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graphcore-research%2Funit-scaling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graphcore-research%2Funit-scaling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/graphcore-research","download_url":"https://codeload.github.com/graphcore-research/unit-scaling/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graphcore-research%2Funit-scaling/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414728,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T08:58:16.417Z","updated_at":"2026-01-14T08:58:17.029Z","avatar_url":"https://github.com/graphcore-research.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Unit-Scaled Maximal Update Parameterization (u-μP)\n\n[![tests](https://github.com/graphcore-research/unit-scaling/actions/workflows/ci.yaml/badge.svg)](https://github.com/graphcore-research/unit-scaling/actions/workflows/ci-public.yaml)\n![PyPI version](https://img.shields.io/pypi/v/unit-scaling)\n[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/graphcore-research/unit-scaling/blob/main/LICENSE)\n[![GitHub Repo stars](https://img.shields.io/github/stars/graphcore-research/unit-scaling)](https://github.com/graphcore-research/unit-scaling/stargazers)\n\nA library for unit scaling in PyTorch, based on the paper [u-μP: The Unit-Scaled Maximal Update Parametrization](https://arxiv.org/abs/2407.17465) and previous work [Unit Scaling: Out-of-the-Box Low-Precision Training](https://arxiv.org/abs/2303.11257).\n\nDocumentation can be found at\n[https://graphcore-research.github.io/unit-scaling](https://graphcore-research.github.io/unit-scaling) and an example notebook at [examples/demo.ipynb](examples/demo.ipynb).\n\n**Note:** The library is currently in its _beta_ release.\nSome features have yet to be implemented and occasional bugs may be present.\nWe're keen to help users with any problems they encounter.\n\n## Installation\n\nTo install the `unit-scaling` library, run:\n\n```sh\npip install unit-scaling\n```\nor for a local editable install (i.e. one which uses the files in this repo), run:\n\n```sh\npip install -e .\n```\n\n## Development\n\nFor development in this repository, we recommend using the provided docker container.\nThis image can be built and entered interactively using:\n\n```sh\ndocker build -t unit-scaling-dev:latest .\ndocker run -it --rm  --user developer:developer -v $(pwd):/home/developer/unit-scaling unit-scaling-dev:latest\n# To use git within the container, add `-v ~/.ssh:/home/developer/.ssh:ro -v ~/.gitconfig:/home/developer/.gitconfig:ro`.\n```\n\nFor vscode users, this repo also contains a `.devcontainer.json` file, which enables the container to be used as a full-featured IDE (see the [Dev Container docs](https://code.visualstudio.com/docs/devcontainers/containers) for details on how to use this feature).\n\nKey development functionality is contained within the `./dev` script. This includes running unit tests, linting, formatting, documentation generation and more. Run `./dev --help` for the available options. Running `./dev` without arguments is equivalent to using the `--ci` option, which runs all of the available dev checks. This is the test used for GitHub CI.\n\nWe encourage pull requests from the community. Please reach out to us with any questions about contributing.\n\n**Releasing:**\n\n1. `git push origin origin/main:refs/tags/v\u003cMAJOR\u003e.\u003cMINOR\u003e.\u003cPATCH\u003e`\n2. Create a release at https://github.com/graphcore-research/unit-scaling/releases/\n3. Check [GitHub Actions](https://github.com/graphcore-research/unit-scaling/actions) and [PyPI](https://pypi.org/project/unit-scaling/), which should auto-publish the project\n\n## What is u-μP?\n\nu-μP inserts scaling factors into the model to make activations, gradients and weights unit-scaled (RMS ≈ 1) at initialisation, and into optimiser learning rates to keep updates stable as models are scaled in width and depth. This results in hyperparameter transfer from small to large models and easy support for low-precision training.\n\nFor a quick intro, see [examples/demo.ipynb](examples/demo.ipynb), for more depth see the [paper](https://arxiv.org/abs/2407.17465) and [library documentation](https://graphcore-research.github.io/unit-scaling/).\n\n## What is unit scaling?\n\nFor a demonstration of the library and an overview of how it works, see\n[Out-of-the-Box FP8 Training](https://github.com/graphcore-research/out-of-the-box-fp8-training/blob/main/out_of_the_box_fp8_training.ipynb)\n(a notebook showing how to unit-scale the nanoGPT model).\n\nFor a more in-depth explanation, consult our paper\n[Unit Scaling: Out-of-the-Box Low-Precision Training](https://arxiv.org/abs/2303.11257).\n\nAnd for a practical introduction to using the library, see our [User Guide](https://graphcore-research.github.io/unit-scaling/user_guide.html).\n\n## License\n\nCopyright (c) 2023 Graphcore Ltd. Licensed under the Apache 2.0 License.\n\nSee [NOTICE.md](NOTICE.md) for further details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraphcore-research%2Funit-scaling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgraphcore-research%2Funit-scaling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraphcore-research%2Funit-scaling/lists"}