{"id":13451291,"url":"https://github.com/alpa-projects/alpa","last_synced_at":"2026-06-05T10:30:14.391Z","repository":{"id":36968380,"uuid":"341067188","full_name":"alpa-projects/alpa","owner":"alpa-projects","description":"Training and serving large-scale neural networks with auto parallelization.","archived":false,"fork":false,"pushed_at":"2023-12-09T16:26:31.000Z","size":7457,"stargazers_count":2990,"open_issues_count":75,"forks_count":343,"subscribers_count":45,"default_branch":"main","last_synced_at":"2024-05-23T01:11:56.792Z","etag":null,"topics":["alpa","auto-parallelization","compiler","deep-learning","distributed-computing","distributed-training","high-performance-computing","jax","llm","machine-learning"],"latest_commit_sha":null,"homepage":"https://alpa.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alpa-projects.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-02-22T03:21:23.000Z","updated_at":"2024-05-21T23:52:46.000Z","dependencies_parsed_at":"2023-12-09T17:38:18.743Z","dependency_job_id":null,"html_url":"https://github.com/alpa-projects/alpa","commit_stats":{"total_commits":666,"total_committers":52,"mean_commits":"12.807692307692308","dds":0.5525525525525525,"last_synced_commit":"b8078a9f75cb4c90cabb4550ee48c99ef394e209"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpa-projects%2Falpa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpa-projects%2Falpa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpa-projects%2Falpa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alpa-projects%2Falpa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alpa-projects","download_url":"https://codeload.github.com/alpa-projects/alpa/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219847178,"owners_count":16556407,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alpa","auto-parallelization","compiler","deep-learning","distributed-computing","distributed-training","high-performance-computing","jax","llm","machine-learning"],"created_at":"2024-07-31T07:00:51.237Z","updated_at":"2026-06-05T10:30:14.294Z","avatar_url":"https://github.com/alpa-projects.png","language":"Python","funding_links":[],"categories":["Data Parallelism + Model Parallelism (or Tensor Parallelism) + Pipeline Parallelism:","Python","Frameworks","[:robot: machine-learning]([robot-machine-learning)](\u003chttps://github.com/stars/ketsapiwiq/lists/robot-machine-learning\u003e))","Open Source Projects","Training","Models and Projects"],"sub_categories":["Survey","Ray + JAX / TPU"],"readme":"**Note: Alpa is not actively maintained currently. It is available as a research artifact. The core algorithm in Alpa has been merged into XLA, which is still being maintained. https://github.com/openxla/xla/tree/main/xla/hlo/experimental/auto_sharding**\n\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/alpa-projects/alpa/blob/main/docs/logo/alpa-logo-cropped.png\" alt=\"logo\" width=\"250\"\u003e\u003c/img\u003e\n\u003cbr\u003e\u003c/br\u003e\n\u003c/div\u003e\n\n[![CI](https://github.com/alpa-projects/alpa/actions/workflows/ci.yml/badge.svg)](https://github.com/alpa-projects/alpa/actions/workflows/ci.yml)\n[![Build Jaxlib](https://github.com/alpa-projects/alpa/actions/workflows/build_jaxlib.yml/badge.svg)](https://github.com/alpa-projects/alpa/actions/workflows/build_jaxlib.yml)\n\n[**Documentation**](https://alpa-projects.github.io) | [**Slack**](https://forms.gle/YEZTCrtZD6EAVNBQ7)\n\nAlpa is a system for training and serving large-scale neural networks.\n\nScaling neural networks to hundreds of billions of parameters has enabled dramatic breakthroughs such as GPT-3, but training and serving these large-scale neural networks require complicated distributed system techniques.\nAlpa aims to automate large-scale distributed training and serving with just a few lines of code.\n\nThe key features of Alpa include:  \n\n💻 **Automatic Parallelization**. Alpa automatically parallelizes users' single-device code on distributed clusters with data, operator, and pipeline parallelism. \n\n🚀 **Excellent Performance**. Alpa achieves linear scaling on training models with billions of parameters on distributed clusters.\n\n✨ **Tight Integration with Machine Learning Ecosystem**. Alpa is backed by open-source, high-performance, and production-ready libraries such as [Jax](https://github.com/google/jax), [XLA](https://www.tensorflow.org/xla), and [Ray](https://github.com/ray-project/ray).\n\n## Serving\nThe code below shows how to use huggingface/transformers interface and Alpa distributed backend for large model inference.\nDetailed documentation is in [Serving OPT-175B using Alpa](https://alpa-projects.github.io/tutorials/opt_serving.html).\n\n```python\nfrom transformers import AutoTokenizer\nfrom llm_serving.model.wrapper import get_model\n\n# Load the tokenizer\ntokenizer = AutoTokenizer.from_pretrained(\"facebook/opt-2.7b\")\ntokenizer.add_bos_token = False\n\n# Load the model. Alpa automatically downloads the weights to the specificed path\nmodel = get_model(model_name=\"alpa/opt-2.7b\", path=\"~/opt_weights/\")\n\n# Generate\nprompt = \"Paris is the capital city of\"\n\ninput_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\noutput = model.generate(input_ids=input_ids, max_length=256, do_sample=True)\ngenerated_string = tokenizer.batch_decode(output, skip_special_tokens=True)\n\nprint(generated_string)\n```\n\n## Training\nUse Alpa's decorator ``@parallelize`` to scale your single-device training code to distributed clusters.\nCheck out the [documentation](https://alpa-projects.github.io) site and\n[examples](https://github.com/alpa-projects/alpa/tree/main/examples) folder\nfor installation instructions, tutorials, examples, and more.\n\n```python\nimport alpa\n\n# Parallelize the training step in Jax by simply using a decorator\n@alpa.parallelize\ndef train_step(model_state, batch):\n    def loss_func(params):\n        out = model_state.forward(params, batch[\"x\"])\n        return jnp.mean((out - batch[\"y\"]) ** 2)\n\n    grads = grad(loss_func)(model_state.params)\n    new_model_state = model_state.apply_gradient(grads)\n    return new_model_state\n\n# The training loop now automatically runs on your designated cluster\nmodel_state = create_train_state()\nfor batch in data_loader:\n    model_state = train_step(model_state, batch)\n```\n\n## Learning more\n- [Papers](docs/publications/publications.rst)\n- [Google AI blog](https://ai.googleblog.com/2022/05/alpa-automated-model-parallel-deep.html)\n- [OSDI 2022 talk slides](https://docs.google.com/presentation/d/1CQ4S1ff8yURk9XmL5lpQOoMMlsjw4m0zPS6zYDcyp7Y/edit?usp=sharing)\n- [ICML 2022 big model tutorial](https://sites.google.com/view/icml-2022-big-model/home)\n- [GTC 2023 talk video](https://www.nvidia.com/en-us/on-demand/session/gtcspring23-s51337/)\n\n## Getting Involved\n- Connect to Alpa developers via the [Alpa slack](https://forms.gle/YEZTCrtZD6EAVNBQ7).\n- Please read the [contributor guide](https://alpa-projects.github.io/developer/developer_guide.html) if you are interested in contributing code.\n\n## License\nAlpa is licensed under the [Apache-2.0 license](https://github.com/alpa-projects/alpa/blob/main/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpa-projects%2Falpa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falpa-projects%2Falpa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falpa-projects%2Falpa/lists"}