{"id":13562360,"url":"https://github.com/higgsfield-ai/higgsfield","last_synced_at":"2025-12-24T21:16:56.785Z","repository":{"id":38752679,"uuid":"134999697","full_name":"higgsfield-ai/higgsfield","owner":"higgsfield-ai","description":"Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters","archived":false,"fork":false,"pushed_at":"2024-05-25T17:43:07.000Z","size":5068,"stargazers_count":3509,"open_issues_count":5,"forks_count":585,"subscribers_count":72,"default_branch":"main","last_synced_at":"2025-12-11T05:25:06.072Z","etag":null,"topics":["cluster-management","deep-learning","distributed","llama","llama2","llm","machine-learning","mlops","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/higgsfield-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-26T22:47:43.000Z","updated_at":"2025-12-10T12:10:20.000Z","dependencies_parsed_at":"2023-12-23T20:50:41.627Z","dependency_job_id":"1c8c4c53-1018-4be6-8eaa-d998c63e34a5","html_url":"https://github.com/higgsfield-ai/higgsfield","commit_stats":{"total_commits":38,"total_committers":4,"mean_commits":9.5,"dds":0.3421052631578947,"last_synced_commit":"d12a36e66024a93d33ec61826a77d5a346c16869"},"previous_names":["higgsfield/higgsfield","higgsfield/rl-adventure-2"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/higgsfield-ai/higgsfield","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/higgsfield-ai%2Fhiggsfield","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/higgsfield-ai%2Fhiggsfield/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/higgsfield-ai%2Fhiggsfield/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/higgsfield-ai%2Fhiggsfield/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/higgsfield-ai","download_url":"https://codeload.github.com/higgsfield-ai/higgsfield/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/higgsfield-ai%2Fhiggsfield/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28008675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-24T02:00:07.193Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-management","deep-learning","distributed","llama","llama2","llm","machine-learning","mlops","pytorch"],"created_at":"2024-08-01T13:01:07.739Z","updated_at":"2025-12-24T21:16:56.758Z","avatar_url":"https://github.com/higgsfield-ai.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook","A01_文本生成_文本对话","By Language","📚 Project Purpose"],"sub_categories":["大语言对话模型及数据","Data Science","Machine Learning (Intermediate-Level"],"readme":"# higgsfield - multi node training without crying\n\n\nHiggsfield is an open-source, fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters, such as Large Language Models (LLMs).\n\n[![PyPI version](https://badge.fury.io/py/higgsfield.svg)](https://badge.fury.io/py/higgsfield)\n\n![architecture](https://raw.githubusercontent.com/higgsfield/higgsfield/main/docs/static/architecture.png)\n\nHiggsfield serves as a GPU workload manager and machine learning framework with five primary functions:\n\n1. Allocating exclusive and non-exclusive access to compute resources (nodes) to users for their training tasks.\n2. Supporting ZeRO-3 deepspeed API and fully sharded data parallel API of PyTorch, enabling efficient sharding for trillion-parameter models.\n3. Offering a framework for initiating, executing, and monitoring the training of large neural networks on allocated nodes.\n4. Managing resource contention by maintaining a queue for running experiments.\n5. Facilitating continuous integration of machine learning development through seamless integration with GitHub and GitHub Actions.\n   Higgsfield streamlines the process of training massive models and empowers developers with a versatile and robust toolset.\n## Install\n\n```bash\n$ pip install higgsfield==0.0.3\n```\n\n\n\n## Train example\n\nThat's all you have to do in order to train LLaMa in a distributed setting:\n\n```python\nfrom higgsfield.llama import Llama70b\nfrom higgsfield.loaders import LlamaLoader\nfrom higgsfield.experiment import experiment\n\nimport torch.optim as optim\nfrom alpaca import get_alpaca_data\n\n@experiment(\"alpaca\")\ndef train(params):\n    model = Llama70b(zero_stage=3, fast_attn=False, precision=\"bf16\")\n\n    optimizer = optim.AdamW(model.parameters(), lr=1e-5, weight_decay=0.0)\n\n    dataset = get_alpaca_data(split=\"train\")\n    train_loader = LlamaLoader(dataset, max_words=2048)\n\n    for batch in train_loader:\n        optimizer.zero_grad()\n        loss = model(batch)\n        loss.backward()\n        optimizer.step()\n\n    model.push_to_hub('alpaca-70b')\n```\n\n## How it's all done?\n\n1. We install all the required tools in your server (Docker, your project's deploy keys, higgsfield binary).\n2. Then we generate deploy \u0026 run workflows for your experiments.\n3. As soon as it gets into Github, it will automatically deploy your code on your nodes.\n4. Then you access your experiments' run UI through Github, which will launch experiments and save the checkpoints.\n\n## Design\n\nWe follow the standard pytorch workflow. Thus you can incorporate anything besides what we provide, `deepspeed`, `accelerate`, or just implement your custom `pytorch` sharding from scratch.\n\n**Enviroment hell**\n\nNo more different versions of pytorch, nvidia drivers, data processing libraries.\nYou can easily orchestrate experiments and their environments, document and track the specific versions and configurations of all dependencies to ensure reproducibility.\n\n**Config hell**\n\nNo need to define [600 arguments for your experiment](https://github.com/huggingface/transformers/blob/aaccf1844eccbb90cc923378e3c37a6b143d03fb/src/transformers/training_args.py#L161). No more [yaml witchcraft](https://hydra.cc/).\nYou can use whatever you want, whenever you want. We just introduce a simple interface to define your experiments. We have even taken it further, now you only need to design the way to interact.\n\n## Compatibility\n\n**We need you to have nodes with:**\n\n- Ubuntu\n- SSH access\n- Non-root user with sudo privileges (no-password is required)\n\n**Clouds we have tested on:**\n\n- Azure\n- LambdaLabs\n- FluidStack\n\nFeel free to open an issue if you have any problems with other clouds.\n\n## Getting started\n\n#### [Setup](./setup.md)\n\nHere you can find the quick start guide on how to setup your nodes and start training.\n\n- [Initialize the project](https://github.com/higgsfield/higgsfield/blob/main/setup.md#initialize-the-project)\n- [Setup the environment](https://github.com/higgsfield/higgsfield/blob/main/setup.md#setup-the-environment)\n- [Setup git](https://github.com/higgsfield/higgsfield/blob/main/setup.md#setup-git)\n- [Time to setup your nodes!](https://github.com/higgsfield/higgsfield/blob/main/setup.md#time-to-setup-your-nodes)\n- [Run your very first experiment](https://github.com/higgsfield/higgsfield/blob/main/setup.md#run-your-very-first-experiment)\n- [Fasten your seatbelt, it's time to deploy!](https://github.com/higgsfield/higgsfield/blob/main/setup.md#fasten-your-seatbelt-its-time-to-deploy)\n\n#### [Tutorial](./tutorial.md)\n\nAPI for common tasks in Large Language Models training.\n\n- [Working with distributed model](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#working-with-distributed-model)\n- [Preparing Data](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#preparing-data)\n- [Optimizing the Model Parameters](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#optimizing-the-model-parameters)\n- [Saving Model](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#saving-model)\n- [Training stabilization techniques](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#training-stabilization-techniques)\n- [Monitoring](https://github.com/higgsfield/higgsfield/blob/main/tutorial.md#monitoring)\n\n| Platform                                                          | Purpose                                                           | Estimated Response Time | Support Level   |\n| ----------------------------------------------------------------- | ----------------------------------------------------------------- | ----------------------- | --------------- |\n| [Github Issues](https://github.com/higgsfield/higgsfield/issues/) | Bug reports, feature requests, install issues, usage issues, etc. | \u003c 1 day                 | Higgsfield Team |\n| [Twitter](https://twitter.com/higgsfield_ai/)                     | For staying up-to-date on new features.                           | Daily                   | Higgsfield Team |\n| [Website](https://higgsfield.ai/)                                 | Discussion, news.                                                 | \u003c 2 days                | Higgsfield Team |\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiggsfield-ai%2Fhiggsfield","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhiggsfield-ai%2Fhiggsfield","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiggsfield-ai%2Fhiggsfield/lists"}