{"id":26655662,"url":"https://github.com/raumberg/myllm","last_synced_at":"2025-06-23T00:34:56.746Z","repository":{"id":279382958,"uuid":"926415064","full_name":"Raumberg/myllm","owner":"Raumberg","description":"LLM Training Framework","archived":false,"fork":false,"pushed_at":"2025-04-21T09:58:06.000Z","size":1495,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-21T10:51:22.515Z","etag":null,"topics":["deep-neural-networks","deepspeed","framework","huggingface","huggingface-transformers","llm","llm-training","python","reinforcement-learning","torch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Raumberg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-03T08:01:21.000Z","updated_at":"2025-04-21T09:58:09.000Z","dependencies_parsed_at":"2025-02-25T09:38:46.315Z","dependency_job_id":"f8e66a4a-4774-4d5c-9bc6-04ad376537c9","html_url":"https://github.com/Raumberg/myllm","commit_stats":null,"previous_names":["raumberg/myllm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Raumberg/myllm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raumberg%2Fmyllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raumberg%2Fmyllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raumberg%2Fmyllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raumberg%2Fmyllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Raumberg","download_url":"https://codeload.github.com/Raumberg/myllm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raumberg%2Fmyllm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261390860,"owners_count":23151653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-neural-networks","deepspeed","framework","huggingface","huggingface-transformers","llm","llm-training","python","reinforcement-learning","torch"],"created_at":"2025-03-25T06:37:50.711Z","updated_at":"2025-06-23T00:34:51.727Z","avatar_url":"https://github.com/Raumberg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ch1 style=\"background: linear-gradient(to right, black, white); -webkit-background-clip: text; -webkit-text-fill-color: transparent; margin: 0;\"\u003e\n        MyLLM\n    \u003c/h1\u003e\n    \u003cdiv style=\"border: 10px solid; border-image: linear-gradient(to right, black, white) 1; padding: 10px; display: inline-block;\"\u003e\n        \u003cimg src=\"assets/myllm.png\" alt=\"gallery\" width=\"600\"/\u003e\n    \u003c/div\u003e\n    \u003cbr\u003e\n    \u003cbr\u003e\n    \u003cp align=\"center\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/issues/Raumberg/myllm?style=for-the-badge\"\u003e\n        \u003cbr\u003e\n        \u003cimg src=\"https://img.shields.io/github/languages/count/Raumberg/myllm?style=for-the-badge\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/repo-size/Raumberg/myllm?style=for-the-badge\"\u003e\n        \u003cbr\u003e\n    \u003c/p\u003e\n\u003c/div\u003e\n\n# LLM Framework | Toolkit for various training stages\nInitially derived from [Effective LLM Alignment](https://github.com/VikhrModels/effective_llm_alignment/) by VikhrModels.  \nMany credits goes to the Vikhr Team.\n\n## 🚀 [Methods and Stages supported]:\n- Supervised Finetuning (Full/LoRa/QLoRa)\n- Distillation (KL Divergence, MSE, Cosine and others)\n- Reinforcement Learning (GRPO, DPO, PPO)\n- Adapters merging\n- Tokenizer extensions\n\n## 🛠️ [Technical details]:\n- Built on top of PyTorch, Transformers, TRL, Peft. No 'magic' libraries like unsloth.\n- Distributed training via Accelerate, FSDP and DeepSpeed (Stage 2, 3).\n- Acceleration with vLLM, FlashAttn, Liger Kernels and fusion.\n- Logging options: wandb, clearml\n- Convenient config management using TOML\n\n## 🧠 [Training an LLM]\n- Everything is available from the root (MyLLM) folder. \n- What you need to do is start any desired script using accelerate:  \n```bash\n# ~/../myllm \u003e\naccelerate launch --config_file \u003cpath-to-cfg.yaml\u003e \u003cpath-to-script.py\u003e \u003cpath-to-model-cfg.toml\u003e\n# example SFT:\naccelerate launch --config_file configs/accelerate/stage3_config.yaml src/train/sft.py configs/train/sft/full-sft-watari.toml\n# example GRPO:\naccelerate launch src/train/grpo.py configs/train/grpo/rl-grpo-zariman-no-vllm.toml\n```  \n- Example launching GRPO with VLLM support:\n```bash\n\u003e CUDA_VISIBLE_DEVICES=1 trl vllm-serve --model \u003cyour-model\u003e --tensor_parallel_size 1 --max_model_len 4096\n\u003e CUDA_VISIBLE_DEVICES=0 accelerate launch src/train/grpo.py configs/train/grpo/\u003cyour-model-config\u003e.toml \n```\n   \n\u003e **⚠️ Disclaimer:**  \n\u003e GRPO scripts can be unstable, the work is still going on. If you encounter any errors, please, open an Issue.  \n\n## 📟 [Useful scripts]:\nThe folder `myllm/src/helpers` contains useful scripts that you can utilize for your models:\n- Merge your LoRA adapters with original model using `adapters.py` by:\n```bash\ncd myllm/src/helpers\npython adapters.py merge --source ../../models/attn-signs-watari-32/checkpoint-5500/ --output ../../models/attn-signs-watari-32/watari-32-merged --dtype bf16\n```\n- Extend model tokenizer by using `tokenizer.py`\n\n# Latest changes:\n- Added lora-sft-watari-32-stage-n.toml training configs from [Attention Signs HuggingFace Page](https://huggingface.co/attn-signs/Watari-32b-v0)\n- Added new [fusion] toml group for fused kernels. Example:  \n```toml\n[fusion]\nuse_liger = true\npatch_dyntanh = true # Nightly function, may be unstable\n```\n- Added new modules: `stdout` and `data_processors` and `liger`.\n    - **stdout:** print your model config, script arguments and training config in table. Example:\n    ```\n        Model Inspection:\n    +----------------------+----------------------------+\n    | Config key           | Config value               |\n    +======================+============================+\n    | Model Architecture   | Qwen2ForCausalLM           |\n    +----------------------+----------------------------+\n    | Total Parameters     | 0                          |\n    +----------------------+----------------------------+\n    | Trainable Parameters | 0                          |\n    +----------------------+----------------------------+\n    | Dtype                | torch.bfloat16             |\n    +----------------------+----------------------------+\n    | Device               | cuda:0                     |\n    +----------------------+----------------------------+\n    | Tokenizer Vocab Size | 147200                     |\n    +----------------------+----------------------------+\n    | Model Embedding Size | 0                          |\n    +----------------------+----------------------------+\n    | Padding Token        | \u003c|endoftext|\u003e (ID: 147075) |\n    +----------------------+----------------------------+\n    | EOS Token            | \u003c|im_end|\u003e (ID: 147077)    |\n    +----------------------+----------------------------+\n    | Max Sequence Length  | 32768                      |\n    +----------------------+----------------------------+\n    | Architecture         | Qwen2ForCausalLM           |\n    +----------------------+----------------------------+\n    | Hidden Size          | 5120                       |\n    +----------------------+----------------------------+\n    | Attention Heads      | 40                         |\n    +----------------------+----------------------------+\n    ```\n    - **data_processors**: moved all tokenizer processing functions to a separate module. Added support for default processing and history processing.\n    - **liger**: moved all liger kernels to a separate module\n- `resume_from` now is a **boolean flag** instead of a string representing a model path. When providing `resume_from=true`, the initial model_name_or_path should be the path to your local checkpoint.\n- Added `construct_history` **boolean flag** that constructs history out of the dataset. If `construct_history=false`, the script will use `default_row_processor` function. \n\nOverall, the training scripts are becoming more easy to read and user-friendly, outsourcing difficult tasks under the hood.\n\n# Nightly | Development functions:\n- Added `fusion` in which native / custom CUDA/Triton kernels will be developed\n- Added Fused Dynamic Tanh kernel, Torch Interface and patching function.  \nWhat is **Dynamic Tanh**?\nDynamic Tanh is the most recent discovery by Meta, attempting to replace LayerNorm with Tanh() to speed up training stages and minimize total parameters.  \nDynTanh is a novel approach, thus, can be unstable (until we release the final version). At the same time it is also a debatable method for now.  \n```\nBased on arxiv paper: https://www.alphaxiv.org/abs/2503.10622\nBased on authors code: https://github.com/jiachenzhu/DyT/tree/main\n```\n\n\u003e [!IMPORTANT]\n\u003e Thank you for your interest in MyLLM! We look forward to your contributions and feedback! 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraumberg%2Fmyllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraumberg%2Fmyllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraumberg%2Fmyllm/lists"}