{"id":14964640,"url":"https://github.com/bobazooba/xllm","last_synced_at":"2025-04-04T17:08:55.271Z","repository":{"id":206946891,"uuid":"717165456","full_name":"BobaZooba/xllm","owner":"BobaZooba","description":"🦖 X—LLM: Cutting Edge \u0026 Easy LLM Finetuning","archived":false,"fork":false,"pushed_at":"2024-01-17T16:43:39.000Z","size":1899,"stargazers_count":400,"open_issues_count":8,"forks_count":21,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-28T15:04:53.689Z","etag":null,"topics":["alpaca","bitsandbytes","cerebras","chatgpt","deep-learning","deep-neural-networks","gpt","gpt-4","gptq","large-language-models","llama","llama2","llm","mistral","openai","pytorch","torch","vicuna","zephyr"],"latest_commit_sha":null,"homepage":"https://t.me/talequestbot","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BobaZooba.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-10T17:55:03.000Z","updated_at":"2025-03-22T19:07:52.000Z","dependencies_parsed_at":"2024-08-11T01:46:03.277Z","dependency_job_id":null,"html_url":"https://github.com/BobaZooba/xllm","commit_stats":{"total_commits":60,"total_committers":2,"mean_commits":30.0,"dds":0.3666666666666667,"last_synced_commit":"8c3580faab6f3d7de0e9739a6f379d7b672e76be"},"previous_names":["bobazooba/xllm"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BobaZooba%2Fxllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BobaZooba%2Fxllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BobaZooba%2Fxllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BobaZooba%2Fxllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BobaZooba","download_url":"https://codeload.github.com/BobaZooba/xllm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217183,"owners_count":20903009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alpaca","bitsandbytes","cerebras","chatgpt","deep-learning","deep-neural-networks","gpt","gpt-4","gptq","large-language-models","llama","llama2","llm","mistral","openai","pytorch","torch","vicuna","zephyr"],"created_at":"2024-09-24T13:33:33.586Z","updated_at":"2025-04-04T17:08:55.249Z","avatar_url":"https://github.com/BobaZooba.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🦖 X—LLM: Cutting Edge \u0026 Easy LLM Finetuning\n\n\u003cdiv align=\"center\"\u003e\n\n[![Build](https://github.com/BobaZooba/xllm/actions/workflows/build.yaml/badge.svg?branch=main)](https://github.com/BobaZooba/xllm/actions/workflows/build.yaml)\n[![Github: License](https://img.shields.io/github/license/BobaZooba/xllm.svg?color=63C462\u0026style=flat-square)](https://github.com/BobaZooba/xllm/blob/main/LICENSE)\n[![Github: Release](https://img.shields.io/github/v/release/BobaZooba/xllm.svg?color=63C462\u0026style=flat-square)](https://github.com/BobaZooba/xllm/releases)\n\n[![PyPI - Version](https://img.shields.io/pypi/v/xllm.svg?logo=pypi\u0026label=PyPI\u0026logoColor=gold\u0026color=63C462\u0026style=flat-square)](https://pypi.org/project/xllm/)\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/xllm.svg?color=63C462\u0026label=Downloads\u0026logo=pypi\u0026logoColor=gold\u0026style=flat-square)](https://pypi.org/project/xllm/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/xllm?logo=python\u0026label=Python\u0026logoColor=gold\u0026color=63C462\u0026style=flat-square)](https://pypi.org/project/xllm/)\n\n[![Pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=gold\u0026color=63C462\u0026style=flat-square)](https://github.com/modelfront/predictor/blob/master/.pre-commit-config.yaml)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?logoColor=gold\u0026color=63C462\u0026style=flat-square)](https://github.com/psf/black)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json?logoColor=gold\u0026color=63C462\u0026style=flat-square)](https://github.com/astral-sh/ruff)\n[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n[![codecov](https://codecov.io/gh/BobaZooba/xllm/graph/badge.svg?token=ZOBMDXVW4B)](https://codecov.io/gh/BobaZooba/xllm)\n\n[![](https://dcbadge.vercel.app/api/server/5znbxBgwZP)](https://discord.gg/5znbxBgwZP)\n\nCutting Edge \u0026 Easy LLM Finetuning using the most advanced methods (QLoRA, DeepSpeed, GPTQ, Flash Attention 2, FSDP,\netc)\n\nDeveloped\nby [Boris Zubarev](https://t.me/BobaZooba) | [CV](https://docs.google.com/document/d/1BhFvIHQ1mpm81P-n2A-lhNac-U2wOGc6F2uS9gKvk88/edit?usp=sharing) | [LinkedIn](https://www.linkedin.com/in/boriszubarev/) | [bobazooba@gmail.com](mailto:bobazooba@gmail.com)\n\n\u003c/div\u003e\n\n# Why you should use X—LLM 🪄\n\nAre you using **Large Language Models (LLMs)** for your work and want to train them more efficiently with advanced\nmethods? Wish to focus on the data and improvements rather than repetitive and time-consuming coding for LLM training?\n\n**X—LLM** is your solution. It's a user-friendly library that streamlines training optimization, so you can **focus on\nenhancing your models and data**. Equipped with **cutting-edge training techniques**, X—LLM is engineered for efficiency\nby engineers who understand your needs.\n\n**X—LLM** is ideal whether you're **gearing up for production** or need a **fast prototyping tool**.\n\n## Features\n\n- Hassle-free training for Large Language Models\n- Seamless integration of new data and data processing\n- Effortless expansion of the library\n- Speed up your training, while simultaneously reducing model sizes\n- Each checkpoint is saved to the 🤗 HuggingFace Hub\n- Easy-to-use integration with your existing project\n- Customize almost any part of your training with ease\n- Track your training progress using `W\u0026B`\n- Supported many 🤗 Transformers models\n  like `Yi-34B`, `Mistal AI`, `Llama 2`, `Zephyr`, `OpenChat`, `Falcon`, `Phi`, `Qwen`, `MPT` and many more\n- Benefit from cutting-edge advancements in LLM training optimization\n  - QLoRA and fusing\n  - Flash Attention 2\n  - Gradient checkpointing\n  - bitsandbytes\n  - GPTQ (including post-training quantization)\n  - DeepSpeed\n  - FSDP\n  - And many more\n\n# Quickstart 🦖\n\n### Installation\n\n`X—LLM` is tested on Python 3.8+, PyTorch 2.0.1+ and CUDA 11.8.\n\n```sh\npip install xllm\n```\n\nVersion which include `deepspeed`, `flash-attn` and `auto-gptq`:\n\n```sh\npip install \"xllm[train]\"\n```\n\nDefault `xllm` version recommended for local development, `xllm[train]` recommended for training.\n\n#### Training recommended environment\n\nCUDA version: `11.8`\nDocker: `huggingface/transformers-pytorch-gpu:latest`\n\n## Fast prototyping ⚡\n\n```python\nfrom xllm import Config\nfrom xllm.datasets import GeneralDataset\nfrom xllm.experiments import Experiment\n\n# Init Config which controls the internal logic of xllm\n# QLoRA example\nconfig = Config(\n  model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n  apply_lora=True,\n  load_in_4bit=True,\n)\n\n# Prepare the data\ntrain_data = [\"Hello!\"] * 100\ntrain_dataset = GeneralDataset.from_list(data=train_data)\n\n# Build Experiment from Config: init tokenizer and model, apply LoRA and so on\nexperiment = Experiment(config=config, train_dataset=train_dataset)\nexperiment.build()\n\n# Run Experiment (training)\nexperiment.run()\n\n# # [Optional] Fuse LoRA layers\n# experiment.fuse_lora()\n\n# [Optional] Or push LoRA weights to HuggingFace Hub\nexperiment.push_to_hub(repo_id=\"YOUR_NAME/MODEL_NAME\")\n```\n\n### How `Config` controls `xllm`\n\n[More about config](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#config)\n\n\u003cdetails\u003e\n  \u003csummary\u003eLoRA\u003c/summary\u003e\n\n#### Simple\n\n  ```python\n  config = Config(\n    model_name_or_path=\"openchat/openchat_3.5\",\n    apply_lora=True,\n)\n  ```\n\n#### Advanced\n\n  ```python\n  config = Config(\n    model_name_or_path=\"openchat/openchat_3.5\",\n    apply_lora=True,\n    lora_rank=8,\n    lora_alpha=32,\n    lora_dropout=0.05,\n    raw_lora_target_modules=\"all\",\n    # Names of modules to apply LoRA. A comma-separated string, for example: \"k,q,v\" or \"all\".\n)\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eQLoRA\u003c/summary\u003e\n\nTo train the `QLoRA` model, we need to load the backbone model using `bitsandbytes` library and int4 (or int8) weights.\n\n#### Simple\n\n  ```python\n  config = Config(\n    model_name_or_path=\"01-ai/Yi-34B\",\n    apply_lora=True,\n    load_in_4bit=True,\n    prepare_model_for_kbit_training=True,\n)\n  ```\n\n#### Advanced\n\n  ```python\n  config = Config(\n    model_name_or_path=\"01-ai/Yi-34B\",\n    apply_lora=True,\n    load_in_4bit=True,\n    prepare_model_for_kbit_training=True,\n    llm_int8_threshold=6.0,\n    llm_int8_has_fp16_weight=True,\n    bnb_4bit_use_double_quant=True,\n    bnb_4bit_quant_type=\"nf4\",\n)\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003ePush checkpoints to the HuggingFace Hub\u003c/summary\u003e\n\nBefore that, you must log in to `Huggingface Hub` or add an `API Token` to the environment variables.\n\n  ```python\n  config = Config(\n    model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n    push_to_hub=True,\n    hub_private_repo=True,\n    hub_model_id=\"BobaZooba/AntModel-7B-XLLM-Demo-LoRA\",\n    save_steps=25,\n)\n  ```\n\n- Checkpoints will be saved locally and in Huggingface Hub each `save_steps`\n- If you train a model with `LoRA`, then only `LoRA` weights will be saved\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eReport to W\u0026B\u003c/summary\u003e\n\nBefore that, you must log in to `W\u0026B` or add an `API Token` to the environment variables.\n\n  ```python\n  config = Config(\n    model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n    report_to_wandb=True,\n    logging_steps=5,\n    wandb_project=\"xllm-demo\",\n)\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eGradient checkpointing\u003c/summary\u003e\n\nThis will help to use `less GPU memory` during training, that is, you will be able to learn more than without this\ntechnique. The disadvantages of this technique is slowing down the forward step, that is, `slowing down training`.\n\nYou will be training larger models (for example 7B in colab), but at the expense of training speed.\n\n  ```python\n  config = Config(\n    model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n    use_gradient_checkpointing=True,\n)\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eFlash Attention 2\u003c/summary\u003e\n\nThis speeds up training and GPU memory consumption, but it does not work with all models and GPUs. You also need to\ninstall `flash-attn` for this. This can be done using:\n\n`pip install \"xllm[train]\"`\n\n  ```python\n  config = Config(\n    model_name_or_path=\"meta-llama/Llama-2-7b-hf\",\n    use_flash_attention_2=True,\n)\n  ```\n\n\u003c/details\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eRecommended setup\u003c/b\u003e\u003c/summary\u003e\n\n#### Recommendations\n\n- Incredibly effective method is LoRA (`apply_lora`). It allows for a tremendous reduction in training costs\n  and, moreover, helps very effectively combat catastrophic forgetting.\n- Then, I advise using `load_in_4bit` and `prepare_model_for_kbit_training` together. This also significantly reduces\n  memory consumption.\n- Lastly, I would recommend apply `use_gradient_checkpointing`. This method also greatly reduces memory consumption, but\n  at the expense of slowing down training.\n- If your project allows, I suggest enabling `push_to_hub` and `hub_private_repo`, also specifying the model name\n  in `hub_model_id` and `save_steps`. Example: \"BobaZooba/SupaDupaLlama-7B-LoRA\". During training, every checkpoint of\n  your model will be saved in the HuggingFace Hub. If you specified `apply_lora`, then only the LoRA weights will be\n  saved, which you can later easily fuse with the main model, for example, using `xllm`.\n- If your GPU allows it add `use_flash_attention_2`\n- I also recommend using `report_to_wandb`, also specifying `wandb_project` (the project name in W\u0026B)\n  and `wandb_entity` (user or organization name in W\u0026B).\n- Note that for `push_to_hub`, you need to log in to the HuggingFace Hub beforehand or specify the\n  token (`HUGGING_FACE_HUB_TOKEN`) in the .env file. Similarly, when using `report_to_wandb`, you will need to log in to\n  W\u0026B. You can either specify the token (`WANDB_API_KEY`) in the .env file or you will be prompted to enter the token on\n  the command line.\n\n#### Features\n\n- QLoRA\n- Gradient checkpointing\n- Flash Attention 2\n- Stabilize training\n- Push checkpoints to HuggingFace Hub\n- W\u0026B report\n\n  ```python\n  config = Config(\n    model_name_or_path=\"meta-llama/Llama-2-7b-hf\",\n    tokenizer_padding_side=\"right\",  # good for llama2\n  \n    warmup_steps=1000,\n    max_steps=10000,\n    logging_steps=1,\n    save_steps=1000,\n    per_device_train_batch_size=2,\n    gradient_accumulation_steps=2,\n    max_length=2048,\n  \n    stabilize=True,\n    use_flash_attention_2=True,\n  \n    apply_lora=True,\n  \n    load_in_4bit=True,\n    prepare_model_for_kbit_training=True,\n  \n    use_gradient_checkpointing=True,\n\n    push_to_hub=False,\n    hub_private_repo=True,\n    hub_model_id=\"BobaZooba/SupaDupaLlama-7B-LoRA\",\n\n    report_to_wandb=False,\n    wandb_project=\"xllm-demo\",\n    wandb_entity=\"bobazooba\",\n  )\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eFuse\u003c/summary\u003e\n\nThis operation is only for models with a LoRA adapter.\n\nYou can explicitly specify to fuse the model after training.\n\n  ```python\n  config = Config(\n    model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n    apply_lora=True,\n    fuse_after_training=True,\n)\n  ```\n\nEven when you are using QLoRa\n\n  ```python\n  config = Config(\n    model_name_or_path=\"HuggingFaceH4/zephyr-7b-beta\",\n    apply_lora=True,\n    load_in_4bit=True,\n    prepare_model_for_kbit_training=True,\n    fuse_after_training=True,\n)\n  ```\n\nOr you can fuse the model yourself after training.\n\n  ```python\n  experiment.fuse_lora()\n  ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDeepSpeed\u003c/summary\u003e\n\n`DeepSpeed` is needed for training models on `multiple GPUs`. `DeepSpeed` allows you\nto `efficiently manage the resources of several GPUs during training`. For example, you\ncan `distribute the gradients and the state of the optimizer to several GPUs`, rather than storing a complete set of\ngradients and the state of the optimizer on each GPU. Starting training using `DeepSpeed` can only happen from\nthe `command line`.\n\n`train.py`\n\n  ```python\nfrom xllm import Config\nfrom xllm.datasets import GeneralDataset\nfrom xllm.cli import cli_run_train\n\nif __name__ == '__main__':\n    train_data = [\"Hello!\"] * 100\n    train_dataset = GeneralDataset.from_list(data=train_data)\n    cli_run_train(config_cls=Config, train_dataset=train_dataset)\n  ```\n\nRun train (in the `num_gpus` parameter, specify as many GPUs as you have)\n\n  ```sh\n  deepspeed --num_gpus=8 train.py --deepspeed_stage 2\n  ```\n\nYou also can pass other parameters\n\n  ```sh\ndeepspeed --num_gpus=8 train.py \\\n    --deepspeed_stage 2 \\\n    --apply_lora True \\\n    --stabilize True \\\n    --use_gradient_checkpointing True\n  ```\n\n\u003c/details\u003e\n\n### Notebooks\n\n| Name                                      | Comment                                                                            | Link                                                                                                                                                                   |\n|-------------------------------------------|------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| X—LLM Prototyping                         | In this notebook you will learn the basics of the library                          | [![xllm_prototyping](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1zsNmJFns1PKZy5VE5p5nsQL-mZF7SwHf?usp=sharing) |\n| Llama2 \u0026 Mistral AI efficient fine-tuning | 7B model training in colab using QLoRA, bnb int4, gradient checkpointing and X—LLM | [![Llama2MistalAI](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1CNNB_HPhQ8g7piosdehqWlgA30xoLauP?usp=sharing)   |\n\n## Production solution 🚀\n\n`X—LLM` enables not only to prototype models, but also facilitates the development of production-ready solutions through\nbuilt-in capabilities and customization.\n\nUsing `X—LLM` to train a model is easy and involves these few steps:\n\n1. `Prepare` — Get the data and the model ready by downloading and preparing them. Saves data locally\n   to `config.train_local_path_to_data` and `config.eval_local_path_to_data` if you are using eval dataset\n2. `Train` — Use the data prepared in the previous step to train the model\n3. `Fuse` — If you used LoRA during the training, fuse LoRA\n4. `Quantize` — Optimize your model's memory usage by quantizing it\n\nRemember, these tasks in `X—LLM` start from the command line. So, when you're all set to go, launching your full project\nwill look something like this:\n\n\u003cdetails\u003e\n  \u003csummary\u003eExample how to run your project\u003c/summary\u003e\n\n1. Downloading and preparing data and model\n\n  ```sh\npython3 MY_PROJECT/cli/prepare.py \\\n    --dataset_key MY_DATASET \\\n    --model_name_or_path mistralai/Mistral-7B-v0.1 \\\n    --path_to_env_file ./.env\n  ```\n\n2. Run train using DeepSpeed on multiple GPUs\n\n  ```sh\ndeepspeed --num_gpus=8 MY_PROJECT/cli/train.py \\\n    --use_gradient_checkpointing True \\\n    --deepspeed_stage 2 \\\n    --stabilize True \\\n    --model_name_or_path mistralai/Mistral-7B-v0.1 \\\n    --use_flash_attention_2 False \\\n    --load_in_4bit True \\\n    --apply_lora True \\\n    --raw_lora_target_modules all \\\n    --per_device_train_batch_size 8 \\\n    --warmup_steps 1000 \\\n    --save_total_limit 0 \\\n    --push_to_hub True \\\n    --hub_model_id MY_HF_HUB_NAME/LORA_MODEL_NAME \\\n    --hub_private_repo True \\\n    --report_to_wandb True \\\n    --path_to_env_file ./.env\n  ```\n\n3. Fuse LoRA\n\n  ```sh\npython3 MY_PROJECT/cli/fuse.py \\\n    --model_name_or_path mistralai/Mistral-7B-v0.1 \\\n    --lora_hub_model_id MY_HF_HUB_NAME/LORA_MODEL_NAME \\\n    --hub_model_id MY_HF_HUB_NAME/MODEL_NAME \\\n    --hub_private_repo True \\\n    --force_fp16 True \\\n    --fused_model_local_path ./fused_model/ \\\n    --path_to_env_file ./.env\n  ```\n\n4. [Optional] GPTQ quantization of the trained model with fused LoRA\n\n  ```sh\npython3 MY_PROJECT/cli/quantize.py \\\n    --model_name_or_path ./fused_model/ \\\n    --apply_lora False \\\n    --stabilize False \\\n    --quantization_max_samples 128 \\\n    --quantized_model_path ./quantized_model/ \\\n    --prepare_model_for_kbit_training False \\\n    --quantized_hub_model_id MY_HF_HUB_NAME/MODEL_NAME_GPTQ \\\n    --quantized_hub_private_repo True \\\n    --path_to_env_file ./.env\n  ```\n\n\u003c/details\u003e\n\nRight now, the `X—LLM` library lets you use only the [SODA dataset](https://huggingface.co/datasets/allenai/soda). We've\nset it up this way for demo purposes, but we're planning to add more datasets soon. You'll need to figure out how to\ndownload and handle your dataset. Simply put, you take care of your data, and `X—LLM` handles the rest. We've done it\nthis\nway on purpose, to give you plenty of room to get creative and customize to your heart's content.\n\nYou can customize your dataset in detail, adding additional fields. All of this will enable you to implement virtually\nany task in the areas of `Supervised Learning` and `Offline Reinforcement Learning`.\n\nAt the same time, you always have an easy way to submit data for language modeling.\n\n\u003cdetails\u003e\n  \u003csummary\u003eExample\u003c/summary\u003e\n\n  ```python\nfrom xllm import Config\nfrom xllm.datasets import GeneralDataset\nfrom xllm.cli import cli_run_train\n\nif __name__ == '__main__':\n    train_data = [\"Hello!\"] * 100\n    train_dataset = GeneralDataset.from_list(data=train_data)\n    cli_run_train(config_cls=Config, train_dataset=train_dataset)\n  ```\n\n\u003c/details\u003e\n\n### Build your own project\n\nTo set up your own project using `X—LLM`, you need to do two things:\n\n1. Implement your dataset (figure out how to download and handle it)\n2. Add `X—LLM`'s command-line tools into your project\n\nOnce that's done, your project will be good to go, and you can start running the steps you need (like prepare, train,\nand so on).\n\nTo get a handle on building your project with `X—LLM`, check out the materials below.\n\n## Useful materials\n\n- [Guide](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md): here, we go into detail about everything the library can\n  do\n- [Demo project](https://github.com/BobaZooba/xllm-demo): here's a step-by-step example of how to use `X—LLM` and fit\n  it\n  into your own project\n- [WeatherGPT](https://github.com/BobaZooba/wgpt): this repository features an example of how to utilize the xllm library. Included is a solution for a common type of assessment given to LLM engineers, who typically earn between $120,000 to $140,000 annually\n- [Shurale](https://github.com/BobaZooba/shurale): project with the finetuned 7B Mistal model\n\n# Config 🔧\n\nThe `X—LLM` library uses a single config setup for all steps like preparing, training and the other steps. It's\ndesigned in a way that\nlets you easily understand the available features and what you can adjust. `Config` has control almost over every\nsingle part of each step. Thanks to the config, you can pick your dataset, set your collator, manage the type of\nquantization during training, decide if you want to use LoRA, if you need to push a checkpoint to the `HuggingFace Hub`,\nand a\nlot more.\n\nConfig path: `src.xllm.core.config.Config`\n\nOr\n\n```python\nfrom xllm import Config\n```\n\n## Useful materials\n\n- [Important config fields for different steps](https://github.com/BobaZooba/xllm#how-config-controls-xllm)\n- [How do I choose the methods for training?](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-do-i-choose-the-methods-for-training)\n- [Detailed description of all config fields](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#detailed-config-explanation)\n\n# Customization options 🛠\n\nYou have the flexibility to tweak many aspects of your model's training: data, how data is processed, trainer, config,\nhow the model is loaded, what happens before and after training, and so much more.\n\nWe've got ready-to-use components for every part of the `xllm` pipeline. You can entirely switch out some components\nlike the dataset, collator, trainer, and experiment.\nFor some components like experiment and config, you have the option to just build on what's already there.\n\n### Useful materials\n\n- [How to implement dataset](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-implement-dataset)\n- [How to add CLI tools to your project](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-add-cli-tools-to-your-project)\n- [How to implement collator](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-implement-collator)\n- [How to implement trainer](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-implement-trainer)\n- [How to implement experiment](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-implement-experiment)\n- [How to extend config](https://github.com/BobaZooba/xllm/blob/main/GUIDE.md#how-to-extend-config)\n\n# Projects using X—LLM 🏆\n\nBuilding something cool with [X—LLM](https://github.com/BobaZooba/xllm)? Kindly reach out to me\nat [bobazooba@gmail.com](mailto:bobazooba@gmail.com). I'd love to hear from you.\n\n## Hall of Fame\n\nWrite to us so that we can add your project.\n\n- [TachyHealth/Thealth](https://huggingface.co/TachyHealth)\n  - [TachyHealth/Thealth-llama-7b_v1](https://huggingface.co/TachyHealth/Thealth-llama-7b_v1)\n  - [TachyHealth/Thealth-llama-13b](https://huggingface.co/TachyHealth/Thealth-llama-13b)\n  - [TachyHealth/Thealth-Yi-6b_v1](https://huggingface.co/TachyHealth/Thealth-Yi-6b_v1)\n  - [TachyHealth/Thealth-Yi-6b_v2](https://huggingface.co/TachyHealth/Thealth-Yi-6b_v2)\n  - [TachyHealth/Thealth-zypher-7b_v1](https://huggingface.co/TachyHealth/Thealth-zypher-7b_v1)\n  - [TachyHealth/Thealth-Zypher-7b_v2](https://huggingface.co/TachyHealth/Thealth-Zypher-7b_v2)\n  - [TachyHealth/Thealth-Falcon-7b](https://huggingface.co/TachyHealth/Thealth-Falcon-7b)\n  - [TachyHealth/Thealth-Mistral-7b](https://huggingface.co/TachyHealth/Thealth-Mistral-7b)\n- [Shurale](https://github.com/BobaZooba/shurale)\n  - [Shurale7B-v1](https://huggingface.co/BobaZooba/Shurale7b-v1)\n  - [Shurale7B-v1-GPTQ](https://huggingface.co/BobaZooba/Shurale7b-v1-GPTQ)\n\n## Badge\n\nConsider adding a badge to your model card.\n\nFor Github project:\n```sh\n[\u003cimg src=\"https://github.com/BobaZooba/xllm/blob/main/static/images/xllm-badge.png\" alt=\"Powered by X—LLM\" width=\"175\" height=\"32\"/\u003e](https://github.com/BobaZooba/xllm)\n```\n\nFor Hugging Face Hub model card:\n```sh\n[\u003cimg src=\"https://cdn-uploads.huggingface.co/production/uploads/6074d5f1134c000d1ae10d42/JudU3rrPP5i87CfwINANO.png\" alt=\"Powered by X—LLM\" width=\"175\" height=\"32\"/\u003e](https://github.com/BobaZooba/xllm)\n```\n\nIt will be looks like this:\n\n[\u003cimg src=\"https://github.com/BobaZooba/xllm/blob/main/static/images/xllm-badge.png\" alt=\"Powered by X—LLM\" width=\"175\" height=\"32\"/\u003e](https://github.com/BobaZooba/xllm)\n\n# Testing 🧪\n\nAt the moment, we don't have Continuous Integration tests that utilize a GPU. However, we might develop these kinds of\ntests in the future. It's important to note, though, that this would require investing time into their development, as\nwell as funding for machine maintenance.\n\n# Future Work 🔮\n\n- Add more tests\n- Add RunPod deploy function\n- Add [DPO](https://arxiv.org/abs/2305.18290) components\n- Add Callbacks to `Experiment`\n- GPU CI using RunPod\n- Solve DeepSpeed + push_to_hub checkpoints bug\n- Make DeepSpeed Stage 3 + bitsandbytes works correc\n- Add multipacking\n- Add adaptive batch size\n- Fix caching in CI\n- Add sequence bucketing\n- Add more datasets\n- Maybe add [tensor_parallel](https://github.com/BlackSamorez/tensor_parallel)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobazooba%2Fxllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbobazooba%2Fxllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobazooba%2Fxllm/lists"}