{"id":13437500,"url":"https://github.com/mosaicml/llm-foundry","last_synced_at":"2025-05-08T22:26:18.744Z","repository":{"id":158463844,"uuid":"634046345","full_name":"mosaicml/llm-foundry","owner":"mosaicml","description":"LLM training code for Databricks foundation models","archived":false,"fork":false,"pushed_at":"2025-05-08T20:52:57.000Z","size":94819,"stargazers_count":4237,"open_issues_count":60,"forks_count":559,"subscribers_count":50,"default_branch":"main","last_synced_at":"2025-05-08T20:55:45.228Z","etag":null,"topics":["deep-learning","llm","neural-networks","nlp","pytorch"],"latest_commit_sha":null,"homepage":"https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mosaicml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-28T22:33:27.000Z","updated_at":"2025-05-08T08:28:44.000Z","dependencies_parsed_at":"2023-09-26T00:42:56.310Z","dependency_job_id":"7b1e7067-140c-4c39-8b47-3f0ed3d9bd5e","html_url":"https://github.com/mosaicml/llm-foundry","commit_stats":{"total_commits":1245,"total_committers":94,"mean_commits":13.24468085106383,"dds":0.7606425702811245,"last_synced_commit":"3269c7399add8ca30842edbeb83d0c82f7906726"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosaicml%2Fllm-foundry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosaicml%2Fllm-foundry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosaicml%2Fllm-foundry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosaicml%2Fllm-foundry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mosaicml","download_url":"https://codeload.github.com/mosaicml/llm-foundry/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253156701,"owners_count":21862986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","llm","neural-networks","nlp","pytorch"],"created_at":"2024-07-31T03:00:57.740Z","updated_at":"2025-05-08T22:26:18.723Z","avatar_url":"https://github.com/mosaicml.png","language":"Python","funding_links":[],"categories":["Statistics","Python","A01_文本生成_文本对话","微调 Fine-Tuning","Tools","7. Training \u0026 Fine-tuning Ecosystem"],"sub_categories":["大语言对话模型及数据"],"readme":"\u003c!-- SETUPTOOLS_LONG_DESCRIPTION_HIDE_BEGIN --\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/mosaicml/llm-foundry\"\u003e\n    \u003cpicture\u003e\n      \u003cimg alt=\"LLM Foundry\" src=\"./assets/llm-foundry.png\" width=\"95%\"\u003e\n    \u003c/picture\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003c!-- SETUPTOOLS_LONG_DESCRIPTION_HIDE_END --\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://pypi.org/project/llm-foundry/\"\u003e\n        \u003cimg alt=\"PyPi Version\" src=\"https://img.shields.io/pypi/pyversions/llm-foundry\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/llm-foundry/\"\u003e\n        \u003cimg alt=\"PyPi Package Version\" src=\"https://img.shields.io/pypi/v/llm-foundry\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://mosaicml.me/slack\"\u003e\n        \u003cimg alt=\"Chat @ Slack\" src=\"https://img.shields.io/badge/slack-chat-2eb67d.svg?logo=slack\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/mosaicml/llm-foundry/blob/main/LICENSE\"\u003e\n        \u003cimg alt=\"License\" src=\"https://img.shields.io/badge/License-Apache%202.0-green.svg\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003cbr /\u003e\n\n# LLM Foundry\n\nThis repository contains code for training, finetuning, evaluating, and deploying LLMs for inference with [Composer](https://github.com/mosaicml/composer) and the [MosaicML platform](https://forms.mosaicml.com/demo?utm_source=github.com\u0026utm_medium=referral\u0026utm_campaign=llm-foundry). Designed to be easy-to-use, efficient _and_ flexible, this codebase enables rapid experimentation with the latest techniques.\n\nYou'll find in this repo:\n* `llmfoundry/` - source code for models, datasets, callbacks, utilities, etc.\n* `scripts/` - scripts to run LLM workloads\n  * `data_prep/` - convert text data from original sources to StreamingDataset format\n  * `train/` - train or finetune HuggingFace and MPT models from 125M - 70B parameters\n    * `train/benchmarking` - profile training throughput and MFU\n  * `inference/` - convert models to HuggingFace or ONNX format, and generate responses\n    * `inference/benchmarking` - profile inference latency and throughput\n  * `eval/` - evaluate LLMs on academic (or custom) in-context-learning tasks\n* `mcli/` - launch any of these workloads using [MCLI](https://docs.mosaicml.com/projects/mcli/en/latest/) and the [MosaicML platform](https://www.mosaicml.com/platform)\n* `TUTORIAL.md` - a deeper dive into the repo, example workflows, and FAQs\n\n# DBRX\n\nDBRX is a state-of-the-art open source LLM trained by Databricks Mosaic team. It uses the Mixture-of-Experts (MoE) architecture and was trained with optimized versions of [Composer](https://github.com/mosaicml/composer), LLM Foundry, and [MegaBlocks](https://github.com/databricks/megablocks). The model has 132B total parameters and 36B active parameters. We have released two DBRX models:\n\n\n| Model              | Context Length | Download                                           |\n| ------------------ | -------------- | -------------------------------------------------- |\n| DBRX Base          | 32768          | https://huggingface.co/databricks/dbrx-base        |\n| DBRX Instruct      | 32768          | https://huggingface.co/databricks/dbrx-instruct    |\n\nOur model weights and code are licensed for both researchers and commercial entities. The Databricks Open Source License can be found at [LICENSE](https://github.com/databricks/dbrx/blob/main/LICENSE), and our Acceptable Use Policy can be found [here](https://www.databricks.com/legal/acceptable-use-policy-open-model).\n\nFor more information about the DBRX models, see https://github.com/databricks/dbrx.\n\n# MPT\n\nMosaic Pretrained Transformers (MPT) are GPT-style models with some special features -- Flash Attention for efficiency, ALiBi for context length extrapolation, and stability improvements to mitigate loss spikes. As part of MosaicML's Foundation series, we have open-sourced several MPT models:\n\n\n| Model              | Context Length | Download                                           | Commercial use? |\n| ------------------ | -------------- | -------------------------------------------------- | --------------- |\n| MPT-30B            | 8192           | https://huggingface.co/mosaicml/mpt-30b            | Yes             |\n| MPT-30B-Instruct   | 8192           | https://huggingface.co/mosaicml/mpt-30b-instruct   | Yes             |\n| MPT-30B-Chat       | 8192           | https://huggingface.co/mosaicml/mpt-30b-chat       | No              |\n| MPT-7b-8k          | 8192           | https://huggingface.co/mosaicml/mpt-7b-8k          | Yes             |\n| MPT-7b-8k-Chat | 8192           | https://huggingface.co/mosaicml/mpt-7b-8k-chat         | No              |\n| MPT-7B             | 2048           | https://huggingface.co/mosaicml/mpt-7b             | Yes             |\n| MPT-7B-Instruct    | 2048           | https://huggingface.co/mosaicml/mpt-7b-instruct    | Yes             |\n| MPT-7B-Chat        | 2048           | https://huggingface.co/mosaicml/mpt-7b-chat        | No              |\n| MPT-7B-StoryWriter | 65536          | https://huggingface.co/mosaicml/mpt-7b-storywriter | Yes             |\n\nTo try out these models locally, [follow the instructions](https://github.com/mosaicml/llm-foundry/tree/main/scripts/inference#interactive-generation-with-modelgenerate) in `scripts/inference/README.md` to prompt HF models using our [hf_generate.py](https://github.com/mosaicml/llm-foundry/blob/main/scripts/inference/hf_generate.py) or [hf_chat.py](https://github.com/mosaicml/llm-foundry/blob/main/scripts/inference/hf_chat.py) scripts.\n\n# MPT Community\n\nWe've been overwhelmed by all the amazing work the community has put into MPT! Here we provide a few links to some of them:\n* [ReplitLM](https://github.com/replit/replitLM): `replit-code-v1-3b` is a 2.7B Causal Language Model focused on Code Completion. The model has been trained on a subset of the Stack Dedup v1.2 dataset covering 20 languages such as Java, Python, and C++\n* [LLaVa-MPT](https://github.com/haotian-liu/LLaVA#LLaVA-MPT-7b): Visual instruction tuning to get MPT multimodal capabilities\n* [ggml](https://github.com/ggerganov/ggml/tree/master): Optimized MPT version for efficient inference on consumer hardware\n* [GPT4All](https://gpt4all.io/index.html): locally running chat system, now with MPT support!\n* [Q8MPT-Chat](https://huggingface.co/spaces/Intel/Q8-Chat): 8-bit optimized MPT for CPU by our friends at Intel\n\nTutorial videos from the community:\n* [Using MPT-7B with Langchain](https://www.youtube.com/watch?v=DXpk9K7DgMo\u0026t=3s) by [@jamesbriggs](https://www.youtube.com/@jamesbriggs)\n* [MPT-7B StoryWriter Intro](https://www.youtube.com/watch?v=O9Y_ZdsuKWQ) by [AItrepreneur](https://www.youtube.com/@Aitrepreneur)\n* [Fine-tuning MPT-7B on a single GPU](https://www.youtube.com/watch?v=KSlWkrByc0o\u0026t=9s) by [@AIology2022](https://www.youtube.com/@AIology2022)\n* [How to Fine-tune MPT-7B-Instruct on Google Colab](https://youtu.be/3de0Utr9XnI) by [@VRSEN](https://www.youtube.com/@vrsen)\n\nSomething missing? Contribute with a PR!\n\n# Latest News\n* [Blog: Introducing DBRX: A New State-of-the-Art Open LLM](https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm)\n* [Blog: LLM Training and Inference with Intel Gaudi2 AI Accelerators](https://www.databricks.com/blog/llm-training-and-inference-intel-gaudi2-ai-accelerators)\n* [Blog: Training LLMs at Scale with AMD MI250 GPUs](https://www.databricks.com/blog/training-llms-scale-amd-mi250-gpus)\n* [Blog: Training LLMs with AMD MI250 GPUs and MosaicML](https://www.mosaicml.com/blog/amd-mi250)\n* [Blog: Announcing MPT-7B-8K: 8K Context Length for Document Understanding](https://www.mosaicml.com/blog/long-context-mpt-7b-8k)\n* [Blog: Training LLMs with AMD MI250 GPUs and MosaicML](https://www.mosaicml.com/blog/amd-mi250)\n* [Blog: MPT-30B: Raising the bar for open-source foundation models](https://www.mosaicml.com/blog/mpt-30b)\n* [Blog: Introducing MPT-7B](https://www.mosaicml.com/blog/mpt-7b)\n* [Blog: Benchmarking LLMs on H100](https://www.mosaicml.com/blog/coreweave-nvidia-h100-part-1)\n* [Blog: Blazingly Fast LLM Evaluation](https://www.mosaicml.com/blog/llm-evaluation-for-icl)\n* [Blog: GPT3 Quality for $500k](https://www.mosaicml.com/blog/gpt-3-quality-for-500k)\n* [Blog: Billion parameter GPT training made easy](https://www.mosaicml.com/blog/billion-parameter-gpt-training-made-easy)\n\n\n\n# Hardware and Software Requirements\nThis codebase has been tested with PyTorch 2.4 with NVIDIA A100s and H100s.\nThis codebase may also work on systems with other devices, such as consumer NVIDIA cards and AMD cards, but we are not actively testing these systems.\nIf you have success/failure using LLM Foundry on other systems, please let us know in a Github issue and we will update the support matrix!\n\n| Device         | Torch Version | Cuda Version | Status                       |\n| -------------- | ------------- | ------------ | ---------------------------- |\n| A100-40GB/80GB | 2.6.0         | 12.4         | :white_check_mark: Supported |\n| H100-80GB      | 2.6.0         | 12.4         | :white_check_mark: Supported |\n\n## MosaicML Docker Images\nWe highly recommend using our prebuilt Docker images. You can find them here: https://hub.docker.com/orgs/mosaicml/repositories.\n\nThe `mosaicml/pytorch` images are pinned to specific PyTorch and CUDA versions, and are stable and rarely updated.\n\nThe `mosaicml/llm-foundry` images are built with new tags upon every commit to the `main` branch.\nYou can select a specific commit hash such as `mosaicml/llm-foundry:2.6.0_cu124-9867a7b` or take the latest one using `mosaicml/llm-foundry:2.6.0_cu124-latest`.\n\n**Please Note:** The `mosaicml/llm-foundry` images do not come with the `llm-foundry` package preinstalled, just the dependencies. You will still need to `pip install llm-foundry` either from PyPi or from source.\n\n| Docker Image                                           | Torch Version | Cuda Version      | LLM Foundry dependencies installed? |\n| ------------------------------------------------------ | ------------- | ----------------- | ----------------------------------- |\n| `mosaicml/pytorch:2.6.0_cu124-python3.12-ubuntu22.04`  | 2.6.0         | 12.4 (Infiniband) | No                                  |\n| `mosaicml/llm-foundry:2.6.0_cu124-latest`              | 2.6.0         | 12.4 (Infiniband) | Yes                                 |\n| `mosaicml/llm-foundry:2.6.0_cu124_aws-latest`          | 2.6.0         | 12.4 (EFA)        | Yes                                 |\n\n\n# Installation\n\nThis assumes you already have PyTorch, CMake, and packaging installed. If not, you can install them with `pip install cmake packaging torch`.\n\nTo get started, clone the repo and set up your environment. Instructions to do so differ slightly depending on whether you're using Docker.\n\n### With Docker (recommended)\n\nWe *strongly* recommend working with LLM Foundry inside a Docker container (see our recommended Docker image above). If you are doing so, follow these steps to clone the repo and install the requirements.\n\n\u003c!--pytest.mark.skip--\u003e\n```bash\ngit clone https://github.com/mosaicml/llm-foundry.git\ncd llm-foundry\npip install -e \".[gpu]\"  # or `pip install -e .` if no NVIDIA GPU.\n```\n\n### Without Docker (not recommended)\n\nIf you choose not to use Docker, you should create and use a virtual environment.\n\n\u003c!--pytest.mark.skip--\u003e\n```bash\ngit clone https://github.com/mosaicml/llm-foundry.git\ncd llm-foundry\n\n# Creating and activate a virtual environment\npython3 -m venv llmfoundry-venv\nsource llmfoundry-venv/bin/activate\n\npip install cmake packaging torch  # setup.py requires these be installed\n\npip install -e \".[gpu]\"  # or `pip install -e .` if no NVIDIA GPU.\n```\n\n### TransformerEngine and amp_fp8 support\nNVIDIA H100 GPUs have FP8 support; we have installed Flash Attention and Transformer in our Docker images already (see above). If you are not using our Docker images, you can install these packages with:\n\u003c!--pytest.mark.skip--\u003e\n```bash\npip install flash-attn --no-build-isolation\npip install git+https://github.com/NVIDIA/TransformerEngine.git@stable\n```\n\nSee [here](https://github.com/mosaicml/llm-foundry/blob/main/TUTORIAL.md#TransformerEngine-and-amp_fp8-support) for more details on enabling TransformerEngine layers and amp_fp8.\n\n### AMD (BETA support)\n\nIn [our testing of AMD GPUs](https://www.mosaicml.com/blog/amd-mi250), the env setup includes:\n\n\u003c!--pytest.mark.skip--\u003e\n```bash\ngit clone https://github.com/mosaicml/llm-foundry.git\ncd llm-foundry\n\n# Creating and activate a virtual environment\npython3 -m venv llmfoundry-venv-amd\nsource llmfoundry-venv-amd/bin/activate\n\n# installs\npip install cmake packaging torch\npip install -e .  # This installs some things that are not needed but they don't hurt\npip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2\n```\n**Lastly**, install the ROCm enabled flash attention (instructions [here](https://github.com/ROCmSoftwarePlatform/flash-attention/tree/flash_attention_for_rocm2#amd-gpurocm-support)).\n\nNotes:\n1. We don't yet have a Docker image where everything works perfectly. You might need to up/downgrade some packages (in our case, we needed to downgrade to `numpy==1.23.5`) before everything works without issue.\n\n### Intel Gaudi\nSupport for LLM Foundry on Intel Gaudi devices is experimental, please use the branch `habana_alpha` and see the [README on that branch](https://github.com/mosaicml/llm-foundry/blob/habana_alpha) which has [install instructions and known issues.](https://github.com/mosaicml/llm-foundry/tree/habana_alpha?tab=readme-ov-file#intel-gaudi)\n\nFor training and inference performance results on Intel Gaudi2 accelerators, see our blog: https://www.databricks.com/blog/llm-training-and-inference-intel-gaudi2-ai-accelerators\n\n\n# Quickstart\n\n\u003e **Note**\n\u003e Make sure to go through the installation steps above before trying the quickstart!\n\nHere is an end-to-end workflow for preparing a subset of the C4 dataset, training an MPT-125M model for 10 batches,\nconverting the model to HuggingFace format, evaluating the model on the Winograd challenge, and generating responses to prompts.\n\n**(Remember this is a quickstart just to demonstrate the tools -- To get good quality, the LLM must be trained for longer than 10 batches 😄)**\n\n\u003c!--pytest.mark.skip--\u003e\n```bash\ncd scripts\n\n# Convert C4 dataset to StreamingDataset format\npython data_prep/convert_dataset_hf.py \\\n  --dataset allenai/c4 --data_subset en \\\n  --out_root my-copy-c4 --splits train_small val_small \\\n  --concat_tokens 2048 --tokenizer EleutherAI/gpt-neox-20b --eos_text '\u003c|endoftext|\u003e'\n\n# Train an MPT-125m model for 10 batches\ncomposer train/train.py \\\n  train/yamls/pretrain/mpt-125m.yaml \\\n  variables.data_local=my-copy-c4 \\\n  train_loader.dataset.split=train_small \\\n  eval_loader.dataset.split=val_small \\\n  max_duration=10ba \\\n  eval_interval=0 \\\n  save_folder=mpt-125m\n\n# Convert the model to HuggingFace format\npython inference/convert_composer_to_hf.py \\\n  --composer_path mpt-125m/ep0-ba10-rank0.pt \\\n  --hf_output_path mpt-125m-hf \\\n  --output_precision bf16 \\\n  # --hf_repo_for_upload user-org/repo-name\n\n# Evaluate the model on a subset of tasks\ncomposer eval/eval.py \\\n  eval/yamls/hf_eval.yaml \\\n  icl_tasks=eval/yamls/copa.yaml \\\n  model_name_or_path=mpt-125m-hf\n\n# Generate responses to prompts\npython inference/hf_generate.py \\\n  --name_or_path mpt-125m-hf \\\n  --max_new_tokens 256 \\\n  --prompts \\\n    \"The answer to life, the universe, and happiness is\" \\\n    \"Here's a quick recipe for baking chocolate chip cookies: Start by\"\n```\n\nNote: the `composer` command used above to train the model refers to the [Composer](https://github.com/mosaicml/composer) library's distributed launcher.\n\nIf you have a write-enabled [HuggingFace auth token](https://huggingface.co/docs/hub/security-tokens), you can optionally upload your model to the Hub! Just export your token like this:\n\n```bash\nexport HF_TOKEN=your-auth-token\n```\n\nand uncomment the line containing `--hf_repo_for_upload ...` in the above call to `inference/convert_composer_to_hf.py`.\n\n# Registry\n\nYou can use the registry to customize your workflows without forking the library. Some components of LLM Foundry are registrable, such as models, loggers, and callbacks. This means that you can register new options for these components, and then use them in your yaml config.\n\n## Discovering registrable components\nTo help find and understand registrable components, you can use the `llmfoundry registry` cli command.\n\nWe provide two commands currently:\n- `llmfoundry registry get [--group]`: List all registries, and their components, optionally specifying a specific registry. Example usage: `llmfoundry registry get --group loggers` or `llmfoundry registry get`\n- `llmfoundry registry find \u003cgroup\u003e \u003cname\u003e`: Get information about a specific registered component. Example usage: `llmfoundry registry find loggers wandb`\n\nUse `--help` on any of these commands for more information.\n\nThese commands can also help you understand what each registry is composed of, as each registry contains a docstring that will be printed out. The general concept is that each registry defines an interface, and components registered to that registry must implement that interface. If there is a part of the library that is not currently extendable, but you think it should be, please open an issue!\n\n## How to register\n\nThere are a few ways to register a new component:\n\n### Python entrypoints\n\nYou can specify registered components via a Python entrypoint if you are building your own package with registered components.\nThis would be the expected usage if you are building a large extension to LLM Foundry, and going to be overriding many components. Note that things registered via entrypoints will override components registered directly in code.\n\nFor example, the following would register the `MyLogger` class, under the key `my_logger`, in the `llm_foundry.loggers` registry:\n\n\u003c!--pytest.mark.skip--\u003e\n```yaml\n[build-system]\nrequires = [\"setuptools\u003e=42\", \"wheel\"]\nbuild-backend = \"setuptools.build_meta\"\n\n[project]\nname = \"foundry_registry\"\nversion = \"0.1.0\"\ndependencies = [\n    \"mosaicml\",\n    \"llm-foundry\",\n]\n\n# Note: Even though in python code, this would be llmfoundry.registry.loggers,\n# when specified in the entry_points, it has to be \"llmfoundry_loggers\". That is,\n# the segments of the name should be joined by an _ in the entry_points section.\n[project.entry-points.\"llmfoundry_loggers\"]\nmy_logger = \"foundry_registry.loggers:MyLogger\"\n```\n\nIf developing new components via entrypoints, it is important to note that Python entrypoints are global to the Python environment. This means that if you have multiple packages that register components with the same key, the last one installed will be the one used. This can be useful for overriding components in LLM Foundry, but can also lead to unexpected behavior if not careful. Additionally, if you change the pyproject.toml, you will need to reinstall the package for the changes to take effect. You can do this quickly by installing with `pip install -e . --no-deps` to avoid reinstalling dependencies.\n\n### Direct call to register\n\nYou can also register a component directly in your code:\n\n\u003c!--pytest.mark.skip--\u003e\n```python\nfrom composer.loggers import LoggerDestination\nfrom llmfoundry.registry import loggers\n\nclass MyLogger(LoggerDestination):\n    pass\n\nloggers.register(\"my_logger\", func=MyLogger)\n```\n\n### Decorators\n\nYou can also use decorators to register components directly from your code:\n\n\u003c!--pytest.mark.skip--\u003e\n```python\nfrom composer.loggers import LoggerDestination\nfrom llmfoundry.registry import loggers\n\n@loggers.register(\"my_logger\")\nclass MyLogger(LoggerDestination):\n    pass\n```\n\nFor both the direct call and decorator approaches, if using the LLM Foundry train/eval scripts, you will need to provide the `code_paths` argument, which is a list of files need to execute in order to register your components. For example, you may have a file called `foundry_imports.py` that contains the following:\n\n\u003c!--pytest.mark.skip--\u003e\n```python\nfrom foundry_registry.loggers import MyLogger\nfrom llmfoundry.registry import loggers\n\nloggers.register(\"my_logger\", func=MyLogger)\n```\n\nYou would then provide `code_paths` to the train/eval scripts in your yaml config:\n\n\u003c!--pytest.mark.skip--\u003e\n```yaml\n...\ncode_paths:\n  - foundry_imports.py\n...\n```\n\nOne of these would be the expected usage if you are building a small extension to LLM Foundry, only overriding a few components, and thus don't want to create an entire package.\n\n# Learn more about LLM Foundry!\n\nCheck out [TUTORIAL.md](https://github.com/mosaicml/llm-foundry/blob/main/TUTORIAL.md) to keep learning about working with LLM Foundry. The tutorial highlights example workflows, points you to other resources throughout the repo, and answers frequently asked questions!\n\n# Contact Us\n\nIf you run into any problems with the code, please file Github issues directly to this repo.\n\nIf you want to train LLMs on the MosaicML platform, reach out to us at [demo@mosaicml.com](mailto:demo@mosaicml.com)!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmosaicml%2Fllm-foundry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmosaicml%2Fllm-foundry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmosaicml%2Fllm-foundry/lists"}