{"id":29075171,"url":"https://github.com/lightning-ai/litmodels","last_synced_at":"2025-06-27T15:10:24.673Z","repository":{"id":281727894,"uuid":"889082530","full_name":"Lightning-AI/LitModels","owner":"Lightning-AI","description":"Save, load, host, and share AI model checkpoints without slowing down training. Host on Lightning AI or your own cloud with enterprise-grade access controls.","archived":false,"fork":false,"pushed_at":"2025-06-17T05:33:27.000Z","size":1540,"stargazers_count":29,"open_issues_count":1,"forks_count":4,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-06-22T18:56:17.890Z","etag":null,"topics":["checkpoint","fine-tuning","model","sharing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lightning-AI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-15T15:20:49.000Z","updated_at":"2025-06-21T09:47:20.000Z","dependencies_parsed_at":"2025-03-10T21:25:06.618Z","dependency_job_id":"6dff7436-ed63-4d0b-b5e2-22e93ce7f8e3","html_url":"https://github.com/Lightning-AI/LitModels","commit_stats":null,"previous_names":["lightning-ai/models","lightning-ai/litmodels"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/Lightning-AI/LitModels","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FLitModels","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FLitModels/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FLitModels/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FLitModels/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lightning-AI","download_url":"https://codeload.github.com/Lightning-AI/LitModels/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FLitModels/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262279297,"owners_count":23286554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["checkpoint","fine-tuning","model","sharing"],"created_at":"2025-06-27T15:10:21.279Z","updated_at":"2025-06-27T15:10:24.652Z","avatar_url":"https://github.com/Lightning-AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align='center'\u003e\n\n# Save, share and host AI model checkpoints Lightning fast ⚡\n\n\u003cimg alt=\"Lightning\" src=\"https://pl-public-data.s3.us-east-1.amazonaws.com/assets_lightning/LitModels.png\" width=\"800px\" style=\"max-width: 100%;\"\u003e\n\n\u003c/div\u003e\n\n______________________________________________________________________\n\nSave, load, host, and share models without slowing down training.\n**LitModels** minimizes training slowdowns from checkpoint saving. Share public links on Lightning AI or your own cloud with enterprise-grade access controls.\n\n\u003cdiv align=\"center\"\u003e\n\n\u003cpre\u003e\n✅ Checkpoint without slowing training.  ✅ Granular access controls.           \n✅ Load models anywhere.                 ✅ Host on Lightning or your own cloud.\n\u003c/pre\u003e\n\n[![Discord](https://img.shields.io/discord/1077906959069626439?label=Get%20help%20on%20Discord)](https://discord.gg/WajDThKAur)\n![CI testing](https://github.com/Lightning-AI/LitModels/actions/workflows/ci-testing.yml/badge.svg?event=push)\n![Cloud integration](https://github.com/Lightning-AI/LitModels/actions/workflows/ci-cloud.yml/badge.svg?event=push)\n[![codecov](https://codecov.io/gh/Lightning-AI/LitModels/graph/badge.svg?token=MQ0PN2cxKo)](https://codecov.io/gh/Lightning-AI/LitModels)\n[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/LitModels/blob/main/LICENSE)\n\n\u003cdiv style=\"text-align: center;\"\u003e\n    \u003ca target=\"_blank\" href=\"#quick-start\" style=\"margin: 0 10px;\"\u003eQuick start\u003c/a\u003e •\n    \u003ca target=\"_blank\" href=\"#examples\" style=\"margin: 0 10px;\"\u003eExamples\u003c/a\u003e •\n    \u003ca target=\"_blank\" href=\"#features\" style=\"margin: 0 10px;\"\u003eFeatures\u003c/a\u003e •\n    \u003ca target=\"_blank\" href=\"#performance\" style=\"margin: 0 10px;\"\u003ePerformance\u003c/a\u003e •\n    \u003ca target=\"_blank\" href=\"#community\" style=\"margin: 0 10px;\"\u003eCommunity\u003c/a\u003e •\n    \u003ca target=\"_blank\" href=\"https://lightning.ai/docs/overview/model-registry\" style=\"margin: 0 10px;\"\u003eDocs\u003c/a\u003e\n\u003c/div\u003e\n\n\u003c/div\u003e\n\n# Quick start\n\nInstall LitModels via pip:\n\n```bash\npip install litmodels\n```\n\nToy example ([see real examples](#examples)):\n\n```python\nimport litmodels as lm\nimport torch\n\n# save a model\nmodel = torch.nn.Module()\nlm.save_model(model=model, name=\"model-name\")\n\n# load a model\nmodel = lm.load_model(name=\"model-name\")\n```\n\n# Examples\n\n\u003cdetails\u003e\n  \u003csummary\u003ePyTorch\u003c/summary\u003e\n\nSave model:\n\n```python\nimport torch\nfrom litmodels import save_model\n\nmodel = torch.nn.Module()\nsave_model(model=model, name=\"your_org/your_team/torch-model\")\n```\n\nLoad model:\n\n```python\nfrom litmodels import load_model\n\nmodel_ = load_model(name=\"your_org/your_team/torch-model\")\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003ePyTorch Lightning\u003c/summary\u003e\n\nSave model:\n\n```python\nfrom lightning import Trainer\nfrom litmodels import upload_model\nfrom litmodels.demos import BoringModel\n\n# Configure Lightning Trainer\ntrainer = Trainer(max_epochs=2)\n# Define the model and train it\ntrainer.fit(BoringModel())\n\n# Upload the best model to cloud storage\ncheckpoint_path = getattr(trainer.checkpoint_callback, \"best_model_path\")\n# Define the model name - this should be unique to your model\nupload_model(model=checkpoint_path, name=\"\u003corganization\u003e/\u003cteamspace\u003e/\u003cmodel-name\u003e\")\n```\n\nLoad model:\n\n```python\nfrom lightning import Trainer\nfrom litmodels import download_model\nfrom litmodels.demos import BoringModel\n\n# Load the model from cloud storage\ncheckpoint_path = download_model(\n    # Define the model name and version - this needs to be unique to your model\n    name=\"\u003corganization\u003e/\u003cteamspace\u003e/\u003cmodel-name\u003e:\u003cmodel-version\u003e\",\n    download_dir=\"my_models\",\n)\nprint(f\"model: {checkpoint_path}\")\n\n# Train the model with extended training period\ntrainer = Trainer(max_epochs=4)\ntrainer.fit(BoringModel(), ckpt_path=checkpoint_path)\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eTensorFlow / Keras\u003c/summary\u003e\n\nSave model:\n\n```python\nfrom tensorflow import keras\n\nfrom litmodels import save_model\n\n# Define the model\nmodel = keras.Sequential(\n    [\n        keras.layers.Dense(10, input_shape=(784,), name=\"dense_1\"),\n        keras.layers.Dense(10, name=\"dense_2\"),\n    ]\n)\n\n# Compile the model\nmodel.compile(optimizer=\"adam\", loss=\"categorical_crossentropy\")\n\n# Save the model\nsave_model(\"lightning-ai/jirka/sample-tf-keras-model\", model=model)\n```\n\nLoad model:\n\n```python\nfrom litmodels import load_model\n\nmodel_ = load_model(\n    \"lightning-ai/jirka/sample-tf-keras-model\", download_dir=\"./my-model\"\n)\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eSKLearn\u003c/summary\u003e\n\nSave model:\n\n```python\nfrom sklearn import datasets, model_selection, svm\nfrom litmodels import save_model\n\n# Load example dataset\niris = datasets.load_iris()\nX, y = iris.data, iris.target\n\n# Split dataset into training and test sets\nX_train, X_test, y_train, y_test = model_selection.train_test_split(\n    X, y, test_size=0.2, random_state=42\n)\n\n# Train a simple SVC model\nmodel = svm.SVC()\nmodel.fit(X_train, y_train)\n\n# Upload the saved model using litmodels\nsave_model(model=model, name=\"your_org/your_team/sklearn-svm-model\")\n```\n\nUse model:\n\n```python\nfrom litmodels import load_model\n\n# Download and load the model file from cloud storage\nmodel = load_model(\n    name=\"your_org/your_team/sklearn-svm-model\", download_dir=\"my_models\"\n)\n\n# Example: run inference with the loaded model\nsample_input = [[5.1, 3.5, 1.4, 0.2]]\nprediction = model.predict(sample_input)\nprint(f\"Prediction: {prediction}\")\n```\n\n\u003c/details\u003e\n\n# Features\n\n\u003cdetails\u003e\n    \u003csummary\u003ePyTorch Lightning Callback\u003c/summary\u003e\n\nEnhance your training process with an automatic checkpointing callback that uploads the model at the end of each epoch.\n\n```python\nimport torch.utils.data as data\nimport torchvision as tv\nfrom lightning import Trainer\nfrom litmodels.integrations import LightningModelCheckpoint\nfrom litmodels.demos import BoringModel\n\ndataset = tv.datasets.MNIST(\".\", download=True, transform=tv.transforms.ToTensor())\ntrain, val = data.random_split(dataset, [55000, 5000])\n\ntrainer = Trainer(\n    max_epochs=2,\n    callbacks=[\n        LightningModelCheckpoint(\n            # Define the model name - this should be unique to your model\n            model_registry=\"\u003corganization\u003e/\u003cteamspace\u003e/\u003cmodel-name\u003e\",\n        )\n    ],\n)\ntrainer.fit(\n    BoringModel(),\n    data.DataLoader(train, batch_size=256),\n    data.DataLoader(val, batch_size=256),\n)\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eSave any Python class as a checkpoint\u003c/summary\u003e\n\nMixin classes streamline model management in Python by modularizing reusable functionalities like saving/loading, enabling consistent, conflict-free, and maintainable code across multiple models.\n\n**Save model:**\n\n```python\nfrom litmodels.integrations.mixins import PickleRegistryMixin\n\n\nclass MyModel(PickleRegistryMixin):\n    def __init__(self, param1, param2):\n        self.param1 = param1\n        self.param2 = param2\n        # Your model initialization code\n        ...\n\n\n# Create and push a model instance\nmodel = MyModel(param1=42, param2=\"hello\")\nmodel.upload_model(name=\"my-org/my-team/my-model\")\n```\n\nLoad model:\n\n```python\nloaded_model = MyModel.download_model(name=\"my-org/my-team/my-model\")\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eSave custom PyTorch models\u003c/summary\u003e\n\nMixin classes centralize serialization logic, eliminating redundant code and ensuring consistent, error-free model persistence across projects.\nThe `download_model` method bypasses constructor arguments entirely, reconstructing the model directly from the registry with pre-configured architecture and weights, eliminating initialization mismatches.\n\nSave model:\n\n```python\nimport torch\nfrom litmodels.integrations.mixins import PyTorchRegistryMixin\n\n\n# Important: PyTorchRegistryMixin must be first in the inheritance order\nclass MyTorchModel(PyTorchRegistryMixin, torch.nn.Module):\n    def __init__(self, input_size, hidden_size=128):\n        super().__init__()\n        self.linear = torch.nn.Linear(input_size, hidden_size)\n        self.activation = torch.nn.ReLU()\n\n    def forward(self, x):\n        return self.activation(self.linear(x))\n\n\n# Create and push the model\nmodel = MyTorchModel(input_size=784)\nmodel.upload_model(name=\"my-org/my-team/torch-model\")\n```\n\nUse the model:\n\n```python\n# Pull the model with the same architecture\nloaded_model = MyTorchModel.download_model(name=\"my-org/my-team/torch-model\")\n```\n\n\u003c/details\u003e\n\n# Performance\n\n\u003c!--\nTODO: show the chart between not using this vs using this and the impact on training (the GPU utilization side-by-side)... also, what are tangible speed ups in training and inference.\n--\u003e\n\n# Community\n\n💬 [Get help on Discord](https://discord.com/invite/XncpTy7DSt)\\\n📋 [License: Apache 2.0](https://github.com/Lightning-AI/litModels/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Flitmodels","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightning-ai%2Flitmodels","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Flitmodels/lists"}