{"id":14964587,"url":"https://github.com/eliahuhorwitz/mother","last_synced_at":"2026-02-28T19:01:58.386Z","repository":{"id":241618018,"uuid":"806976571","full_name":"eliahuhorwitz/MoTHer","owner":"eliahuhorwitz","description":"Official PyTorch Implementation for the \"Model Tree Heritage Recovery\" paper.","archived":false,"fork":false,"pushed_at":"2024-07-07T08:26:00.000Z","size":2632,"stargazers_count":55,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-10-30T00:52:01.451Z","etag":null,"topics":["deep-learning","heritage-recovery","huggingface","llama","llama2","machine-learning","model-graph","model-tree","stable-diffusion"],"latest_commit_sha":null,"homepage":"https://vision.huji.ac.il/mother/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eliahuhorwitz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-28T08:58:26.000Z","updated_at":"2024-09-23T07:57:21.000Z","dependencies_parsed_at":"2024-07-27T19:14:24.145Z","dependency_job_id":"37165281-3bd6-4c1b-90a9-5885bfed5ae9","html_url":"https://github.com/eliahuhorwitz/MoTHer","commit_stats":{"total_commits":4,"total_committers":2,"mean_commits":2.0,"dds":0.5,"last_synced_commit":"82fa4e29f865590e90168dc9fc42ff06666f1158"},"previous_names":["eliahuhorwitz/mother"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FMoTHer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FMoTHer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FMoTHer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FMoTHer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eliahuhorwitz","download_url":"https://codeload.github.com/eliahuhorwitz/MoTHer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230788236,"owners_count":18280301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","heritage-recovery","huggingface","llama","llama2","machine-learning","model-graph","model-tree","stable-diffusion"],"created_at":"2024-09-24T13:33:27.900Z","updated_at":"2025-10-01T01:31:05.669Z","avatar_url":"https://github.com/eliahuhorwitz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Unsupervised Model Tree Heritage Recovery\nOfficial PyTorch Implementation for the \"Unsupervised Model Tree Heritage Recovery\" paper (ICLR 2025).  \n\u003cp align=\"center\"\u003e\n    🌐 \u003ca href=\"https://horwitz.ai/mother\" target=\"_blank\"\u003eProject\u003c/a\u003e | 📃 \u003ca href=\"https://arxiv.org/abs/2405.18432\" target=\"_blank\"\u003ePaper\u003c/a\u003e| 🤗 \u003ca href=\"https://huggingface.co/MoTHer-VTHR\" target=\"_blank\"\u003eDataset\u003c/a\u003e \u003cbr\u003e\n\u003c/p\u003e\n\n![](imgs/header.gif)\n\nOur proposed *Model Graphs* and *Model Trees* are new data structures for describing the heredity training relations between models.\nIn these structures, heredity relations are represented as directed edges. \nWe introduce the task of *Model Tree Heritage Recovery* (MoTHer Recovery), its goal is to uncover the \nunknown structure of Model Graphs based on the weights of a set of input models.\n___\n![](imgs/poster.png) \n\u003e **Unsupervised Model Tree Heritage Recovery**\u003cbr\u003e\n\u003e Eliahu Horwitz, Asaf Shul, Yedid Hoshen\u003cbr\u003e\n\u003e \u003ca href=\"https://arxiv.org/abs/2405.18432\" target=\"_blank\"\u003ehttps://arxiv.org/abs/2405.18432 \u003cbr\u003e\n\u003e\n\u003e**Abstract:** The number of models shared online has recently skyrocketed, with over one \n\u003e million public models available on Hugging Face. Sharing models allows other users to build \n\u003e on existing models, using them as initialization for fine-tuning, improving accuracy, and \n\u003e saving compute and energy. However, it also raises important intellectual property issues, \n\u003e as fine-tuning may violate the license terms of the original model or that of its training data. \n\u003e A Model Tree, i.e., a tree data structure rooted at a foundation model and having directed \n\u003e edges between a parent model and other models directly fine-tuned from it (children), would \n\u003e settle such disputes by making the model heritage explicit. Unfortunately, current models are \n\u003e not well documented, with most model metadata (e.g., \"model cards\") not providing accurate \n\u003e information about heritage. In this paper, we introduce the task of \n\u003e *Unsupervised Model Tree Heritage Recovery* (Unsupervised MoTHer Recovery) for collections \n\u003e of neural networks. For each pair of models, this task requires: i) determining if they are \n\u003e directly related, and ii) establishing the direction of the relationship. Our hypothesis is \n\u003e that model weights encode this information, the challenge is to decode the underlying tree \n\u003e structure given the weights. We discover several properties of model weights that allow us to \n\u003e perform this task. By using these properties, we formulate the MoTHer Recovery task as \n\u003e finding a directed minimal spanning tree. In extensive experiments we demonstrate that our \n\u003e method successfully reconstructs complex Model Trees.\n\n## Installation \n1.  Clone the repo:\n```bash\ngit clone https://github.com/eliahuhorwitz/MoTHer.git\ncd MoTHer\n```\n2. Create a new environment and install the libraries:\n```bash\npython3 -m venv mother_venv\nsource mother_venv/bin/activate\npip install -r requirements.txt\n```\n\n\n## The VTHR Dataset \nThe ViT Tree Heritage Recovery (VTHR) dataset is a dataset that was created for the purpose of evaluating the MoTHer Recovery task. \nThe dataset contains three splits: i) FT - Fully fine-tuned models, ii) LoRA-V - ViT models that were fine-tuned with LoRA with varying ranks, \nand iii) LoRA-F - ViT models that were fine-tuned with LoRA of rank 16. \n\nEach split contains a Model Graph with 105 models in 3 levels of hierarchy and with 5 Model Trees. \nAll the models for the VTHR dataset are hosted on Hugging Face under the [https://huggingface.co/MoTHer-VTHR](https://huggingface.co/MoTHer-VTHR) organization.\nTo easily process all the models of a Model Graph, we provide a pickle file per script that contains the \noriginal tree structure and the paths for each model. The pickle files are located in the `dataset` directory.\n\nEach of the splits is roughly 30GB in size, there is **no need** to download the dataset in advance, the code will take care of this for you.\n\n## Running MoTHer on the VTHR Dataset \nBelow are instructions to run MoTHer Recovery on the different splits. \nWe start by assuming the models are already clustered into the different Model Trees. \nWe will later discuss how to perform this clustering.\n\n### Running on Model Graphs with known model clusters\n\n#### Running on the FT Split\nRun the MoTHer Recovery on the FT split:\n```bash\npython MoTHer_FullFT.py\n```\n\n#### Running on the LoRA Splits\nRun the MoTHer Recovery on the LoRA-V and LoRA-F splits:\n```bash\npython MoTHer_LoRA.py\n```\n\n\n### Running on Model Graphs with multiple Model Trees \nWhen running on models from different Model Trees (i.e., a Model Graph), running the clustering is needed.\nWe provide a script that shows the clustering accuracy for both the LoRA-V and the FT splits.\nYou can change the 'LORA' flag to switch between the two splits.\n\n```bash\npython clustering.py  \n```\n\n\n\n## Citation\nIf you find this useful for your research, please use the following.\n\n```\n@inproceedings{\nhorwitz2025unsupervised,\ntitle={Unsupervised Model Tree Heritage Recovery},\nauthor={Eliahu Horwitz and Asaf Shul and Yedid Hoshen},\nbooktitle={The Thirteenth International Conference on Learning Representations},\nyear={2025},\nurl={https://openreview.net/forum?id=QVj3kUvdvl}\n}\n```\n\n\n## Acknowledgments\n- The project makes extensive use of the different Hugging Face libraries (e.g. [Diffusers](https://huggingface.co/docs/diffusers/en/index), [PEFT](https://huggingface.co/docs/peft/en/index), [Transformers](https://huggingface.co/docs/transformers/en/index)).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fmother","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feliahuhorwitz%2Fmother","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fmother/lists"}