{"id":19320911,"url":"https://github.com/eliahuhorwitz/spectral-detuning","last_synced_at":"2025-04-30T22:23:32.915Z","repository":{"id":222748672,"uuid":"755959286","full_name":"eliahuhorwitz/Spectral-DeTuning","owner":"eliahuhorwitz","description":"Official PyTorch Implementation for the \"Recovering the Pre-Fine-Tuning Weights of Generative Models\" paper (ICML 2024).","archived":false,"fork":false,"pushed_at":"2025-04-15T08:20:32.000Z","size":14069,"stargazers_count":79,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-30T02:40:52.497Z","etag":null,"topics":["deep-learning","jailbreak","llama2","llm","lora","machine-learning","mistral","stable-diffusion"],"latest_commit_sha":null,"homepage":"https://horwitz.ai/spectral_detuning/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eliahuhorwitz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-11T15:33:33.000Z","updated_at":"2025-04-15T08:20:35.000Z","dependencies_parsed_at":"2024-06-03T12:31:30.405Z","dependency_job_id":"8f81a396-ed68-42f7-add2-356fb2e1823d","html_url":"https://github.com/eliahuhorwitz/Spectral-DeTuning","commit_stats":{"total_commits":6,"total_committers":2,"mean_commits":3.0,"dds":"0.16666666666666663","last_synced_commit":"1b0c471943cc96a18223ebb7ef4f940fb1b8dcd6"},"previous_names":["eliahuhorwitz/spectral_detuning"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FSpectral-DeTuning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FSpectral-DeTuning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FSpectral-DeTuning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FSpectral-DeTuning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eliahuhorwitz","download_url":"https://codeload.github.com/eliahuhorwitz/Spectral-DeTuning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251790363,"owners_count":21644209,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","jailbreak","llama2","llm","lora","machine-learning","mistral","stable-diffusion"],"created_at":"2024-11-10T01:33:48.899Z","updated_at":"2025-04-30T22:23:32.909Z","avatar_url":"https://github.com/eliahuhorwitz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Recovering the Pre-Fine-Tuning Weights of Generative Models (ICML 2024)\nOfficial PyTorch Implementation for the \"Recovering the Pre-Fine-Tuning Weights of Generative Models\" paper (ICML 2024).  \n\u003cp align=\"center\"\u003e\n    🌐 \u003ca href=\"https://horwitz.ai/spectral_detuning/\" target=\"_blank\"\u003eProject\u003c/a\u003e | 📃 \u003ca href=\"https://arxiv.org/abs/2402.10208\" target=\"_blank\"\u003ePaper\u003c/a\u003e | 🤗 \u003ca href=\"https://huggingface.co/datasets/Eliahu/LoWRA-Bench\" target=\"_blank\"\u003eDataset\u003c/a\u003e \u003cbr\u003e\n\u003c/p\u003e\n\n![](imgs/header.gif)\n\n\n***Pre-Fine-Tuning Weight Recovery Attack Setting:***  We uncover a vulnerability in LoRA fine-tuned models wherein an attacker is \nable to undo the fine-tuning process and recover the weights of the original pre-trained model. \nThe setting for the vulnerability is as follows: \n\n(a) The attacker only has access to n different LoRA fine-tuned models. \n\n(b) The attacker assumes that all n models originated from the same source model.  \n\n(c) Using only the n visible models, the attacker attempts to recover the original source model.\n\nOur method, *Spectral DeTuning*, can perform the attack in an unsupervised and data-free manner on real models such as Stable Diffusion and Mistral. \nFor simplicity, we illustrate the attack on a single layer, in reality, the attack is carried out independently on all the fine-tuned layers.\n\n**Note: The attacker has no access to the low-rank decomposition of the fine-tuned models.**\n___\n\n\u003e **Recovering the Pre-Fine-Tuning Weights of Generative Models**\u003cbr\u003e\n\u003e Eliahu Horwitz, Jonathan Kahana, Yedid Hoshen\u003cbr\u003e\n\u003e \u003ca href=\"https://arxiv.org/abs/2402.10208\" target=\"_blank\"\u003ehttps://arxiv.org/abs/2402.10208\u003c/a\u003e \u003cbr\u003e\n\u003e\n\u003e**Abstract:** The dominant paradigm in generative modeling consists of two steps: \n\u003e i) pre-training on a large-scale but unsafe dataset, ii) aligning the pre-trained model with human values via fine-tuning.\n\u003e This practice is considered safe, as no current method can recover the unsafe, *pre-fine-tuning* model weights. \n\u003e In this paper, we demonstrate that this assumption is often false. Concretely, we present *Spectral DeTuning*, \n\u003e a method that can recover the weights of the pre-fine-tuning model using a few low-rank (LoRA) fine-tuned models. \n\u003e In contrast to previous attacks that attempt to recover pre-fine-tuning capabilities, \n\u003e our method aims to recover the exact pre-fine-tuning weights. \n\u003e Our approach exploits this new vulnerability against large-scale models such as a personalized Stable Diffusion and an aligned Mistral.\n\n\n## Project Structure\nThis project consists of:\n- `spectral_detuning.py` - main file for recovering the Pre-FT weights using Spectral DeTuning.\n- `distributed_spectral_detuning.py` - Distributing Spectral DeTuning across multiple CPU cores of a single machine.\n- `increase_rank_on_plateau_scheduler.py` - rank scheduler class.\n- [`slurm`](./slurm/) - Examples for distributing Spectral DeTuning across a slurm cluster.   \n- [`lowra_bench`](./lowra_bench/) - Scripts for running inference and evaluation of the recovered weights.   \n\n\n## Installation \n1.  Clone the repo:\n```bash\ngit clone https://github.com/eliahuhorwitz/spectral_detuning.git\ncd spectral_detuning\n```\n2. Create a new environment and install the libraries:\n```bash\npython3 -m venv spectral_detuning_venv\nsource spectral_detuning_venv/bin/activate\npip install -r requirements.txt\n```\n\n\n\n## Running Spectral DeTuning for Pre-Fine-Tuning Weight Recovery \nThe `spectral_detuning.py` script is the main script in this project. \nIt handles the downloading of the LoWRA Bench dataset that is hosted \non Hugging Face.\n\nBelow are examples for running runs Spectral DeTuning for Pre-FT weight recovery on the \nLoWRA Bench dataset subset using different distribution strategies.   \n\n### Single GPU Execution\nThese use a single GPU to recover all the layers one by one *sequentially*.\n\n#### ViT\n```bash\npython spectral_detuning.py --subset=\"vit\" --output_path=\"./recovered_weights/vit/\" \\\n--start_layer=0 --n_layers_to_recover=-1 --sched_end_rank=16 --n_loras=5 \n```\n\u003e [!TIP] \n\u003e ViT contains 24 layers to recover and can be recovered *sequentially* in a few minutes on a desktop grade GPU.\n\n#### Stable Diffusion\n```bash\npython spectral_detuning.py --subset=\"stable-diffusion-1.5\" \\ \n--output_path=\"./recovered_weights/stable_diffusion_15/\" --start_layer=0 \\\n--n_layers_to_recover=-1 --sched_end_rank=32 --n_loras=5 \n```\n\u003e [!IMPORTANT] \n\u003e Stable Diffusion contains 264 layers to recover. See below for a faster option.\n\n#### Mistral SFT\n```bash\npython spectral_detuning.py --subset=\"mistral-7b-v0.1-sft\" \\\n--output_path=\"./recovered_weights/mistral7b_01_sft/\" --start_layer=0 \\\n--n_layers_to_recover=-1 --sched_end_rank=64 --n_loras=12 --n_iters=1000 \n```\n\n#### Mistral DPO\n```bash\npython spectral_detuning.py --subset=\"mistral-7b-v0.1-dpo\" \\\n--output_path=\"./recovered_weights/mistral7b_01_dpo/\" --start_layer=0 \\\n--n_layers_to_recover=-1 --sched_end_rank=64 --n_loras=8 --n_iters=1000\n```\n\u003e [!IMPORTANT] \n\u003e Mistral contains 128 layers to recover, some of them are of high dimensions (up to 4096x4096), see below for a faster option.\n\n\n### Distributed Multiprocess CPU Execution\nSince Spectral DeTuning does not require gradients or running \ninference on the model, it can run quickly even on a CPU. \nBelow are options for distributing Spectral DeTuning across the CPU cores\nof a single machine using multiple processes.     \n\nTo run using this strategy, run `distributed_spectral_detuning.py` with the same arguments as above.\nTo control the number of CPU cores to distribute across use the `--n_cpus` argument,\nset `--n_cpus=-1` to use all available core.\n\u003e [!TIP] \n\u003e ViT contains 24 layers to recover and can be recovered in minutes when distributed across desktop CPU cores.\n\n\n### Distributed Execution on a Compute Cluster\nIn cases where the model has many layers (e.g., Stable Diffusion and Mistral),\nit is recommended to distribute the recovery across a compute cluster (GPU or CPU).\nWe provide example slurm scripts under the [`slurm`](./slurm/) dir. \n\nThe main difference is the `--n_layers_to_recover` argument which controls how many layers\neach machine will recover.\n\n\u003e [!TIP] \n\u003e Spectral DeTuning can recover a *single layer* of a large model (e.g. Mistral-7B)\n\u003e in under 5 minutes on a *single desktop GPU* (e.g. RTX2080). \n\u003e The recovery speed of the entire model is a function of the number of machines in your cluster.   \n\n\n\n## Using the Recovered Pre-Fine-Tuning Weights\nTo run inference on the Pre-FT recovered weights use the following scripts:\n#### ViT: \n```bash\npython lowra_bench/inference/vit_inference.py --input_path=\"./recovered_weights/vit/\"\n```\n\n#### Stable Diffusion: \n```bash\npython lowra_bench/inference/stable_diffusion_inference.py \\\n--input_path=\"./recovered_weights/stable_diffusion/\"\n```\n\n#### Mistral SFT: \n```bash\npython lowra_bench/inference/mistral_inference.py \\\n--input_path=\"./recovered_weights/mistral7b_01_sft/\" --subset=\"mistral-7b-v0.1-sft\"\n```\n\n#### Mistral DPO: \n```bash\npython lowra_bench/inference/mistral_inference.py \\\n--input_path=\"./recovered_weights/mistral7b_01_dpo/\" --subset=\"mistral-7b-v0.1-dpo\"\n```\n\n\n## Using a Custom Dataset of Fine-tuned LoRAs and Pre-FT Models\nComing soon...\n- [ ] Preprocessing scripts for constructing a LoRA dataset similar to the LoWRA Bench one.\n\n\n\n## Citation\nIf you find this useful for your research, please use the following.\n\n```\n@inproceedings{horwitz2024recovering,\n  title={Recovering the Pre-Fine-Tuning Weights of Generative Models},\n  author={Horwitz, Eliahu and Kahana, Jonathan and Hoshen, Yedid},\n  booktitle={International Conference on Machine Learning},\n  pages={18882--18904},\n  year={2024},\n  organization={PMLR}\n}\n```\n\n\n## Acknowledgments\n- The project makes extensive use of the different Hugging Face libraries (e.g. [Diffusers](https://huggingface.co/docs/diffusers/en/index), [PEFT](https://huggingface.co/docs/peft/en/index), [Transformers](https://huggingface.co/docs/transformers/en/index)).\n- The [LoWRA Bench dataset](https://huggingface.co/datasets/Eliahu/LoWRA-Bench) is hosted on Hugging Face.\n- The fine-tuning of Mistral was performed based on the Zephyr model as seen [here](https://github.com/huggingface/alignment-handbook/tree/main).\n- The fine-tuned LoRA models for Stable Diffusion are taken from civitai and were fine-tuned by [RalFinger](https://civitai.com/user/RalFinger).\n- The rank scheduler is based on the PyTorch [ReduceLROnPlateau Scheduler](https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ReduceLROnPlateau.html).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fspectral-detuning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feliahuhorwitz%2Fspectral-detuning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fspectral-detuning/lists"}