{"id":14964641,"url":"https://github.com/pratyushasharma/laser","last_synced_at":"2025-11-17T03:23:05.345Z","repository":{"id":214017470,"uuid":"732543215","full_name":"pratyushasharma/laser","owner":"pratyushasharma","description":"The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction","archived":false,"fork":false,"pushed_at":"2024-07-09T08:10:35.000Z","size":2362,"stargazers_count":389,"open_issues_count":13,"forks_count":35,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-11-03T01:04:04.100Z","etag":null,"topics":["gpt-j","interpretability","laser","llama2","llm","llms","model-compression","transformers"],"latest_commit_sha":null,"homepage":"https://pratyushasharma.github.io/laser/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pratyushasharma.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-17T02:50:10.000Z","updated_at":"2025-10-30T01:22:14.000Z","dependencies_parsed_at":"2024-01-18T00:57:00.116Z","dependency_job_id":"aad1cdcb-6b97-48a6-a587-91c857a438fd","html_url":"https://github.com/pratyushasharma/laser","commit_stats":{"total_commits":45,"total_committers":4,"mean_commits":11.25,"dds":"0.15555555555555556","last_synced_commit":"d391375b0e49f9ed0133bed38f36570c2d1d2043"},"previous_names":["pratyushasharma/laser"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/pratyushasharma/laser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratyushasharma%2Flaser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratyushasharma%2Flaser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratyushasharma%2Flaser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratyushasharma%2Flaser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pratyushasharma","download_url":"https://codeload.github.com/pratyushasharma/laser/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratyushasharma%2Flaser/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":282383958,"owners_count":26660308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-02T02:00:06.609Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpt-j","interpretability","laser","llama2","llm","llms","model-compression","transformers"],"created_at":"2024-09-24T13:33:33.679Z","updated_at":"2025-11-17T03:23:05.329Z","avatar_url":"https://github.com/pratyushasharma.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Layer-Selective Rank Reduction\n\nThis repository contains code for the paper _\"The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction,\"_ by Pratyusha Sharma, Jordan T. Ash and Dipendra Misra [ICLR 2024](https://arxiv.org/pdf/2312.13558.pdf). \n\n**Website:** [https://pratyushasharma.github.io/laser/](https://pratyushasharma.github.io/laser)\n\n**Updates:**\n- **Jan 18th, 2024:** Refactoring is happening in the [refactor](https://github.com/pratyushasharma/laser/tree/refactor) branch. We are working to release it quickly and thank you for your patience.\n- **Jan 7th, 2024:** Results table has been created on the [website](https://pratyushasharma.github.io/laser/index.html#results). \n- **Jan 4th, 2024:** Discussions page is open. Feel free to use it to suggest new topics/ideas/results that are not covered by issues.\n\n**This is an early development release. We will do a major refactor in Jan 2024 to make the code easier to use and more flexible.** \n\nWe welcome issues and pull requests. If you report a new result using LASER on a given LLM and NLP task, please issue a pull request and we'll add it to the website's leaderboard.\n\n## What is Layer-Selective Rank Reduction?\n\n**LA**yer-**SE**lective **R**ank-Reduction, abbreviated as LASER, is an intervention where we replace a selected weight matrix in the transformer architecture of an LLM with its low-rank approximation. A single LASER transformation consists of 3 hyperparameters: the layer number to modify (\u0026ell;) such as 16th layer, the parameter type (\u0026tau;) such as the first MLP layer, and the fraction of the maximum rank to retain (\u0026rho;) such as 0.01 fraction of the rank. We can write this transformation as (\u0026ell;, \u0026tau;, \u0026rho;) and we can compose these transformations and apply them in parallel. The low-rank approximation is performed using SVD. Figure below from our paper shows an illustration.\n\n![LASER illustration](https://pratyushasharma.github.io/laser/images/main.png)\n\nLASER can give significant performance improvements on question-answerting tasks without additional model training. Our paper presents various results related to evaluating LASER on 3 different LLMs and several LLM benchmarks. This repository contains the code to reproduce these results.\n\n## How to run a sample code\n\nWe first discuss installing the code and then discuss how to run an experiment.\n\n### Installation\n\nTo install the experiment, please install the pip file. We chiefly just need pytorch and the datasets and transformers package from huggingface. It might be a good idea to create a conda environment.\n\n```bash\npip3 install -r requirements.txt\n```\n\nOptionally, if you want to experiment with the CounterFact dataset then run the following script to download it. All other datasets are available on HuggingFace.\n\n```bash\npython scripts/get_counterfact.py\n```\n\n### Run a sample code\n\nAt the moment, each setup is its own file. To run an experiment that performs a single LASER transformer to GPTJ on the Fever dataset, you can run:\n\n```bash\npython3 intervention_gptj_fever.py --lname fc_in --rate 9.9 --lnum 26\n```\n\nhere _lnum_ is \u0026ell;, _lname_ is \u0026tau;, and _rate_ is related to \u0026rho; by \u0026rho; = 1 - 0.1 * rate. The rate is a value between [0, 10.0] and measures how many components to throw away with 10 means all components are thrown away and we get a 0 matrix and 0 means all components are retained and we retain the original matrix. The use of rate is for legacy reasons and we will refactor the code to directly use \u0026rho; in the future. The mapping for _lname_ that we use is:\n\n**lname** | **description**| \n--- | --- |\ndont | use the base model and dont perform intervention |\nfc_in | first layer of MLP |\nfc_out | second layer of MLP | \nfc_up | a third MLP weight matrix in some LLM, used for Hadamard multiplication | \nmlp | all MLP weight matrices {fc_in, fc_up, fc_out} | \nk_proj | key matrix in self attention | \nv_proj | value matrix in self attention | \nq_proj | query matrix in self attention | \nout_proj | output matrix in self attention |\nattn | all attention weight matrices |\n\n**Please do note that if you add a new LLM, then you have to adapt the laser package to implement mappings.** For example, see the mappings for Llama2 [here](https://github.com/pratyushasharma/laser/blob/main/src/laser/llama2_laser.py#L22). You also need to update the Laser wrapper to work with the new LLM [here](https://github.com/pratyushasharma/laser/blob/main/src/laser/LaserWrapper.py#L20).\n\nNote that the above experiments will save accuracies and log-losses for each datapoint. In some files, one has to take the validation set (first 20% examples) and do hyperparameter selection separately, and then compute the accuracy on the test set (remaining 80% examples) with the chose hyperparameters. In the future, we will refactor the code to make this very easy to do.\n\n## Code Organization\n\nCode is inside the `src` folder. The main experiment files are top-level inside the `src`. The filename convention is `intervention_\u003cllm-name\u003e_\u003cdataset-name\u003e.py` where `\u003cllm-name\u003e` is the name of the LLM and `\u003cdataset-name\u003e` is the name of the dataset. For BigBench, the dataset split is often specified with an additional flag --split. Please see the codebase for details of command line arguments. We will provide a comprehensive tutorial later.\n\nThe code for performing laser is inside the `laser` package. We use PyTorch to do SVD and compute low-rank approximation. The code for low-rank approximation happens [here](https://github.com/pratyushasharma/laser/blob/main/src/laser/matrix_utils.py#L39). The code for reading and processing dataset is inside `dataset_util`. Finally, metrics and logging are done using the `study_utils`.  \n\n## Citation\n\nIf you find this codebase useful, then please cite the following paper. Additionally, feel free to send a PR or an email and we will cite your result/paper on the leaderboard.\n\n```bash\n@article{sharma2023truth,\n \n  title={The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction},\n\n  author={Sharma, Pratyusha and Ash, Jordan T and Misra, Dipendra},\n\n journal={arXiv preprint arXiv:2312.13558},\n\n   year={2023}\n }\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpratyushasharma%2Flaser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpratyushasharma%2Flaser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpratyushasharma%2Flaser/lists"}