{"id":15653108,"url":"https://github.com/csinva/iprompt","last_synced_at":"2025-04-30T17:12:10.741Z","repository":{"id":65420615,"uuid":"528558477","full_name":"csinva/iprompt","owner":"csinva","description":"Finding semantically meaningful and accurate prompts.","archived":false,"fork":false,"pushed_at":"2023-10-30T19:11:54.000Z","size":86414,"stargazers_count":46,"open_issues_count":0,"forks_count":8,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-30T17:11:29.139Z","etag":null,"topics":["ai","autoprompt","deep-learning","explainability","explainable-ai","interpretability","iprompt","language-model","large-language-models","ml","natural-language-processing","neural-network","prompting","text","text-classification","xai"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2210.01848","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csinva.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-24T19:03:34.000Z","updated_at":"2024-09-21T14:34:21.000Z","dependencies_parsed_at":"2024-10-03T12:54:51.435Z","dependency_job_id":null,"html_url":"https://github.com/csinva/iprompt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fiprompt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fiprompt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fiprompt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fiprompt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csinva","download_url":"https://codeload.github.com/csinva/iprompt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251748951,"owners_count":21637418,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","autoprompt","deep-learning","explainability","explainable-ai","interpretability","iprompt","language-model","large-language-models","ml","natural-language-processing","neural-network","prompting","text","text-classification","xai"],"created_at":"2024-10-03T12:44:42.268Z","updated_at":"2025-04-30T17:12:10.696Z","avatar_url":"https://github.com/csinva.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e  Interpretable autoprompting \u003c/h1\u003e\n\u003cp align=\"center\"\u003e Natural language explanations of a \u003ci\u003edataset\u003c/i\u003e via language-model autoprompting.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"http://csinva.io/imodelsX/iprompt/api.html#imodelsx.iprompt.api.explain_dataset_iprompt\"\u003e📚 sklearn-friendly api\u003c/a\u003e •\n  \u003ca href=\"https://github.com/csinva/interpretable-autoprompting/blob/master/demo.ipynb\"\u003e📖 demo notebook\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/license-mit-blue.svg\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/python-3.6+-blue\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/v/imodelsx?color=green\"\u003e\n\u003c/p\u003e  \n\n\n\u003cb\u003eOfficial code for using / reproducing iPrompt from the paper \"Explaining Patterns in Data  with  Language Models via Interpretable Autoprompting\" (\u003ca href=\"https://arxiv.org/abs/2210.01848\"\u003eSingh*, Morris*, Aneja, Rush, \u0026 Gao, 2022\u003c/a\u003e) \u003c/b\u003e iPrompt generates a human-interpretable prompt that explains patterns in data while still inducing strong generalization performance.\n\nhttps://user-images.githubusercontent.com/4960970/197355573-e5a1af4c-0784-4344-a314-79793f284b97.mov\n\n# Quickstart\n**Installation**: `pip install imodelsx` (or, for more control, clone and install from source)\n\n**Usage example** (see [imodelsX](https://github.com/csinva/imodelsX) for more details):\n\n```python\nfrom imodelsx import explain_dataset_iprompt, get_add_two_numbers_dataset\n\n# get a simple dataset of adding two numbers\ninput_strings, output_strings = get_add_two_numbers_dataset(num_examples=100)\nfor i in range(5):\n    print(repr(input_strings[i]), repr(output_strings[i]))\n\n# explain the relationship between the inputs and outputs\n# with a natural-language prompt string\nprompts, metadata = explain_dataset_iprompt(\n    input_strings=input_strings,\n    output_strings=output_strings,\n    checkpoint='EleutherAI/gpt-j-6B', # which language model to use\n    num_learned_tokens=3, # how long of a prompt to learn\n    n_shots=3, # shots per example\n\n    n_epochs=15, # how many epochs to search\n    verbose=0, # how much to print\n    llm_float16=True, # whether to load the model in float_16\n)\n--------\nprompts is a list of found natural-language prompt strings\n```\n\n# Docs\n\u003cblockquote\u003e\n\u003cb\u003eAbstract\u003c/b\u003e: Large language models (LLMs) have displayed an impressive ability to harness natural language to perform complex tasks. In this work, we explore whether we can leverage this learned ability to find and explain patterns in data. Specifically, given a pre-trained LLM and data examples, we introduce interpretable autoprompting (iPrompt), an algorithm that generates a natural-language string explaining the data. iPrompt iteratively alternates between generating explanations with an LLM and reranking them based on their performance when used as a prompt. Experiments on a wide range of datasets, from synthetic mathematics to natural-language understanding, show that iPrompt can yield meaningful insights by accurately finding groundtruth dataset descriptions. Moreover, the prompts produced by iPrompt are simultaneously human-interpretable and highly effective for generalization: on real-world sentiment classification datasets, iPrompt produces prompts that match or even improve upon human-written prompts for GPT-3. Finally, experiments with an fMRI dataset show the potential for iPrompt to aid in scientific discovery.\n\u003c/blockquote\u003e\n\n- the main api requires simply importing `imodelsx`\n- the `experiments` and `experiments/scripts` folders contain hyperparameters for running sweeps contained in the paper\n  - note: args that start with `use_` are boolean\n- the `notebooks` folder contains notebooks for analyzing the outputs + making figures\n\n# Related work\n\n- **fMRI data experiment**: Uses scientific data/code from https://github.com/HuthLab/speechmodeltutorial linked to the paper \"Natural speech reveals the semantic maps that tile human cerebral cortex\" [Huth, A. G. et al., (2016) _Nature_.](https://www.nature.com/articles/nature17637)\n- AutoPrompt: find an (uninterpretable) prompt using input-gradients ([paper](https://arxiv.org/abs/2010.15980); [github](https://github.com/ucinlp/autoprompt))\n- Aug-imodels: Explain a dataset by fitting an interpretable linear model/decision tree leveraging a pre-trained language model ([paper](https://arxiv.org/abs/2209.11799); [github](https://github.com/csinva/emb-gam))\n\n## Testing\n- to check if the pipeline seems to work, install pytest then run `pytest` from the repo's root directory\n\nIf this package is useful for you, please cite the following!\n\n```r\n@article{singh2022iprompt,\n  title = {Explaining Patterns in Data with Language Models via Interpretable Autoprompting},\n  author = {Singh, Chandan and Morris, John X. and Aneja, Jyoti and Rush, Alexander M. and Gao, Jianfeng},\n  year = {2022},\n  url = {https://arxiv.org/abs/2210.01848},\n  publisher = {arXiv},  \n  doi = {10.48550/ARXIV.2210.01848}  \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Fiprompt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsinva%2Fiprompt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Fiprompt/lists"}