{"id":13795544,"url":"https://github.com/Shark-NLP/OpenICL","last_synced_at":"2025-05-12T23:32:25.714Z","repository":{"id":120889349,"uuid":"606288940","full_name":"Shark-NLP/OpenICL","owner":"Shark-NLP","description":"OpenICL is an open-source framework to facilitate research, development, and prototyping of in-context learning.","archived":false,"fork":false,"pushed_at":"2023-10-03T13:22:36.000Z","size":341,"stargazers_count":556,"open_issues_count":7,"forks_count":30,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-21T13:53:12.906Z","etag":null,"topics":["in-context-learning","language-model","nlp"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Shark-NLP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-02-25T03:32:01.000Z","updated_at":"2025-04-21T09:01:56.000Z","dependencies_parsed_at":"2024-01-15T10:35:41.599Z","dependency_job_id":"39ec1ac4-0d0d-4fd2-baae-b4112cd1d2c4","html_url":"https://github.com/Shark-NLP/OpenICL","commit_stats":{"total_commits":93,"total_committers":7,"mean_commits":"13.285714285714286","dds":"0.30107526881720426","last_synced_commit":"82845a535370cf2dc4bcc6797e8a43fa3019ffe6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shark-NLP%2FOpenICL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shark-NLP%2FOpenICL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shark-NLP%2FOpenICL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shark-NLP%2FOpenICL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Shark-NLP","download_url":"https://codeload.github.com/Shark-NLP/OpenICL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253841020,"owners_count":21972580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["in-context-learning","language-model","nlp"],"created_at":"2024-08-03T23:00:58.483Z","updated_at":"2025-05-12T23:32:25.248Z","avatar_url":"https://github.com/Shark-NLP.png","language":"Python","funding_links":[],"categories":["Repositories"],"sub_categories":["Framework"],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://s1.ax1x.com/2023/03/07/ppZfEmq.png\" border=\"0\" width=600px/\u003e\n\u003c/div\u003e\n\n------\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#overview\"\u003eOverview\u003c/a\u003e •\n  \u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e •\n  \u003ca href=\"https://arxiv.org/abs/2303.02913\"\u003ePaper\u003c/a\u003e •\n  \u003ca href=\"https://github.com/Shark-NLP/OpenICL/tree/main/examples\"\u003eExamples\u003c/a\u003e •\n  \u003ca href=\"https://openicl.readthedocs.io/en/latest/index.html\"\u003eDocs\u003c/a\u003e •\n  \u003ca href=\"#citation\"\u003eCitation\u003c/a\u003e \n\u003c/p\u003e\n\n![version](https://img.shields.io/badge/version-0.1.8-blue)\n\n\n## Overview\nOpenICL provides an easy interface for in-context learning, with many state-of-the-art retrieval and inference methods built in to facilitate systematic comparison of LMs and fast research prototyping. Users can easily incorporate different retrieval and inference methods, as well as different prompt instructions into their workflow. \n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://s1.ax1x.com/2023/03/07/ppZWjmt.jpg\"  border=\"0\" /\u003e\n\u003c/div\u003e\n\n## What's News\n+ **v0.1.8** Support LLaMA and self-consistency\n\n## Installation\nNote: OpenICL requires Python 3.8+\n\n**Using Pip**\n```\npip install openicl\n```\n\n\n**Installation for local development:**\n```\ngit clone https://github.com/Shark-NLP/OpenICL\ncd OpenICL\npip install -e .\n```\n\n## Quick Start\nFollowing example shows you how to perform ICL on sentiment classification dataset.  More examples and tutorials can be found at [examples](https://github.com/Shark-NLP/OpenICL/tree/main/examples)\n\n#### Step 1: Load and prepare data\n```python\nfrom datasets import load_dataset\nfrom openicl import DatasetReader\n\n# Loading dataset from huggingface\ndataset = load_dataset('gpt3mix/sst2')\n\n# Define a DatasetReader, with specified column names where input and output are stored.\ndata = DatasetReader(dataset, input_columns=['text'], output_column='label')\n```\n\n#### Step 2: Define the prompt template (Optional)\n```python\nfrom openicl import PromptTemplate\ntp_dict = {\n    0: \"\u003c/E\u003ePositive Movie Review: \u003c/text\u003e\",\n    1: \"\u003c/E\u003eNegative Movie Review: \u003c/text\u003e\" \n}\n\ntemplate = PromptTemplate(tp_dict, {'text': '\u003c/text\u003e'}, ice_token='\u003c/E\u003e')\n```\nThe placeholder `\u003c/E\u003e` and `\u003c/text\u003e` will be replaced by in-context examples and testing input, respectively. For more detailed information about `PromptTemplate` (such as string-type template) , please see [tutorial1](https://github.com/Shark-NLP/OpenICL/blob/main/examples/tutorials/openicl_tutorial1_getting_started.ipynb).\n\n#### Step 3: Initialize the Retriever\n```python\nfrom openicl import TopkRetriever\n# Define a retriever using the previous `DataLoader`.\n# `ice_num` stands for the number of data in in-context examples.\nretriever = TopkRetriever(data, ice_num=8)\n```\nHere we use the popular \u003ca href=\"https://arxiv.org/abs/2101.06804\"\u003eTopK\u003c/a\u003e method to build the retriever. \n\n#### Step 4: Initialize the Inferencer \n```python\nfrom openicl import PPLInferencer\ninferencer = PPLInferencer(model_name='distilgpt2')\n```\n\n#### Step 5: Inference and scoring\n```python\nfrom openicl import AccEvaluator\n# the inferencer requires retriever to collect in-context examples, as well as a template to wrap up these examples.\npredictions = inferencer.inference(retriever, ice_template=template)\n# compute accuracy for the prediction\nscore = AccEvaluator().score(predictions=predictions, references=data.references)\nprint(score)\n```\n\n\n\n## Docs\n**(updating...)**\n\n[OpenICL Documentation](https://openicl.readthedocs.io/en/latest/index.html)\n\n## Citation\nIf you find this repository helpful, feel free to cite our paper:\n```bibtex\n@article{wu2023openicl,\n  title={OpenICL: An Open-Source Framework for In-context Learning},\n  author={Zhenyu Wu, Yaoxiang Wang, Jiacheng Ye, Jiangtao Feng, Jingjing Xu, Yu Qiao, Zhiyong Wu},\n  journal={arXiv preprint arXiv:2303.02913},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShark-NLP%2FOpenICL","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FShark-NLP%2FOpenICL","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShark-NLP%2FOpenICL/lists"}