{"id":31211528,"url":"https://github.com/cyberagentailab/x2-icl","last_synced_at":"2025-09-21T05:30:43.178Z","repository":{"id":297130218,"uuid":"993348071","full_name":"CyberAgentAILab/x2-icl","owner":"CyberAgentAILab","description":"[ACL 2025] Code for \"Exploring Explanations Improves the Robustness of In-Context Learning\"","archived":false,"fork":false,"pushed_at":"2025-08-14T04:55:59.000Z","size":178,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-10T07:42:49.199Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CyberAgentAILab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-30T16:26:06.000Z","updated_at":"2025-08-14T04:56:02.000Z","dependencies_parsed_at":"2025-06-04T08:11:09.563Z","dependency_job_id":"4119f8e0-9b2a-42a5-b83a-792e0e88126a","html_url":"https://github.com/CyberAgentAILab/x2-icl","commit_stats":null,"previous_names":["cyberagentailab/x2-icl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CyberAgentAILab/x2-icl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fx2-icl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fx2-icl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fx2-icl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fx2-icl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CyberAgentAILab","download_url":"https://codeload.github.com/CyberAgentAILab/x2-icl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyberAgentAILab%2Fx2-icl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276195627,"owners_count":25601152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-21T05:30:41.476Z","updated_at":"2025-09-21T05:30:43.172Z","avatar_url":"https://github.com/CyberAgentAILab.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Exploring Explanations Improves the Robustness of In-Context Learning\n\nThis repository implements the main experiments of our ACL 2025 paper, [Exploring Explanations Improves the Robustness of In-Context Learning](https://aclanthology.org/2025.acl-long.1155/).\nIn our paper, we introduce an advanced ICL framework, **X^2-ICL**, which systematically **explores explanations** for all possible labels during inference.\nThis approach enables more comprehensive and robust decision-making, especially when faced with out-of-distribution data.\n\n\u003cimg src=\"image/x2icl_overview.png\" width=\"400\"\u003e\n\nThe code is intended solely for reproducing the experiments.\nIt is based on the [X-ICL](https://github.com/xlhex/acl2024_xicl.git) repository, and we gratefully acknowledge the authors of that work.\n\n\n## Environment\n\nWe tested the code in the following environment.\n\n* OS: Ubuntu 22.04.3 LTS\n* Python: 3.12\n* CUDA: 12.2\n* GPU: NVIDIA A100 (40GB)\n\n\n## Setup\n\n### Python and Packages\n\nWe used [uv](https://github.com/astral-sh/uv) to manage Python and its packages.\n```bash\ngit clone https://github.com/CyberAgentAILab/x2-icl.git\ncd x2-icl\n\n# (Optional) If this version of Python is not installed\nuv python pin 3.12\nuv python install\n\n# PyTorch needs to be installed separately\nuv venv --python 3.12 --python-preference managed\nuv pip install --no-cache --force-reinstall -r requirements.txt\nuv pip install --no-cache --force-reinstall torch==2.5.1 --index-url https://download.pytorch.org/whl/cu121\n```\n\n### API Keys\n\nThe code assumes that the API keys are stored as environment variables.\nFill in the `.envrc` file and load it.\n\n```bash\ncp .envrc_example .envrc\n\n`Fill in the .envrc file with your API keys`\n\n# If you are using direnv\ndirenv allow\n# Otherwise\nsource .envrc\n```\n\n### Data\n\nDownload the test data from the repository of [X-ICL](https://github.com/xlhex/acl2024_xicl.git) and place them under `data/testset`.\n```bash\ngit clone https://github.com/xlhex/acl2024_xicl.git\ncp -r acl2024_xicl/data/testset ./data\nrm -rf acl2024_xicl\n```\n\nThe demonstration files under [`data/prompt`](data/prompt) are named as follows:\n* `xx.txt`: Demonstrations for vanilla **ICL**. `xx` corresponds to one of {`anli_v1`, `esnli_no`, `qqp_no`}.\n* `xx_one_reason.txt`: Demonstrations for **X-ICL**. `_no` omitted.\n* `xx_all_reason.txt`: Demonstrations for our **X^2-ICL**. `_no` omitted.\n\n\n## Run\n\nBelow are sample commands for running the experiments. See [`script`](script) for more examples.  \n\n**NOTE:**\n* We used Azure OpenAI API for GPT-4o.\n* We used the specific version `gpt-4o-2024-08-06`, and the code assumes that the deployment name is the same.\n  - If you want to use a different deployment name, modify `model_libs` in [`code/utils_api.py`](code/utils_api.py).\n* We set the content filters to the least restrictive setting.\n\n```bash\nMODEL=\"gpt-4o-2024-08-06\"\nTASK=\"ANLI\"\nTEST=\"data/testset/anli_v3_hard.jsonl\"\nSEED=1000\n\n# Vanilla ICL\nuv run code/run_gpt.py \\\n  --model_name $MODEL \\\n  --train_file data/prompt/anli/seed$SEED/anli_v1.txt \\\n  --test_file $TEST \\\n  --task $TASK \\\n  --seed $SEED\n\n# X-ICL\nuv run code/run_gpt.py \\\n  --model_name $MODEL \\\n  --train_file data/prompt/anli/seed$SEED/anli_v1_one_reason.txt \\\n  --test_file $TEST \\\n  --task $TASK \\\n  --seed $SEED\n\n# X^2-ICL\nuv run code/run_gpt.py \\\n  --model_name $MODEL \\\n  --train_file data/prompt/anli/seed$SEED/anli_v1_all_reason.txt \\\n  --test_file $TEST \\\n  --task $TASK \\\n  --seed $SEED\n```\n\n\n## Citation\n\n```bibtex\n@inproceedings{honda-oka-2025-exploring,\n    title = \"Exploring Explanations Improves the Robustness of In-Context Learning\",\n    author = \"Honda, Ukyo and Oka, Tatsushi\",\n    editor = \"Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher\",\n    booktitle = \"Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\",\n    month = jul,\n    year = \"2025\",\n    address = \"Vienna, Austria\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2025.acl-long.1155/\",\n    doi = \"10.18653/v1/2025.acl-long.1155\",\n    pages = \"23693--23714\",\n    ISBN = \"979-8-89176-251-0\",\n}\n\n@inproceedings{he-etal-2024-using,\n    title = \"Using Natural Language Explanations to Improve Robustness of In-context Learning\",\n    author = \"He, Xuanli and Wu, Yuxiang and Camburu, Oana-Maria and Minervini, Pasquale and Stenetorp, Pontus\",\n    editor = \"Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek\",\n    booktitle = \"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\",\n    month = aug,\n    year = \"2024\",\n    address = \"Bangkok, Thailand\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2024.acl-long.728/\",\n    doi = \"10.18653/v1/2024.acl-long.728\",\n    pages = \"13477--13499\",\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyberagentailab%2Fx2-icl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyberagentailab%2Fx2-icl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyberagentailab%2Fx2-icl/lists"}