{"id":18960460,"url":"https://github.com/google-research-datasets/swim-ir","last_synced_at":"2026-04-01T03:30:19.108Z","repository":{"id":207038893,"uuid":"715047220","full_name":"google-research-datasets/swim-ir","owner":"google-research-datasets","description":"SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 languages, generated using PaLM 2 and summarize-then-ask prompting.","archived":false,"fork":false,"pushed_at":"2023-11-13T23:42:22.000Z","size":206,"stargazers_count":45,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-01-01T04:33:06.806Z","etag":null,"topics":["cross-lingual","datasets","deep-learning","information-retrieval","machine-learning","multilingual","natural-language-processing","neural-information-retrieval","nlp","training-data"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2311.05800","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-research-datasets.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-11-06T11:26:39.000Z","updated_at":"2024-11-27T16:59:33.000Z","dependencies_parsed_at":"2023-11-13T20:25:49.444Z","dependency_job_id":"415a2e1d-9153-46f4-bbc5-6bfe5e36ac1a","html_url":"https://github.com/google-research-datasets/swim-ir","commit_stats":{"total_commits":8,"total_committers":2,"mean_commits":4.0,"dds":0.125,"last_synced_commit":"51cb43da80a877faac76761e66f4bc0389ac0275"},"previous_names":["google-research-datasets/swim-ir"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fswim-ir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fswim-ir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fswim-ir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fswim-ir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-research-datasets","download_url":"https://codeload.github.com/google-research-datasets/swim-ir/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239958283,"owners_count":19724925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cross-lingual","datasets","deep-learning","information-retrieval","machine-learning","multilingual","natural-language-processing","neural-information-retrieval","nlp","training-data"],"created_at":"2024-11-08T14:07:03.592Z","updated_at":"2026-04-01T03:30:18.989Z","avatar_url":"https://github.com/google-research-datasets.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# SWIM-IR\n\n\u003ch1 align=\"center\"\u003e\n\u003cimg style=\"vertical-align:middle\" width=\"200px\" height=\"296px\" src=\"https://github.com/google-research-datasets/swim-ir/blob/main/SWIM-IR-logo.png\" /\u003e\n\u003c/h1\u003e\n\n\u003ch4 align=\"center\"\u003e\n    \u003cp\u003e\n        \u003ca href=\"https://arxiv.org/abs/2311.05800\"\u003ePaper\u003c/a\u003e |\n        \u003ca href=\"#download\"\u003eDownload\u003c/a\u003e |\n        \u003ca href=\"swim-ir-datacard.md\"\u003eDataCard\u003c/a\u003e |\n        \u003ca href=\"#prompts\"\u003ePrompts\u003c/a\u003e\n    \u003cp\u003e\n\u003c/h4\u003e\n\n## Overview\n\nSWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training dataset consisting of 28 million query-passage pairs spanning 33 languages. Multilingual passages are sampled from Wikipedia and are paired with queries generated by PaLM-2 using a novel summarize-then-ask prompting (SAP) generation method.\n\nModels trained on SWIM-IR achieve good performance on XOR-Retrieve (cross-lingual), and MIRACL (multilingual). SWIM-IR based models achieved a new state-of-the-art on XTREME-UP, a cross-lingual retrieval benchmark for under-represented and scarce-data languages.\n\n## Announcements\n- [Nov 2023] SWIM-IR v1.0 currently covers a portion (10 of the 18) of the MIRACL languages in the SWIM-IR monolingual set. The cross-lingual SWIM-IR data contains synthetic training pairs for all of the languages in XOR-Retrieve and XTREME-UP.\n\n## Dataset Generation\n\n![\"Figure illustrating how the SMIM-IR dataset was created\"](SWIM-IR-Diagram-Updated.drawio.png \"SWIM-IR dataset creation.\")\n**Figure 1:** SWIM-IR dataset generation process. Sampled Wikipedia passages are provided to an LLM (PaLM-2) using the novel summarize-then-ask prompting (SAP) method.\n\nThe SWIM-IR dataset is generated by first sampling passages from Wikipedia. The passages are then provided to PaLM-2 along with a prompt that asks the model to summarize the passage. The model is then prompted to ask a question that can be answered by the passage. The end-to-end process is illustrated in Figure 1. Summarize-then-ask prompting (SAP) aids the model in generating good information seeking queries for each specific input passage.\n\n\n## Download\n\nThe SWIM-IR dataset can be downloaded using the links below:\n\n* [SWIM-IR v1.0](\nhttp://storage.googleapis.com/gresearch/swim-ir/swim_ir_v1.tar.gz\n) (Nov 9, 2023)\n\n### Data Format\n\nSWIM-IR is partitioned into three sections (/directories): `cross_lingual`,  `cross_lingual_ext`, and `monolingual`. The `cross_lingual` section contains training data that can be used for evaluation on XOR Retrieve, while the `cross_lingual_ext` section can be used for evaluations on XTERME-UP. The `monolingual` section can be used for MIRACL evaluation.\n\nEach section contains language specific JSONL files with the fields:  `_id`, `lang`, `code`, `query`, `title` and `text`. Synthetic questions generated by PaLM-2 about the passage are stored in the  `query` field. The `text` field contains a sampled passage from Wikipedia, while `title` is the title of the passage's article. \n\nFor the `monolingual` data, `lang` is the language of both the query and passage with the corresponding langauge code being stored in `code` (e.g., 'fr'). For the `cross_lingual` and `cross_lingual_ext` data, the queries are in English, while `lang` and `code` indicate the language and language code of the passage.\n\n\n\nBelow is a JSON example from SWIM-IR for a question in Chinese, \"托马斯·爱迪生在哪里发明了留声机？\" [Where did Thomas Edison invent the phonograph?]. \n\n```javascript\n{'_id': '10770836',\n'lang': 'Chinese',\n'code': 'zh',\n'query': '托马斯·爱迪生在哪里发明了留声机？', \n'title': 'Menlo Park, New Jersey',\n'text': 'Menlo Park is an unincorporated community located \\\nwithin Edison Township in Middlesex County, New Jersey, United \\\nStates. In 1876, Thomas Edison set up his home and research \\\nlaboratory in Menlo Park, which at the time was the site of an \\ unsuccessful real estate development named after the town of \\\nMenlo Park, California. While there, he earned the nickname \\\n\"the Wizard of Menlo Park\". The Menlo Park lab was significant \\\nin that it was one of the first laboratories to pursue practical \\\ncommercial applications of research. It was in his Menlo Park \\\nlaboratory that Thomas Edison invented the phonograph and developed'\n}\n```\nNote that within the SWIM-IR dataset, JSON examples are stored as JSONL with one JSON example per line. Multiple line JSON is used above to make the example more readable.\n\n### Prompts\n\nPrompts given to PaLM-2 to generate the three parts of our dataset are provided below:\n\n* [Cross-Lingual Prompts (XOR Retrieve)](XOR-Retrieve-prompts.csv)\n* [Multi-Monolingual Prompts (MIRACL)](MIRACL-prompts.csv)\n* [Cross-Lingual Extended Prompts (XTREME-UP)](xtreme-up-prompts.csv)\n\n## Paper\n\nSWIM-IR is described in datail in the paper [Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval](https://arxiv.org/abs/2311.05800) by Nandan Thakur, Jianmo Ni, Gustavo Hernández Ábrego, John Wieting, Jimmy Lin and Daniel Cer. Please cite our paper in research work that uses or discusses SWIM-IR.\n\n### BibTeX\n\n```shell\n@article{swim-ir-dataset,\n  author    = {Nandan Thakur and\n               Jianmo Ni and\n               Gustavo Hern\\'andez \\'Abrego$^\\lozenge$ and\n               John Wieting and\n               Jimmy Lin and\n               Daniel Cer},\n  title     = {Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval},\n  journal   = {CoRR},\n  volume    = {abs/2311.05800},\n  year      = {2023},\n  url       = {https://arxiv.org/abs/2311.05800},\n  eprinttype = {arXiv},\n  primaryClass={cs.IR},\n  eprint    = {2311.05800},\n}\n \n```\n\n## Contact\n\nQuestions about the SWIM-IR dataset can asked by creating an issue on this repository or by sending them to \u003ca href=\"mailto:swim-ir-dataset@googlegroups.com\u003c\"\u003e\nswim-ir-dataset@googlegroups.com\u003c/a\u003e\n\n## License\n\n\u003cp xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dct=\"http://purl.org/dc/terms/\"\u003e\u003cspan property=\"dct:title\"\u003eThe SWIM-IR dataset\u003c/span\u003e is licensed under \u003ca href=\"http://creativecommons.org/licenses/by-sa/4.0/?ref=chooser-v1\" target=\"_blank\" rel=\"license noopener noreferrer\" style=\"display:inline-block;\"\u003eCC BY-SA 4.0 \u003cimg style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1\"\u003e\u003cimg style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1\"\u003e\u003cimg style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1\"\u003e\u003c/a\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research-datasets%2Fswim-ir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-research-datasets%2Fswim-ir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research-datasets%2Fswim-ir/lists"}