{"id":19763933,"url":"https://github.com/khuangaf/concrete","last_synced_at":"2025-04-30T14:32:30.859Z","repository":{"id":61708718,"uuid":"525586073","full_name":"khuangaf/CONCRETE","owner":"khuangaf","description":"Official implementation of \"CONCRETE: Improving Cross-lingual Fact Checking with Cross-lingual Retrieval\" (COLING'22)","archived":false,"fork":false,"pushed_at":"2022-10-13T01:45:10.000Z","size":140,"stargazers_count":7,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-05-04T00:58:51.151Z","etag":null,"topics":["cross-lingual-transfer","fact-checking","low-resource-languages","multilinguality","retrieval"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/khuangaf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-17T00:31:43.000Z","updated_at":"2023-05-04T00:58:51.151Z","dependencies_parsed_at":"2022-10-20T09:00:51.780Z","dependency_job_id":null,"html_url":"https://github.com/khuangaf/CONCRETE","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuangaf%2FCONCRETE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuangaf%2FCONCRETE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuangaf%2FCONCRETE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuangaf%2FCONCRETE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/khuangaf","download_url":"https://codeload.github.com/khuangaf/CONCRETE/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224214059,"owners_count":17274524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cross-lingual-transfer","fact-checking","low-resource-languages","multilinguality","retrieval"],"created_at":"2024-11-12T04:11:42.019Z","updated_at":"2024-11-12T04:11:42.987Z","avatar_url":"https://github.com/khuangaf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CONCRETE\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/concrete-improving-cross-lingual-fact/zero-shot-cross-lingual-fact-checking-on-x)](https://paperswithcode.com/sota/zero-shot-cross-lingual-fact-checking-on-x?p=concrete-improving-cross-lingual-fact)\n\n\nSource code for the COLING'22 paper: [CONCRETE: Improving Cross-lingual Fact Checking with Cross-lingual Retrieval](https://aclanthology.org/2022.coling-1.86/).\n\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"framework_overview.png\"   style=\"margin:auto\"\u003e\u003c/p\u003e\nWe propose a fact checking framework augmented with cross-lingual retrieval for zero-shot cross-lingual fact checking, as depicted in the above figure.\n\n## Dependencies\nAll required packages are listed in `requirements.txt`.\n\n```\nconda create -n concrete python=3.7\nconda activate concrete\npip install -r requirements.txt\n```\n\n## Data\nPlease create a `data` folder under the root directory of this project and download the X-Fact data from [their repo](https://github.com/utahnlp/x-fact/tree/main/data) under this `data` directory (an example path of the tsv file is `/data/x-fact/train.all.tsv`). Please also create another directory `CORA/mDPR/retrieved_docs` and download the retrieved multilingual passages from [here](https://drive.google.com/drive/folders/11Yx4eu-ZitKPOwAbyz7JmcnNz_wURA5D?usp=sharing) into this folder (an example path of the json file is `retrieved_docs/zeroshot.xict.json`). These passages were retrieved from a multilingual passage collection using the proposed cross-lingual retriever.\n\n\n\n## Cross-lingual Retrieval\nWe develop our cross-lingual retriever based on [mDPR](https://github.com/AkariAsai/CORA/tree/main/mDPR). Our cross-lingual scripts can be found under `CORA/mDPR`. The BBC passages in the passage collection can be downloaded from [here](https://drive.google.com/drive/folders/1q4riRjPTKGD-Ji0h8Dx2E1qRTHZ1A3NV?usp=sharing). This folder contains 7 sub-folders, each denotes passages in a different langauge. The articles are downloaded using [news-please](https://github.com/fhamborg/news-please). If you want to crawl news articles with a custom range of news, please refer to [this example script](https://github.com/fhamborg/news-please/blob/master/newsplease/examples/commoncrawl.py) provided by news-please. Our crawling script is also provided in the root directory of this repo `commoncrawl.py`. Below, we illustrate our scripts for running X-ICT and how to perform inference. (This step is not needed if you only want to experiment with our retrieved passages, which is described in the previous section.)\n\n### Create X-ICT samples\nSince machine translation is computationally expensive if performed on the fly, we pre-computed all translation and created X-ICT samples before running X-ICT. To do so, head over to `CORA/mDPR` and run\n\n```\nfor idx in 0 1 2;\n    python create_ict_samples.py --passage_dir ../../data/bbc_passages/ --out_file ../../data/bbc_passages/all_ict_samples.jsonl --num_shards 4 --shard_id $idx\n```\n\n\n```\npython create_ict_samples.py --passage_dir ../../data/bbc_passages/ --out_file ../../data/bbc_passages/all_ict_samples-trans100.jsonl --num_shards 4 --shard_id 3 --is_eval\n```\n\nThis will create X-ICT samples split into 4 shards, stored under `bbc_passages/`. Note that we use the last shard to do evaluation, hence the `--is_eval` argument in computing the last shard.\n\n### Learning X-ICT\nTo learn X-ICT, simply run the following script\n\n```\nbash run_xict.sh\n```\nThis will store the checkpoint files under `CORA/mDPR/xict_outputs`. Optionally, we provided the [trained checkpoints of X-ICT](https://drive.google.com/file/d/1pY9gNieMpEIadRHe8Lxq-tRmloxbf0ZJ/view?usp=sharing).\n\n### Inference\nTo perform retrieval, we first compute the representations for all passages,\n\n```\nbash generate_multilingual_embeddings.sh\n```\nThis will generate embeddings in `.pkl` files and store them under `CORA/mDPR/embeddings_multilingual`.\n\n\nThen we perform retrieval by querying CONCRETE with each claim in X-Fact.\n\n```\nbash dense_retriever.sh\n```\n\n\n\n## Retrieval-augmented Fact Checking\n\n### Training\nTo train our cross-lingual fact checker, go to the `src` directory via `cd src`. Then, run the following command:\n\n```\npython train.py --output_dir outputs --batch_size 2 --eval_batch_size 2 --max_epoch 12 --accumulate_step 32 --model_name bert-base-multilingual-cased\n```\n\nThis will train our fact checking framework augmented with passages retrieved from our proposed retriever, CONCERETE.\n\nWe provide the checkpoint of the trained model for reproduction purposes. The weights can be downloaded [here](https://drive.google.com/file/d/16LO6R-Xjei08YyJxi1kzT-QBxY8Nrpf_/view?usp=sharing).\n\n### Evaluation\nFollowing the X-Fact paper, we use Macro F1 as the evaluation metric. To run evaluation on trained models, execute the `test.py` script as follows:\n\n```\npython test.py --checkpoint_path PATH_TO_CHECKPOINT --test_path ../data/x-fact/zeroshot.tsv --model_name bert-base-multilingual-cased\n```\n\nHere, `PATH_TO_CHECKPOINT` is the path to the checkpoint you have just trained, or you can use the weights we provided above.\n\n```bibtex\n@inproceedings{huang-etal-2022-concrete,\n    title = \"CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval\",\n    author = \"Huang, Kung-Hsiang  and\n      Zhai, ChengXiang  and\n      Ji, Heng\",\n    booktitle = \"Proceedings of the 29th International Conference on Computational Linguistics\",\n    month = oct,\n    year = \"2022\",\n    address = \"Gyeongju, Republic of Korea\",\n    publisher = \"International Committee on Computational Linguistics\",\n    url = \"https://aclanthology.org/2022.coling-1.86\",\n    pages = \"1024--1035\",\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhuangaf%2Fconcrete","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkhuangaf%2Fconcrete","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhuangaf%2Fconcrete/lists"}