{"id":13754257,"url":"https://github.com/GeekDream-x/SemEval2022-Task8-TonyX","last_synced_at":"2025-05-09T22:31:40.196Z","repository":{"id":62824188,"uuid":"462539271","full_name":"GeekDream-x/SemEval2022-Task8-TonyX","owner":"GeekDream-x","description":"Deep-learning system proposed by HFL for SemEval-2022 Task 8: Multilingual News Similarity","archived":false,"fork":false,"pushed_at":"2022-07-15T05:42:33.000Z","size":2958,"stargazers_count":40,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-16T07:33:17.199Z","etag":null,"topics":["computational-linguistics","cross-lingual","crosslingual","deep-learning","machine-learning","multi-lingual","multilingual","natural-language-processing","nlp","paper","semantic-similarity","semeval-2022","xlm-roberta"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GeekDream-x.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-02-23T01:35:19.000Z","updated_at":"2024-02-20T02:37:31.000Z","dependencies_parsed_at":"2022-11-07T11:01:48.700Z","dependency_job_id":null,"html_url":"https://github.com/GeekDream-x/SemEval2022-Task8-TonyX","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeekDream-x%2FSemEval2022-Task8-TonyX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeekDream-x%2FSemEval2022-Task8-TonyX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeekDream-x%2FSemEval2022-Task8-TonyX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GeekDream-x%2FSemEval2022-Task8-TonyX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GeekDream-x","download_url":"https://codeload.github.com/GeekDream-x/SemEval2022-Task8-TonyX/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253335719,"owners_count":21892718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-linguistics","cross-lingual","crosslingual","deep-learning","machine-learning","multi-lingual","multilingual","natural-language-processing","nlp","paper","semantic-similarity","semeval-2022","xlm-roberta"],"created_at":"2024-08-03T09:01:52.330Z","updated_at":"2025-05-09T22:31:36.751Z","avatar_url":"https://github.com/GeekDream-x.png","language":"Python","funding_links":[],"categories":["文本匹配 文本检索 文本相似度"],"sub_categories":["其他_文本生成、文本对话"],"readme":"# HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity\n\n\n# Introduction\n\nHere, we provide the implementation of our winning system in Semeval-2022 Task8 —— Multilingual News Article Similarity. This is a competition about assessing the similarity of multilingual and crosslingual news articles which covers 18 language pairs.\n\nWe proposed a linguistics-inspired model trained with a few task-specific strategies. The main techniques of our system are: 1) data augmentation, 2) multi-label loss, 3) adapted R-Drop, 4) samples reconstruction with the head-tail combination. We also present a brief analysis of some negative methods like two-tower architecture in our paper. Our system ranked 1st on the leaderboard while achieving a Pearson's Correlation Coefficient of 0.818 on the official evaluation set.\n\nFor more imformation about the contest, please refer to the official site [Semeval2022-Task8](https://competitions.codalab.org/competitions/33835).\n\nFor more detailed description of our system, please refer to our paper [HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity](https://aclanthology.org/2022.semeval-1.157/).\n\nCitation:\n```\n@inproceedings{xu-etal-2022-hfl,\n    title = \"{HFL} at {S}em{E}val-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity\",\n    author = \"Xu, Zihang  and\n      Yang, Ziqing  and\n      Cui, Yiming  and\n      Chen, Zhigang\",\n    booktitle = \"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)\",\n    month = jul,\n    year = \"2022\",\n    address = \"Seattle, United States\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2022.semeval-1.157\",\n    pages = \"1114--1120\",\n    abstract = \"This paper describes our system designed for SemEval-2022 Task 8: Multilingual News Article Similarity. We proposed a linguistics-inspired model trained with a few task-specific strategies. The main techniques of our system are: 1) data augmentation, 2) multi-label loss, 3) adapted R-Drop, 4) samples reconstruction with the head-tail combination. We also present a brief analysis of some negative methods like two-tower architecture. Our system ranked 1st on the leaderboard while achieving a Pearson{'}s Correlation Coefficient of 0.818 on the official evaluation set.\",\n}\n\n```\n\n# System Overview\n\n### System Structure\n\n\n\u003cp align=\"center\"\u003e\n    \u003cbr\u003e\n    \u003cimg src=\"./imgs/systemdiagram-q.png\" width=\"50%\"/\u003e\n    \u003cbr\u003e\n\u003c/p\u003e\n\n\n### System Performance\n\nFinally, our system got 0.818 on the evaluation set according to the official scoring system and ranked 1st out of more than 30 teams. The performance of our system on individual language pairs on the official evaluation set is as displayed below:\n\n| **Language** | en | de | es | pl | tr | ar |  ru | zh | fr | it | esen | deen | plen | zhen | esit | defr | depl | frpl |\n| :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: |\n| **Pearson's CC** | 87.19 | 84.96 | 86.64 | 75.29 | 83.54 | 79.42 | 78.47 | 76.78 | 86.53 | 86.17 | 86.35 | 85.98 | 88.18 | 81.00 | 81.97 | 68.89 | 64.31 | 82.68 |\n\n# Project Structure\n\n- `data/`\n  - `training_set.csv`: an example of training set\n  - `testing_set.csv`: an example of testing set\n- `log/`\n  - `mmregression.log`: an example of training log\n- `pretrained_model/`: pretrained model files like `pytorch_model.bin` and `config.json`\n- `saved_models`: models saved during training\n- `config.json`: configuration file for training parameters\n- `models.py`: main classes\n- `run_reg.py`: project entry\n- `utils.py`: helpful functions for data processing\n\n\n# Preparation \n\n### Model\n\nIn this project, the foundation model we choose is XLM-RoBERTa and it is easily accessible on [Hugging Face](https://huggingface.co/).\n\n### Data\n\n1. Download dataset files where the news is provided via links. \n  \n  These files are provided on the official website of this [task](https://competitions.codalab.org/competitions/33835#learn_the_details-timetable).\n\n2. Crawl the news based on the files downloaded. \n  \n  The task organizers offered [a python script](https://github.com/euagendas/semeval_8_2022_ia_downloader) that helps with this. \n\n3. Make the training and testing datasets. \n  \n  With the help of the functions provided in `utils.py`, you can easily transfer the data crawled into datasets for training. As for the dataset format, you can refer to the files in `data/`.\n\n4. Clean the texts.\n  \n  Feel free to customize your own function to clean the data like removing the URLs in the texts.\n\n5. Combine the head and tail parts. \n  \n  For the detailed description of this operation, please read our [paper](https://arxiv.org/abs/2204.04844). Helpful functions are also provided in `utils.py`.\n\n### Requirements\n\nMain tools and libraries:\n\n- NumPy: 1.21.2\n- pandas: 1.2.4\n- Python: 3.7.10\n- PyTorch: 1.9.0\n- Transformers: 4.5.1\n  \n\n# Training\n\n### Parameters Configuration\n\nCustomize the training parameters in `config.json` as you need. This is a Json dictionary like:\n\n```json\n{\n    \"learning_rate\":5e-6,\n    \"epoch\":2,\n    \"gradient_acc\":8,\n    \"batch_size\":4,\n    \"max_len\":512,\n    \"seed\":8824,\n    \"weight_decay\":1e-4,\n    \"warmup_rate\":0.1,\n    \"overall_weight\":0.75,\n    \"rdrop_weight\":0.1,\n    \"model_pretrain_dir\":\"pretrained_models/xlmr-large\", \n    \"model_save_path\":\"saved_models/best_mmregressor.pth\",\n    \"training_set_path\":\"data/training_set.csv\",\n    \"testing_set_path\":\"data/testing_set.csv\",\n    \"log_path\":\"log/mmregression.log\"\n}\n```\n\n### Run command\n\n```shell\npython  run_reg.py --params config.json\n```\nAfter running this program, you could check the log messages and model testing results in `log/mmregression.log`.\n\n\n\n# Notice\n\nFor copyright reasons, the complete datasets including the augmented one will not be provided here but the method of generating it has been introduced in our paper detailedly.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGeekDream-x%2FSemEval2022-Task8-TonyX","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FGeekDream-x%2FSemEval2022-Task8-TonyX","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGeekDream-x%2FSemEval2022-Task8-TonyX/lists"}