{"id":18885029,"url":"https://github.com/mindful/mweaswsd","last_synced_at":"2025-04-14T21:30:49.355Z","repository":{"id":126671339,"uuid":"611144049","full_name":"Mindful/MWEasWSD","owner":"Mindful","description":"Repo for the paper \"MWE as WSD: Solving Multi-Word Expression Identification with Word Sense Disambiguation\"","archived":false,"fork":false,"pushed_at":"2024-08-24T04:54:19.000Z","size":1174,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-24T05:43:37.084Z","etag":null,"topics":["multiword-expressions","mwe","nlp","word-sense-disambiguation","wsd"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mindful.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.MD","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-08T07:56:16.000Z","updated_at":"2024-08-24T04:54:23.000Z","dependencies_parsed_at":"2023-11-12T07:27:36.109Z","dependency_job_id":null,"html_url":"https://github.com/Mindful/MWEasWSD","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mindful%2FMWEasWSD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mindful%2FMWEasWSD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mindful%2FMWEasWSD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mindful%2FMWEasWSD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mindful","download_url":"https://codeload.github.com/Mindful/MWEasWSD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223644608,"owners_count":17178766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multiword-expressions","mwe","nlp","word-sense-disambiguation","wsd"],"created_at":"2024-11-08T07:16:03.703Z","updated_at":"2024-11-08T07:16:04.379Z","avatar_url":"https://github.com/Mindful.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MWE as WSD\nRepo for the paper `MWE as WSD: Solving Multiword Expression Identification with Word\nSense Disambiguation`.\n\n## Installation\n```shell\ncd MWEasWSD\npip install -e .\n```\n\n## Data\n\n```shell\ncd data \u0026\u0026 ./get-data.sh\n```\n\nWhile our repository contains all the code necessary to recreate the modified SemCor data\nused in our experiments, we also make the fully processed data with synthetic negatives and \nour annotations available in the `data/augmented` directory.\nNote that the data has been converted to a JSON format. \nIf you use the SemCor data in any capacity, please cite the original \nauthors as mentioned [here](http://lcl.uniroma1.it/wsdeval/training-data).\n\n### Preprocessing\n\nGenerate `.jsonl` files and split data. \n```shell\n# Convert the data into .jsonl files\npython scripts/preprocessing/wsd_xml.py data/WSD_Evaluation_Framework/Training_Corpora/SemCor\ncp data/WSD_Evaluation_Framework/Evaluation_Datasets/ALL/ALL.data.xml data/WSD_Evaluation_Framework/Evaluation_Datasets/ALL/all.data.xml\ncp data/WSD_Evaluation_Framework/Evaluation_Datasets/ALL/ALL.gold.key.txt data/WSD_Evaluation_Framework/Evaluation_Datasets/ALL/all.gold.key.txt\nfor d in \"data/WSD_Evaluation_Framework/Evaluation_Datasets/\"*\"/\";\n  do python scripts/preprocessing/wsd_xml.py \"$d\"\ndone\n\npython scripts/preprocessing/connl_u.py --data_type cupt data/parseme_mwe/EN/train.cupt data/cupt_train.jsonl\npython scripts/preprocessing/connl_u.py --data_type cupt data/parseme_mwe/EN/test.cupt data/cupt_test.jsonl\n\npython scripts/preprocessing/dimsum.py data/dimsum-data/dimsum16.test data/dimsum_test.jsonl\npython scripts/preprocessing/dimsum.py data/dimsum-data/dimsum16.train data/dimsum_train.jsonl\n\n\npython scripts/preprocessing/random_split.py data/cupt_train.jsonl\npython scripts/preprocessing/random_split.py data/dimsum_train.jsonl\npython scripts/preprocessing/fix_existing_mwes.py data/WSD_Evaluation_Framework/Training_Corpora/SemCor/semcor.jsonl \n```\n\nApply annotations, add automatic negatives\n```shell\npython scripts/preprocessing/apply_mwe_annotations.py data/annotations.jsonl data/WSD_Evaluation_Framework/Training_Corpora/SemCor/semcor.fixed.jsonl\npython scripts/preprocessing/auto_add_negative_mwes.py data/WSD_Evaluation_Framework/Training_Corpora/SemCor/semcor.fixed.annotated.jsonl --target_percent 0.55\n```\n\n## Model download\nPre-trained models can be downloaded from the Hugging Face Hub. \n\n| Data                    | Architecture     | Download                                        |\n|-------------------------|------------------|-------------------------------------------------|\n| SemCor                  | Bi-encoder       | https://huggingface.co/Jotanner/mweaswsd        |\n| SemCor + PARSEME/DiMSUM | Bi-encoder       | https://huggingface.co/Jotanner/mweaswsd-ft     |\n| SemCor                  | DCA Poly-encoder | https://huggingface.co/Jotanner/mweaswsd-dca    |\n| SemCor + PARSEME/DiMSUM | DCA Poly-encoder | https://huggingface.co/Jotanner/mweaswsd-dca-ft |\n\n## Training\n\nTo replicate our top run Bi-encoder run:\n```shell\npython scripts/training/train.py \\\n--max_epochs 15 \\\n--batch_size 16 \\\n--accumulate_grad_batches 2 \\\n--gpus 1 \\\n--swa true \\\n--gradient_clip_val 1.0 \\\n--lr 0.00001 \\\n--run_name replicate-top \\\n--encoder bert-base-uncased \\\n--enable_checkpointing true \\\n--mwe_processing true \\\n--train_data_suffix fixed.annotated.autoneg\n```\n\nFor the distinct codes attention Poly-encoder:\n```shell\npython scripts/training/train.py \\\n--max_epochs 15 \\\n--batch_size 16 \\\n--accumulate_grad_batches 2 \\\n--gpus 1 \\\n--swa false \\\n--gradient_clip_val 1.0 \\\n--lr 0.00001 \\\n--run_name replicate-top-dca \\\n--encoder bert-base-uncased \\\n--enable_checkpointing true \\\n--weight_decay 0.01 \\\n--dropout 0.1 \\\n--mwe_processing true \\\n--head_config configs/poly_distinct_codes_128.yaml \\\n--train_data_suffix fixed.annotated.autoneg\n```\n\n\n### Finetune on DiMSUM/PARSEME data\n```shell\nexport BASE_MODEL=checkpoints/replicate-top/ep14_0.73f1.ckpt # set this to the base model you want to fine tune from\n\n# add candidates using the model we want to finetune, so it learns from its own mistakes\npython scripts/preprocessing/auto_add_negative_mwes.py data/dimsum_train_train.jsonl \\\n--target_percent 1.0 --pipeline dimsum_train --no_filter_candidates --model $BASE_MODEL\npython scripts/preprocessing/assign_gold_senses.py data/dimsum_train_train.autoneg.jsonl \nmkdir data/dimsum_train_train\nmv data/dimsum_train_train.* data/dimsum_train_train\n\npython scripts/preprocessing/auto_add_negative_mwes.py data/cupt_train_train.jsonl \\\n--target_percent 1.0 --pipeline cupt_train --no_filter_candidates --model $BASE_MODEL\npython scripts/preprocessing/assign_gold_senses.py data/cupt_train_train.autoneg.jsonl \nmkdir data/cupt_train_train\nmv data/cupt_train_train.* data/cupt_train_train\n \npython scripts/training/train.py \\\n--data mixed_finetune \\\n--load_model $BASE_MODEL \\\n--max_epochs 3 \\\n--batch_size 16 \\\n--accumulate_grad_batches 2 \\\n--gpus 1 \\\n--swa true \\\n--gradient_clip_val 1.0 \\\n--lr 0.00001 \\\n--run_name mixed_finetune \\\n--enable_checkpointing true \\\n--mwe_processing true \\\n--wsd_processing false \\\n--limit_val_batches 0 \\\n--mwe_eval_pipelines cupt_sample dimsum_sample \\\n--checkpoint_metric val/cupt_sample/mwe_pipeline_f1 \\\n--train_data_suffix autoneg.sense \\\n--limit_key_candidates False\n```\n\nTo finetune on only a single dataset, just change the data argument to `--data cupt` or `--data dimsum`.\n\n## Evaluation\n\n```shell\nexport MODEL=checkpoints/replicate-top/ep14_0.73f1.ckpt # set this to the base model you want to evaluate\n\npython scripts/training/wsd_eval.py --data data/WSD_Evaluation_Framework/Evaluation_Datasets/ALL/ --model $MODEL --batch_size 2\n\npython scripts/training/mwe_out.py cupt_test --model $MODEL  --output compare/results.cupt\npython scripts/training/mwe_out.py dimsum_test --model $MODEL  --output compare/results.dimsum \n\npython data/parseme_mwe/bin/evaluate.py --pred compare/results.cupt  --gold data/parseme_mwe/EN/test.cupt \nconda activate py27 # dimsum eval requires python 2.7\npython data/dimsum-data/scripts/dimsumeval.py data/dimsum-data/dimsum16.test compare/results.dimsum \n```\n\nIf the dimsum scorer errors out, it may be necessary to comment out lines 204, 206, 456 and 458-472. \nThis does not change MWE scoring, but prevents errors from happening. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindful%2Fmweaswsd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmindful%2Fmweaswsd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindful%2Fmweaswsd/lists"}