{"id":19168382,"url":"https://github.com/bloomberg/emnlp21_fewrel","last_synced_at":"2025-05-07T14:41:30.945Z","repository":{"id":66125162,"uuid":"424726731","full_name":"bloomberg/emnlp21_fewrel","owner":"bloomberg","description":"Code to reproduce the results of the paper 'Towards Realistic Few-Shot Relation Extraction' (EMNLP 2021)","archived":false,"fork":false,"pushed_at":"2023-06-12T21:29:58.000Z","size":80950,"stargazers_count":10,"open_issues_count":1,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-19T23:31:57.864Z","etag":null,"topics":["emnlp","emnlp2021","few-shot-learning","nlp","relation-extraction"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bloomberg.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-04T20:10:53.000Z","updated_at":"2025-04-17T11:57:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"4a8a2dc0-125d-42d5-b865-4b823fa82125","html_url":"https://github.com/bloomberg/emnlp21_fewrel","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Femnlp21_fewrel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Femnlp21_fewrel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Femnlp21_fewrel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Femnlp21_fewrel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bloomberg","download_url":"https://codeload.github.com/bloomberg/emnlp21_fewrel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252897366,"owners_count":21821430,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["emnlp","emnlp2021","few-shot-learning","nlp","relation-extraction"],"created_at":"2024-11-09T09:42:29.002Z","updated_at":"2025-05-07T14:41:30.920Z","avatar_url":"https://github.com/bloomberg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Realistic Few-Shot Relation Extraction\nThis repository contains code to reproduce the results in the paper \"Towards Realistic Few-Shot Relation Extraction\" to appear in The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021). This code is not intended to be modified or reused. It is a fork of an existing [FewRel repository](https://github.com/thunlp/FewRel) with some modifications.\n\n## Fine-tuning\n\nThe following command is to fine-tune a pre-trained model on a training dataset complying with the FewRel's format (see the [Dataset section](#dataset) below).\n\n```bash\npython -m fewrel.fewrel_eval \\\n  --train train_wiki \\\n  --test val_wiki \\\n  --encoder {\"cnn\", \"bert\", \"roberta\", \"luke\"} \\\n  --pool {\"cls\", \"cat_entity_reps\"} \\\n  --data_root data/fewrel \\\n  --pretrain_ckpt {pretrained_model_path} \\\n  --train_iter 10000 \\\n  --val_iter 1000 \\\n  --val_step 2000 \\\n  --test_iter 2000\n```\n\nThe above command will dump the fine-tuned model under `./checkpoint`. The following command can be used to get the overall accuracy for the fine-tuned model. \n\n## Overall accuracy\n```bash\npython -m fewrel.fewrel_eval \\\n  --only_test \\\n  --test val_wiki \\\n  --encoder {\"cnn\", \"bert\", \"roberta\", \"luke\"} \\\n  --pool {\"cls\", \"cat_entity_reps\"} \\\n  --data_root data/fewrel \\\n  --pretrain_ckpt {pretrained_model_path} \\ # needed for getting model config\n  --load_ckpt {trained_checkpoint_path} \\\n  --test_iter 2000\n```\n\n## P@50 for individual relations\n\nPrecision at 50 can be calculated using the following command\n\n```bash\npython -m fewrel.alt_eval \\\n  --test {test_file_name_without_extension} \\ # e.g., tacred_org \n  --encoder {\"cnn\", \"bert\", \"roberta\", \"luke\"} \\\n  --pool {\"cls\", \"cat_entity_reps\"} \\\n  --data_root {path_to_data_folder} \\\n  --pretrain_ckpt {pretrained_model_path} \\ # needed for getting model config\n  --load_ckpt {trained_checkpoint_path}\n```\n\n## Pre-trained models\n\nIn this work, several encoders are experimented with including _CNN_, _BERT_, _SpanBERT_, _RoBERTa-base_, _RoBERTa-large_, and _LUKE-base_. Most pre-trained models can be downloaded from [Hugging Face Transformers](https://huggingface.co/transformers/pretrained_models.html), and _LUKE-base_ can be downloaded from its original [GitHub repository](https://github.com/studio-ousia/luke).\n\n**Note:** the original LUKE code depends on an older version of HuggingFace Transformers, which is not compatible with the version used in this repository. To experiment with LUKE, please run script `./checkout_out_luke.sh`. This will first clone the original LUKE repository, apply the necessary changes to make luke compatible with this repo, and move the LUKE module to the correct place to make sure the code runs correctly.\n\n## Dataset\n\nThe original FewRel dataset has already be contained in the github repo (here)[./data/fewrel]. To convert other dataset (e.g., [TACRED](https://nlp.stanford.edu/projects/tacred/)) to the FewRel format, one could use [./scripts/prep_more_data.py](./scripts/prep_more_data.py).\n\n[./scripts/select_rel.py](./scripts/select_rel.py) is a script to augment an existing dataset with relations from another dataset. For example, to add a list of relations from dataset `source.json` to `destination.json` and dump the merged dataset to a file `output.json`, one can use the following command:\n\n```bash\npython scripts/select_rel.py add_rel \\\n  --src source.json \\\n  --dst destination.json \\\n  --output output.json \\\n  --rels {relations_delimitated_by_space}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomberg%2Femnlp21_fewrel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbloomberg%2Femnlp21_fewrel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomberg%2Femnlp21_fewrel/lists"}