{"id":21441760,"url":"https://github.com/dmis-lab/gener","last_synced_at":"2025-07-14T17:32:15.266Z","repository":{"id":45056093,"uuid":"438859733","full_name":"dmis-lab/GeNER","owner":"dmis-lab","description":"Simple Questions Generate Named Entity Recognition Datasets (EMNLP 2022)","archived":false,"fork":false,"pushed_at":"2023-04-10T06:26:04.000Z","size":86507,"stargazers_count":73,"open_issues_count":1,"forks_count":9,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-05-14T00:23:18.509Z","etag":null,"topics":["dataset","named-entity-recognition","ner","nlp"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2112.08808","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmis-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-16T04:30:22.000Z","updated_at":"2024-01-04T17:04:13.000Z","dependencies_parsed_at":"2023-01-23T02:30:41.769Z","dependency_job_id":null,"html_url":"https://github.com/dmis-lab/GeNER","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FGeNER","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FGeNER/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FGeNER/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FGeNER/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmis-lab","download_url":"https://codeload.github.com/dmis-lab/GeNER/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225990493,"owners_count":17556152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataset","named-entity-recognition","ner","nlp"],"created_at":"2024-11-23T01:42:04.346Z","updated_at":"2024-11-23T01:42:04.963Z","avatar_url":"https://github.com/dmis-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GeNER\nThis repository provides the official code for **GeNER** (an automated dataset **Ge**neration framework for **NER**).\n\n### Updates\n\n* **Nov. 2022:** Synthetic datasets generated by GeNER can be downloaded (see below).\n```bash\nwget http://nlp.dmis.korea.edu/projects/gener-kim-et-al-2022/synthetic_datasets-221105.zip\n```\n* **Nov. 2022:** Several hyperparameters have been updated for better reproducibility. Please check out the changes in Makefile and configuration files.\n* **Oct. 2022:** Our paper (GeNER) is accepted to [EMNLP 2022](https://2022.emnlp.org/)! Camera-ready version is now available online ([link](https://arxiv.org/abs/2112.08808)).\n\n## Overview of GeNER\n\nGeNER allows you to build NER models for specific entity types of interest. The core idea is to **ask simple natural language questions** to an open-domain question answering (QA) system and then **retrieve phrases and sentences**, as shown in the query formulation and retrieval stages in the figure below. Please see our paper ([Simple Questions Generate Named Entity Recognition Datasets](https://arxiv.org/abs/2112.08808)) for details.\n\n\u003cdiv align=\"center\"\u003e\n      \u003cimg src=\"docs/images/overview.PNG\" width=\"95%\"\u003e\n\u003c/div\u003e\n\n## Requirements\n\nPlease follow the instructions below to set up your environment and install GeNER.\n\n```bash\n# Create a conda virtual environment\nconda create -n GeNER python=3.8\nconda activate GeNER\n\n# Install PyTorch v1.9.0\npip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html\n\n# Install GeNER\ngit clone https://github.com/dmis-lab/GeNER.git\ncd GeNER\npip install -r requirements.txt\n```\n\n### NER Benchmarks\n\nRun `unzip data/benchmarks.zip -d ./data` to unpack (pre-processed) NER benchmarks.\n\n### QA Model and Phrase Index: DensePhrases\n\nWe use DensePhrases and a Wikipedia index precomputed by DensePhrases in order to automatically generate NER datasets.\nAfter installing [DensePhrases v1.0.0](https://github.com/princeton-nlp/DensePhrases#installation), please download **the DensePhrases model** (\u003cem\u003edensephrases-multi-query-multi\u003c/em\u003e) and **the phrase index** (\u003cem\u003edensephrases-multi_wiki-20181220\u003c/em\u003e) in the official DensePhrases repository.\n\n* [[GitHub](https://github.com/princeton-nlp/DensePhrases)] [[Paper](https://arxiv.org/abs/2012.12624)] [[Demo](http://densephrases.korea.ac.kr/)]\n\n### AutoPhrase (Optional)\n\nUsing AutoPhrase in the dictionary matching stage usually improves final NER performance.\nIf you are using AutoPhrase to apply Rule 10 (i.e., refining entity boundaries), please check the system requirements in the AutoPhrase repository.\nIf you are not using AutoPhrase, set `refine_boundary` to `false` in a configuration file in the `configs` directory.\n\n* [[GitHub](https://github.com/shangjingbo1226/AutoPhrase)] [[Paper](https://arxiv.org/abs/1702.04457)]\n\n### Computational Resource\nPlease see the resource requirement of DensePhrases and self-training, and check available resources of your machine. \n\n* **100GB RAM** and **a single 11G GPU** to run DensePhrases \n* **Single 6G GPU** to perform self-training (based on batch size 16)\n\n## Reproducing Experiments\n\nGeNER is implemented as a pipeline of DensePhrases, dictionary matching, and AutoPhrase.\nThe entire pipeline is controlled by configuration files located in the `configs` directory.\nPlease see `configs/README.md` for details.\n\nWe have already set up configuration files and optimal hyperparameters for all benchmarks and experiments so that you can easily reproduce similar or better performance to those presented in our paper. Just follow the instructions below for reproduction!\n\n### Example: low-resource NER (CoNLL-2003)\n\nThis example is intended to reproduce the experiment in the low-resource NER setting on the CoNLL-2003 benchmark. If you want to reproduce other experiments, you will need to change some arguments including `--gener_config_path` according to the target benchmark.\n\n#### Retrieval\n\nRunning `retrieve.py` will create `*.json` and `*.raw` files in the `data/retrieved/conll-2003` directory.\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\nexport DENSEPHRASES_PATH={enter your densephrases path here}\nexport CONFIG_PATH=./configs/conll-2003.json\n\npython retrieve.py \\\n      --run_mode eval \\\n      --model_type bert \\\n      --cuda \\\n      --aggregate \\\n      --truecase \\\n      --return_sent \\\n      --pretrained_name_or_path SpanBERT/spanbert-base-cased \\\n      --dump_dir $DENSEPHRASES_PATH/outputs/densephrases-multi_wiki-20181220/dump/ \\\n      --index_name start/1048576_flat_OPQ96  \\\n      --load_dir $DENSEPHRASES_PATH/outputs/densephrases-multi-query-multi/  \\\n      --gener_config_path $CONFIG_PATH\n```\n\n#### Applying AutoPhrase (optional)\n\n`apply_autophrase.sh` takes as input all `*.raw` files in the `data/retrieved/conll-2003` directory and outputs `*.autophrase` files in the same directory.\n\n```bash\nbash autophrase/apply_autophrase.sh data/retrieved/conll-2003\n```\n\n#### Dictionary matching\n\nRunning `annotate.py` will create `train_hf.json` in the `data/annotated/conll-2003` directory.\n\n```bash\npython annotate.py --gener_config_path $CONFIG_PATH\n```\n\n#### Self-training\n\nFinally, you can get the final NER model and see its performance. \nRunning `make conll-low` creates the `train.json` file in the `data/annotated/conll-2003` directory, which is pre-processed and derived from `train_tf.json`.\nThe model and training logs are stored in the `./outputs` directory. See the Makefile file for running experiments on other benchmarks.\n\n```bash\nmake conll-low\n```\n\n## Fine-tuning GeNER\n\nWhile GeNER performs well without any human-labeled data, you can further boost GeNER's performance using some training examples.\nThe way to do this is very simple: load a trained GeNER model from the `./outputs` directory and fine-tune it on training examples you have by a standard NER objective (i.e., token classification).\nWe provide a fine-tuning **script** in this repository (`self-training/run_ner.py`) and **datasets** to reproduce fine-grained and few-shot NER experiments (`data/fine-grained` and `data/few-shot` directories).\n\n```bash\nexport CUDA_VISIBLE_DEVICES=0\n\npython self_training/run_ner.py \\\n      --train_dir data/few-shot/conll-2003/conll-2003_0 \\\n      --eval_dir data/few-shot/conll-2003/conll-2003_0 \\\n      --model_type roberta \\\n      --model_name_or_path outputs/{enter GeNER model path here} \\\n      --output_dir outputs/{enter GeNER model path here} \\\n      --num_train_epochs 100 \\\n      --per_gpu_train_batch_size 64 \\\n      --per_gpu_eval_batch_size 64 \\\n      --learning_rate 1e-5 \\\n      --do_train \\\n      --do_eval \\\n      --do_predict \\\n      --evaluate_during_training\n\n# Note that this hyperparameter setup may not be optimal. It is recommended to search for more effective hyperparameters, especially the learning rate.\n```\n\n## Building NER Models for Your Specific Needs\n\nThe main benefit of GeNER is that you can create NER datasets of new and different entity types you want to extract.\nSuppose you want to extract \u003cem\u003efighter aircraft\u003c/em\u003e names.\nThe first thing you have to do is to formulate your needs as natural language questions such as \"\u003cem\u003eWhich fighter aircraft?\u003c/em\u003e.\"\nAt this stage, we recommend using the [DensePhrases demo](http://densephrases.korea.ac.kr/) to manually check the feasibility of your questions.\nIf relevant phrases are retrieved well, you can proceed to the next step.\n\nNext, you should make a configuration file (e.g., fighter_aircraft_config.json) and set up its values.\nYou can reflect questions you made in the configuration file as follows: `\"subtype\": \"fighter aircraft\"`.\nAlso, you can fine-tune some hyperparameters such as `top_k` and normalization rules.\nSee `configs/README.md` for detailed descriptions of configuration files.\n\n```bash\n{\n    \"retrieved_path\": \"data/retrieved/{file name}\",\n    \"annotated_path\": \"data/annotated/{file name}\",\n    \"add_abbreviation\": true,\n    \"refine_boundary\" : true,\n    \"subquestion_configs\": [\n        {\n            \"type\": \"{the name of pre-defined entity type}\",\n            \"subtype\" : \"fighter aircraft\",\n            \"top_k\" : 5000,\n            \"split_composite_mention\": true,\n            \"remove_lowercase_phrase\": true,\n            \"remove_the\": false,\n            \"skip_lowercase_ngram\": 1\n        }\n    ]\n}\n```\n\nFor subsequent steps (i.e., retrieval, dictionary matching, and self-training), refer to the CoNLL-2003 example described above.\n\n## References\n\nPlease cite our paper if you consider GeNER to be related to your work. Thanks!\n\n```bibtex\n@inproceedings{kim-etal-2022-simple,\n    title = \"Simple Questions Generate Named Entity Recognition Datasets\",\n    author = \"Kim, Hyunjae  and\n      Yoo, Jaehyo  and\n      Yoon, Seunghyun  and\n      Lee, Jinhyuk  and\n      Kang, Jaewoo\",\n    booktitle = \"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing\",\n    month = dec,\n    year = \"2022\",\n    address = \"Abu Dhabi, United Arab Emirates\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2022.emnlp-main.417\",\n    pages = \"6220--6236\"\n}\n```\n\n## Contact\n\nFeel free to email Hyunjae Kim `(hyunjae-kim@korea.ac.kr)` if you have any questions.\n\n## License\n\nSee the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fgener","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmis-lab%2Fgener","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fgener/lists"}