{"id":13595244,"url":"https://github.com/ShannonAI/mrc-for-flat-nested-ner","last_synced_at":"2025-04-09T10:33:16.855Z","repository":{"id":38430574,"uuid":"264440589","full_name":"ShannonAI/mrc-for-flat-nested-ner","owner":"ShannonAI","description":"Code for ACL 2020 paper `A Unified MRC Framework for Named Entity Recognition`","archived":false,"fork":false,"pushed_at":"2023-06-12T21:30:11.000Z","size":114,"stargazers_count":659,"open_issues_count":58,"forks_count":118,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-06T17:46:48.611Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShannonAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-16T13:13:58.000Z","updated_at":"2024-10-28T03:15:34.000Z","dependencies_parsed_at":"2024-01-16T22:19:32.901Z","dependency_job_id":"f4dee529-ada1-4fa5-aa80-fa334c716416","html_url":"https://github.com/ShannonAI/mrc-for-flat-nested-ner","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShannonAI%2Fmrc-for-flat-nested-ner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShannonAI%2Fmrc-for-flat-nested-ner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShannonAI%2Fmrc-for-flat-nested-ner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShannonAI%2Fmrc-for-flat-nested-ner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShannonAI","download_url":"https://codeload.github.com/ShannonAI/mrc-for-flat-nested-ner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248020593,"owners_count":21034459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T16:01:46.318Z","updated_at":"2025-04-09T10:33:11.842Z","avatar_url":"https://github.com/ShannonAI.png","language":"Python","funding_links":[],"categories":["Python","实体识别NER、意图识别、槽位填充"],"sub_categories":["其他_文本生成、文本对话"],"readme":"# A Unified MRC Framework for Named Entity Recognition \nThe repository contains the code of the recent research advances in [Shannon.AI](http://www.shannonai.com). \n\n**A Unified MRC Framework for Named Entity Recognition** \u003cbr\u003e\nXiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu and Jiwei Li\u003cbr\u003e\nIn ACL 2020. [paper](https://arxiv.org/abs/1910.11476)\u003cbr\u003e\nIf you find this repo helpful, please cite the following:\n```latex\n@article{li2019unified,\n  title={A Unified MRC Framework for Named Entity Recognition},\n  author={Li, Xiaoya and Feng, Jingrong and Meng, Yuxian and Han, Qinghong and Wu, Fei and Li, Jiwei},\n  journal={arXiv preprint arXiv:1910.11476},\n  year={2019}\n}\n```\nFor any question, please feel free to post Github issues. \u003cbr\u003e\n\n## Install Requirements\n\n* The code requires Python 3.6+.\n\n* If you are working on a GPU machine with CUDA 10.1, please run `pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html` to install PyTorch. If not, please see the [PyTorch Official Website](https://pytorch.org/) for instructions.\n\n* Then run the following script to install the remaining dependenices: `pip install -r requirements.txt`\n\nWe build our project on [pytorch-lightning.](https://github.com/PyTorchLightning/pytorch-lightning)\nIf you want to know more about the arguments used in our training scripts, please \nrefer to [pytorch-lightning documentation.](https://pytorch-lightning.readthedocs.io/en/latest/)\n\n### Baseline: BERT-Tagger \n\nWe release code, [scripts](./scripts/bert_tagger/reproduce) and [datafiles](./ner2mrc/download.md) for fine-tuning BERT and treating NER as a sequence labeling task. \u003cbr\u003e\n\n### MRC-NER: Prepare Datasets\n\nYou can [download](./ner2mrc/download.md) the preprocessed MRC-NER datasets used in our paper. \u003cbr\u003e\nFor flat NER datasets, please use `ner2mrc/mrsa2mrc.py` to transform your BMES NER annotations to MRC-format. \u003cbr\u003e\nFor nested NER datasets, please use `ner2mrc/genia2mrc.py` to transform your start-end NER annotations to MRC-format. \u003cbr\u003e\n\n### MRC-NER: Training\n\nThe main training procedure is in `train/mrc_ner_trainer.py`\n\nScripts for reproducing our experimental results can be found in the `./scripts/mrc_ner/reproduce/` folder. \nNote that you need to change `DATA_DIR`, `BERT_DIR`, `OUTPUT_DIR` to your own dataset path, bert model path and log path, respectively.  \u003cbr\u003e \nFor example, run `./scripts/mrc_ner/reproduce/ace04.sh` will start training MRC-NER models and save intermediate log to `$OUTPUT_DIR/train_log.txt`. \u003cbr\u003e \nDuring training, the model trainer will automatically evaluate on the dev set every `val_check_interval` epochs,\nand save the topk checkpoints to `$OUTPUT_DIR`. \u003cbr\u003e \n\n### MRC-NER: Evaluation\n\nAfter training, you can find the best checkpoint on the dev set according to the evaluation results in `$OUTPUT_DIR/train_log.txt`. \u003cbr\u003e \nThen run `python3 evaluate/mrc_ner_evaluate.py $OUTPUT_DIR/\u003cbest_ckpt_on_dev\u003e.ckpt  $OUTPUT_DIR/lightning_logs/\u003cversion_0/hparams.yaml\u003e` to evaluate on the test set with the best checkpoint chosen on dev. \n\n### MRC-NER: Inference \n\nCode for inference using the trained MRC-NER model can be found in `inference/mrc_ner_inference.py` file. \u003cbr\u003e\nFor flat NER, we provide the inference script in [flat_inference.sh](./scripts/mrc_ner/flat_inference.sh) \u003cbr\u003e\nFor nested NER, we provide the inference script in [nested_inference.sh](./scripts/mrc_ner/nested_inference.sh) ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShannonAI%2Fmrc-for-flat-nested-ner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FShannonAI%2Fmrc-for-flat-nested-ner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShannonAI%2Fmrc-for-flat-nested-ner/lists"}