{"id":17132066,"url":"https://github.com/dayyass/qaner","last_synced_at":"2025-04-13T05:34:49.148Z","repository":{"id":38363003,"uuid":"504114030","full_name":"dayyass/QaNER","owner":"dayyass","description":"Unofficial implementation of QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition.","archived":false,"fork":false,"pushed_at":"2022-10-15T09:53:58.000Z","size":2403,"stargazers_count":66,"open_issues_count":10,"forks_count":6,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-11T02:14:17.335Z","etag":null,"topics":["data-science","machine-learning","named-entity-recognition","natural-language-processing","ner","nlp","python","python3","question-answering"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2203.01543","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dayyass.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-16T10:37:47.000Z","updated_at":"2025-04-03T12:25:09.000Z","dependencies_parsed_at":"2022-07-10T22:47:59.396Z","dependency_job_id":null,"html_url":"https://github.com/dayyass/QaNER","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2FQaNER","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2FQaNER/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2FQaNER/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2FQaNER/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dayyass","download_url":"https://codeload.github.com/dayyass/QaNER/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670517,"owners_count":21142896,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","machine-learning","named-entity-recognition","natural-language-processing","ner","nlp","python","python3","question-answering"],"created_at":"2024-10-14T19:25:54.521Z","updated_at":"2025-04-13T05:34:48.132Z","avatar_url":"https://github.com/dayyass.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![tests](https://github.com/dayyass/qaner/actions/workflows/tests.yml/badge.svg)](https://github.com/dayyass/qaner/actions/workflows/tests.yml)\n[![linter](https://github.com/dayyass/qaner/actions/workflows/linter.yml/badge.svg)](https://github.com/dayyass/qaner/actions/workflows/linter.yml)\n\u003c!-- [![codecov](https://codecov.io/gh/dayyass/qaner/branch/main/graph/badge.svg?token=S3UKX8BFP3)](https://codecov.io/gh/dayyass/qaner) --\u003e\n\n[![python 3.7](https://img.shields.io/badge/python-3.7-blue.svg)](https://github.com/dayyass/qaner#requirements)\n[![release (latest by date)](https://img.shields.io/github/v/release/dayyass/qaner)](https://github.com/dayyass/qaner/releases/latest)\n[![license](https://img.shields.io/github/license/dayyass/qaner?color=blue)](https://github.com/dayyass/qaner/blob/main/LICENSE)\n\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-black)](https://github.com/dayyass/qaner/blob/main/.pre-commit-config.yaml)\n[![code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n[![pypi version](https://img.shields.io/pypi/v/qaner)](https://pypi.org/project/qaner)\n[![pypi downloads](https://img.shields.io/pypi/dm/qaner)](https://pypi.org/project/qaner)\n\nMy public talk about this project at Sberloga:\u003cbr\u003e\n[**QaNER - NER via Exractive QA**](https://youtu.be/JRec8FpjhpM)\n\n# QaNER\nUnofficial implementation of [*QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition*](https://arxiv.org/abs/2203.01543).\n\nYou can adopt this pipeline for arbitrary [BIO-markup](https://github.com/dayyass/QaNER/tree/main/data/conll2003) data.\n\n## Installation\n```\npip install qaner\n```\n\n## CoNLL-2003\nPipeline results on CoNLL-2003 dataset:\n- [Metrics](https://tensorboard.dev/experiment/FEsbNJdmSd2LGVhga8Ku0Q/)\n- [Trained Hugging Face model](https://huggingface.co/dayyass/qaner-conll-bert-base-uncased)\n\n## How to use\n### Training\nScript for training QaNER model:\n```\nqaner-train \\\n--bert_model_name 'bert-base-uncased' \\\n--path_to_prompt_mapper 'data/conll2003/prompt_mapper.json' \\\n--path_to_train_data 'data/conll2003/train.bio' \\\n--path_to_test_data 'data/conll2003/test.bio' \\\n--path_to_save_model 'dayyass/qaner-conll-bert-base-uncased' \\\n--n_epochs 2 \\\n--batch_size 128 \\\n--learning_rate 1e-5 \\\n--seed 42 \\\n--log_dir 'runs/qaner'\n```\n\nRequired arguments:\n- **--bert_model_name** - base bert model for QaNER fine-tuning\n- **--path_to_prompt_mapper** - path to prompt mapper json file\n- **--path_to_train_data** - path to train data ([BIO-markup](https://github.com/dayyass/QaNER/tree/main/data/conll2003))\n- **--path_to_test_data** - path to test data ([BIO-markup](https://github.com/dayyass/QaNER/tree/main/data/conll2003))\n- **--path_to_save_model** - path to save trained QaNER model\n- **--n_epochs** - number of epochs to fine-tune\n- **--batch_size** - batch size\n- **--learning_rate** - learning rate\n\nOptional arguments:\n- **--seed** - random seed for reproducibility (default: 42)\n- **--log_dir** - tensorboard log_dir (default: 'runs/qaner')\n\n### Infrerence\nScript for inference trained QaNER model:\n```\nqaner-inference \\\n--context 'EU rejects German call to boycott British lamb .' \\\n--question 'What is the organization?' \\\n--path_to_prompt_mapper 'data/conll2003/prompt_mapper.json' \\\n--path_to_trained_model 'dayyass/qaner-conll-bert-base-uncased' \\\n--n_best_size 1 \\\n--max_answer_length 100 \\\n--seed 42\n```\n\nResult:\n```\nquestion: What is the organization?\n\ncontext: EU rejects German call to boycott British lamb .\n\nanswer: [Span(token='EU', label='ORG', start_context_char_pos=0, end_context_char_pos=2)]\n```\n\nRequired arguments:\n- **--context** - sentence to extract entities from\n- **--question** - question prompt with entity name to extract (examples below)\n- **--path_to_prompt_mapper** - path to prompt mapper json file\n- **--path_to_trained_model** - path to trained QaNER model\n- **--n_best_size** - number of best QA answers to consider\n\nOptional arguments:\n- **--max_answer_length** - entity max length to eliminate very long entities (default: 100)\n- **--seed** - random seed for reproducibility (default: 42)\n\nPossible inference questions for CoNLL-2003:\n- What is the location? (LOC)\n- What is the person? (PER)\n- What is the organization? (ORG)\n- What is the miscellaneous entity? (MISC)\n\n## Requirements\nPython \u003e= 3.7\n\n## Citation\n```bibtex\n@misc{liu2022qaner,\n    title         = {QaNER: Prompting Question Answering Models for Few-shot Named Entity Recognition},\n    author        = {Andy T. Liu and Wei Xiao and Henghui Zhu and Dejiao Zhang and Shang-Wen Li and Andrew Arnold},\n    year          = {2022},\n    eprint        = {2203.01543},\n    archivePrefix = {arXiv},\n    primaryClass  = {cs.LG}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdayyass%2Fqaner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdayyass%2Fqaner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdayyass%2Fqaner/lists"}