{"id":19398821,"url":"https://github.com/kyubyong/bert_ner","last_synced_at":"2025-07-19T19:07:25.539Z","repository":{"id":85026540,"uuid":"172331006","full_name":"Kyubyong/bert_ner","owner":"Kyubyong","description":"Ner with Bert","archived":false,"fork":false,"pushed_at":"2019-10-20T02:50:01.000Z","size":496,"stargazers_count":281,"open_issues_count":10,"forks_count":56,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-09T23:36:25.384Z","etag":null,"topics":["bert","bert-model","named-entity-recognition","ner"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kyubyong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-02-24T11:47:55.000Z","updated_at":"2025-03-05T06:05:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"fad5d755-1155-4617-9a62-905eb26f0256","html_url":"https://github.com/Kyubyong/bert_ner","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Kyubyong/bert_ner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kyubyong%2Fbert_ner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kyubyong%2Fbert_ner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kyubyong%2Fbert_ner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kyubyong%2Fbert_ner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kyubyong","download_url":"https://codeload.github.com/Kyubyong/bert_ner/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kyubyong%2Fbert_ner/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265992659,"owners_count":23860945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","bert-model","named-entity-recognition","ner"],"created_at":"2024-11-10T11:07:22.974Z","updated_at":"2025-07-19T19:07:25.509Z","avatar_url":"https://github.com/Kyubyong.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyTorch Implementation of NER with pretrained Bert\n\nI know that you know [BERT](https://arxiv.org/abs/1810.04805).\nIn the great paper, the authors claim that the pretrained models do great in NER.\nIt's even impressive, allowing for the fact that they don't use any prediction-conditioned algorithms like CRFs.\nWe try to reproduce the result in a simple manner.\n\n## Requirements\n* python\u003e=3.6 (Let's move on to python 3 if you still use python 2)\n* pytorch==1.0\n* pytorch_pretrained_bert==0.6.1\n* numpy\u003e=1.15.4\n\n## Training \u0026 Evaluating\n\n* STEP 1. Run the command below to download conll 2003 NER dataset.\n```\nbash download.sh\n```\nIt should be extracted to `conll2003/` folder automatically.\n\n* STEP 2a. Run the command if you want to do the feature-based approach.\n```\npython train.py --logdir checkpoints/feature --batch_size 128 --top_rnns --lr 1e-4 --n_epochs 30\n```\n\n* STEP 2b. Run the command if you want to do the fine-tuning approach.\n```\npython train.py --logdir checkpoints/finetuning --finetuning --batch_size 32 --lr 5e-5 --n_epochs 3\n```\n\n## Results in the paper\n\n* Feature-based approach\n\n\u003cimg src=\"bert_ner.png\"\u003e\n\n* Fine-tuning\n\n\u003cimg src=\"bert_ner_finetuning.png\"\u003e\n\n## Results\n\n* F1 scores on conll2003 valid dataset are reported.\n* You can check the classification outputs in [checkpoints](checkpoints).\n\n|epoch|feature-based|fine-tuning|\n|--|--|--|\n|1|0.2|0.95|\n|2|0.75|0.95|\n|3|0.84|0.96|\n|4|0.88|\n|5|0.89|\n|6|0.90|\n|7|0.90|\n|8|0.91|\n|9|0.91|\n|10|0.92|\n|11|0.92|\n|12|0.93|\n|13|0.93|\n|14|0.93|\n|15|0.93|\n|16|0.92|\n|17|0.93|\n|18|0.93|\n|19|0.93|\n|20|0.93|\n|21|**0.94**|\n|22|**0.94**|\n|23|0.93|\n|24|0.93|\n|25|0.93|\n|26|0.93|\n|27|0.93|\n|28|0.93|\n|29|**0.94**|\n|30|0.93|\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyubyong%2Fbert_ner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyubyong%2Fbert_ner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyubyong%2Fbert_ner/lists"}