{"id":21441766,"url":"https://github.com/dmis-lab/bern2","last_synced_at":"2025-09-22T21:58:00.019Z","repository":{"id":40537816,"uuid":"445094328","full_name":"dmis-lab/BERN2","owner":"dmis-lab","description":"BERN2: an advanced neural biomedical namedentity recognition and normalization tool","archived":false,"fork":false,"pushed_at":"2024-03-28T00:52:14.000Z","size":1856,"stargazers_count":185,"open_issues_count":15,"forks_count":43,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-09T23:18:18.731Z","etag":null,"topics":["biomedical-text-mining"],"latest_commit_sha":null,"homepage":"http://bern2.korea.ac.kr","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmis-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-06T08:22:09.000Z","updated_at":"2025-04-01T03:32:11.000Z","dependencies_parsed_at":"2023-11-13T07:28:33.623Z","dependency_job_id":"11a3733f-fd2f-4629-a3ce-961755bf2e47","html_url":"https://github.com/dmis-lab/BERN2","commit_stats":{"total_commits":102,"total_committers":4,"mean_commits":25.5,"dds":0.303921568627451,"last_synced_commit":"b508e01fd6b1afff8d017110b5e653d800e2c802"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FBERN2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FBERN2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FBERN2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmis-lab%2FBERN2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmis-lab","download_url":"https://codeload.github.com/dmis-lab/BERN2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248125593,"owners_count":21051771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["biomedical-text-mining"],"created_at":"2024-11-23T01:42:12.676Z","updated_at":"2025-09-22T21:57:54.969Z","avatar_url":"https://github.com/dmis-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BERN2\n\nWe present **BERN2** (Advanced **B**iomedical **E**ntity **R**ecognition and **N**ormalization), a tool that improves the previous neural network-based NER tool by employing a multi-task NER model and neural network-based NEN models to achieve much faster and more accurate inference. This repository provides a way to host your own BERN2 server. Currently, BERN2 is running on a hosting server with 64-core CPU, 512GB Memory, and 12GB GPU. See our [paper](https://arxiv.org/abs/2201.02080) for more details.\n\n***** **Try BERN2 at [http://bern2.korea.ac.kr](http://bern2.korea.ac.kr)** ***** \n\n### Updates\n* \\[**Apr 26, 2023**\\] The BERN2 server will be shut down due to an internal server issue and is expected to be restored by 2022-04-26 12:00 PM (UTC-0). Sorry for the inconvenience.\n* \\[**Feb 13, 2023**\\] The BERN2 server is currently down due to an internal server issue and is expected to be restored by 2022-02-19 12:00 PM (UTC-0). Sorry for the inconvenience.\n* \\[**Jun 26, 2022**\\] We updated our resource file ([resources_v1.1.b.tar.gz](http://nlp.dmis.korea.edu/projects/bern2-sung-et-al-2022/resources_v1.1.b.tar.gz)) to address the issue regarding CRF++. (issue https://github.com/dmis-lab/BERN2/issues/17). \n* \\[**Apr 14, 2022**\\] We updated our resource file ([resources_v1.1.a.tar.gz](http://nlp.dmis.korea.edu/projects/bern2-sung-et-al-2022/resources_v1.1.a.tar.gz)) to address the issue where BERN2 is not working on Windows (issue https://github.com/dmis-lab/BERN2/issues/4). \n* \\[**Apr 14, 2022**\\] We increased the API limit of our [web service](http://bern2.korea.ac.kr) from '100 reqeusts per 100 seconds' to '**300** requests per 100 seconds' per user.\n* \\[**Mar 18, 2022**\\] On the [web service](http://bern2.korea.ac.kr), we set the API limit of 100 requests per 100 seconds per user. For bulk requests, we highly recommend you to use the local installation.\n* \\[**Mar 17, 2022**\\] [BERN2 v1.1](https://github.com/dmis-lab/BERN2/releases/tag/v1.1.0) has been released. Please see the release page for more information on what's new in this version.\n* \\[**Feb 15, 2022**\\] [Bioregistry](https://bioregistry.io/) is used to standardize prefixes for normalized entity identifiers.\n\n| old | new |\n| :---:   | :-: |\n| MESH:D009369  | [mesh:D009369](https://bioregistry.io/mesh:D009369)  |\n| OMIM:608627 | [mim:608627](https://bioregistry.io/mim:608627) |\n| CL_0000021 | [CL:0000021](https://bioregistry.io/CL:0000021) |\n| CVCL_J260 | [cellosaurus:CVCL_J260](https://bioregistry.io/cellosaurus:CVCL_J260) |\n| NCBI:txid10095 | [NCBITaxon:10095](https://bioregistry.io/NCBITaxon:10095) |\n| EntrezGene:10533 |[NCBIGene:10533](https://bioregistry.io/NCBIGene:10533) |\n\n* \\[**Jan 07, 2022**\\] [BERN2 v1.0](https://github.com/dmis-lab/BERN2/releases/tag/v1.0.0) has been released.\n\n## Installing BERN2\n\nYou first need to install BERN2 and its dependencies.\n\n```bash\n# Install torch with conda (please check your CUDA version)\nconda create -n bern2 python=3.7\nconda activate bern2\nconda install pytorch==1.9.0 cudatoolkit=10.2 -c pytorch\nconda install faiss-gpu libfaiss-avx2 -c conda-forge\n\n# Check if cuda is available\npython -c \"import torch;print(torch.cuda.is_available())\"\n\n# Install BERN2\ngit clone git@github.com:dmis-lab/BERN2.git\ncd BERN2\npip install -r requirements.txt\n\n```\n\n(Optional) If you want to use mongodb as a caching database, you need to install and run it.\n```\n# https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/#install-mongodb-community-edition-using-deb-packages\nsudo systemctl start mongod\nsudo systemctl status mongod\n```\n\nThen, you need to download resources (e.g., external modules or dictionaries) for running BERN2. Note that you will need 70GB of free disk space. You can also download the resource file from [google drive](https://drive.google.com/file/d/147b3OhU4IdQi121ZBUSqO1XKdKoXE5DK/view?usp=sharing).\n\n```\nwget http://nlp.dmis.korea.edu/projects/bern2-sung-et-al-2022/resources_v1.1.b.tar.gz\ntar -zxvf resources_v1.1.b.tar.gz\nmd5sum resources_v1.1.b.tar.gz\n# make sure the md5sum is 'c0db4e303d1ccf6bf56b42eda2fe05d0'\nrm -rf resources_v1.1.b.tar.gz\n\n# (For Linux Users) install CRF \ncd resources/GNormPlusJava\ntar -zxvf CRF++-0.58.tar.gz\nmv CRF++-0.58 CRF\ncd CRF\n./configure --prefix=\"$HOME\"\nmake\nmake install\ncd ../../..\n\n# (For Windows Users) install CRF \ncd resources/GNormPlusJava\nunzip -zxvf CRF++-0.58.zip\nmv CRF++-0.58 CRF\ncd ../..\n```\n\n## Running BERN2\n\nThe minimum memory requirement for running BERN2 on GPU is 63.5GB of RAM \u0026 5.05GB of GPU. \nThe following command runs BERN2.\n```\nexport CUDA_VISIBLE_DEVICES=0\ncd scripts\n\n# For Linux\nbash run_bern2.sh\n\n# For Windows\nbash run_bern2_windows.sh\n```\n\n(Optional) To restart BERN2, you need to run the following commands.\n```\nexport CUDA_VISIBLE_DEVICES=0\ncd scripts\nbash stop_bern2.sh\nbash run_bern2.sh\n```\n\n## Using BERN2\n\nAfter successfully running BERN2 in your local environment, you can access it via RESTful API. \nIf you want to use BERN2 without installing it locally, please see [here](http://bern2.korea.ac.kr/documentation#api_content) for instructions on how to use the web service.\n\n### Plain Text as Input\n```\nimport requests\n\ndef query_plain(text, url=\"http://localhost:8888/plain\"):\n    return requests.post(url, json={'text': text}).json()\n\nif __name__ == '__main__':\n    text = \"Autophagy maintains tumour growth through circulating arginine.\"\n    print(query_plain(text))\n```\n\n### PubMed ID (PMID) as Input\n```\nimport requests\n\ndef query_pmid(pmids, url=\"http://localhost:8888/pubmed\"):\n    return requests.get(url + \"/\" + \",\".join(pmids)).json()\n\nif __name__ == '__main__':\n    pmids = [\"30429607\", \"29446767\"]\n    print(query_pmid(pmids))\n```\n\n## Annotations\n\n```\nwget http://nlp.dmis.korea.edu/projects/bern2-sung-et-al-2022/annotation_v1.1.tar.gz\n```\n\nNER and normalization for 33.4+ millions of PubMed articles (pubmed22n0001 ~ pubmed22n1114 (2021.12.12)) generated by [BERN2 v1.1](https://github.com/dmis-lab/BERN2/releases/tag/v1.1.0) (Compressed, 22 GB). The data provided by BERN2 is post-processed and may differ from the most current/accurate data available from [U.S. National Library of Medicine (NLM)](https://www.nlm.nih.gov/).\n\n## Citation\n```bibtex\n@article{sung2022bern2,\n    title={BERN2: an advanced neural biomedical namedentity recognition and normalization tool}, \n    author={Sung, Mujeen and Jeong, Minbyul and Choi, Yonghwa and Kim, Donghyeon and Lee, Jinhyuk and Kang, Jaewoo},\n    year={2022},\n    eprint={2201.02080},\n    archivePrefix={arXiv},\n    primaryClass={cs.CL}\n}\n```\n\n## Contact Information\nFor help or issues using BERN2, please submit a GitHub issue. Please contact Mujeen Sung (`mujeensung (at) korea.ac.kr`), or Minbyul Jeong (`minbyuljeong (at) korea.ac.kr`) for communication related to BERN2.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fbern2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmis-lab%2Fbern2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmis-lab%2Fbern2/lists"}