{"id":38674993,"url":"https://github.com/wglab/phenogpt","last_synced_at":"2026-01-17T10:00:53.969Z","repository":{"id":174715180,"uuid":"637938749","full_name":"WGLab/PhenoGPT","owner":"WGLab","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-15T04:49:01.000Z","size":16056,"stargazers_count":25,"open_issues_count":4,"forks_count":8,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-09-01T01:35:31.248Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WGLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-08T18:28:02.000Z","updated_at":"2025-08-06T11:59:18.000Z","dependencies_parsed_at":"2024-04-02T20:47:03.409Z","dependency_job_id":"0da60231-eca6-4782-85d4-74e72d86c018","html_url":"https://github.com/WGLab/PhenoGPT","commit_stats":null,"previous_names":["wglab/phenogpt"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/WGLab/PhenoGPT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGLab%2FPhenoGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGLab%2FPhenoGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGLab%2FPhenoGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGLab%2FPhenoGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WGLab","download_url":"https://codeload.github.com/WGLab/PhenoGPT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGLab%2FPhenoGPT/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28505570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T06:57:29.758Z","status":"ssl_error","status_checked_at":"2026-01-17T06:56:03.931Z","response_time":85,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-17T10:00:29.840Z","updated_at":"2026-01-17T10:00:53.910Z","avatar_url":"https://github.com/WGLab.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PhenoGPT\n\nPhenoGPT is an advanced phenotype recognition model, leveraging the robust capabilities of large language models. It employs a fine-tuned implementation on the publicly accessible [BiolarkGSC+ dataset](https://github.com/lasigeBioTM/IHP), to enhance prediction accuracy and alignments. Like GPT's broad utilization, PhenoGPT can process diverse clinical abstracts for improved flexibility. For enhanced model precision and specialization, you have the option to further fine-tune the proposed PhenoGPT model on your own clinical datasets. This process is elaborated in the subsequent [section](##Fine-tuning). \n\nLlama 2 is the default model as it performs the best compared to other models such as GPT-J and Falcon.\n\nPhenoGPT is distributed under the [MIT License by Wang Genomics Lab](https://wglab.mit-license.org/).\n\n**Update in 2024**: The latest development of PhenoGPT is [PhenoGPT2](https://github.com/WGLab/PhenoGPT2). Compared to PhenoGPT, the main differences are: (1) we have used Llama 3.1 as the base model for fine-tuning HPO recognition model, with noticeable improvements in accuracy of phenotype recognition as evaluated on several data sets. (2) we have implemented the ability of direct tokenization of HPO ID in the model (i.e., \"HP:1234567\" is treated as one token, rather than several tokens, in the model), to minimize the HPO hallucination problem. (3) We used a larger and more comprehensive corpus to fine-tune the model, with the ability to extract demographic data (such as sex, age and ethnicity/race) from clinical notes.\n\n\n\n## Installation\nWe need to install the required packages for model fine-tuning and inference. \n```\nconda create -n llm_phenogpt python=3.11\nconda activate llm_phenogpt\nconda install pandas numpy scikit-learn matplotlib seaborn requests joblib\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121\nconda install nvidia/label/cuda-12.1.0::cuda-tools\nconda install -c conda-forge jupyter\nconda install intel-openmp blas\nconda install mpi4py\npip install transformers datasets\npip install fastobo sentencepiece einops protobuf\npip install evaluate sacrebleu scipy accelerate deepspeed\npip install git+https://github.com/huggingface/peft.git\npip install flash-attn --no-build-isolation\npip install xformers\npip install bitsandbytes\nconda install -c anaconda ipykernel\npython -m ipykernel install --user --name=llm_phenogpt\n```\nIn the command above, we utilize the accelerate package for model sharding. PEFT package is used for efficient fine-tuning like LORA.\nbitsandbytes package is used for model quantization.\nPlease pip uninstalll package and pip install package if you encounter any running issues.\n\nWe need to install the required packages for BioSent2Vec model to convert medical terms to HPO ID\n```\npip install nltk\nconda install scipy\n```\nPlease follow the steps in the [BioSent2Vec tutorial](https://github.com/ncbi-nlp/BioSentVec/tree/master) and [issue](https://github.com/ncbi-nlp/BioSentVec/issues/16#issuecomment-1222629369) to install BioSent2Vec properly.\n\n## Set Up Model, Input, and Output directories\n1. Models:\n    - To use LLaMA 2 model, please apply for access first and download it into the local drive. [Instruction](https://huggingface.co/docs/transformers/main/model/llama2)\n    - Save model in the ./model/llama2/llama2_base/\n    - Download the updated fine-tuning LoRA weights in the release section on GitHub (Latest version: v1.1.0)\n    - Save LoRA weights in the ./model/llama2/\n    - Setups for Falcon 70B and Llama 1 7B models are similar.\n2. Input:\n    - Input files should be txt files\n    - Input argument can be either a single txt file or a whole directory containing all input txt files\n    - Please see the input and output directories for reference\n3. BioSent2Vec:\n    - To use BioSent2Vec model, please see the BioSent2Vec tutorial above. Then, do the following steps:\n    - ```mkdir ./BioSent2Vec/model```\n    - ```cd ./Biosent2Vec/model```\n    - ```wget https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/BioSentVec/BioSentVec_PubMed_MIMICIII-bigram_d700.bin```\n\n## Fine-tuning\nYou can reproduce PhenoGPT model with different base models on BiolarkGSC+ dataset. To fine-tune a specialized phenotype recognition language model, we recommend to follow this [notebook](https://github.com/WGLab/PhenoGPT/blob/main/run_phenogpt.ipynb) script for details. (The notebook is for both llama and falcon model implementation. For gpt-j, please refer to this [script](https://github.com/WGLab/PhenoGPT/blob/main/model/gpt-j/Finetune_gpt_j_6B_8bit_biolark.ipynb).)\n\nFor training (fine-tuning) of PhenoBCBERT on labelled clinical notes, please refer to this [notebook](https://github.com/WGLab/PhenoGPT/blob/main/PhenoBCBERT_Finetune.ipynb).\n\n## Inference\nIf you want to simply implement PhenoGPT on your local machine for inference, the fine-tuned models are saved in the [model](https://github.com/WGLab/PhenoGPT/tree/main/model) directory. Please follow the inference section of the [script](https://github.com/WGLab/PhenoGPT/blob/main/inference.py) to run your model.\n\nPlease use the following command:\n```\npython inference.py -i your_input_folder_directory -o your_output_folder_directory -id yes\n```\n-id: specify 'yes' if you want to obtain the corresponding HPO ID to the detected phenotypes, otherwise 'no' (default: 'yes')\n\n## Regarding PhenoBCBERT\nSince PhenoBCBERT was fine-tuned on the CHOP Proprietary dataset, we cannot publish the model publicly. Please refer to the [paper](https://doi.org/10.1016%2Fj.patter.2023.100887) for results.\n\n## Citation\nYang, J., Liu, C., Deng, W., Wu, D., Weng, C., Zhou, Y., \u0026 Wang, K. (2023). Enhancing phenotype recognition in clinical notes using large language models: PhenoBCBERT and PhenoGPT. Patterns (New York, N.Y.), 5(1), 100887. https://doi.org/10.1016/j.patter.2023.100887\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwglab%2Fphenogpt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwglab%2Fphenogpt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwglab%2Fphenogpt/lists"}