https://github.com/scai-bio/tau-kg-benchmarking-gpt

Benchmarking the performance of GPT-based LLMs against human curated Tau KG from Human Brain Pharmacome (HBP)
https://github.com/scai-bio/tau-kg-benchmarking-gpt

Last synced: about 2 months ago
JSON representation

Benchmarking the performance of GPT-based LLMs against human curated Tau KG from Human Brain Pharmacome (HBP)

• Host: GitHub
• URL: https://github.com/scai-bio/tau-kg-benchmarking-gpt
• Owner: SCAI-BIO
• License: apache-2.0
• Created: 2023-11-13T12:28:12.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-13T08:28:58.000Z (over 1 year ago)
• Last Synced: 2024-05-14T00:19:15.030Z (about 1 year ago)
• Language: Jupyter Notebook
• Homepage:
• Size: 2.89 MB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Rationalism in the face of GPT hypes: Benchmarking the output of Large Language Models against human expert-curated biomedical knowledge graphs

This repository provides the data and source code for our research regarding the benchmarking the output of LLMs against human expert-curated biomedical disease-specific knowledge graphs

## Relevant data

The different dataset and KG used in this study are located in data directory. This includes the abstracts as well as full-text documents stored in small chunks. The extracted triples using different tools are stored as an excel file for each abstract and full-text experiment, separately. 

## Scripts

The code directory contains the scripts for extractin triples as well as querying and analysis of loaded KGs in Neo4j.