{"id":18256400,"url":"https://github.com/baderlab/geneeval","last_synced_at":"2025-07-16T22:32:02.676Z","repository":{"id":37856190,"uuid":"279940413","full_name":"BaderLab/GeneEval","owner":"BaderLab","description":"A Python library for evaluating gene embeddings.","archived":false,"fork":false,"pushed_at":"2023-03-06T15:57:21.000Z","size":288,"stargazers_count":4,"open_issues_count":13,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-29T08:40:55.896Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BaderLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-15T17:58:40.000Z","updated_at":"2020-11-30T21:48:10.000Z","dependencies_parsed_at":"2024-11-05T10:34:53.416Z","dependency_job_id":"a8325554-20aa-480b-ba9c-72cf47dcb78f","html_url":"https://github.com/BaderLab/GeneEval","commit_stats":{"total_commits":104,"total_committers":3,"mean_commits":"34.666666666666664","dds":"0.16346153846153844","last_synced_commit":"ad26540d999e7f0562ad1d67d574e08edcadae57"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/BaderLab/GeneEval","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BaderLab%2FGeneEval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BaderLab%2FGeneEval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BaderLab%2FGeneEval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BaderLab%2FGeneEval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BaderLab","download_url":"https://codeload.github.com/BaderLab/GeneEval/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BaderLab%2FGeneEval/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265545954,"owners_count":23785885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T10:21:41.430Z","updated_at":"2025-07-16T22:32:02.659Z","avatar_url":"https://github.com/BaderLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![build](https://github.com/BaderLab/GeneEval/workflows/build/badge.svg)\n[![codecov](https://codecov.io/gh/BaderLab/GeneEval/branch/master/graph/badge.svg)](https://codecov.io/gh/BaderLab/GeneEval)\n![GitHub](https://img.shields.io/github/license/JohnGiorgi/allennlp-multi-label-classification?color=blue)\n\n# GeneEval\n\nA Python library for benchmarking gene function prediction.\n\n## Installation\n\nLatest PyPI release\n\n```bash\npip install geneeval\n```\n\nFrom source\n\n```bash\n# Install poetry for your system: https://python-poetry.org/docs/#installation\ncurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python\n\n# Clone and move into the repo\ngit clone https://github.com/BaderLab/GeneEval.git\ncd GeneEval\n\n# Install the package with poetry\npoetry install\n```\n\nIf you plan on evaluating fixed-length feature vectors (see [Usage](#usage)), please install with `pip install \"geneeval[features]\"` (or `poetry install -E \"features\"` if installing from source).\n\n## Usage\n\nFirst, download the benchmark with the `prepare` command\n\n```bash\ngeneeval prepare \"./benchmark.json\"\n```\n\nThere are two ways to run the evaluation, depending on your method.\n\n### Methods that produce fixed-length feature vectors\n\nIf your method produces a fixed-length feature vector for each gene ID in the benchmark, collect these in a comma-separated file, e.g.\n\n```\nQ8W5R2, 0.2343, -0.1242, 0.5431, -0.3475, 0.9373\nQ99732, -0.9323, 0.2212, -0.4331, -0.8634, 0.8373\nP83774, 0.5633, -0.6242, 0.3723, -0.2375, -0.1673\nQ1ENB6, 0.1433, -0.3242, 0.5323, -0.9975, -0.4573\nQ9XF19, 0.5621, -0.4272, 0.9743, -0.1373, -0.2173\n```\n\n\u003e You can prepare a `.csv`, `.tsv`, `.txt` (separated by spaces) or a `.json` file (where the vectors are keyed by gene IDs). We will correctly parse the file based on its file extension.\n\nand then call the `evaluate features` command\n\n```bash\ngeneeval evaluate features \"./features.csv\"\n```\n\nThese features will be used as input to simple classifiers, which will be evaluated with a grid search over the benchmark tasks.\n\n### Methods that do not produce fixed-length feature vectors\n\nFor all other methods, you simply need to produce predictions for each task in the benchmark that you wish to evaluate on. Predictions should be collected in a `.json` file keyed by task name, data partition, and gene ID, e.g.\n\n```json\n{\n    \"subcellular_localization\": {\n      \"train\": {\n        \"Q8W5R2\": \"M\",\n        \"Q99732\": \"M\",\n        \"P83774\": \"S\"\n      },\n      \"valid\": {\n        \"Q1ENB6\": \"S\"\n      },\n      \"test\": {\n        \"Q9XF19\": \"S\"\n      }\n    }\n}\n```\n\n\u003e We make no assumptions about how these predictions are obtained.\n\nthen, the `evaluate predictions` command can be used to obtain a score on the tasks\n\n```bash\ngeneeval predictions \"./predictions.json\"\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaderlab%2Fgeneeval","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbaderlab%2Fgeneeval","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaderlab%2Fgeneeval/lists"}