{"id":47829363,"url":"https://github.com/basf/neural-fingerprint-uncertainty","last_synced_at":"2026-04-03T20:07:19.412Z","repository":{"id":245420376,"uuid":"800094147","full_name":"basf/neural-fingerprint-uncertainty","owner":"basf","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-25T16:43:24.000Z","size":66610,"stargazers_count":8,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-03T10:46:54.185Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/basf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-13T17:31:44.000Z","updated_at":"2025-02-19T13:03:06.000Z","dependencies_parsed_at":"2024-07-09T20:11:27.086Z","dependency_job_id":null,"html_url":"https://github.com/basf/neural-fingerprint-uncertainty","commit_stats":null,"previous_names":["basf/neural-fingerprint-uncertainty"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/basf/neural-fingerprint-uncertainty","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basf%2Fneural-fingerprint-uncertainty","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basf%2Fneural-fingerprint-uncertainty/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basf%2Fneural-fingerprint-uncertainty/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basf%2Fneural-fingerprint-uncertainty/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/basf","download_url":"https://codeload.github.com/basf/neural-fingerprint-uncertainty/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basf%2Fneural-fingerprint-uncertainty/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31374103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T17:53:18.093Z","status":"ssl_error","status_checked_at":"2026-04-03T17:53:17.617Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-03T20:07:19.329Z","updated_at":"2026-04-03T20:07:19.398Z","avatar_url":"https://github.com/basf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Analysis of Uncertainty of Neural Fingerprint-based Models\nThis repository contains the code to reproduce the results of the paper \"Analysis of Uncertainty of Neural Fingerprint-based Models\" (under review).\n## Abstract\nEstimating the uncertainty of model predictions is crucial in a wide range of cheminformatics applications, not only to better understand machine learning models but also to establish trust in deployed models.\nUncertainty estimates for many standard machine learning models, like Random Forest, are well studied. However, their predictive performances can be inferior to deep learning models, like graph neural networks (GNNs).\nWe investigated whether the neural fingerprint extracted from a GNN can be combined with classical machine learning models to achieve good prediction performance and reliable uncertainty estimates.\n## Reproducing the results\n### DVC\nThe experiments are managed using [DVC](https://dvc.org/), where each step is specified in the [`dvc.yaml`](https://dvc.org/doc/user-guide/project-structure/dvcyaml-files#dvcyaml) file.\nRunning the pipeline will create a dvc.lock file, which contains the hashes of the scripts, input files, and output files, ensuring that the results originate from the provided code and data.\nThe cache is available from this repository as a tarball.\nThe following sections describe how to set up the project and reproduce the results.\n### Commands to reproduce the results\n1. Clone the repository\n```bash\ngit clone https://github.com/basf/neural-fingerprint-uncertainty.git\ncd neural-fingerprint-uncertainty\n```\n2. Install the requirements\n```bash\npip install -r requirements.txt\n```\n3. Unzip the dvc cache\n```bash\ntar -xf dvc_cache.tar.gz .dvc/\n```\n4. Pull the data\n```bash\ndvc pull\n```\n5. Reproduce the results\n```bash\ndvc repro\n```\nThe generated results, figures, etc are saved in the [data](data) folder.\n## Workflow of the experiments\n### Molecular standardization\nThe molecular standardization is performed using [molpipeline](https://github.com/basf/molpipeline).\nDetails of the standardization are provided in the [01_preprocess_data.py](scripts%2F01_preprocess_data.py) script.\n### Creating the folds\nThe data is split into 5 folds using the `StratifiedKFold` method and the `GroupKFold` method, where the group is determined by Agglomerative Clustering.\nThe details of the fold creation are provided in the [02_assign_groups.py](scripts%2F02_assign_groups.py) script.\n### ML experiments with Morgan fingerprints\nThe Morgan fingerprints are used to train the classical machine learning models.\nThe details of the experiments are provided in the [03_ml_experiments.py](scripts%2F03_ml_experiments.py) script.\n### ML experiments with neural fingerprints\nThe neural fingerprints are extracted from a pre-trained [Chemprop](https://github.com/chemprop/chemprop) model.\nIn addition to the neural fingerprints, the GNN is also used to predict the target values.\nThe details of the experiments are provided in the [04_neural_fingerprint_predictions.py](scripts%2F04_neural_fingerprint_predictions.py) script.\n### Create plots for each endpoint\nThe results of the experiments are visualized using matplotlib, where the plots are saved in the [figures](data%2Ffigures) folder.\nFigures have to be loaded using the commands provided [above](#Commands to reproduce the results).\nCode for the plots is provided in the [05_create_plots.py](scripts%2F05_create_plots.py) script.\n### Plots used in the paper\nThe plots used in the paper are saved in the [final_figures](data%2Ffigures%2Ffinal_figures) folder.\nThe code for the plots is provided in the [06_create_final_figures.py](scripts%2F06_create_final_figures.py) script.\n### Tables used in the paper\nThe tables used in the paper were logged and directly extracted from the console.\nA copy of the console output is provided in the file [07_create_final_tables.log](logs%2F07_create_final_tables.log) (again, only available after executing the commands [above](#Commands to reproduce the results)).\n\n## License\n\nThis software is licensed under the MIT license. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasf%2Fneural-fingerprint-uncertainty","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbasf%2Fneural-fingerprint-uncertainty","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasf%2Fneural-fingerprint-uncertainty/lists"}