{"id":25922061,"url":"https://github.com/hfooladi/alinemol","last_synced_at":"2026-06-06T12:31:28.360Z","repository":{"id":279587198,"uuid":"736027112","full_name":"HFooladi/ALineMol","owner":"HFooladi","description":"Exploring performance of machine learning model on out-of-distribution data in chemical domain","archived":false,"fork":false,"pushed_at":"2026-05-23T17:05:05.000Z","size":22105,"stargazers_count":7,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-05-23T18:11:57.014Z","etag":null,"topics":["cheminformatics","generalization","molecular-property-prediction","out-of-distribution"],"latest_commit_sha":null,"homepage":"https://hfooladi.github.io/ALineMol/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HFooladi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-12-26T19:18:02.000Z","updated_at":"2026-05-23T17:05:09.000Z","dependencies_parsed_at":"2025-02-26T11:27:26.685Z","dependency_job_id":"dc1efbd1-961d-473b-bd58-806b8d8ea967","html_url":"https://github.com/HFooladi/ALineMol","commit_stats":null,"previous_names":["hfooladi/alinemol"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/HFooladi/ALineMol","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFooladi%2FALineMol","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFooladi%2FALineMol/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFooladi%2FALineMol/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFooladi%2FALineMol/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HFooladi","download_url":"https://codeload.github.com/HFooladi/ALineMol/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFooladi%2FALineMol/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33983046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-06T02:00:07.033Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cheminformatics","generalization","molecular-property-prediction","out-of-distribution"],"created_at":"2025-03-03T16:17:11.576Z","updated_at":"2026-06-06T12:31:28.355Z","avatar_url":"https://github.com/HFooladi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ALineMol: Evaluating Machine Learning Models for Molecular Property Prediction on OOD Data\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![CI](https://github.com/HFooladi/ALineMol/actions/workflows/ci.yml/badge.svg)](https://github.com/HFooladi/ALineMol/actions/workflows/ci.yml)\n[![JCIM](https://img.shields.io/badge/JCIM-2025-blue)](https://doi.org/10.1021/acs.jcim.5c00475)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HFooladi/ALineMol/blob/main/notebooks/colab_splitter_quickstart.ipynb)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/banner/alinemol_banner.png\" alt=\"ALineMol Banner\" style=\"max-width:100%;\"\u003e\n\u003c/p\u003e\n\n## Overview\n\n**ALineMol** is a comprehensive research framework for evaluating and quantitatively assessing the relationship between machine learning model performance on in-distribution (ID) and out-of-distribution (OOD) data in the molecular domain. This work addresses critical questions in AI-driven drug discovery about model generalization to novel chemical structures.\n\n### Key Contributions\n\n🔬 **Comprehensive Evaluation**: Systematic assessment of ML models (classical ML + GNNs) across multiple datasets using different splitting strategies\n\n📊 **Distribution Shift Analysis**: Quantitative investigation of what constitutes \"out-of-distribution\" data in molecular property prediction\n\n🎯 **ID-OOD Relationship**: Deep analysis of correlation between in-distribution and out-of-distribution performance across different scenarios\n\n⚗️ **Drug Discovery Focus**: Practical insights for molecular property prediction and bioactivity classification in pharmaceutical research\n\n### Setup\n\n#### Option 1: Using uv (Recommended)\n\n[uv](https://docs.astral.sh/uv/) is a fast Python package installer.\n\n**Quick Install (using script):**\n```bash\n# Clone the repository\ngit clone https://github.com/HFooladi/ALineMol.git\ncd ALineMol\n\n# Install with CPU (default)\n./install.sh\n\n# Or install with CUDA support\n./install.sh cu121  # CUDA 12.1\n./install.sh cu118  # CUDA 11.8\n./install.sh cu124  # CUDA 12.4\n\n# Activate the environment\nsource .venv/bin/activate\n```\n\n**Manual Install:**\n```bash\n# Install uv\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Clone the repository\ngit clone https://github.com/HFooladi/ALineMol.git\ncd ALineMol\n\n# Create virtual environment\nuv venv --python 3.11\nsource .venv/bin/activate  # On Windows: .venv\\Scripts\\activate\n\n# CPU-only installation\nuv pip install -e \".[all,dev,test]\" -f https://download.pytorch.org/whl/cpu -f https://data.dgl.ai/wheels/repo.html\n\n# Or CUDA 12.1 installation\nuv pip install -e \".[all,dev,test]\" -f https://download.pytorch.org/whl/cu121 -f https://data.dgl.ai/wheels/repo.html\n```\n\n#### Option 2: Using conda\n\n```bash\n# Clone the repository\ngit clone https://github.com/HFooladi/ALineMol.git\ncd ALineMol\n\n# Create and activate conda environment\nconda env create -f environment.yml\nconda activate alinemol\n\n# Install ALineMol package\npip install --no-deps -e .\n```\n\n#### Optional extras\n\nThe default install (`pip install alinemol`) is lean — only the splitter\nAPI and `SplitAnalyzer`. Pull in heavier components on demand:\n\n| Extra | Pulls in | When you need it |\n|---|---|---|\n| `[gnn]` | torch, dgl, dgllife, torch-geometric | Training/inference with GNN models |\n| `[ml]` | statsmodels, POT, astartes | `alinemol.utils.utils`, OT-based graph utilities, `astartes`-backed splitters |\n| `[datasail]` | datasail | The `datasail` splitter |\n| `[all]` | gnn + ml + datasail | Everything (used by `install.sh`) |\n| `[dev]` | ruff, pre-commit, mypy | Contributing |\n| `[test]` | pytest, pytest-cov | Running the test suite |\n\nExample: `pip install \"alinemol[gnn]\"` for splitters + GNN training only.\n\n## Quick Start\n\n### Basic Usage\n\n```python\nimport pandas as pd\nfrom alinemol.preprocessing import standardization_pipeline\nfrom alinemol.splitters import ScaffoldSplit, MolecularWeightSplit\nfrom alinemol.utils import compute_similarities\n\n# Load and preprocess data\ndf = pd.read_csv(\"your_dataset.csv\")  # Columns: 'smiles', 'label'\ndf_clean = standardization_pipeline(df)\n\n# Create different types of splits\nscaffold_splitter = ScaffoldSplit(test_size=0.2)\nweight_splitter = MolecularWeightSplit(test_size=0.2, generalize_to_larger=True)\n\n# Evaluate different splitting strategies\nfor train_idx, test_idx in scaffold_splitter.split(df_clean['smiles']):\n    train_data = df_clean.iloc[train_idx]\n    test_data = df_clean.iloc[test_idx]\n    \n    # Compute molecular similarities\n    similarities = compute_similarities(\n        train_data['smiles'], \n        test_data['smiles'],\n        fingerprint='ecfp',\n        fprints_hopts={'radius': 2, 'fpSize': 1024}\n    )\n    print(f\"Average train-test similarity: {similarities.mean():.3f}\")\n```\n\n### Comprehensive Evaluation Pipeline\n\n```python\nfrom alinemol.utils import load_dataset, split_dataset, compute_ID_OOD\nfrom alinemol.utils.plot_utils import plot_ID_OOD_sns, heatmap_plot\n\n# Evaluate multiple models across different split types\nresults = compute_ID_OOD(\n    dataset_category=\"TDC\",\n    dataset_names=\"CYP2C19\", \n    split_type=\"scaffold\",\n    num_of_splits=10\n)\n\n# Visualize ID vs OOD performance\nplot_ID_OOD_sns(results, dataset_name=\"CYP2C19\", save=True)\n\n# Create performance heatmaps\nheatmap_plot(results, metric=\"roc_auc\", save=True)\n```\n\n## Splitting Strategies\n\nALineMol provides a unified API for molecular dataset splitting with multiple strategies to simulate different types of distribution shift.\n\n### Factory Function (Recommended)\n\nThe easiest way to create splitters is using the `get_splitter()` factory function:\n\n```python\nfrom alinemol.splitters import get_splitter, list_splitters, get_splitter_names\n\n# List all available splitters\nprint(get_splitter_names())\n# ['butina', 'datasail', 'hi', 'kmeans', 'lo', 'max_dissimilarity',\n#  'molecular_logp', 'molecular_weight', 'molecular_weight_reverse',\n#  'perimeter', 'random', 'scaffold', 'scaffold_generic',\n#  'scaffold_kmeans', 'umap']\n\n# Create a splitter via factory function\nsplitter = get_splitter(\"scaffold\", make_generic=True, n_splits=5, test_size=0.2)\n\n# Use with SMILES directly\nsmiles = [\"CCO\", \"c1ccccc1\", \"CCN\", ...]\nfor train_idx, test_idx in splitter.split(smiles):\n    train_smiles = [smiles[i] for i in train_idx]\n    test_smiles = [smiles[i] for i in test_idx]\n```\n\n### 1. Structure-Based Splits\n```python\nfrom alinemol.splitters import get_splitter, ScaffoldSplit, PerimeterSplit\n\n# Bemis-Murcko scaffold splitting (via factory)\nscaffold_split = get_splitter(\"scaffold\", make_generic=True)\n\n# Or direct class instantiation\nscaffold_split = ScaffoldSplit(make_generic=True)\n\n# Perimeter-based clustering\nperimeter_split = get_splitter(\"perimeter\", n_clusters=10)\n```\n\n### 2. Property-Based Splits\n```python\nfrom alinemol.splitters import get_splitter, MolecularWeightSplit, MolecularLogPSplit\n\n# Split by molecular weight (test on larger molecules)\nmw_split = get_splitter(\"molecular_weight\", generalize_to_larger=True)\n\n# Split by lipophilicity\nlogp_split = get_splitter(\"molecular_logp\", generalize_to_larger=True)\n```\n\n### 3. Similarity-Based Splits\n```python\nfrom alinemol.splitters import get_splitter, HiSplit, LoSplit\n\n# Hi-split: ensures low similarity between train/test\nhi_split = get_splitter(\"hi\",\n    similarity_threshold=0.4,\n    train_min_frac=0.7,\n    test_min_frac=0.15\n)\n\n# Lo-split: for lead optimization scenarios\nlo_split = get_splitter(\"lo\",\n    threshold=0.4,\n    min_cluster_size=5,\n    std_threshold=0.6\n)\n```\n\n### 4. Clustering-Based Splits\n```python\nfrom alinemol.splitters import get_splitter, UMAPSplit, KMeansSplit\n\n# UMAP + clustering split\numap_split = get_splitter(\"umap\",\n    n_clusters=20,\n    n_neighbors=100,\n    min_dist=0.1\n)\n\n# K-means clustering split\nkmeans_split = get_splitter(\"kmeans\", n_clusters=10, metric=\"jaccard\")\n\n# Butina clustering (Taylor-Butina algorithm)\nbutina_split = get_splitter(\"butina\", cutoff=0.65)\n\n# Scaffold extraction + k-means on scaffold ECFP fingerprints\nscaffold_kmeans_split = get_splitter(\"scaffold_kmeans\", n_clusters=10, make_generic=True)\n```\n\n### Split Quality Analysis\n\n`SplitAnalyzer` quantifies what a split actually produced: train↔test Tanimoto similarity\ndistribution, scaffold overlap, property divergence, and size metrics. Useful for\nvalidating that an \"OOD\" splitter behaves more strictly than the random baseline.\n\n```python\nfrom alinemol.splitters import SplitAnalyzer, get_splitter\n\nanalyzer = SplitAnalyzer(smiles_list)\n\n# Analyze a single split\nsplitter = get_splitter(\"scaffold\")\ntrain_idx, test_idx = next(splitter.split(smiles_list))\nreport = analyzer.analyze_split(train_idx, test_idx, splitter_name=\"scaffold\")\nprint(f\"Mean train-test similarity: {report.similarity_metrics.mean_sim:.3f}\")\n\n# Compare multiple splitters in one DataFrame\ncomparison = analyzer.compare_splitters([\"scaffold\", \"kmeans\", \"random\"])\n```\n\nFor large datasets, point `SplitAnalyzer` at a precomputed Jaccard distance matrix to\nskip recomputing per-call similarity (the TDC datasets ship one as `Jaccard_distance.npy`):\n\n```python\nanalyzer = SplitAnalyzer(\n    smiles_list,\n    precomputed_distance_matrix=\"datasets/TDC/CYP2C9/Jaccard_distance.npy\",\n)\n```\n\nThe SMILES list must be in the same order as the matrix (use `valid_canonical_smiles.txt`\nfrom the same dataset folder, or let `scripts/analyze_splits.py` auto-detect and validate\nthe alignment).\n\n### Command-Line Splitting Tool\n\nALineMol includes a production-ready CLI tool for dataset splitting:\n\n```bash\n# Basic scaffold splitting\npython scripts/splitting.py -f data/molecules.csv -sp scaffold --save\n\n# Run all splitters at once\npython scripts/splitting.py -f data/molecules.csv -sp all --save\n\n# Preview without saving (dry run)\npython scripts/splitting.py -f data/molecules.csv -sp kmeans --dry-run\n\n# List available splitters\npython scripts/splitting.py --list-splitters\n```\n\n## Development\n\n### Tests\n\nRun the test suite with pytest:\n\n```bash\npytest\n```\n\n### Code Style\n\nWe use `ruff` for linting and formatting:\n\n```bash\n# Check code style\nruff check\n\n# Format code\nruff format\n```\n\n### Documentation\n\nBuild and serve the documentation locally:\n\n```bash\nmkdocs serve\n```\n\n### Continuous Integration\n\nThis project uses GitHub Actions for continuous integration and deployment:\n\n- **CI Workflow**: Automatically runs tests and linting on all pull requests and pushes to the main branch\n- **Release Workflow**: Automatically builds and publishes the package to PyPI when a new release is created\n\nTo create a new release:\n\n1. Update the version in `_version.py`\n2. Create a new tag and GitHub release\n3. The release workflow will automatically publish to PyPI\n\n## Citation\n\nIf you find ALineMol useful in your research, please cite the following paper:\n\n```bibtex\n@article{fooladi2025evaluating,\n  title={Evaluating Machine Learning Models for Molecular Property Prediction: Performance and Robustness on Out-of-Distribution Data},\n  author={Fooladi, Hosein and Vu, Thi Ngoc Lan and Mathea, Miriam and Kirchmair, Johannes},\n  journal={Journal of Chemical Information and Modeling},\n  volume={65},\n  number={19},\n  pages={9871--9891},\n  year={2025},\n  publisher={ACS Publications},\n  doi={https://doi.org/10.1021/acs.jcim.5c00475}\n}\n```\n\n## Related Work\n\n- **Splito**: Molecular splitting library - [GitHub](https://github.com/datamol-io/splito)\n- **TDC**: Therapeutics Data Commons - [Website](https://tdcommons.ai/)\n- **DGL-LifeSci**: Deep Graph Library for Life Sciences - [GitHub](https://github.com/awslabs/dgl-lifesci)\n\n## Documentation\n\n- 📖 [Full Documentation](docs/)\n- 📝 [API Reference](docs/api/)\n- 🎓 [Tutorials](docs/tutorials/)\n- 📊 [Paper](https://doi.org/10.1021/acs.jcim.5c00475)\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details on:\n- Reporting bugs\n- Suggesting enhancements  \n- Submitting pull requests\n- Code style guidelines\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhfooladi%2Falinemol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhfooladi%2Falinemol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhfooladi%2Falinemol/lists"}