{"id":29225673,"url":"https://github.com/aidos-lab/rings","last_synced_at":"2026-02-05T10:01:46.968Z","repository":{"id":301321477,"uuid":"974086210","full_name":"aidos-lab/rings","owner":"aidos-lab","description":"Relevant Information in Node features and Graph Structure","archived":false,"fork":false,"pushed_at":"2025-07-22T11:40:48.000Z","size":501,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-22T12:24:49.265Z","etag":null,"topics":["data-centric","evaluation-framework","geometric-deep-learning","graph-learning","icml-2025"],"latest_commit_sha":null,"homepage":"https://aidos.group/rings/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aidos-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-28T08:32:24.000Z","updated_at":"2025-07-22T11:40:52.000Z","dependencies_parsed_at":"2025-06-26T09:35:45.604Z","dependency_job_id":"68d8b766-9f6d-4f78-9c24-aeab84812dc5","html_url":"https://github.com/aidos-lab/rings","commit_stats":null,"previous_names":["aidos-lab/rings"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aidos-lab/rings","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidos-lab%2Frings","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidos-lab%2Frings/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidos-lab%2Frings/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidos-lab%2Frings/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aidos-lab","download_url":"https://codeload.github.com/aidos-lab/rings/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidos-lab%2Frings/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29119218,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T09:40:36.738Z","status":"ssl_error","status_checked_at":"2026-02-05T09:36:49.977Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-centric","evaluation-framework","geometric-deep-learning","graph-learning","icml-2025"],"created_at":"2025-07-03T07:37:37.862Z","updated_at":"2026-02-05T10:01:46.931Z","avatar_url":"https://github.com/aidos-lab.png","language":"Python","readme":"# RINGS\n\nThis is the official repository for our ICML paper:\n\n**[No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets](https://arxiv.org/abs/2502.02379)**\n\nwhich introduces **RINGS**: a perturbation framework for attributed graphs, designed to enable more principled evaluations of graph-learning benchmarks from first principles.\n\n---\n\n## 🚧 Repository Status\n\nThis repository is **under active development**.  \nWe’re making it public early to invite feedback, discussion, and **transparency** as we move from research prototypes to a stable, user-friendly package.\n\nIn the coming weeks, we’ll release updates, architectural notes, and implementation details via a series of pull requests. You're welcome to follow along, open issues, or suggest improvements!\n\n---\n\n## 🚀 Current MVP Release\n\nThis initial Minimum Viable Product (MVP) includes:\n\n- A set of **graph perturbation transformations** for manipulating node features and graph structure\n- The **SeparabilityFunctor**, which enables statistical comparisons between distributions\n- The **ComplementarityFunctor**, which computes mode complementarity between node features and graph structure\n- Example scripts demonstrating usage with PyTorch Geometric datasets and toy performance distributions\n\n---\n\n## 💍 Framework Overview\n\nWe are developing a community-friendly implementation of the **RINGS** framework introduced in the paper. Our goal is to make it easy for the graph-learning community to:\n\n- Apply dataset perturbations tailored to graph-learning datasets\n- Conduct more rigorous and insightful evaluations of both datasets and models\n- Promote better dataset practices and evaluation hygiene across the field\n\nIf you have feedback on the paper or suggestions for how this package could better integrate with popular frameworks, please feel free to reach out to the authors.\n\n---\n\n## 📦 Installation\n\nRINGS uses [uv](https://github.com/astral-sh/uv) as the package manager, which provides faster dependency resolution and installation.\n\n### Prerequisites\n\nInstall `uv` if you don’t have it already:\n\n```bash\npip install uv\n```\n\n### Installing RINGS\n\nClone the repository and install dependencies using `uv`:\n\n```bash\n# Clone the repository\ngit clone https://github.com/aidos-lab/rings.git\ncd rings\n\n# Install dependencies\nuv sync\n\n# Activate environment\nsource .venv/bin/activate\n```\n\n---\n\n## 🔑 Key Components\n\n### Mode Perturbations\n\nRINGS provides several perturbation transforms that can be applied to graph datasets:\n\n**Node Feature Perturbations:**\n\n- `Original`: Keeps node features unchanged (baseline)\n- `EmptyFeatures`: Replaces node features with zero vectors\n- `RandomFeatures`: Replaces node features with random values\n- `CompleteFeatures`: Assigns unique one-hot vectors to nodes\n\n**Graph Structure Perturbations:**\n\n- `EmptyGraph`: Removes all edges from the graph\n- `CompleteGraph`: Connects every pair of nodes\n- `RandomGraph`: Generates a random graph structure\n- `RandomConnectedGraph`: Generates a random graph that is guaranteed to be connected\n\n### SeparabilityFunctor\n\nThe `SeparabilityFunctor` computes statistically rigorous comparisons between multiple distributions to determine if they differ significantly. This is useful for:\n\n- Evaluating whether different graph perturbations produce statistically distinct model performances\n- Identifying which perturbations most impact model behavior\n- Making rigorous, statistically valid claims about distribution separability\n\nIt employs statistical tests with permutation testing and built-in correction for multiple hypotheses (Bonferroni correction).\n\nAvailable comparators include:\n\n- `KSComparator`: Kolmogorov–Smirnov test for comparing distributions\n- `WilcoxonComparator`: Wilcoxon signed-rank test for paired comparisons\n\n### ComplementarityFunctor\n\nThe `ComplementarityFunctor` measures the alignment between node features and graph structure by comparing their induced metric spaces. It can help you understand:\n\n- Whether node features and graph structure contain complementary information\n- How different perturbations affect this complementarity\n- The distribution of information content across modalities in graph datasets\n\n---\n\n## 🔍 Example Usage\n\nThe repository includes example scripts that demonstrate how to use RINGS to analyze graph datasets.\n\n### Performance Separability\n\nMeasure distances between performance distributions to test whether the original dataset statistically outperforms perturbed versions.\n\n```bash\n# Run a basic separability analysis with the default KS comparator\npython -m examples.separability --comparator ks --alpha 0.05\n```\n\n```bash\n# Use the Wilcoxon test comparator\npython -m examples.separability --comparator wilcoxon --alpha 0.01\n```\n\n```bash\n# Get help and see all available options\npython -m examples.separability --help\n```\n\nThe script analyzes and compares distributions from synthetic data, showing how to determine if differences are statistically significant.\n\n---\n\n### Mode Complementarity\n\nAssess the complementarity of geometric information in the metric spaces of node features and graph structure.\n\n```bash\n# Run the example on the MUTAG dataset with original (unperturbed) graphs\npython -m examples.complementarity --dataset MUTAG --perturbation original\n```\n\n```bash\n# Try different perturbations\npython -m examples.complementarity --dataset MUTAG --perturbation random-features\n```\n\n```bash\n# Get help and see all available options\npython -m examples.complementarity --help\n```\n\nThe script outputs complementarity statistics that measure how well node features align with graph structure in the dataset.\n\n## 📚 Citation\n\nIf you use RINGS in your research, please cite our paper:\n\n```bibtex\n@inproceedings{\ncoupette2025metric,\ntitle={No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets},\nauthor={Corinna Coupette and Jeremy Wayland and Emily Simons and Bastian Rieck},\nbooktitle={Forty-second International Conference on Machine Learning},\nyear={2025},\nurl={https://openreview.net/forum?id=XbmBNwrfG5}\n}\n```\n\nStay tuned for more examples and documentation!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidos-lab%2Frings","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faidos-lab%2Frings","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidos-lab%2Frings/lists"}