{"id":21432490,"url":"https://github.com/gagolews/genieclust","last_synced_at":"2025-04-04T13:13:09.210Z","repository":{"id":54483248,"uuid":"133134262","full_name":"gagolews/genieclust","owner":"gagolews","description":"Genie: Fast and Robust Hierarchical Clustering with Noise Point Detection - in Python and R","archived":false,"fork":false,"pushed_at":"2025-03-20T16:29:05.000Z","size":83012,"stargazers_count":61,"open_issues_count":27,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-28T12:09:47.552Z","etag":null,"topics":["cluster-analysis","clustering","clustering-algorithm","data-analysis","data-mining","data-science","genie","hdbscan","hierarchical-clustering","hierarchical-clustering-algorithm","machine-learning","machine-learning-algorithms","mlpack","nmslib","python","python3","r","sparse"],"latest_commit_sha":null,"homepage":"https://genieclust.gagolewski.com","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gagolews.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-12T10:01:53.000Z","updated_at":"2025-03-20T16:29:08.000Z","dependencies_parsed_at":"2023-09-25T01:51:36.778Z","dependency_job_id":"7266db66-5540-4398-80b4-e751c55c5732","html_url":"https://github.com/gagolews/genieclust","commit_stats":{"total_commits":454,"total_committers":5,"mean_commits":90.8,"dds":0.05947136563876654,"last_synced_commit":"5f049495becbac919792730ed3a40e046b154135"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gagolews%2Fgenieclust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gagolews%2Fgenieclust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gagolews%2Fgenieclust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gagolews%2Fgenieclust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gagolews","download_url":"https://codeload.github.com/gagolews/genieclust/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247182401,"owners_count":20897381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-analysis","clustering","clustering-algorithm","data-analysis","data-mining","data-science","genie","hdbscan","hierarchical-clustering","hierarchical-clustering-algorithm","machine-learning","machine-learning-algorithms","mlpack","nmslib","python","python3","r","sparse"],"created_at":"2024-11-22T23:18:45.360Z","updated_at":"2025-04-04T13:13:09.183Z","avatar_url":"https://github.com/gagolews.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://genieclust.gagolewski.com\"\u003e\u003cimg src=\"https://www.gagolewski.com/_static/img/genieclust.png\" align=\"right\" height=\"128\" width=\"128\" /\u003e\u003c/a\u003e\n# [**genieclust**](https://genieclust.gagolewski.com/) Package for R and Python\n\n### *Genie*: Fast and Robust Hierarchical Clustering with Noise Point Detection\n\n\n![genieclust for Python](https://github.com/gagolews/genieclust/workflows/genieclust%20for%20Python/badge.svg)\n![genieclust for R](https://github.com/gagolews/genieclust/workflows/genieclust%20for%20R/badge.svg)\n\n\n\u003e **Genie finds meaningful clusters quickly – even on large data sets.**\n\u003e\n\u003e A comprehensive tutorial, benchmarks, and a reference manual is available\nat \u003chttps://genieclust.gagolewski.com/\u003e.\n\nWhen using **genieclust** in research publications, please\ncite (Gagolewski, 2021) and (Gagolewski, Bartoszuk, Cena, 2016)\nas specified below. Thank you.\n\n\n## About\n\nA faster and more powerful version of *Genie* – a robust and outlier\nresistant clustering algorithm (see Gagolewski, Bartoszuk, Cena, 2016),\noriginally included in the R package\n[**genie**](https://CRAN.R-project.org/package=genie).\n\nThe idea behind Genie is beautifully simple. First, make each individual\npoint the only member of its own cluster. Then, keep merging pairs\nof the closest clusters, one after another. However, to **prevent\nthe formation of clusters of highly imbalanced sizes** a point group of\nthe *smallest* size will sometimes be matched with its nearest neighbour.\n\nGenie's appealing simplicity goes hand in hand with its usability;\nit **often outperforms other clustering approaches**\nsuch as K-means, BIRCH, or average, Ward, and complete linkage\non [benchmark data](https://github.com/gagolews/clustering-benchmarks).\nOf course, there is no, nor will there ever be, a single best\nuniversal clustering approach for every kind of problem, but Genie\nis definitely worth a try!\n\nThanks to its being based on minimal spanning trees of the pairwise distance\ngraphs, Genie is also **very fast** – determining the whole cluster hierarchy\nfor datasets of millions of points can be completed within minutes. Therefore,\nit is nicely suited for solving **extreme clustering tasks** (large datasets\nwith any number of clusters to detect) for data (also sparse) that fit into\nmemory. Thanks to the use of [**nmslib**](https://github.com/nmslib/nmslib)\n(if available), sparse or string inputs are also supported.\n\nIt also allows clustering with respect to mutual reachability distances\nso that it can act as a **noise point detector** or a\nrobustified version of *HDBSCAN\\**  (see Campello et al., 2013)\nthat is able to detect a predefined\nnumber of clusters and hence it doesn't dependent on the *DBSCAN*'s somewhat\ndifficult-to-set `eps` parameter.\n\n\nThe package also features an implementation of economic inequality indices\n(the Gini, Bonferroni index), external cluster validity measures\n(e.g., the normalised clustering accuracy and partition similarity scores\nsuch as the adjusted Rand, Fowlkes-Mallows, adjusted mutual information,\nand the pair sets index), and internal cluster validity indices\n(e.g., the Calinski-Harabasz, Davies-Bouldin, Ball-Hall, Silhouette,\nand generalised Dunn indices).\n\n\n## Author and Contributors\n\n**Author and Maintainer**: [Marek Gagolewski](https://www.gagolewski.com/)\n\nContributors:\nMaciej Bartoszuk,\n[Anna Cena](https://cena.rexamine.com/) (R packages\n[**genie**](https://CRAN.R-project.org/package=genie)\nand [**CVI**](https://github.com/gagolews/optim_cvi)),\n[Peter M. Larsen](https://github.com/pmla)\n([rectangular_lsap](https://github.com/scipy/scipy/blob/main/scipy/optimize/rectangular_lsap/rectangular_lsap.cpp)).\n\n\n\n\n## Examples, Tutorials, and Documentation\n\nR's interface is compatible with `stats::hclust()`, but there is more.\n\n```r\nX \u003c- ... # some data\nh \u003c- gclust(X)\nplot(h) # plot cluster dendrogram\ncutree(h, k=2)\n# or genie(X, k=2)\n```\n\n\n*To learn more about R, check out Marek's open-access (free!) textbook*\n[Deep R Programming](https://deepr.gagolewski.com/).\n\n\n\nThe Python language version of **genieclust** follows a **scikit-learn**-like API:\n\n```python\nimport genieclust\nX = ... # some data\ng = genieclust.Genie(n_clusters=2)\nlabels = g.fit_predict(X)\n```\n\nTutorials and the package documentation are available\n[here](https://genieclust.gagolewski.com/).\n\n*To learn more about Python, check out Marek's recent open-access (free!) textbook*\n[Minimalist Data Wrangling in Python](https://datawranglingpy.gagolewski.com/).\n\n\n## How to Install\n\n\n### Python Version\n\nTo install via `pip` (see [PyPI](https://pypi.org/project/genieclust)):\n\n```bash\npip3 install genieclust\n```\n\nThe package requires Python 3.7+ together with **cython** as well as\n**numpy**, **scipy**, **matplotlib**, **scikit-learn**, and **mlpack**.\nOptional dependency: **nmslib**.\n\n\n\n\n\n\n\n### R Version\n\n\nTo install the most recent release, call:\n\n```r\ninstall.packages(\"genieclust\")\n```\n\nSee the package entry on\n[CRAN](https://CRAN.R-project.org/package=genieclust).\n\n\n\n\n### Other\n\nThe core functionality is implemented in the form of a header-only\nC++ library. It can thus be easily adapted for use in other environments.\n\nNew contributions are welcome (e.g., Julia, Matlab, ...).\n\n\n\n\n## License\n\nCopyright (C) 2018–2025 Marek Gagolewski \u003chttps://www.gagolewski.com/\u003e\n\nThis program is free software: you can redistribute it and/or modify it\nunder the terms of the GNU Affero General Public License Version 3, 19\nNovember 2007, published by the Free Software Foundation.\n\nThis program is distributed in the hope that it will be useful, but\nWITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero\nGeneral Public License Version 3 for more details. You should have\nreceived a copy of the License along with this program. If not, see\n(https://www.gnu.org/licenses/).\n\n--------------\n\nThe file `src/c_scipy_rectangular_lsap.h` is adapted from the\n**scipy** project (https://scipy.org/scipylib), source:\n`/scipy/optimize/rectangular_lsap/rectangular_lsap.cpp`.\nAuthor: Peter M. Larsen. Distributed under the BSD-3-Clause license.\n\nThe implementation of internal cluster validity measures\nwere adapted from our previous project (Gagolewski, Bartoszuk, Cena, 2021);\nsee [optim_cvi](https://github.com/gagolews/optim_cvi).\nOriginally distributed under the GNU Affero General Public License Version 3.\n\n\n## References\n\nGagolewski M., genieclust: Fast and robust hierarchical clustering,\n*SoftwareX* **15**, 2021, 100722.\n[DOI: 10.1016/j.softx.2021.100722](https://doi.org/10.1016/j.softx.2021.100722).\n\u003chttps://genieclust.gagolewski.com/\u003e.\n\nGagolewski M., Bartoszuk M., Cena A., Genie: A new, fast, and\noutlier-resistant hierarchical clustering algorithm, *Information\nSciences* **363**, 2016, 8–23.\n[DOI: 10.1016/j.ins.2016.05.003](https://doi.org/10.1016/j.ins.2016.05.003).\n\nGagolewski M., Bartoszuk M., Cena A., Are cluster validity measures (in)valid?,\n*Information Sciences* **581**, 2021, 620–636.\n[DOI: 10.1016/j.ins.2021.10.004](https://doi.org/10.1016/j.ins.2021.10.004).\n\nGagolewski M., Cena A., Bartoszuk M., Brzozowski L.,\nClustering with minimum spanning trees: How good can it be?,\n*Journal of Classification*, 2024, in press,\n[DOI: 10.1007/s00357-024-09483-1](https://doi.org/10.1007/s00357-024-09483-1).\n\nGagolewski M., Normalised clustering accuracy: An asymmetric external\ncluster validity measure, *Journal of Classification*, 2024, in press,\n[DOI: 10.1007/s00357-024-09482-2](https://doi.org/10.1007/s00357-024-09482-2).\n\nGagolewski M., A framework for benchmarking clustering algorithms,\n*SoftwareX* **20**, 2022, 101270.\n[DOI: 10.1016/j.softx.2022.101270](https://doi.org/10.1016/j.softx.2022.101270).\n\u003chttps://clustering-benchmarks.gagolewski.com/\u003e.\n\nCampello R.J.G.B., Moulavi D., Sander J.,\nDensity-based clustering based on hierarchical density estimates,\n*Lecture Notes in Computer Science* **7819**, 2013, 160–172.\n[DOI: 10.1007/978-3-642-37456-2_14](https://doi.org/10.1007/978-3-642-37456-2_14).\n\nMueller A., Nowozin S., Lampert C.H., Information theoretic clustering\nusing minimum spanning trees, *DAGM-OAGM*, 2012.\n\nRezaei M., Fränti P., Set matching measures for external cluster validity,\n*IEEE Transactions on Knowledge and Data Engineering* **28**(8), 2016,\n2173–2186 [DOI: 10.1109/TKDE.2016.2551240](https://doi.org/10.1109/TKDE.2016.2551240).\n\nSee the package's [homepage](https://genieclust.gagolewski.com/) for more\nreferences.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgagolews%2Fgenieclust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgagolews%2Fgenieclust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgagolews%2Fgenieclust/lists"}