{"id":15482778,"url":"https://github.com/ap6yc/icvi-examples","last_synced_at":"2026-02-05T04:03:04.949Z","repository":{"id":128626506,"uuid":"350850618","full_name":"AP6YC/ICVI-Examples","owner":"AP6YC","description":"Example usage of the Incremental Cluster Validity Indices (ICVI) implemented in the AdaptiveResonance.jl julia package.","archived":false,"fork":false,"pushed_at":"2021-04-12T14:40:05.000Z","size":136,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-26T11:39:23.494Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AP6YC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-23T20:37:17.000Z","updated_at":"2021-04-12T14:40:08.000Z","dependencies_parsed_at":"2023-04-13T05:01:02.285Z","dependency_job_id":null,"html_url":"https://github.com/AP6YC/ICVI-Examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AP6YC/ICVI-Examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AP6YC%2FICVI-Examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AP6YC%2FICVI-Examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AP6YC%2FICVI-Examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AP6YC%2FICVI-Examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AP6YC","download_url":"https://codeload.github.com/AP6YC/ICVI-Examples/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AP6YC%2FICVI-Examples/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29110589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T03:44:17.043Z","status":"ssl_error","status_checked_at":"2026-02-05T03:44:12.077Z","response_time":65,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T05:09:59.236Z","updated_at":"2026-02-05T04:03:04.932Z","avatar_url":"https://github.com/AP6YC.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ICVI-Examples\n\n| **Build Status** | **Coverage** |\n|:----------------:|:------------:|\n| [![Build Status][ci-img]][ci-url] [![Build Status][appveyor-img]][appveyor-url] | [![Codecov][codecov-img]][codecov-url] [![Coveralls][coveralls-img]][coveralls-url] |\n\n[ci-img]: https://github.com/AP6YC/ICVI-Examples/workflows/CI/badge.svg\n[ci-url]: https://github.com/AP6YC/ICVI-Examples/actions?query=workflow%3ACI\n\n[appveyor-img]: https://ci.appveyor.com/api/projects/status/github/AP6YC/ICVI-Examples?svg=true\n[appveyor-url]: https://ci.appveyor.com/project/AP6YC/ICVI-Examples\n\n[codecov-img]: https://codecov.io/gh/AP6YC/ICVI-Examples/branch/main/graph/badge.svg\n[codecov-url]: https://codecov.io/gh/AP6YC/ICVI-Examples\n\n[coveralls-img]: https://coveralls.io/repos/github/AP6YC/ICVI-Examples/badge.svg?branch=main\n[coveralls-url]: https://coveralls.io/github/AP6YC/ICVI-Examples?branch=main\n\nExample usage of the Incremental Cluster Validity Indices (ICVI) implemented in the ClusterValidityIndices.jl julia package.\n\nThis package is developed and maintained by [Sasha Petrenko](https://github.com/AP6YC) with sponsorship by the [Applied Computational Intelligence Laboratory (ACIL)](https://acil.mst.edu/). This project is supported by grants from the [Night Vision Electronic Sensors Directorate](https://c5isr.ccdc.army.mil/inside_c5isr_center/nvesd/), the [DARPA Lifelong Learning Machines (L2M) program](https://www.darpa.mil/program/lifelong-learning-machines), [Teledyne Technologies](http://www.teledyne.com/), and the [National Science Foundation](https://www.nsf.gov/).\nThe material, findings, and conclusions here do not necessarily reflect the views of these entities.\n\n## Table of Contents\n\n- [ICVI-Examples](#icvi-examples)\n  - [Table of Contents](#table-of-contents)\n  - [Outline](#outline)\n  - [Quickstart](#quickstart)\n  - [Structure](#structure)\n  - [Usage](#usage)\n    - [Data](#data)\n    - [Instantiation](#instantiation)\n    - [Incremental vs. Batch](#incremental-vs-batch)\n    - [Updating](#updating)\n    - [Criterion Values](#criterion-values)\n    - [Porcelain](#porcelain)\n  - [Authors](#authors)\n\n## Outline\n\nThis Julia project contains an outline of the conceptual usage of CVIs along with many example scripts.\n[Quickstart](##Quickstart) provides an overview of how to use this project, while [Structure](##Structure) outlines the project file structure, giving context to the locations of every component of the project.\n[Usage](##Usage) outlines the general syntax and workflow of the ICVIs, while [Authors](##Authors) gives credit to the author(s).\n\n## Quickstart\n\nThis section provides a quick overview of how to use the project.\nFor more detailed code usage, please see [Usage](##Usage).\n\nThis project has several example scripts to demonstrate the functionality of CVIs in the ClusterValidityIndices.jl package.\nIn `ICVI-Examples/src/examples/`, the scripts `db.jl`, `ps.jl`, and `xb.jl` demonstrate usage of the Davies-Boudin (DB), Partition Separation (PS), and Xie-Beni (XB) metrics, respectively.\n\n**NOTE** Each of these scripts must be run at the top level of the project to correctly point to the datasets.\nFor example, they can be run in the shell with\n\n```sh\njulia src/examples/db.jl\n```\n\nor in a Julia REPL session with\n\n```sh\ninclude(\"src/examples/db.jl\")\n```\n\nThree preprocessed datasets are provided under `data/` to demonstrate the correct partitioning, over partitioning, and under partitioning of samples by a clustering algorithm to illustrate how the CVIs behave in each case.\nThe data consists of 2000 samples of 2-element features with the clustering label appended in the third column.\nYou can change which dataset is used in each script above.\n\nLastly, there is a large experiment script `src/examples/combined.jl` that runs every CVI with all three datasets.\nThe common code for all scripts is contained under `src/common.jl`, while the experiment subroutines referenced in these scripts are under `src/experiments.jl`, so feel free to modify them to further explore the behavior and usage of these CVIs.\n\n## Structure\n\n```console\nICVI-Examples\n├── .github/workflows       // GitHub: workflows for testing and documentation.\n├── data                    // Data: CI and example data location.\n├── src                     // Source: scripts and common helper functions.\n│   └───examples            //      Example scripts for CVI usage.\n├── test                    // Test: unit, integration, and environment tests.\n├── .gitignore              // Git: .gitignore for the whole project.\n├── LICENSE                 // Doc: the license to the project.\n├── Manifest.toml           // Julia: the explicit package versions used.\n├── Project.toml            // Julia: the Pkg.jl dependencies of the project.\n└── README.md               // Doc: this document.\n```\n\n## Usage\n\nThe usage of these CVIs requires an understanding of:\n- [Data](###Data) assumptions of the CVIs.\n- [How to instantiate](###Instantiation) the CVIs.\n- [Incremental vs. batch](###Incremental-vs.-Batch) evaluation.\n- [Updating](###Updating) internal CVI parameters.\n- [Computing and extracting](###Criterion-Values) the criterion values.\n- [Porcelain functions](###Porcelain) that are available to simplify operation.\n\n### Data\n\nBecause Julia is programmed in a column-major fashion, all CVIs make the assumption that the first dimension (columns) contains features, while the second dimension (rows) contains samples.\nThis is more important for batch operation, as incremental operation accepts 1-D sample of features at each time step by definition.\n\nFor example,\n\n```julia\n# Load data from somewhere\ndata = load_data()\n# The data shape is dimsion x samples\ndim, n_samples = size(data)\n```\n\n**NOTE**: As of ClusterValidityIndices.jl v0.1.3, all the CVIs assume that the labels are presented sequentially initially, starting with index 1 (e.g., 1, 1, 2, 2, 3, 2, 2, 1, 3, 4, 4 ...).\nYou may repeat previously seen label indices, but skipping label indices (e.g., 1, 2, 4) results in undefined behavior.\nIn this project, this is ameliorated with the function\n\n```julia\nrelabel_cvi_data(labels::Array{M, 1}) where {M\u003c:Int}\n```\n\nFor example,\n\n```julia\ndata_file = \"path/to/data.csv\"\ndata, labels = get_cvi_data(data_file)\nlabels = relabel_cvi_data(labels)\n```\n\nAlternatively, you may pairwise sort the entirety of the data with\n\n```julia\nsort_cvi_data(data::Array{N, 2}, labels::Array{M, 1}) where {N\u003c:Real, M\u003c:Int}\n```\n\n**NOTE*** `sort_cvi_data` reorders the input data as well, which will lead to different ICVI results than with `relabel_cvi_data`.\n\n### Instantiation\n\nThe names of each CVI are capital abbreviations of their literature names, often based upon the surname of the principal authors of the papers that introduce the metrics.\nAll CVIs are implemented with the default constructor, such as\n\n```julia\ncvi = DB()\n```\n\n### Incremental vs. Batch\n\nThe CVIs in this project all contain *incremental* and *batch* implementations.\nWhen evaluated in incremental mode, they are often called ICVIs (incremental cluster validity indices).\nIn documentation, CVI refers to both modalities (as in the literature), but in code, CVI means batch and ICVI means incremental.\n\nThe funtions that differ between the two modes are how they are updated\n\n```julia\n# Incremental\nparam_inc!(...)\n# Batch\nparam_batch!(...)\n```\n\nand their respective porcelain functions\n\n```julia\n# Incremental\nget_icvi!(...)\n# Batch\nget_cvi!(...)\n```\n\nThey both compute their most recent criterion values with\n\n```julia\nevaluate!(...)\n```\n\n**NOTE**: Any CVI can switch to be updated incrementally or in batch, as the CVI data structs are update mode agnostic.\n\n### Updating\n\nThe CVIs in this project all contain internal *parameters* that must be updated.\nEach update function modifies the CVI, so they use the Julia nomenclature convention of appending an exclamation point to indicate as much.\n\nIn both incremental and batch modes, the parameter update requires:\n\n- The CVI being updates\n- The sample (or array of samples)\n- The label(s) that was/were prescribed by the clustering algorithm to the sample(s)\n\nMore concretely, they are\n\n```julia\n# Incremental updating\nparam_inc!(cvi::C, sample::Array{T, 1}, label::I) where {C\u003c:AbstractCVI, T\u003c:Real, I\u003c:Int}\n# Batch updating\nparam_batch!(cvi::C, data::Array{T, 2}, labels::Array{I, 1}) where {C\u003c:AbstractCVI, T\u003c:Real, I\u003c:Int}\n```\n\nEvery CVI is a subtype of the abstract type `AbstractCVI`.\nFor example, we may instantiate and load our data\n\n```julia\ncvi = DB()\ndata = load_data()\nlabels = get_cluster_labels(data)\ndim, n_samples = size(data)\n```\n\nthen update the parameters incrementally with\n\n```julia\n# Iterate over all samples\nfor ix = 1:n_samples\n    sample = data[:, ix]\n    label = labels[ix]\n    param_inc!(cvi, sample, labels)\nend\n```\n\nor in batch with\n\n```julia\nparam_batch!(cvi, data, labels)\n```\n\nFurthermore, any CVI can alternate between being updated in incremental or batch modes, such as\n\n```julia\n# Create a new CVI\ncvi_mixed = DB()\n\n# Update on half of the data incrementally\ni_split = n_samples/2\nfor ix = 1:i_split\n    param_inc!(cvi, data[:, ix], labels[ix])\nend\n\n# Update on the other half all at once\nparam_batch!(cvi, data[:, (i_split+1):end])\n```\n\n### Criterion Values\n\nThe CVI parameters are separate from the criterion values that they produce.\nThis is partly because in batch mode computing the criterion value is only relevant at the last step, which eliminates unnecessarily computing it at every step.\nThis is also provide granularity to the user that may only which to extract the criterion value occasionally during incremental mode.\n\nBecause the criterion values only depend on the internal CVI parameters, they are computed (and internally stored) with\n\n```julia\nevaluate!(cvi::C) where {C\u003c:AbstractCVI}\n```\n\nTo extract them, you must then simply grab the criterion value from the CVI struct with\n\n```julia\ncriterion_value = cvi.criterion_value\n```\n\nFor example, after loading the data\n\n```julia\ncvi = DB()\ndata = load_data()\nlabels = get_cluster_labels(data)\ndim, n_samples = size(data)\n```\n\nwe may extract and return the criterion value at every step with\n\n```julia\ncriterion_values = zeros(n_samples)\nfor ix = 1:n_samples\n    param_inc!(cvi, data[:, ix], labels[ix])\n    evaluate!(cvi)\n    criterion_values[ix] = cvi.criterion_value\nend\n```\n\nor we may get it at the end in batch mode with\n\n```julia\nparam_batch!(cvi, data, labels)\nevaluate!(cvi)\ncriterion_value = cvi.criterion_value\n```\n\n### Porcelain\n\nTaken from the `git` convention of calling low-level operations *plumbing* and high-level user-land functions *porcelain*, the package comes with a small set of *porcelain* function that do common operations all at once for the user.\n\nFor example, you may compute, evalute, and return the criterion value all at once with the functions\n\n```julia\n# Incremental\nget_icvi!(...)\n# Batch\nget_cvi!(...)\n```\n\nExactly as in the usage for updating the parameters, the functions take the cvi, sample(s), and clustered label(s) as input:\n\n```julia\n# Incremental\nget_icvi!(cvi::C, x::Array{N, 1}, y::M) where {C\u003c:AbstractCVI, N\u003c:Real, M\u003c:Int}\n# Batch\nget_cvi!(cvi::C, x::Array{N, 2}, y::Array{M, 1}) where {C\u003c:AbstractCVI, N\u003c:Real, M\u003c:Int}\n```\n\nFor example, after loading the data you may get the criterion value at each step with\n\n```julia\ncriterion_values = zeros(n_samples)\nfor ix = 1:n_samples\n    criterion_values = get_icvi!(cvi, data[:, ix], labels[ix])\nend\n```\n\nor you may get the final criterion value in batch mode with\n\n```julia\ncriterion_value = get_cvi!(cvi, data, labels)\n```\n\n## Authors\n\n- Sasha Petrenko \u003csap625@mst.edu\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fap6yc%2Ficvi-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fap6yc%2Ficvi-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fap6yc%2Ficvi-examples/lists"}