{"id":25472854,"url":"https://github.com/gchq/coreax","last_synced_at":"2025-11-05T04:30:31.345Z","repository":{"id":222827587,"uuid":"662956038","full_name":"gchq/coreax","owner":"gchq","description":"A library for coreset algorithms, written in Jax for fast execution and GPU support.","archived":false,"fork":false,"pushed_at":"2025-02-10T12:23:37.000Z","size":14380,"stargazers_count":26,"open_issues_count":67,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-10T12:33:20.905Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gchq.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-06T08:45:08.000Z","updated_at":"2025-02-10T12:23:36.000Z","dependencies_parsed_at":"2024-12-20T14:37:28.433Z","dependency_job_id":"971a51d8-ce28-419d-a28d-337e41b7dc5d","html_url":"https://github.com/gchq/coreax","commit_stats":null,"previous_names":["gchq/coreax"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2Fcoreax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2Fcoreax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2Fcoreax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2Fcoreax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gchq","download_url":"https://codeload.github.com/gchq/coreax/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239449574,"owners_count":19640534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-18T10:02:02.423Z","updated_at":"2025-11-05T04:30:31.282Z","avatar_url":"https://github.com/gchq.png","language":"Python","funding_links":[],"categories":["Uncategorized","Recently Updated","Libraries"],"sub_categories":["Uncategorized","[Feb 16, 2025](/content/2025/02/16/README.md)","New Libraries"],"readme":"\u003cdiv align=\"center\"\u003e\n    \u003cimg alt=\"Coreax logo\" src=\"https://raw.githubusercontent.com/gchq/coreax/main/documentation/assets/Logo.svg\"\u003e\n\u003c/div\u003e\n\n# Coreax\n\n[![Unit Tests](https://github.com/gchq/coreax/actions/workflows/unittests.yml/badge.svg)](https://github.com/gchq/coreax/actions/workflows/unittests.yml)\n[![Coverage](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fgchq%2Fcoreax-metadata%2Frefs%2Fheads%2Fmain%2Fcoverage%2Fcoreax_coverage.json)](https://github.com/gchq/coreax/actions/workflows/coverage.yml)\n[![Pre-commit Checks](https://github.com/gchq/coreax/actions/workflows/pre_commit_checks.yml/badge.svg)](https://github.com/gchq/coreax/actions/workflows/pre_commit_checks.yml)\n[![linting: pylint](https://img.shields.io/badge/linting-pylint-yellowgreen)](https://github.com/pylint-dev/pylint)\n[![Python version](https://img.shields.io/pypi/pyversions/coreax.svg)](https://pypi.org/project/coreax)\n[![PyPI](https://img.shields.io/pypi/v/coreax)](https://pypi.org/project/coreax)\n![Beta](https://img.shields.io/badge/pre--release-beta-red)\n\n_© Crown Copyright GCHQ_\n\nCoreax is a library for **coreset algorithms**, written in \u003ca href=\"https://jax.readthedocs.io/en/latest/notebooks/quickstart.html\" target=\"_blank\"\u003eJAX\u003c/a\u003e for fast execution and GPU support.\n\n## About Coresets\n\nFor $n$ points in $d$ dimensions, a coreset algorithm takes an $n \\times d$ data set and\nreduces it to $m \\ll n$ points whilst attempting to preserve the statistical properties\nof the full data set. The algorithm maintains the dimension of the original data set.\nThus the $m$ points, referred to as the **coreset**, are also $d$-dimensional.\n\nThe $m$ points need not be in the original data set. We refer to the special case where\nall selected points are in the original data set as a **coresubset**.\n\nSome algorithms return the $m$ points with weights, so that importance can be\nattributed to each point in the coreset. The weights, $w_i$ for $i=1,...,m$, are often\nchosen from the simplex. In this case, they are non-negative and sum to 1:\n$w_i \u003e0$ $\\forall i$ and $\\sum_{i} w_i =1$.\n\nPlease see [the documentation](https://coreax.readthedocs.io/en/latest/quickstart.html) for some in-depth examples.\n\n\n##  Example applications\n\n### Choosing pixels from an image\n\nIn the example below, we reduce the original 180x215\npixel image (38,700 pixels in total) to a coreset approximately 20% of this size.\n(Left) original image.\n(Centre) 8,000 coreset points chosen using Stein kernel herding, with point size a\nfunction of weight.\n(Right) 8,000 points chosen randomly.\nRun `examples/david_map_reduce_weighted.py` to  replicate.\n\n![](https://raw.githubusercontent.com/gchq/coreax/main/examples/data/david_coreset.png)\n\n\n### Video event detection\n\nHere we identify representative frames such that most of the\nuseful information in a video is preserved.\nRun `examples/pounce.py` to replicate.\n\n|                                 Original                                 |                                     Coreset                                      |\n|:------------------------------------------------------------------------:|:--------------------------------------------------------------------------------:|\n| ![](https://raw.githubusercontent.com/gchq/coreax/main/examples/pounce/pounce.gif) | ![](https://raw.githubusercontent.com/gchq/coreax/main/examples/pounce/pounce_coreset.gif) |\n\n\n# Setup\n\nInstall Coreax from PyPI by adding `coreax` to your project dependencies or running\n```shell\npip install coreax\n```\n\nCoreax uses JAX. It installs the CPU version by default, but if you have a GPU or TPU,\nsee the\n[JAX installation instructions](https://jax.readthedocs.io/en/latest/installation.html)\nfor options available to take advantage of the power of your system. For example, if you\nhave an NVIDIA GPU on Linux, add `jax[cuda12]` to your project dependencies or run\n```shell\npip install jax[cuda12]\n```\n\nThere are optional sets of additional dependencies:\n* `coreax[test]` is required to run the tests;\n* `coreax[example]` contains all dependencies for the example scripts;\n* `coreax[benchmark]` is required to run benchmarking;\n* `coreax[doc]` is for compiling the Sphinx documentation;\n* `coreax[dev]` includes all tools and packages a developer of Coreax might need.\n\nNote that the `test` and `dev` dependencies include `opencv-python-headless`, which is\nthe headless version of OpenCV and is incompatible with other versions of OpenCV. If you\nwish to use an alternative version, remove `opencv-python-headless` and select an\nalternative from the\n[OpenCV documentation](https://pypi.org/project/opencv-python-headless/).\n\nShould the installation of Coreax fail, you can see the versions used by the Coreax\ndevelopment team in `uv.lock`. You can transfer these to your own project as follows.\nFirst, [install UV](https://docs.astral.sh/uv/getting-started/installation/). Then,\nclone the repo from [GitHub](https://github.com/gchq/coreax). Next, run\n```shell\nuv export --format requirements-txt\n```\nwhich will generate a `requirements.txt`. Install this in your own project before trying\nto install Coreax itself,\n```shell\npip install -r requirements.txt\npip install coreax\n```\n\n# Release cycle\n\nWe anticipate two release types: feature releases and security releases. Security\nreleases will be issued as needed in accordance with the\n[security policy](https://github.com/gchq/coreax/security/policy). Feature releases will\nbe issued as appropriate, dependent on the feature pipeline and development priorities.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgchq%2Fcoreax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgchq%2Fcoreax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgchq%2Fcoreax/lists"}