https://github.com/sea-shunned/hawks
A package for generating synthetic clusters with control over "difficulty"
https://github.com/sea-shunned/hawks
clustering evolutionary-algorithms python synthetic-data
Last synced: about 2 months ago
JSON representation
A package for generating synthetic clusters with control over "difficulty"
- Host: GitHub
- URL: https://github.com/sea-shunned/hawks
- Owner: sea-shunned
- License: mit
- Created: 2019-06-25T15:23:13.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-05-16T09:52:57.000Z (almost 4 years ago)
- Last Synced: 2025-09-21T22:52:07.276Z (5 months ago)
- Topics: clustering, evolutionary-algorithms, python, synthetic-data
- Language: Python
- Homepage: https://hawks.readthedocs.io/
- Size: 14.5 MB
- Stars: 22
- Watchers: 0
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- License: LICENSE.txt
Awesome Lists containing this project
README
HAWKS Data Generator
====================
.. image:: docs/source/images/hawks_animation.gif
:alt: Example gif of HAWKS
:scale: 65 %
:align: center
.. summary-marker-1-start
HAWKS is a tool for generating controllably difficult synthetic data,
used primarily for clustering.
.. summary-marker-1-end
This `repo `_ is associated with the following papers:
.. paper-marker-1-start
1. `Shand, C. `_, `Allmendinger, R. `_, `Handl, J. `_, `Webb, A. `_, & Keane, J. (2019, July). Evolving controllably difficult datasets for clustering. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 463-471). https://doi.org/10.1145/3321707.3321761 **(Nominated for best paper on the evolutionary machine learning track at GECCO'19)**
2. `Shand, C. `_, `Allmendinger, R. `_, `Handl, J. `_, `Webb, A. `_, & Keane, J. (2021, Dec.). HAWKS: Evolving Challenging Benchmark Sets for Cluster Analysis. Published in **IEEE Transactions on Evolutionary Computation**. https://doi.org/10.1109/TEVC.2021.3137369
Related BibTeX info:
.. code-block:: bibtex
@inproceedings{shand2019evolving,
title={Evolving controllably difficult datasets for clustering},
author={Shand, Cameron and Allmendinger, Richard and Handl, Julia and Webb, Andrew and Keane, John},
booktitle={Proceedings of the Genetic and Evolutionary Computation Conference},
pages={463--471},
year={2019},
url={https://doi.org/10.1145/3321707.3321761}
}
@article{shand2021hawks,
author={Shand, Cameron and Allmendinger, Richard and Handl, Julia and Webb, Andrew and Keane, John},
journal={IEEE Transactions on Evolutionary Computation},
title={HAWKS: Evolving Challenging Benchmark Sets for Cluster Analysis}, year={2021},
volume={},
number={},
pages={1-1},
doi={10.1109/TEVC.2021.3137369}
}
The academic/technical details can be found there. What follows here is
a practical guide to using HAWKS to generate synthetic data.
.. paper-marker-1-end
If you use HAWKS to generate data that forms part of a paper, please
cite the paper above and link to this repo.
.. installation-marker-start
Installation
------------
Installation is available through pip by:
``pip install hawks``
.. installation-marker-end
or by cloning this repo (and installing locally using
``pip install .``). HAWKS was written for Python 3.6+. Other dependencies are specified in the `setup.py `_ file.
Running HAWKS
-------------
The parameters of hawks are configured via a config file system. Details
of the parameters are found in the `documentation `_. For any parameters
that are not specified, default values will be used (as defined in
``hawks/defaults.json``).
.. example-marker-start
The example below illustrates how to run ``hawks``. Either a dictionary
or a path to a JSON config can be provided to override any of the
default values. Further examples can be found `here `_.
.. Need to turn the bit below into an example file and then just include that
.. code-block:: python
"""Single, simple HAWKS run, with KMeans applied to the best dataset
"""
import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score
import hawks
# Set the magic seed number
SEED_NUM = 42
# Set the seed number in the config
config = {
"hawks": {
"folder_name": "simple_example",
"seed_num": SEED_NUM
},
"dataset": {
"num_clusters": 5
},
"objectives": {
"silhouette": {
"target": 0.9
}
}
}
# Any missing parameters will take from hawks/defaults.json
generator = hawks.create_generator(config)
# Run the generator
generator.run()
# Let's plot the best individual found
generator.plot_best_indivs(show=True)
# Get the best dataset found and it's labels
datasets, label_sets = generator.get_best_dataset()
# Stored as a list for multiple runs
data, labels = datasets[0], label_sets[0]
# Run KMeans on the data
km = KMeans(
n_clusters=len(np.unique(labels)), random_state=SEED_NUM
).fit(data)
# Plot the output of KMeans
hawks.plotting.scatter_prediction(data, km.labels_)
# Get the Adjusted Rand Index for KMeans on the data
ari = adjusted_rand_score(labels, km.labels_)
print(f"ARI: {ari}")
.. example-marker-end
Documentation
-------------
For further information about how to use HAWKS, including examples, please see the `documentation `__.
Issues
------
As this work is still in development, plain sailing is not guaranteed.
If you encounter an issue, first ensure that ``hawks`` is running as
intended by navigating to the tests directory, and running
``python tests.py``. If any test fails, please add details of this
alongside your original problem to an issue on the `GitHub repo `__.
Contributing
------------
.. contributing-marker-start
At present, this is primarily academic work, so future developments will be released here after they have been published. If you have any suggestions or simple feature requests for HAWKS as a tool to use, please raise that on the `GitHub repo `__.
I have various directions for HAWKS at present, and can only work on a subset of them, and so involvement with more people would be great. If you would like to extend this work or collaborate, please `contact me `__.
.. contributing-marker-end