{"id":18734206,"url":"https://github.com/nico-curti/plasticity","last_synced_at":"2025-04-12T18:32:10.843Z","repository":{"id":139598690,"uuid":"310383408","full_name":"Nico-Curti/plasticity","owner":"Nico-Curti","description":"Unsupervised Neural Networks with biological-inspired learning rules","archived":false,"fork":false,"pushed_at":"2022-08-03T16:42:27.000Z","size":47173,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-26T13:21:22.915Z","etag":null,"topics":["bcm","biological-neuron-model","deep-learning","machine-learning","neural-network"],"latest_commit_sha":null,"homepage":"https://plasticity.readthedocs.io/en/latest/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Nico-Curti.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-05T18:22:02.000Z","updated_at":"2023-10-11T05:59:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"4e4cb3be-6a32-4eab-a7d2-08628ba81b21","html_url":"https://github.com/Nico-Curti/plasticity","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nico-Curti%2Fplasticity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nico-Curti%2Fplasticity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nico-Curti%2Fplasticity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nico-Curti%2Fplasticity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Nico-Curti","download_url":"https://codeload.github.com/Nico-Curti/plasticity/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248613630,"owners_count":21133558,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bcm","biological-neuron-model","deep-learning","machine-learning","neural-network"],"created_at":"2024-11-07T15:12:44.645Z","updated_at":"2025-04-12T18:32:10.829Z","avatar_url":"https://github.com/Nico-Curti.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"| **Authors**  | **Project** |  **Documentation** | **Build Status** | **Code Quality** |\n|:------------:|:-----------:|:------------------:|:----------------:|:----------------:|\n| [**N. Curti**](https://github.com/Nico-Curti) \u003cbr/\u003e [**L. Squadrani**](https://github.com/lorenzosquadrani) \u003cbr/\u003e [**S. Gasperini**](https://github.com/SimoneGasperini) \u003cbr/\u003e [**M. Ceccarelli**](https://github.com/Mat092)  | **plasticity** \u003cbr/\u003e [![Entropy](https://img.shields.io/badge/Entropy-10.3390/e24050682-g.svg)](https://www.mdpi.com/1099-4300/24/5/682) | [![Doxygen Sphinx](https://github.com/Nico-Curti/plasticity/actions/workflows/docs.yml/badge.svg)](https://github.com/Nico-Curti/plasticity/actions/workflows/docs.yml) \u003cbr/\u003e [![ReadTheDocs](https://readthedocs.org/projects/plasticity/badge/?version=latest)](https://plasticity.readthedocs.io/en/latest/?badge=latest) | [![Windows](https://github.com/Nico-Curti/plasticity/actions/workflows/windows.yml/badge.svg)](https://github.com/Nico-Curti/plasticity/actions/workflows/windows.yml) \u003cbr/\u003e [![Linux](https://github.com/Nico-Curti/plasticity/actions/workflows/linux.yml/badge.svg)](https://github.com/Nico-Curti/plasticity/actions/workflows/linux.yml) \u003cbr/\u003e [![MacOS](https://github.com/Nico-Curti/plasticity/actions/workflows/macos.yml/badge.svg)](https://github.com/Nico-Curti/plasticity/actions/workflows/macos.yml) \u003cbr/\u003e [![Python](https://github.com/Nico-Curti/plasticity/actions/workflows/python.yml/badge.svg)](https://github.com/Nico-Curti/plasticity/actions/workflows/python.yml) | [![Codacy](https://app.codacy.com/project/badge/Grade/9879f0e8f90140eab79c338b46c00420)](https://www.codacy.com/gh/Nico-Curti/plasticity/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=Nico-Curti/plasticity\u0026amp;utm_campaign=Badge_Grade) \u003cbr/\u003e [![codebeat](https://codebeat.co/badges/941ebbcf-de5a-4ff0-b4c4-9674bfb20c69)](https://codebeat.co/projects/github-com-nico-curti-plasticity-main) |\n\n**Appveyor:** [![appveyor](https://ci.appveyor.com/api/projects/status/djnkyxc64dlm4r6p/branch/main?svg=true)](https://ci.appveyor.com/project/Nico-Curti/plasticity-9jr6a/branch/main)\n\n[![GitHub pull-requests](https://img.shields.io/github/issues-pr/Nico-Curti/plasticity.svg?style=plastic)](https://github.com/Nico-Curti/plasticity/pulls)\n[![GitHub issues](https://img.shields.io/github/issues/Nico-Curti/plasticity.svg?style=plastic)](https://github.com/Nico-Curti/plasticity/issues)\n\n[![GitHub stars](https://img.shields.io/github/stars/Nico-Curti/plasticity.svg?label=Stars\u0026style=social)](https://github.com/Nico-Curti/plasticity/stargazers)\n[![GitHub watchers](https://img.shields.io/github/watchers/Nico-Curti/plasticity.svg?label=Watch\u0026style=social)](https://github.com/Nico-Curti/plasticity/watchers)\n\n\u003ca href=\"https://github.com/UniboDIFABiophysics\"\u003e\n  \u003cdiv class=\"image\"\u003e\n    \u003cimg src=\"https://cdn.rawgit.com/physycom/templates/697b327d/logo_unibo.png\" width=\"90\" height=\"90\"\u003e\n  \u003c/div\u003e\n\u003c/a\u003e\n\n# Plasticity\n\n## Unsupervised Neural Networks with biological-inspired learning rules\n\nImplementation and optimization of biological-inspired Neural Network models for the features encoding.\n\n* [Overview](#overview)\n* [Theory](#theory)\n* [Prerequisites](#prerequisites)\n* [Installation](#installation)\n* [Efficiency](#efficiency)\n* [Usage](#usage)\n* [Testing](#testing)\n* [Table of contents](#table-of-contents)\n* [Contribution](#contribution)\n* [References](#references)\n* [FAQ](#faq)\n* [Authors](#authors)\n* [License](#license)\n* [Acknowledgments](#acknowledgments)\n* [Citation](#citation)\n\n## Overview\n\nDespite the great success of back-propagation algorithm in deep learning, a question remains to what extent the computational properties of artificial neural networks are comparable to the plasticity rules of the human brain.\nIndeed, even if the architectures of real and artificial neural networks are similar, the supervised training based on back-propagation and the neurobiology learning rules are unrelated.\n\nIn the paper by [D. Krotov and J. J. Hopfield](https://arxiv.org/abs/1806.10181), it is proposed an unusual learning rule, which has a degree of biological plausibility and which is motivated by different well known ideas in neuroplasticity theory:\n\n* Hebb's rule: changes of the synapse strength depend only on the activities of the pre- and post-synaptic neurons and so the learning is **physically local** and describable by local mathematics\n\n* the core of the learning procedure is **unsupervised** because it is believed to be mainly observational, with few or no labels and no explicit task\n\nStarting from these concepts, they were able to design an algorithm (based on an extension of the *Oja rule*) capable of learning early feature detectors in a completely unsupervised way and then use them to train higher-layer weights in a usual supervised neural network.\nIn particular, the Hopfield model has the structure of a 2-layers neural network which can be described by the following equations:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=h_j=f(\\sum_iw_{ij}v_i)\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=c_k=tanh(\\sum_js_{jk}h_j)\"\u003e\n\u003c/p\u003e\n\nwhere\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=f(x)=\\left\\{\\begin{matrix}x^n,x\\geq0\\\\0,x\u003c0\\end{matrix}\\right.\"\u003e\n\u003c/p\u003e\n\nis the activation function of the unsupervised layer (ReLu for n=1), v\u003csub\u003ei\u003c/sub\u003e, h\u003csub\u003ej\u003c/sub\u003e, c\u003csub\u003ek\u003c/sub\u003e are respectively the input, hidden and output neurons and w\u003csub\u003eij\u003c/sub\u003e, s\u003csub\u003ejk\u003c/sub\u003e are the receptive fields of the hidden layer (learned by the local unsupervised algorithm) and the weights learned by conventional supervised technique.\n\n## Theory\n\nIn this project, a parallel approach founded on the same concepts is proposed.\nIn particular, it has been developed a model based on the *BCM theory* (E. Bienenstock, L. Cooper, and P. Munro).\nAn exhaustive theoretical description is provided by the original paper of [Castellani et al.](https://pubmed.ncbi.nlm.nih.gov/10378187/).\nThe proposed implementation is discussed in the work of [Squadrani \u0026 Curti et al.](https://www.mdpi.com/1099-4300/24/5/682).\n\nIn general terms, BCM model proposes a sliding threshold for long-term potentiation (LTP) or long-term depression (LTD) induction, and states that synaptic plasticity is stabilized by a dynamic adaptation of the time-averaged post-synaptic activity.\nThe BCM learning rule is described by the following equations:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\frac{dw_i}{dt}=\\frac{y(y-\\theta)x_i}{\\theta}\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\theta=\\mathbb{E}[y^2]\"\u003e\n\u003c/p\u003e\n\nwhere \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\mathbb{E}[\\cdot]\"\u003e is the time-average operator and, taking the input `x`, the output `y` is computed as:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://render.githubusercontent.com/render/math?math=y=\\sigma\\left(\\sum_iw_ix_i\\right)\"\u003e\n\u003c/p\u003e\n\nIn the classical model the activation function \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\sigma\"\u003e is a given by a sigmoid.\n\nSee [here](https://github.com/Nico-Curti/plasticity/blob/main/docs/source/theory.rst) for further details about the models.\n\n## Prerequisites\n\nC++ supported compilers:\n\n![gcc version](https://img.shields.io/badge/gcc-4.8.5%20|%204.9.*%20|%205.*%20|%206.*%20|%207.*%20|%208.*%20|%209.*%20|10.*-yellow.svg)\n\n![clang version](https://img.shields.io/badge/clang-3.8+%20|%204.*%20|%205.*%20|%206.*%20|%207.*%20|%208.*%20|%209.*%20|%2010.*-red.svg)\n\n![msvc version](https://img.shields.io/badge/msvc-vs2017%20x86%20|%20vs2017%20x64|%20vs2019%20x86%20|%20vs2019%20x64-blue.svg)\n\nThe `plasticity` project is written in `C++` using simple c++14 features.\nThe package installation can be performed via [`CMake`](https://github.com/Nico-Curti/plasticity/blob/main/CMakeLists.txt).\n\nThe only requirement for the installation of the `C++` library is the `Eigen3` library.\nYou can easily install the `Eigen3` library with the following commands:\n\n| **OS**       |  **Command**                      |\n|:------------:|:----------------------------------|\n| **Linux**    | `sudo apt install libeigen3-dev`  |\n| **MacOS**    | `brew install eigen`              |\n| **Windows**  | `vcpkg install eigen3`            |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| For Windows users we suggest to use [`vcpkg`](https://github.com/microsoft/vcpkg) for the library installation/management. |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| You can easily `Eigen3` install the library from source at this [link](https://gitlab.com/libeigen/eigen.git) to get the latest (more efficient) release. |\n\nIf you want visualize the model weights you have to build the library with the `-DVIEW:BOOL=ON`: in this case the `OpenCV` support is required for the installation.\n\nThe `CMake` installer provides also a `plasticity.pc`, useful if you want link to the `plasticity` using `pkg-config`.\n\nYou can also use the `plasticity` package in `Python` using the `Cython` wrap provided inside this project.\nThe only requirements are the following:\n\n* numpy \u003e= 1.15\n* cython \u003e= 0.29\n* scikit-learn \u003e= 0.20.3\n* tqdm\n* matplotlib\n\nThe `Cython` version can be built and installed via `CMake` enabling the `-DPYWRAP` variable.\nThe `Python` wrap guarantees also a good integration with the other common Machine Learning tools provided by `scikit-learn` `Python` package; in this way you can use the `plasticity` algorithm as an equivalent alternative also in other pipelines.\nLike other Machine Learning algorithm also the `plasticity` one depends on many parameters, i.e its hyper-parameters, which has to be tuned according to the given problem.\nThe `Python` wrap of the library was written according to `scikit-optimize` `Python` package to allow an easy hyper-parameters optimization using the already implemented classical methods.\n\n## Installation\n\nA complete list of instructions \"for beginners\" is also provided for both [c++](https://github.com/Nico-Curti/plasticity/blob/main/docs/source/CMake.rst) and [python](https://github.com/Nico-Curti/plasticity/blob/main/docs/source/Python.rst) versions.\n\n### CMake C++ installation\n\nWe recommend to use `CMake` for the installation since it is the most automated way to reach your needs.\nFirst of all make sure you have a sufficient version of `CMake` installed (3.9 minimum version required).\nIf you are working on a machine without root privileges and you need to upgrade your `CMake` version a valid solution to overcome your problems is provided [here](https://github.com/Nico-Curti/Shut).\n\nWith a valid `CMake` version installed first of all clone the project as:\n\n```bash\ngit clone https://github.com/Nico-Curti/plasticity\ncd plasticity\n```\n\nThe you can build the `plasticity` package with\n\n```bash\nmkdir -p build\ncd build \u0026\u0026 cmake .. \u0026\u0026 cmake --build . --target install\n```\n\n`plasticity` could be built with the *build* scripts in the project, as:\n\n|              |  **Linux**    |  **MacOS**    |  **Windows**  |\n|:------------:|:--------------|:--------------|:--------------|\n| **Script**   | `./build.sh`  | `./build.sh`  | `./build.ps1` |\n\nThe `CMake` command line can be customized according to the following parameters:\n\n* `-DOMP:BOOL` : Enable/Disable the OpenMP support for multi-threading computation\n* `-DBUILD_DOCS:BOOL` : Enable/Disable the build of docs using Doxygen and Sphinx\n* `-DPYWRAP:BOOL` : Enable/Disable the build of Python wrap of the library via Cython (see next section for Python requirements)\n* `-DBUILD_TEST:BOOL` : Enable/Disable the build of C++ testing scripts\n* `-DVERBOSE:BOOL` : Enable/Disable the progress bar during training epochs\n* `-DVIEW:BOOL` : Enable/Disable the visualization of neurons' weights during training\n\n| :warning: WARNING |\n|:------------------|\n| The `-DVIEW:BOOL` option is available **only** with the support of OpenCV library! Pay attention to install it before the building of the library! |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| The only requirement of the library is `Eigen3`. Please pay attention to install this dependency before running the CMake installation to avoid any issue. |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| We support all the versions of the `Eigen3` library but we strongly recommend a version \u003e= 3.3.90. |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| If you want enable the `Cython` support compile the library with `-DPYWRAP=ON`. The `Cython` packages will be compiled and correctly positioned in the `plasticity` Python package **BUT** you need to run also the setup before use it. An alternative is to install the `Python` package directly with the setup script: in this way the `CMake` is called inside the package building and all the dependencies automatically checked. |\n\n### Python installation\n\nPython version supported : ![Python version](https://img.shields.io/badge/python-3.5|3.6|3.7|3.8|3.9-blue.svg)\n\nThe `Python` installation can be performed with or without the `C++` installation.\nThe `Python` installation is always executed using [`setup.py`](https://github.com/Nico-Curti/DNetPRO/blob/master/setup.py) script.\n\nIf you have already build the `plasticity` `C++` library the installation is performed faster and the `Cython` wrap directly links to the last library installed.\nOtherwise the full list of dependencies is build.\n\nIn both cases the installation steps are:\n\n```mermaid\ngraph LR;\n    A(Install\u003cbr\u003eRequirements) --\u003e|python -m pip install -r requirements.txt| B(Install\u003cbr\u003eplasticity)\n    B --\u003e|python setup.py install| C(Package\u003cbr\u003eInstall)\n    B --\u003e|python setup.py develop --user| D(Development\u003cbr\u003eMode)\n```\n\n| :warning: WARNING |\n|:------------------|\n| The installation of the `Python` modules requires the `CMake` support and all the listed above libraries.\u003cbr\u003eIf you are working under *Window OS* we require the usage of `VCPKG` for the installation of the libraries and a precise configuration of the environment variables.\u003cbr\u003eIn particular you need to set the variables `VCPKG_ROOT=/path/to/vcpkg/rootdir/` and `VCPKG_DEFAULT_TRIPLET=x64-windows`.\u003cbr\u003eA full working example of OS configuration can be found in the CI actions of the project, available [here](https://github.com/Nico-Curti/plasticity/blob/main/.github/workflows/) |\n\n| :warning: WARNING |\n|:------------------|\n| All the `CMake` flags are set internally in the `setup.py` script with default values.\u003cbr\u003eYou can manually turn on/off the multi-threading support passing the flag `--omp` at the setup command line, *i.e.* `python setup.py develop --user --omp` |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| The requirement of the `Eigen3` library is mandatory also for the `Cython` installation! Make sure to have installed all the requirements before running the `setup.py` command. |\n\n## Efficiency\n\n| BCM | Hopfield |\n| --- | -------- |\n| ![BCM](./img/BCM_timing.png) | ![Hopfield](./img/Hopfield_timing.png) |\n| Comparison of time performances between the pure-`Python` implementation and `Cython` version of the `BCM` model varying the input dimension sizes (number of samples and number of features). For each input configuration 10 runs of both algorithm were performed keeping fixed all the other parameters. In the simulation we used 10 epochs and a SGD optimization algorithm. | Comparison of time performances between the pure-`Python` implementation and `Cython` version of the `Hopfield` model varying the input dimension sizes (number of samples and number of features). For each input configuration 10 runs of both algorithm were performed keeping fixed all the other parameters. In the simulation we used 10 epochs and a SGD optimization algorithm. |\n\nWe test the computational efficiency of both the implementation (pure-`Python` and `Cython` with multi-threading enabled).\nThe tests were performed keeping fixed all the training parameters and varying just the input dimension (number of samples and number of features).\n\nAs expected we have the most significant improvements enlarging the number of samples, while varying the number of features the scalability of the code is quite stable.\nBoth the algorithms spend the same time for the simulations and therefore their use is not constrained by any computational limitation.\nWe however encourage the use of the `Cython` version since it is obviously faster than the `Python` counterpart.\n\n## Usage\n\nYou can use the `plasticity` library into pure-Python modules or inside your C++ application.\n\n### C++ Version\n\nThe easiest usage of `plasticity` library is given by the two examples provided in the [example](https://github.com/Nico-Curti/plasticity/blob/main/example) folder.\nLets see in detail how you can use the models.\n\n**Load the MNIST dataset** (ref. [run_mnist](https://github.com/Nico-Curti/plasticity/blob/main/example/run_mnist.cpp))\n\nFirst of all you need to load your dataset.\nIn all the [example](https://github.com/Nico-Curti/plasticity/blob/main/example) scripts we use the MNIST digit dataset as toy model.\nThe `plasticity` library provides a simple class object for the MNIST dataset loading called `data_loader :: MNIST`: this object class allows to load both training and testing images/labels binary files related to the MNIST dataset.\nThe core implementation was inspired to the [`mnist`](https://github.com/wichtounet/mnist) package: in the original project folder you can also find the required binary files for the MNIST dataset.\n\nThe [`run_mnist`](https://github.com/Nico-Curti/plasticity/blob/main/example/run_mnist.cpp) allows also the visualization of a subset of images sampled by the MNIST dataset using the OpenCV support.\nYou can enable the OpenCV support building the library with the `-DVIEW:BOOL=ON` define.\n\nThe most important thing to take in mind is that all the `plasticty` models work with a 1D buffer of floating-point data as input.\nIn the MNIST dataset case this buffer of data is already exposed by the `data_loader :: MNIST` class as the \"ravel\" buffer of image pixels.\n\n| :warning: WARNING |\n|:------------------|\n| For sake of clarity the `data_loader :: MNIST` class exposes a `uint8_t` buffer of data which must be converted into a floating-point buffer. |\n\n**Load the CIFAR-10 dataset** (ref. [`run_cifar10`](https://github.com/Nico-Curti/plasticity/blob/main/example/run_cifar10.cpp))\n\nThe same procedure proposed for the MNIST dataset can be applied also for the CIFAR-10 dataset.\nA specialization of `data_loader :: BaseData` class (mother class also of the `MNIST` object) provides an equivalent interface for the management of the CIFAR-10 dataset.\nThe `data_loader :: CIFAR10` class allows to load both training and testing images/labels binary files related to the CIFAR-10 dataset.\nThe core implemenation was inspired to the [`cifar-10`](https://github.com/wichtounet/cifar-10/) package.\nThe set of APIs and member functions/variables are the same of the `MNIST` class.\n\n| :warning: WARNING |\n|:------------------|\n| The train/test data include in the same file both labels and images. The required format of the binary file is the same of the original implementation of the CIFAR-10 dataset (available [here](https://www.cs.toronto.edu/~kriz/cifar.html)). In the current implementation we have just concatenate together the full set of batch-files! |\n\n**Train the model** (ref. [run_bcm_mnist](https://github.com/Nico-Curti/plasticity/blob/main/example/run_bcm_mnist.cpp))\n\nThe [`run_bcm_mnist`](https://github.com/Nico-Curti/plasticity/blob/main/example/run_bcm_mnist.cpp) and [`run_hopfield_mnist`](https://github.com/Nico-Curti/plasticity/blob/main/example/run_hopfield_mnist.cpp) scripts show two possible usage-examples of the BCM and Hopfield models, respectively.\nThe model simulations can be performed using a simple configuration file as the following one:\n\n```\n# Dataset parameters\n\nMNIST_training_image = /path/to/train-images-idx3-ubyte\nMNIST_training_label = /path/to/train-labels-idx1-ubyte\nMNIST_testing_image = /path/to/t10k-images-idx3-ubyte\nMNIST_testing_label = /path/to/t10k-labels-idx1-ubyte\nnormalize = 1\nbinarize = 0\n\n# Model parameters\n\noutputs = 100\nbatch_size = 1000\nepochs_for_convergency = 100000\nconvergency_atol = 1e10\ninteraction_strength = 0.0\nseed = 42\nepochs = 20\nweights_decay = 0.0\n\n# BCM\nmemory_factor = 0.9\n\n# Hopfield\np = 4\ndelta = 0.4\nk = 2\n\n# Activation function\n\nactivation = relu\n\n# Optimizer parameters\n\noptimizer = sgd\nlearning_rate = 2e-2\nmomentum = 0.9\ndecay = 0\nB1 = 0.9\nB2 = 0.999\nrho = 0.0\n\n# Weights parameters\n\nweights = normal\nmean = 0.0\nstd = 1.0\nscale = 1.0\nweights_seed = 42\n```\n\nThe configuration file includes all the required parameters for the model training and the instructions related to the MNIST dataset.\nThe usage of the configuration file is not mandatory for the model usage but strongly recommended for the reproducibility.\n\nThe model initialization is performed for all the `plasticity` models into the constructors.\nIn particular, the BCM model is defined as\n\n```c++\nBCM (const int \u0026 outputs, const int \u0026 batch_size, int activation=transfer_t :: logistic,\n     update_args optimizer=update_args(optimizer_t :: sgd),\n     weights_initialization weights_init=weights_initialization(weights_init_t :: normal),\n     int epochs_for_convergency=1, float convergency_atol=1e-2f,\n     float decay=0.f, float memory_factor=0.5f,\n     float interaction_strength=0.f)\n```\n\nwhile the Hopfield model is defined as\n\n```c++\nHopfield (const int \u0026 outputs, const int \u0026 batch_size,\n          update_args optimizer=update_args(optimizer_t :: sgd),\n          weights_initialization weights_init=weights_initialization(weights_init_t :: normal),\n          int epochs_for_convergency=1, float convergency_atol=1e-2f,\n          float decay=0.f,\n          float delta=.4f, float p=2.f,\n          int k=2)\n```\n\nref. to the [documentation](https://plasticity.readthedocs.io/en/latest/cppAPI/modules.html) for a deeper explanation of the model parameters.\n\nAfter the model initialization the core of the simulation is performed inside the `fit` member-function.\nThe function signature is the following:\n\n```c++\ntemplate \u003c class Callback = std :: function \u003c void (BasePlasticity *) \u003e \u003e\nvoid fit (float * X, const int \u0026 n_samples, const int \u0026 n_features, const int \u0026 num_epochs, int seed=42, Callback callback=[](BasePlasticity *){});\n```\n\nwhere `X` is the buffer of data (in ravel format) and (`n_samples`, `n_features`) is the shape of the `X` matrix.\nThe last variable of the function is a (possible) `callback` function which takes the model object as input.\nThis function will be called as each batch subdivision and for each epoch of the training.\nIn the example we use as `callback` function a lambda-function for the visualization of the neurons weight matrix.\n\nIn summary, a minimal working example of a simulation on the MNIST dataset can be written as\n\n```c++\n#include \u003cmnist.h\u003e\n#include \u003cbcm.h\u003e\n\nint main (int argc, char ** argv)\n{\n  data_loader :: MNIST dataset;\n  dataset.load_training_images(training_file);\n\n  std :: unique_ptr \u003c float [] \u003e training(new float[dataset.train_size()]);\n\n  for (int i = 0; i \u003c dataset.train_size(); ++i)\n    training[i] = static_cast \u003c float \u003e(dataset.training_images[i]) / 255.f;\n\n  BCM bcm (100, 100, transfer_t :: relu, update_args(optimizer_t :: adam),\n           weights_initialization(weights_init_t :: he_normal));\n\n  bcm.fit(training.get(), dataset.num_train_sample, dataset.rows * dataset.cols, 10);\n  bcm.save_weights(\"BCM_MNIST_simulation.bin\")\n\n  return 0;\n}\n```\n\n### Python Version\n\nThe `plasticity` classes are totally equivalent to a `scikit-learn` feature-encoder object and thus they provide the member functions `fit` (to train your model) and `predict` (to test a trained model on new samples).\nFirst of all you need to import the desired `plasticity` model.\nThe you can call the `fit` member function with the desired parameters.\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| Following the scikit-learn philosophy all the class parameters (a lot of possible parameters!) are initialized by default values. In this way you can build the object without any preliminary knowledge about the model. However, we strongly recommend to read the full list of possible variables using the `help` command. |\n\n```python\nfrom plasticity.model import BCM\nfrom plasticity.model.optimizer import Adam\nfrom plasticity.model.weights import HeNormal\n\nfrom sklearn.datasets import fetch_openml\n\n# Download the MNIST dataset\nX, y = fetch_openml(name='mnist_784', version=1, data_id=None, return_X_y=True)\n\n# normalize the sample into [0, 1]\nX *= 1. / 255\n\nmodel = BCM(outputs=100, num_epochs=10, batch_size=100, interaction_strength=0.,\n            optimizer=Adam(lr=1e-3), activation='relu', weights_init=HeNormal())\nmodel.fit(X)\n```\n\nNow you have trained the model on the MNIST digit dataset and thus the internal neurons have reached a precise configuration state.\nYou can easily visualize the neuron-weights matrix using the (utility) function `view_weights` provided by the `utils` submodule.\n\n```python\n\nfrom plasticity.utils import view_weights\n\nview_weights (model.weights, dims=(28, 28))\n```\n\nThe results should appear like this\n\n| \u003cimg src=\"img/BCM_weights.gif\" width=\"392\" height=\"392\"\u003e |\n| -- |\n| Weights learned by 100 arbitrarily chosen BCM neurons after convergence on MNIST dataset. The bitmap shows the progression of neurons synapses starting from a random configuration (initial condition) along 10 training epochs. |\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| The above image was generated by a series of simulations! The `view_weights` function just plots the final version of the weights matrix. You can obtained an animation plot of the neuron convergency using the C++ version of the code with an appropriated callback function (ref. [here](https://github.com/Nico-Curti/plasticity/blob/main/example/run_bcm.cpp) for an example code) or with a little hack of the library code. We intentionally do not provide a callback support in the Python version of the model since its user interface must be as much as possible equivalent to a scikit-learn object. |\n\n## Testing\n\n`plasticity` uses CMake to build a full list of tests.\nYou can enable/disable tests setting the `-DBUILD_TEST:BOOL=ON` during the building.\nAll the test are performed using the [`Catch2`](https://github.com/catchorg/Catch2/) (v2.11.0) library.\n\nThe test scripts can be found [here](https://github.com/Nico-Curti/plasticity/blob/main/testing).\nYou can easily run the full list of C++ tests using the scripts [run_test.sh](https://github.com/Nico-Curti/plasticity/blob/main/testing/run_test.sh) and [run_test.ps1](https://github.com/Nico-Curti/plasticity/blob/main/testing/run_test.ps1) if you are working on a Unix-like or Windows machine, rispectively.\n\n## Table of contents\n\nDescription of the folders related to the `C++` version.\n\n| **Directory**                                                           |  **Description**                                                                          |\n|:-----------------------------------------------------------------------:|:------------------------------------------------------------------------------------------|\n| [hpp](https://github.com/Nico-Curti/plasticity/blob/main/hpp)           | Implementation of the C++ template functions and objects used in the `plasticity` library |\n| [include](https://github.com/Nico-Curti/plasticity/blob/main/include)   | Definition of the C++ function and objects used in the `plasticity` library               |\n| [src](https://github.com/Nico-Curti/plasticity/blob/main/src)           | Implementation of the C++ functions and objects used in the `plasticity` library          |\n\nDescription of the folders related to the `Python` version.\n\n| **Directory**                                                                      |  **Description**                                                               |\n|:----------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------|\n| [example](https://github.com/Nico-Curti/plasticity/blob/main/plasticity/example)   | `Jupyter` notebook with some examples on the MNIST (digit) dataset.            |\n| [lib](https://github.com/Nico-Curti/plasticity/blob/main/plasticity/lib)           | List of `Cython` definition files                                              |\n| [source](https://github.com/Nico-Curti/plasticity/blob/main/plasticity/source)     | List of `Cython` implementation objects                                        |\n| [model](https://github.com/Nico-Curti/plasticity/blob/main/plasticity/model)       | pure-`Python` implementation of the classes                                    |\n| [cython](https://github.com/Nico-Curti/plasticity/blob/main/plasticity/cython)     | `Cython`-wraps of the classes                                                  |\n\n## Contribution\n\nAny contribution is more than welcome :heart:. Just fill an [issue](https://github.com/Nico-Curti/plasticity/blob/main/.github/ISSUE_TEMPLATE/ISSUE_TEMPLATE.md) or a [pull request](https://github.com/Nico-Curti/plasticity/blob/main/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md) and we will check ASAP!\n\nSee [here](https://github.com/Nico-Curti/plasticity/blob/main/.github/CONTRIBUTING.md) for further informations about how to contribute with this project.\n\n## References\n\n\u003cblockquote\u003e1- Squadrani L, Curti N, Giampieri E, Remondini D, Blais B, Castellani G. Effectiveness of Biologically Inspired Neural Network Models in Learning and Patterns Memorization. Entropy. 2022; 24(5):682. https://doi.org/10.3390/e24050682 \u003c/blockquote\u003e\n\n\u003cblockquote\u003e2- Castellani G., Intrator N., Shouval H.Z., Cooper L.N. Solutions of the BCM learning rule in a network of lateral interacting nonlinear neurons, Network Computation in Neural Systems, https://doi.org/10.1088/0954-898X/10/2/001. \u003c/blockquote\u003e\n\n\u003cblockquote\u003e3- Blais B., Shouval H., Cooper L.N. Time Dependence of Visual Deprivation: A Comparison between Models of Plasticity and Experimental Results, Psychology, 1996, https://doi.org/10.21236/ada316967. \u003c/blockquote\u003e\n\n\u003cblockquote\u003e4- Blais B. DEMONSTRATION Plasticity: A Synaptic Modification Simulation Environment, 1986, https://github.com/bblais/plasticity \u003c/blockquote\u003e\n\n\u003cblockquote\u003e5- Yeung, Luk Chong and Blais, B.S. and Cooper, L.N and Shouval, Harel, Metaplasticity and the Unified Calcium Model Lead to Input Selectivity in Spiking Neurons (February 2003). Science Direct Working Paper No S1574-034X(04)70246-X, Available at SSRN: https://ssrn.com/abstract=2978356 \u003c/blockquote\u003e\n\n\u003cblockquote\u003e6- Blais B., Cooper L.N. BCM theory (January 2008), https://doi.org/10.4249/scholarpedia.1570 \u003c/blockquote\u003e\n\n\u003cblockquote\u003e7- Dmitry Krotov, and John J. Hopfield. Unsupervised learning by competing hidden units, PNAS, 2019, www.pnas.org/cgi/doi/10.1073/pnas.1820458116. \u003c/blockquote\u003e\n\n## FAQ\n\n* **How can I properly set the C++ compiler for the Python installation?**\n\nIf you are working on a Ubuntu machine pay attention to properly set the environment variables related to the `C++` compiler.\nFirst of all take care to put the compiler executable into your environmental path:\n\n```bash\nls -ltA /usr/bin | grep g++\n```\n\nThen you can simply use the command to properly set the right aliases/variables\n\n```bash\nexport CXX=/usr/bin/g++\nexport CC=/usr/bin/gcc\n```\n\nbut I suggest you to put those lines into your `.bashrc` file (one for all):\n\n```bash\necho \"export CC=/usr/bin/gcc\" \u003e\u003e ~/.bashrc\necho \"export CXX=/usr/bin/g++\" \u003e\u003e ~/.bashrc\n```\n\nI suggest you to not use the default `Python` compiler (aka `x86_64-linux-gnu-g++`) since it can suffer of many issues during the compilation if it is not manually customized.\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| If you are working under Windows OS a complete guide on how to properly configure your MSVC compiler can be found [here](https://github.com/physycom/sysconfig). |\n\n* **I installed the `plasticity` Python package following the instructions but I have an `ImportError` when I try to import the package as in the examples**\n\nThis error is due a missing environment variable (which is not automatically set by the installation script).\nAll the `C++` libraries are searched into the OS directory tree starting from the information/paths hinted by the `LD_LIBRARY_PATH` environment variable.\nWhen you install the `plasticity` library the produced `.so`, `.dll`, `.dylib` files are saved into the `lib` directory created into the project root directory.\nAfter the installation you must add this directory into the searching path.\nYou can add this information editing the configuration file of your `Unix`-like system, i.e\n\n```bash\necho \"export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/plasticity/project/directory/lib/\" \u003e\u003e ~/.bashrc\necho \"export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/path/to/plasticity/project/directory/lib/\" \u003e\u003e ~/.bashrc\n```\n\nor adding the `LD_LIBRARY_PATH` to your set of environment variables (especially for `Windows` users).\n\n* **Where can I find the binary files related to the MNIST dataset?**\n\nYou can get the binary files of the MNIST dataset at this [link](https://github.com/wichtounet/mnist).\nThe object class implemented in the `plasticity` library was inspired to the (original) implementation provided by the author of the above repository.\nYou can just download (or clone) the repository and move the file wherever you want, paying attention to use the correct path to the example scripts.\n\n* **Where can I find the binary files related to the CIFAR-10 dataset?**\n\nYou can get the binary files of the CIFAR-10 dataset at this [link](https://www.cs.toronto.edu/~kriz/cifar.html).\nThe object class implemented in the `plasticity` library was inspired to the (original) implementation provided [here](https://github.com/wichtounet/cifar-10/).\nFor a more user-friendly interface we consider a processed version of the original set of files for our simulation.\nThe original data are split into 5 files related to 5 different batches.\nFurthermore, the stored images are in CHW format (ref. [here](https://www.cs.toronto.edu/~kriz/cifar.html) for the file format description).\n\nTo obtain a version of the data compatible with the `plasticity` library you need to concatenate the full list of files (`data_batch_*.bin`) and transpose the images into HWC format (OpenCV compatible).\nYou can manually perform this transformation or just use the following snippet:\n\n```python\nimport numpy as np\nimport struct\n\nfilenames = ['path/to/data_batch_{:d}.bin'.format(i) for i in range(6)]\n\nlabels = []\nimages = []\nw, h, c = (32, 32, 3)\n\nfor file in filenames:\n  with open(file, 'rb') as fp:\n    buffer = fp.read()\n    data_batch = struct.unpack('B'*(c*h*w*10000 + 10000), buffer)\n    data_batch = np.asarray(data_batch)\n\n    label = data_batch[0::c*h*w + 1]\n\n    image = np.delete(data_batch, np.arange(0, data_batch.size, w*h*c + 1))\n    image = image.reshape(10000, c, h, w).transpose(0, 2, 3, 1)\n\n  images.append(image)\n  labels.append(label)\n\nimages = np.concatenate(images).astype('uint8')\nlabels = np.concatenate(labels).astype('uint8')\n\nwith open('/path/to/whole_train_cifar10.bin', 'wb') as fp:\n  for lbl, img in zip(labels, images):\n    fp.write(struct.pack('B', lbl))\n    fp.write(struct.pack('B'*h*w*c, *img.ravel()))\n```\n\n* **How can I install the library via `VCPKG` dependency manager?**\n\nThe `plasticity` library is not yet supported via `vcpkg` (I have not submitted any PR yet).\nHowever, in the [`cmake`](https://github.com/Nico-Curti/plasticity/blob/master/cmake) folder you can find a complete directory-tree named `vcpkg`.\nYou can simply copy\u0026paste the entire `vcpkg` folder over the original (cloned [here](https://github.com/microsoft/vcpkg)) project to manage the entire installation of the library *also* via vcpkg.\n\n| :triangular_flag_on_post: Note |\n|:-------------------------------|\n| Since no releases have been published yet, the [`portfile`](https://github.com/Nico-Curti/plasticity/blob/master/cmake/ports/plasticity/portfile.cmake) is not complete and you need to manually set the `REF` and `SHA512` variables! |\n\n## Authors\n\n* \u003cimg src=\"https://avatars0.githubusercontent.com/u/24650975?s=400\u0026v=4\" width=\"25px\"\u003e **Nico Curti** [git](https://github.com/Nico-Curti), [unibo](https://www.unibo.it/sitoweb/nico.curti2)\n\n* \u003cimg src=\"https://avatars.githubusercontent.com/u/55916809?v=4\" width=\"25px\"\u003e **Lorenzo Squadrani** [git](https://github.com/lorenzosquadrani)\n\n* \u003cimg src=\"https://avatars2.githubusercontent.com/u/71086758?s=400\u0026v=4\" width=\"25px;\"/\u003e **Simone Gasperini** [git](https://github.com/SimoneGasperini)\n\n* \u003cimg src=\"https://avatars0.githubusercontent.com/u/41483077?s=400\u0026v=4\" width=\"25px;\"/\u003e **Mattia Ceccarelli** [git](https://github.com/Mat092), [unibo](https://www.unibo.it/sitoweb/mattia.ceccarelli5/)\n\nSee also the list of [contributors](https://github.com/Nico-Curti/plasticity/contributors) [![GitHub contributors](https://img.shields.io/github/contributors/Nico-Curti/plasticity.svg?style=plastic)](https://github.com/Nico-Curti/plasticity/graphs/contributors/) who participated in this project.\n\n## License\n\nThe `plasticity` package is licensed under the MIT \"Expat\" License. [![License](https://img.shields.io/github/license/mashape/apistatus.svg)](https://github.com/Nico-Curti/plasticity/blob/main/LICENSE)\n\n## Acknowledgments\n\nThanks goes to all contributors of this project.\n\nWe thank also the author(s) of [Catch2](https://github.com/catchorg/Catch2) library: we have used it in the testing procedure of our C++ version and it is amazing!\n\n## Citation\n\nIf you have found `plasticity` helpful in your research, please consider citing the original paper\n\n\n```BibTex\n@article{10.3390/e24050682,\n  author = {Squadrani, Lorenzo and Curti, Nico and Giampieri, Enrico and Remondini, Daniel and Blais, Brian and Castellani, Gastone},\n  title = {Effectiveness of Biologically Inspired Neural Network Models in Learning and Patterns Memorization},\n  journal = {Entropy},\n  volume = {24},\n  year = {2022},\n  number = {5},\n  article-NUMBER = {682},\n  url = {https://www.mdpi.com/1099-4300/24/5/682},\n  pubmedid = {35626566},\n  issn = {1099-4300},\n  doi = {10.3390/e24050682}\n}\n```\n\nor just this repository\n\n```BibTeX\n@misc{plasticity,\n  author = {Curti, Nico and Squadrani, Lorenzo and Gasperini, Simone and Ceccarelli, Mattia},\n  title = {plasticity - Unsupervised Neural Networks with biological-inspired learning rules},\n  year = {2020},\n  publisher = {GitHub},\n  howpublished = {\\url{https://github.com/Nico-Curti/plasticity}}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnico-curti%2Fplasticity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnico-curti%2Fplasticity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnico-curti%2Fplasticity/lists"}