{"id":21186530,"url":"https://github.com/thieu1995/evorbf","last_synced_at":"2025-07-10T01:31:16.968Z","repository":{"id":208546791,"uuid":"676160338","full_name":"thieu1995/EvoRBF","owner":"thieu1995","description":"EvoRBF: A Nature-inspired Algorithmic Framework for Evolving Radial Basis Function Networks","archived":false,"fork":false,"pushed_at":"2025-06-05T08:38:11.000Z","size":4204,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-05T08:39:08.482Z","etag":null,"topics":["classification","evolutionary-algorithms","genetic-algorithm","global-search","gradient-free-optimization","gradient-free-rbf","local-search","machine-learning","mealpy","metaheuristic-algorithms","nature-inspired-algorithms","neural-network","particle-swarm-optimization","radial-basis-function","radial-basis-function-network","regression","swarm-intelligence","whale-optimization-algorithm"],"latest_commit_sha":null,"homepage":"https://evorbf.readthedocs.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thieu1995.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-08T15:05:23.000Z","updated_at":"2025-06-05T08:38:12.000Z","dependencies_parsed_at":"2024-04-26T09:28:45.853Z","dependency_job_id":"ed00d294-6cdf-4a38-9728-cb96da446a2c","html_url":"https://github.com/thieu1995/EvoRBF","commit_stats":null,"previous_names":["thieu1995/evorbf"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/thieu1995/EvoRBF","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thieu1995%2FEvoRBF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thieu1995%2FEvoRBF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thieu1995%2FEvoRBF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thieu1995%2FEvoRBF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thieu1995","download_url":"https://codeload.github.com/thieu1995/EvoRBF/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thieu1995%2FEvoRBF/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263778551,"owners_count":23510014,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","evolutionary-algorithms","genetic-algorithm","global-search","gradient-free-optimization","gradient-free-rbf","local-search","machine-learning","mealpy","metaheuristic-algorithms","nature-inspired-algorithms","neural-network","particle-swarm-optimization","radial-basis-function","radial-basis-function-network","regression","swarm-intelligence","whale-optimization-algorithm"],"created_at":"2024-11-20T18:24:15.995Z","updated_at":"2025-07-10T01:31:16.963Z","avatar_url":"https://github.com/thieu1995.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003cp align=\"center\"\u003e\n\u003cimg style=\"max-width:100%;\" src=\"https://thieu1995.github.io/post/2023-08/evorbf1.png\" alt=\"EvoRBF\"/\u003e\n\u003c/p\u003e\n\n---\n\n[![GitHub release](https://img.shields.io/badge/release-2.1.0-yellow.svg)](https://github.com/thieu1995/evorbf/releases)\n[![Wheel](https://img.shields.io/pypi/wheel/gensim.svg)](https://pypi.python.org/pypi/evorbf) \n[![PyPI version](https://badge.fury.io/py/evorbf.svg)](https://badge.fury.io/py/evorbf)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/evorbf.svg)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/evorbf.svg)\n[![Downloads](https://static.pepy.tech/badge/evorbf)](https://pepy.tech/project/evorbf)\n[![Tests \u0026 Publishes to PyPI](https://github.com/thieu1995/evorbf/actions/workflows/publish-package.yml/badge.svg)](https://github.com/thieu1995/evorbf/actions/workflows/publish-package.yaml)\n[![Documentation Status](https://readthedocs.org/projects/evorbf/badge/?version=latest)](https://evorbf.readthedocs.io/en/latest/?badge=latest)\n[![Chat](https://img.shields.io/badge/Chat-on%20Telegram-blue)](https://t.me/+fRVCJGuGJg1mNDg1)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.11136007.svg)](https://doi.org/10.5281/zenodo.11136007)\n[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n\n## 🌟 Introduction\n\n**EvoRBF** is a powerful Python library for training Radial Basis Function (RBF) networks using nature-inspired algorithms (NIAs). \nIt provides a wide range of RBF models from traditional to advanced, and supports hyperparameter tuning using optimizers \nlike Whale Optimization Algorithm (WOA), Genetic Algorithm (GA), and more.\n\n---\n\n## 🚀 Key Features\n\n| **EvoRBF**                           | **Evolving Radial Basis Function Network**             |\n|--------------------------------------|--------------------------------------------------------|\n| **Free software**                    | GNU General Public License (GPL) V3 license            |\n| **Traditional RBF models**           | `RbfRegressor`, `RbfClassifier`                        |\n| **Advanced RBF models**              | `AdvancedRbfRegressor`, `AdvancedRbfClassifier`        | \n| **Nature-inspired RBF models**       | `NiaRbfRegressor`, `NiaRbfClassifier`                  |\n| **Tuner for traditional RBF models** | `NiaRbfTuner`                                          | \n| **Provided total ML models**         | \\\u003e 400 Models                                          |\n| **Supported total metrics**          | \\\u003e= 67 (47 regressions and 20 classifications)         |\n| **Supported loss functions**         | \\\u003e= 61 (45 regressions and 16 classifications)         |\n| **Documentation**                    | https://evorbf.readthedocs.io                          | \n| **Python versions**                  | \\\u003e= 3.8.x                                              |  \n| **Dependencies**                     | numpy, scipy, scikit-learn, pandas, mealpy, permetrics |\n\n\n## 🧾 Citation\n\nPlease include these citations if you plan to use this library:\n\n```bibtex\n@software{thieu_2024_11136008,\n  author       = {Nguyen Van Thieu},\n  title        = {EvoRBF: A Nature-inspired Algorithmic Framework for Evolving Radial Basis Function Networks},\n  month        = June,\n  year         = 2025,\n  publisher    = {Zenodo},\n  doi          = {10.5281/zenodo.11136007},\n  url          = {https://doi.org/10.5281/zenodo.11136007}\n}\n\n@article{van2023mealpy,\n  title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},\n  author={Van Thieu, Nguyen and Mirjalili, Seyedali},\n  journal={Journal of Systems Architecture},\n  year={2023},\n  publisher={Elsevier},\n  doi={10.1016/j.sysarc.2023.102871}\n}\n```\n\n## 📦 Installation\n\nInstall the latest version from PyPI:\n\n```bash\npip install evorbf\n```\n\nVerify installation:\n\n```bash\n$ python\n\u003e\u003e\u003e import evorbf\n\u003e\u003e\u003e evorbf.__version__\n```\n\n\nWe have provided above several ways to import and call the proposed classes. If you need more details how to \nuse each of them, please check out the folder [examples](/examples). In this short demonstration, we will use \nWhale Optimization Algorithm to optimize the `sigmas` (in non-linear Gaussian kernel) and `reg_lambda` of \nL2 regularization in RBF network (WOA-RBF model) for Diabetes prediction problem.\n\n```python\nimport numpy as np\nfrom evorbf import Data, NiaRbfRegressor\nfrom sklearn.datasets import load_diabetes\n\n## Load data object\n# total samples = 442, total features = 10\nX, y = load_diabetes(return_X_y=True)\ndata = Data(X, y)\n\n## Split train and test\ndata.split_train_test(test_size=0.2, random_state=2)\nprint(data.X_train.shape, data.X_test.shape)\n\n## Scaling dataset\ndata.X_train, scaler_X = data.scale(data.X_train, scaling_methods=(\"standard\"))\ndata.X_test = scaler_X.transform(data.X_test)\n\ndata.y_train, scaler_y = data.scale(data.y_train, scaling_methods=(\"standard\", ))\ndata.y_test = scaler_y.transform(np.reshape(data.y_test, (-1, 1)))\n\n## Create model\nopt_paras = {\"name\": \"WOA\", \"epoch\": 50, \"pop_size\": 20}\nmodel = NiaRbfRegressor(size_hidden=25,             # Set up big enough hidden size \n                        center_finder=\"kmeans\",     # Use KMeans to find the centers\n                        regularization=True,        # Use L2 regularization \n                        obj_name=\"MSE\",             # Mean squared error as fitness function for NIAs\n                        optim=\"OriginalWOA\",        # Use Whale Optimization\n                        optim_params={\"epoch\": 50, \"pop_size\": 20},  # Set up parameter for Whale Optimization\n                        verbose=True, seed=42)\n\n## Train the model\nmodel.fit(data.X_train, data.y_train)\n\n## Test the model\ny_pred = model.predict(data.X_test)\n\nprint(model.optimizer.g_best.solution)\n## Calculate some metrics\nprint(model.score(X=data.X_test, y=data.y_test))\nprint(model.scores(X=data.X_test, y=data.y_test, list_metrics=[\"R2\", \"R\", \"KGE\", \"MAPE\"]))\nprint(model.evaluate(y_true=data.y_test, y_pred=y_pred, list_metrics=[\"MSE\", \"RMSE\", \"R2S\", \"NSE\", \"KGE\", \"MAPE\"]))\n```\n\n---\n\n## 📚 Brief Theory Combine With Deep Usage\n\n**EvoRBF** is mind-blowing framework for Radial Basis Function (RBF) networks.\nWe explain several keys components and provide several types of RBF networks that you will never see in other places.\n\nYou can read several papers by using Google Scholar search. There are many ways we can use Nature-inspired Algorithms \nto optimize Radial Basis Function network, for example, you can read [this paper](https://doi.org/10.1016/B978-0-443-18764-3.00015-1).\nHere we will walk through some basic concepts and parameters that matter to this network.\n\n### Structure\n\nThe RBF network consists of three layers:\n\n1. **Input layer**: Accepts input features.\n2. **Hidden layer**: Applies radial basis functions (e.g., Gaussian).\n3. **Output layer**: Computes linear combinations of hidden outputs.\n\n---\n\n### 🛠️ Model Training\n\n#### Traditional RBF models\n\nIn case of traditional RBF model. There are a few parameters need to identify to get the best model.\n```code\n1. The number of hidden nodes in hidden layer\n2. The centers and widths (sigmas) of Gaussian function\n3. The output weights\n4. The regularization factor (lambda) L2\n```\n\nTo train their parameters, \n```code\n1. Using hyper-parameter tuning model such as GridSearchCV or RandomizedSearchCV to get the best hidden nodes\n2. The centers can be calculated by Random or KMeans or unsupervised learning algorithms\n3. The widths (sigmas) can be computed by hyper-parameter tuning process.\n   + Width can be a single value that represent all hidden nodes has the same curve of Gaussian function\n   + Width can be multiple values that each hidden node has a different value.\n4. The output weights can be calculated by Moore-Penrose inverse (Matrix multiplication). Do not use Gradient Descent.\n5. When setting regularization L2. lambda can be computed by hyper-parameter tuning process.\n```\n\nExample,\n```python\nfrom evorbf import RbfRegressor, RbfClassifier\n\nmodel = RbfClassifier(size_hidden=10, center_finder=\"kmeans\", sigmas=2.0, reg_lambda=0.1, seed=None)\nmodel = RbfRegressor(size_hidden=4, center_finder=\"random\", sigmas=(1.5, 2, 2, 2.5), reg_lambda=0, seed=42)\n\nmodel.fit(X=X_train, y=y_train)\ny_pred = model.predict(X_test)\ny_pred_prob = model.predict_proba(X_test)\n```\n\n---\n\n#### Advanced RBF models\n\nIn case of advanced RBF model. User can have so many different options.\n```code\n1. Choice different RBF kernel function such as Multiquadric (MQ), Inverse Multiquadric (IMQ), Thin Plate Spline (TPS), Exponential, Power,...\n2. Choice different unsupervised learning algorithms to calculate the centers, and may be the number of hidden nodes.\n   + For example, KMeans, or random algorithms, you need to set up the number of hidden nodes.\n   + But, for MeanShift or DBSCAN algorithms, you don't need to set that value. They can automatically identify the number of cluters (number of hidden nodes).\n3. This version may have the bias in output layer. \n```\n\nExamples,\n```python\nfrom evorbf import AdvancedRbfClassifier, AdvancedRbfRegressor\n\nmodel = AdvancedRbfClassifier(center_finder=\"random\", finder_params={\"n_centers\": 15},\n                 rbf_kernel=\"gaussian\", kernel_params={\"sigma\": 1.5},\n                 reg_lambda=0.1, has_bias=True, seed=42)\n\nmodel = AdvancedRbfClassifier(center_finder=\"random\", finder_params=None,        # Default n_centers = 10\n                 rbf_kernel=\"gaussian\", kernel_params=None,                     # Default sigma = 1.0\n                 reg_lambda=0.1, has_bias=False, seed=42)\n\nmodel = AdvancedRbfClassifier(center_finder=\"kmeans\", finder_params={\"n_centers\": 20},\n                 rbf_kernel=\"multiquadric\", kernel_params=None,\n                 reg_lambda=0.1, has_bias=False, seed=42)\n\nmodel = AdvancedRbfClassifier(center_finder=\"meanshift\", finder_params={\"bandwidth\": 0.6},      # Give us 28 hidden nodes\n                 rbf_kernel=\"inverse_multiquadric\", kernel_params={\"sigma\": 1.5},\n                 reg_lambda=0.5, has_bias=True, seed=42)\n\nmodel = AdvancedRbfClassifier(center_finder=\"dbscan\", finder_params={\"eps\": 0.2},      # Give us 42 hidden nodes\n                 rbf_kernel=\"multiquadric\", kernel_params={\"sigma\": 1.5},\n                 reg_lambda=0.5, has_bias=True, seed=42)\n\nmodel = AdvancedRbfClassifier(center_finder=\"dbscan\", finder_params={\"eps\": 0.175},      # Give us 16 hidden nodes\n                 rbf_kernel=\"multiquadric\", kernel_params={\"sigma\": 1.5},\n                 reg_lambda=None, has_bias=False, seed=42)\n\nmodel.fit(X=X_train, y=y_train)\ny_pred = model.predict(X_test)\ny_pred_prob = model.predict_proba(X_test)\n```\n\n---\n\n#### Nature-inspired Algorithm-based RBF models\n\nThis is the main purpose of this library. In this type of models,\n\n```code\n1. We use Nature-inspired Algorithm (NIA) to train widths (sigmas) value for each hidden node.\n2. If you set up the Regularization technique, then NIA is automatically calculated the lambda factor\n```\n\nExamples,\n```python\nfrom evorbf import NiaRbfRegressor, NiaRbfClassifier\n\nmodel = NiaRbfClassifier(size_hidden=25, center_finder=\"kmeans\", \n                         regularization=False, obj_name=\"F1S\",\n                         optim=\"OriginalWOA\", \n                         optim_params={\"epoch\": 50, \"pop_size\": 20}, \n                         verbose=True, seed=42)\n\nmodel = NiaRbfRegressor(size_hidden=10, center_finder=\"random\", \n                         regularization=True, obj_name=\"AS\",\n                         optim=\"BaseGA\", \n                         optim_params={\"epoch\": 50, \"pop_size\": 20}, \n                         verbose=True, seed=42)\n\nmodel.fit(X=X_train, y=y_train)\ny_pred = model.predict(X_test)\ny_pred_prob = model.predict_proba(X_test)\n```\n\n---\n\n#### 🎯 Nature-inspired Algorithm-based hyperparameter RBF tuning model\n\nIn this case, user can use NIA to tune hyper-parameters of traditional RBF models.\n\n```python\nfrom evorbf import NiaRbfTuner, IntegerVar, StringVar, FloatVar\n\n# Design the boundary (for hyper-parameters)\nmy_bounds = [\n    IntegerVar(lb=5, ub=21, name=\"size_hidden\"),\n    StringVar(valid_sets=(\"kmeans\", \"random\"), name=\"center_finder\"),\n    FloatVar(lb=(0.01,), ub=(3.0,), name=\"sigmas\"),\n    FloatVar(lb=(0, ), ub=(1.0, ), name=\"reg_lambda\"),\n]\n\nmodel = NiaRbfTuner(problem_type=\"classification\", bounds=my_bounds, cv=3, scoring=\"AS\",\n                    optim=\"OriginalWOA\", optim_params={\"epoch\": 10, \"pop_size\": 20}, \n                    verbose=True, seed=42)\n```\n\n---\n\n### My notes\n\n1. RBF networks require training of both the centers and the widths of the Gaussian activation functions (this is the 1st phase of training).\n2. RBF typically uses KMeans to find the centers:\n   + This increases both complexity and computation time.\n   + In this case, users need to define the widths: You can use a single global width or assign individual widths per hidden node.\n   + Alternatively, centers can be chosen randomly, but this often fails to properly separate the data into meaningful clusters.\n3. RBF also requires training the output weights (this is the 2nd phase).\n4. Unlike MLP networks, RBF does not use gradient descent to compute output weights. Instead, it uses the Moore–Penrose pseudoinverse \n(via matrix multiplication and the least squares method) → This makes it faster than MLPs.\n5. The Moore–Penrose inverse finds an exact solution, so there is no need for gradient descent or approximation algorithms in this step.\n6. If overfitting occurs, you can apply L2 regularization to control the model's complexity.\n7. For large-scale datasets, you should:\n   + Increase the number of hidden nodes.\n   + Increase the L2 regularization parameter to avoid overfitting.\n\n\n```code\n1. RbfRegressor, RbfClassifier:\n   - You need to configure 4 types of hyperparameters.\n   \n2. AdvancedRbfRegressor, AdvancedRbfClassifier:\n   - These models require setting 6 types of hyperparameters.\n   - However, they offer much more flexibility—you can design your own custom RBF architectures.\n   - For example:\n     + RBF with bias in the output layer.\n     + RBF using DBSCAN for center initialization and an exponential kernel function.\n\n3. NiaRbfRegressor, NiaRbfClassifier:\n   - You only need to set the hidden size.\n   - These are the best-performing classes in this library.\n     + Widths (sigmas) are automatically computed for each hidden node.\n     + The regularization factor is automatically tuned to find the optimal value.\n\n4. NiaRbfTuner:\n   - Extremely useful for traditional RBF models.\n   - It can automatically tune the hidden size.\n   - However, only a single sigma value will be used across all hidden nodes.\n```\n\n\n## 📎 Official channels\n\n* 🔗 [Official source code repository](https://github.com/thieu1995/evorbf)\n* 📘 [Official document](https://evorbf.readthedocs.io/)\n* 📦 [Download releases](https://pypi.org/project/evorbf/) \n* 🐞 [Issue tracker](https://github.com/thieu1995/evorbf/issues) \n* 📝 [Notable changes log](/ChangeLog.md)\n* 💬 [Official discussion group](https://t.me/+fRVCJGuGJg1mNDg1)\n\n---\n\nDeveloped by: [Thieu](mailto:nguyenthieu2102@gmail.com?Subject=EvoRBF_QUESTIONS) @ 2025\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthieu1995%2Fevorbf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthieu1995%2Fevorbf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthieu1995%2Fevorbf/lists"}