{"id":22416824,"url":"https://github.com/hoangthangta/BSRBF_KAN","last_synced_at":"2025-08-01T01:32:11.041Z","repository":{"id":243044632,"uuid":"811308705","full_name":"hoangthangta/BSRBF_KAN","owner":"hoangthangta","description":"Combine B-Splines (BS) and Radial Basis Functions (RBF) in Kolmogorov-Arnold Networks (KANs)","archived":false,"fork":false,"pushed_at":"2024-11-15T13:21:33.000Z","size":14034,"stargazers_count":22,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-15T14:26:35.824Z","etag":null,"topics":["b-splines","efficientkan","fast-kan","kans","kolmogorov-arnold-networks","mnist","radial-basis-function"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hoangthangta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-06T10:52:22.000Z","updated_at":"2024-11-15T13:21:36.000Z","dependencies_parsed_at":"2024-07-10T02:14:21.990Z","dependency_job_id":"7aea680d-24af-43ac-9e9e-3eb6acae45f0","html_url":"https://github.com/hoangthangta/BSRBF_KAN","commit_stats":null,"previous_names":["hoangthangta/bsrbf_kan"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoangthangta%2FBSRBF_KAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoangthangta%2FBSRBF_KAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoangthangta%2FBSRBF_KAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoangthangta%2FBSRBF_KAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hoangthangta","download_url":"https://codeload.github.com/hoangthangta/BSRBF_KAN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228321215,"owners_count":17901604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["b-splines","efficientkan","fast-kan","kans","kolmogorov-arnold-networks","mnist","radial-basis-function"],"created_at":"2024-12-05T15:16:41.579Z","updated_at":"2025-08-01T01:32:10.664Z","avatar_url":"https://github.com/hoangthangta.png","language":"Python","funding_links":[],"categories":["Library"],"sub_categories":["Theorem"],"readme":"**For novel KANs, see: https://github.com/hoangthangta/All-KAN.**\n\n# BSRBF_KAN\n\nIn this repo, we use Efficient KAN (https://github.com/Blealtan/efficient-kan/ and FAST-KAN (https://github.com/ZiyaoLi/fast-kan/) to create BSRBF_KAN, which combines B-Splines (**BS**) and Radial Basis Functions (**RBF**) for Kolmogorov-Arnold Networks (KANs). \n\nOur paper's name is misspelled: \"BSRBF-KAN: A combination of B-splines and Radial Basi~~c~~s (**s, not c**) Functions in Kolmogorov-Arnold Networks.\" Please cite our paper correctly; thank you!\n\nOur paper is available at https://arxiv.org/abs/2406.11173 (**BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networks**) or https://www.researchgate.net/publication/381471539_BSRBF-KAN_A_combination_of_B-splines_and_Radial_Basis_Functions_in_Kolmogorov-Arnold_Networks.\n\nThe official version is in: https://link.springer.com/chapter/10.1007/978-981-96-4288-5_1.\n\n# Requirements \n* numpy==1.26.4\n* numpyencoder==0.3.0\n* torch==2.3.0+cu118\n* torchvision==0.18.0+cu118\n* tqdm==4.66.4\n  \n# How to combine?\nWe start with layer normalization of the input and then merge three outputs: base_output, bs_output, and rbf_output. Although this method appears simple, finding an optimal combined KAN that is better than the available KANs in terms of something (accuracy, speed, convergence, etc) is time-consuming. We hope our research will lead to the development of various combined KANs using mathematical functions.\n\n```\ndef forward(self, x):\n        # layer normalization\n        x = self.layernorm(x)\n        \n        # base\n        base_output = F.linear(self.base_activation(x), self.base_weight)\n        \n        # b_splines\n        bs_output = self.b_splines(x).view(x.size(0), -1)\n        \n        # rbf\n        rbf_output = self.rbf(x).view(x.size(0), -1)\n        \n        # combine\n        bsrbf_output = bs_output + rbf_output\n        bsrbf_output = F.linear(bsrbf_output, self.spline_weight)\n\n        return base_output + bsrbf_output\n```\n# Training\n\n## Parameters\n* *mode*: working mode (\"train\" or \"test\").\n* *ds_name*: dataset name (\"mnist\" or \"fashion_mnist\").\n* *model_name*: type of model (bsrbf_kan, efficient_kan, fast_kan, faster_kan).\n* *epochs*: the number of epochs.\n* *batch_size*: the training batch size.\n* *n_input*: The number of input neurons.\n* *n_hidden*: The number of hidden neurons. We use only 1 hidden layer. You can modify the code (run.py) for more layers.\n* *n_output*: The number of output neurons (classes). For MNIST, there are 10 classes.\n* *grid_size*: The size of grid (default: 5). Use with bsrbf_kan and efficient_kan.\n* *spline_order*: The order of spline (default: 3). Use with bsrbf_kan and efficient_kan.\n* *num_grids*: The number of grids, equals grid_size + spline_order (default: 8). Use with fast_kan and faster_kan.\n* *device*: use \"cuda\" or \"cpu\".\n* *n_examples*: the number of examples in the training set used for training (default: -1, mean use all training data)\n\n## Commands\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"bsrbf_kan\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10 --grid_size 5 --spline_order 3```\n\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"efficient_kan\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10 --grid_size 5 --spline_order 3```\n\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"fast_kan\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10 --num_grids 8```\n\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"faster_kan\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10 --num_grids 8```\n\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"gottlieb_kan\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10 --spline_order 3```\n\n```python run.py --mode \"train\" --ds_name \"mnist\" --model_name \"mlp\" --epochs 25 --batch_size 64 --n_input 784 --n_hidden 64 --n_output 10```\n\n# Test on MNIST + Fashion MNIST + SL MNIST\nWe trained the models **on GeForce RTX 3060 Ti** (with other default parameters; see Commands). The results are in our updated paper, **we are working to update them**.\n\n# References\n* https://github.com/Blealtan/efficient-kan\n* https://github.com/AthanasiosDelis/faster-kan\n* https://github.com/ZiyaoLi/fast-kan/\n* https://github.com/seydi1370/Basis_Functions\n* https://github.com/KindXiaoming/pykan (the original KAN)\n\n# Acknowledgements\nWe especially thank the contributions of https://github.com/Blealtan/efficient-kan, https://github.com/ZiyaoLi/fast-kan/, and https://github.com/seydi1370/Basis_Functions for their great work in KANs.\n\nAlso, give me a star if you like this repo. Thanks!\n\n# Paper\n```\n@inproceedings{ta2024bsrbf,\n  title={BSRBF-KAN: A Combination of B-Splines and Radial Basis Functions in Kolmogorov-Arnold Networks},\n  author={Ta, Hoang-Thang},\n  booktitle={International Symposium on Information and Communication Technology},\n  pages={3--15},\n  year={2024},\n  organization={Springer}\n}\n```\n\n# Contact\nIf you have any questions, please contact: tahoangthang@gmail.com. If you want to know more about me, please visit website: https://tahoangthang.com.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoangthangta%2FBSRBF_KAN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhoangthangta%2FBSRBF_KAN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoangthangta%2FBSRBF_KAN/lists"}