{"id":22416788,"url":"https://github.com/Indoxer/LKAN","last_synced_at":"2025-08-01T01:31:59.643Z","repository":{"id":238381244,"uuid":"795998079","full_name":"Indoxer/LKAN","owner":"Indoxer","description":"Variations of Kolmogorov-Arnold Networks","archived":false,"fork":false,"pushed_at":"2024-05-13T19:43:41.000Z","size":3472,"stargazers_count":82,"open_issues_count":0,"forks_count":11,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-05-14T17:22:39.095Z","etag":null,"topics":["machine-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Indoxer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-04T16:21:22.000Z","updated_at":"2024-05-14T14:26:30.000Z","dependencies_parsed_at":"2024-05-13T16:06:43.828Z","dependency_job_id":null,"html_url":"https://github.com/Indoxer/LKAN","commit_stats":null,"previous_names":["indoxer/lkan"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Indoxer%2FLKAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Indoxer%2FLKAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Indoxer%2FLKAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Indoxer%2FLKAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Indoxer","download_url":"https://codeload.github.com/Indoxer/LKAN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228321215,"owners_count":17901604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning"],"created_at":"2024-12-05T15:16:39.420Z","updated_at":"2024-12-05T15:16:57.807Z","avatar_url":"https://github.com/Indoxer.png","language":"Python","funding_links":[],"categories":["Library"],"sub_categories":["Theorem"],"readme":"# Large Kolmogorov-Arnold Networks\nImplementations of KAN variations.\n\n## Installation\n1. Install [conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html)\n2. Install CUDA (If you can run CUDA pytorch code, then it should works)\n3. Run `bash -l install.sh` (in future `install.cmd`)\n\n## Running\n\nActivate conda env `conda activate lkan`\n\nTo run mnist select config in `main.py` and run `main.py`.\n\nTo view charts, run `tensorboard --logdir ./.experiments/`\n\n## Docs\n\nSee examples/\n\nOnly done so far:\n\n- `continual_training_adam.ipynb`, `continual_training_lbfgs.ipynb`\n\n## Contribution/Development\n\n#### Contact\nIf you have any questions, please create issue or contact indoxer.mk@gmail.com\n\n#### Additional development packages/apps:\n- cuda-toolkit (nsight compute, etc.)\n\n#### Good to know:\n1. To run nvidia nsight compute on kernels:\n- Add conda python interpreter as executable and python file as args.\n- Install lkancpp `CUDA_LINEINFO=1 pip install ./lkancpp/` to see kernels code.\n2. Installing packages:\n- `pip install .` to update package after modifying `lkan/`\n- `pip install ./lkancpp` to recompile lkancpp.\n\n\n## TODO/Ideas:\n- [ ] Use cmake for lkancpp build.\n- [ ] remove unnecessary dependencies in requirements.txt\n- [ ] test update_grid and \"Other possibilities are: (a) the grid is learnable with gradient descent\" from paper. \n- [ ] Implement and test (examples notebook) Regularization\n- [ ] Implement and test (examples notebook) grid extension\n- [ ] MNIST (yaml config to run, model + readme with results)\n- [ ] CIFAR10 (yaml config to run, model + readme with results)\n- [ ] Test KAN convolution\n- [ ] Test KAN as patches encoder in VIT.\n- [ ] Implement KAN in latent space (... -\u003e Linear -\u003e KAN -\u003e Linear -\u003e ...)\n- [ ] Implement plotting. (plot layers and KAN model)\n- [ ] Test scaling behavior on toy dataset (examples notebook)\n- [ ] Test scaling behavior on MNIST/CIFAR10 (examples notebook)\n- [ ] Test Legendre and Chebyshev polynomials\n- [ ] Test Gaussian KAN\n- [ ] Write functions to prune KAN layers.\n- [ ] Write unit tests for used methods (CUDA version matching pytorch version, pytorch version tests on simple examples)\n\n## Problems\n- [ ] update_grid on cuda raise error (torch.linalg.lstsq assume full rank on cuda, only one algorithm) - solved temporary, moved calculating lstsq to cpu\n- [ ] update_grid_from_samples in original KAN run model multiple times, is it necessary? \n- [ ] parameters counting, is grid parameter or not?\n- [ ] MLP training is almost instant, but KAN train slow on start\n\n## Citations\n```python\n@misc{liu2024kan,\n      title={KAN: Kolmogorov-Arnold Networks}, \n      author={Ziming Liu and Yixuan Wang and Sachin Vaidya and Fabian Ruehle and James Halverson and Marin Soljačić and Thomas Y. Hou and Max Tegmark},\n      year={2024},\n      eprint={2404.19756},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG}\n}\n```\n[Original KAN repo](https://github.com/KindXiaoming/pykan) - base idea\n\n[efficient-kan](https://github.com/Blealtan/efficient-kan) - KANLinear and optimizations\n\nThanks to [Paluzki](https://github.com/paluzki) for proposing Gaussian KAN (KANLinearG - in progress)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FIndoxer%2FLKAN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FIndoxer%2FLKAN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FIndoxer%2FLKAN/lists"}