{"id":35324286,"url":"https://github.com/donkamilo00/fusionkan","last_synced_at":"2026-04-10T11:01:53.277Z","repository":{"id":326374560,"uuid":"1105279446","full_name":"DonKamilo00/FusionKAN","owner":"DonKamilo00","description":null,"archived":false,"fork":false,"pushed_at":"2025-11-29T01:12:29.000Z","size":226,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-30T04:40:35.694Z","etag":null,"topics":["artificial-intelligence","cuda","cuda-kernels","kolmogorov-arnold-networks","machine-learning","neural-network","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DonKamilo00.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-27T11:36:13.000Z","updated_at":"2025-11-29T01:12:33.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/DonKamilo00/FusionKAN","commit_stats":null,"previous_names":["donkamilo00/fusionkan"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/DonKamilo00/FusionKAN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DonKamilo00%2FFusionKAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DonKamilo00%2FFusionKAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DonKamilo00%2FFusionKAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DonKamilo00%2FFusionKAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DonKamilo00","download_url":"https://codeload.github.com/DonKamilo00/FusionKAN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DonKamilo00%2FFusionKAN/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31639524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T07:40:12.752Z","status":"ssl_error","status_checked_at":"2026-04-10T07:40:11.664Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","cuda","cuda-kernels","kolmogorov-arnold-networks","machine-learning","neural-network","python"],"created_at":"2025-12-31T01:02:57.382Z","updated_at":"2026-04-10T11:01:53.272Z","avatar_url":"https://github.com/DonKamilo00.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FusionKAN: High-Performance CUDA Kolmogorov-Arnold Networks\n\n**FusionKAN** is a highly optimized PyTorch library for Kolmogorov-Arnold Networks (KANs). By fusing B-spline basis computation, coefficient gathering, and linear combination into a single CUDA kernel, it achieves **33x speedups** and **Constant Memory Scaling** compared to standard implementations.\n\n![FusionKAN Speedup](kan_paper_benchmark.png)\n\n## 🚀 Key Highlights\n\n### 1. Constant Memory Cost ($O(1)$)\nStandard KAN implementations expand input tensors to size $[Batch, In, Grid]$, causing linear memory growth. FusionKAN computes basis functions on-the-fly in registers.\n*   **Original KAN:** Crashes (OOM) at Grid=500 with \u003e16GB VRAM usage.\n*   **FusionKAN:** Stays constant at **~70 MB** VRAM regardless of grid size.\n\n### 2. Massive Throughput (33x Speedup)\nFusionKAN leverages the GPU's Read-Only Cache (`__ldg`) and minimizes memory round-trips.\n*   **Latency:** Reduced from **570ms** to **17ms** per step (Width=2048, Batch=8192).\n*   **Throughput:** Processes \u003e475,000 samples/sec on a consumer GPU (T4).\n\n### 3. Learnable Grids\nUnlike standard implementations that require manual grid updates to handle data distribution shifts, FusionKAN treats grid boundaries (`min`, `max`) as learnable parameters. They are updated automatically via gradient descent, ensuring **smooth convergence** without loss spikes.\n\n## 📊 Benchmark Results\n\nBenchmarks run on NVIDIA T4, Float32, Batch Size 4096.\n\n### Experiment 1 Memory Scalability and Computational Throughput at High Grid Resolutions\n\nThe benchmark is trying to learn a specific 2D function:\n\n$$ f(x, y) = \\exp(\\sin(\\pi x) + y^2) $$\n\n| Model | Grid Size ($G$) | Peak VRAM | Time per Step |\n| :--- | :--- | :--- | :--- |\n| **PyKAN / MultKAN** | 200 | 12.5 GB | 565 ms |\n| **FusionKAN** | 200 | **68 MB** | **3.2 ms** |\n| | | | |\n| **PyKAN / MultKAN** | 500 | **OOM (Crash)** | N/A |\n| **FusionKAN** | 500 | **70 MB** | **3.2 ms** |\n\n## 🛠️ Features\n\n- **Fused CUDA Kernels:** Performs Grid Mapping + Basis computation + Gather + Multiply in a single kernel launch.\n- **Scalable Backward Pass:** Uses direct Global Atomic accumulation to handle arbitrary layer widths (up to 2048+) without overflowing shared memory.\n- **Physics Ready:** Supports exact input gradients ($\\nabla x$) via analytical cubic spline derivatives, ideal for PINNs and Eikonal loss functions.\n- **Drop-in Replacement:** Matches the `nn.Module` API for easy integration.\n\n## 📦 Installation\n\n**Prerequisites:**\n- NVIDIA GPU\n- CUDA Toolkit (nvcc)\n- PyTorch\n\n```bash\ngit clone https://github.com/yourusername/FusionKAN\ncd FusionKAN\npip install .\n\n\n💻 Usage\nFusionKAN can be used just like a standard PyTorch linear layer, but with B-splines.\n\nimport torch\nimport torch.nn as nn\nfrom fusion_kan import FusionKANLayer\n\n# Define a model\nmodel = nn.Sequential(\n    # Layer 1: 2 Inputs -\u003e 128 Hidden\n    FusionKANLayer(2, 128, grid_size=100, spline_order=3),\n    nn.LayerNorm(128),\n    nn.SiLU(),\n    \n    # Layer 2: 128 Hidden -\u003e 1 Output\n    FusionKANLayer(128, 1, grid_size=100, spline_order=3)\n).cuda()\n\n# Dummy Data\nx = torch.randn(8192, 2).cuda()\n\n# Forward Pass\ny = model(x)\n\n# Backward Pass (Gradients propagate to weights AND grid bounds)\nloss = y.sum()\nloss.backward()\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonkamilo00%2Ffusionkan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdonkamilo00%2Ffusionkan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonkamilo00%2Ffusionkan/lists"}