{"id":29606858,"url":"https://github.com/mr-ravin/aptxneuron","last_synced_at":"2025-07-20T17:37:33.417Z","repository":{"id":305198732,"uuid":"1020327704","full_name":"mr-ravin/APTxNeuron","owner":"mr-ravin","description":"A PyTorch implementation of the APTx Neuron architecture, featuring experiments and results on the MNIST handwritten digit dataset.","archived":false,"fork":false,"pushed_at":"2025-07-18T17:20:33.000Z","size":1306,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-18T21:55:59.054Z","etag":null,"topics":["aptx-neuron","artificial-intelligence","deep-learning","deep-neural-networks","neural-network","neuron","python","python3","pytorch-implementation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mr-ravin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-15T17:35:01.000Z","updated_at":"2025-07-18T17:20:36.000Z","dependencies_parsed_at":"2025-07-18T22:08:02.285Z","dependency_job_id":null,"html_url":"https://github.com/mr-ravin/APTxNeuron","commit_stats":null,"previous_names":["mr-ravin/aptxneuron"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/mr-ravin/APTxNeuron","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mr-ravin%2FAPTxNeuron","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mr-ravin%2FAPTxNeuron/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mr-ravin%2FAPTxNeuron/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mr-ravin%2FAPTxNeuron/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mr-ravin","download_url":"https://codeload.github.com/mr-ravin/APTxNeuron/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mr-ravin%2FAPTxNeuron/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266167035,"owners_count":23886914,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aptx-neuron","artificial-intelligence","deep-learning","deep-neural-networks","neural-network","neuron","python","python3","pytorch-implementation"],"created_at":"2025-07-20T17:37:20.416Z","updated_at":"2025-07-20T17:37:33.410Z","avatar_url":"https://github.com/mr-ravin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## APTx Neuron \nThis repository offers a Python package for the PyTorch implementation of the APTx Neuron, as introduced in the paper \"APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation\".\n\n**Paper Title**: APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation\n\n**Author**: [Ravin Kumar](https://mr-ravin.github.io)\n\n\n#### Github Repositories: \n- **Github Repository** (Pytorch Implementation): [Python Package](https://github.com/mr-ravin/APTxNeuron)\n\n---\n### APTx Neuron\n\u003cb\u003eAbstract\u003c/b\u003e: We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the [APTx activation function](https://arxiv.org/abs/2209.06119), thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \\sum_{i=1}^{n} ((\\alpha_i + \\tanh(\\beta_i x_i)) \\cdot \\gamma_i x_i) + \\delta$, where all parameters $\\alpha_i$, $\\beta_i$, $\\gamma_i$, and $\\delta$ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to 96.69\\% test accuracy in just 20 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and computational efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it.\n\nThe APTx Neuron is a novel computational unit that unifies linear transformation and non-linear activation into a single, expressive formulation. Inspired by the parametric APTx activation function, this neuron architecture removes the strict separation between computation and activation, allowing both to be learned as a cohesive entity. It is designed to enhance representational flexibility while reducing architectural redundancy.\n\n#### Mathematical Formulation\n\nTraditionally, a neuron computes the output as:\n\n$y = \\phi\\left( \\sum_{i=1}^{n} w_i x_i + b \\right)$\n\nwhere: \n- $x_i$ are the inputs,\n- $w_i$ are the weights,\n- $b$ is the bias,\n- and $\\phi$ is an activation function such as ReLU, Swish, or Mish.\n\n\nThe APTx Neuron merges these components into a unified trainable expression as:\n\n$y = \\sum_{i=1}^{n} \\left[ (\\alpha_i + \\tanh(\\beta_i x_i)) \\cdot \\gamma_i x_i \\right] + \\delta$\n\nwhere:\n- $x_i$ is the $i$-th input feature,\n- $\\alpha_i$, $\\beta_i$, and $\\gamma_i$ are trainable parameters for each input,\n- $\\delta$ is a trainable scalar bias.\n\nThis equation allows the neuron to modulate each input through a learned, per-dimension non-linearity and scaling operation. The term $(\\alpha_i + \\tanh(\\beta_i x_i))$ introduces adaptive gating, and $\\gamma_i x_i$ provides multiplicative control.\n\n----\n#### Experimentation on MNIST\nRun the below code to automatically run the APTx Neuron based fully-connected neural network on MNIST and save the `loss` and `accuracy` values in `./result/` directory.\n\n```python\npython3 run.py --total_epoch 20\n```\n\n#### Visualise\n1. ##### Visual analysis of train and test loss values.\n![image](https://github.com/mr-ravin/APTxNeuron//blob/main/mnist_loss.png?raw=true)\n\n2. ##### Visual analysis of train and test accuracy values.\n![image](https://github.com/mr-ravin/APTxNeuron//blob/main/mnist_accuracy.png?raw=true)\n\n----\n#### Conclusion\nThis work introduced the APTx Neuron, a unified, fully trainable neural unit that integrates linear transformation and non-linear activation into a single expression, extending the APTx activation function. By learning per-input parameters $\\alpha_i$, $\\beta_i$, $\\gamma_i$, and $\\delta$ for each input $x_i$ within a neuron, the APTx Neuron removes the need for separate activation layers and enables fine-grained input transformation. APTx Neuron generalizes traditional neurons and activations, offering greater representational power. Our MNIST experiments show that a fully connected APTx Neuron-based feedforward neural network achieves $96.69\\%$ test accuracy in 20 epochs with approximately 332K trainable parameters, demonstrating rapid convergence and high efficiency. This design lays the groundwork for extending APTx Neurons to CNNs and transformers, paving the way for more compact and adaptive deep learning architectures.\n\n----\n\n### 📜 Copyright License\n```python\nCopyright (c) 2025 Ravin Kumar\nWebsite: https://mr-ravin.github.io\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation \nfiles (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, \nmodify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the \nSoftware is furnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the \nSoftware.\n\nTHE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE \nWARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR \nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, \nARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmr-ravin%2Faptxneuron","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmr-ravin%2Faptxneuron","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmr-ravin%2Faptxneuron/lists"}