{"id":22202470,"url":"https://github.com/mytechnotalent/renn","last_synced_at":"2026-02-25T17:05:49.805Z","repository":{"id":264096955,"uuid":"892356516","full_name":"mytechnotalent/RENN","owner":"mytechnotalent","description":"Inspired by Andrej Karpathy’s micrograd, this lecture builds a neural network from scratch, manually deriving gradients, automating backpropagation, and leveraging the TANH activation for nonlinearity. We bridge to PyTorch, demonstrating gradient descent’s power to minimize loss and reveal neural network fundamentals.","archived":false,"fork":false,"pushed_at":"2024-11-22T00:52:47.000Z","size":1380,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T04:53:44.654Z","etag":null,"topics":["ai","artificial-intelligence","artificial-neural-networks","machine-learning","neural-network","neural-networks","neural-networks-from-scratch","nn","nnfs","pytorch","reverse-engineering","stem"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mytechnotalent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-22T00:43:22.000Z","updated_at":"2024-11-23T19:00:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"ae414508-5902-4fad-a5a5-94a2d7f50103","html_url":"https://github.com/mytechnotalent/RENN","commit_stats":null,"previous_names":["mytechnotalent/renn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mytechnotalent/RENN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mytechnotalent%2FRENN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mytechnotalent%2FRENN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mytechnotalent%2FRENN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mytechnotalent%2FRENN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mytechnotalent","download_url":"https://codeload.github.com/mytechnotalent/RENN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mytechnotalent%2FRENN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259794444,"owners_count":22912356,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","artificial-neural-networks","machine-learning","neural-network","neural-networks","neural-networks-from-scratch","nn","nnfs","pytorch","reverse-engineering","stem"],"created_at":"2024-12-02T16:26:01.898Z","updated_at":"2025-10-26T12:34:51.530Z","avatar_url":"https://github.com/mytechnotalent.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"```python\nfrom IPython.display import Image\n```\n\n\n```python\nImage(filename = 'RENN.jpeg')\n```\n\n\n\n\n    \n![jpeg](README_files/README_1_0.jpg)\n    \n\n\n\n# Reverse Engineering a Neural Network\n\n#### Inspired by Andrej Karpathy’s micrograd, this lecture builds a neural network from scratch, manually deriving gradients, automating backpropagation, and leveraging the TANH activation for nonlinearity. We bridge to PyTorch, demonstrating gradient descent’s power to minimize loss and reveal neural network fundamentals.\n\n### [Original Video from Andrej Karpathy](https://youtu.be/VMj-3S1tku0?si=-Tzz2Y2-iv5FLDtY)\n\n#### Credit: [Andrej Karpathy](mailto:karpathy@eurekalabs.ai)\n#### Instructor: [Kevin Thomas](mailto:ket189@pitt.edu)\n\n## Imports\n\n\n```python\nimport math\nimport random\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n```\n\n## Foundational Classes \u0026 Functions\n\n\n```python\nclass Value:\n    \"\"\"\n    A class representing a scalar value in a computational graph.\n\n    This class supports operations such as addition, subtraction, multiplication, \n    division, exponentiation, and hyperbolic tangent (tanh), along with automatic \n    differentiation through backpropagation. It is designed to build and manipulate \n    computational graphs for neural network computations.\n    \n    Attributes:\n        data (float): The scalar value this object holds.\n        grad (float): The gradient of this value with respect to some scalar loss.\n        _backward (function): A function to compute the local gradient contribution during backpropagation.\n        _prev (set[Value]): The set of `Value` objects that are inputs to this value.\n        _op (str): The operation that produced this value (e.g., '+', '*', 'tanh', '**').\n        label (str): A label for identifying the value (useful for visualization and debugging).\n    \"\"\"\n\n    def __init__(self, data, _children=(), _op='', label=''):\n        \"\"\"\n        Initializes a `Value` object.\n\n        Args:\n            data (float): The scalar value to store.\n            _children (tuple[Value], optional): The `Value` objects used to produce this value. Default is an empty tuple.\n            _op (str, optional): The operation that produced this value. Default is an empty string.\n            label (str, optional): A label for this value. Default is an empty string.\n        \"\"\"\n        self.data = data\n        self.grad = 0.0\n        self._backward = lambda: None\n        self._prev = set(_children)\n        self._op = _op\n        self.label = label\n\n    def __repr__(self):\n        \"\"\"\n        Returns a string representation of the `Value` object.\n\n        Returns:\n            str: A string showing the data value.\n        \"\"\"\n        return f'Value(data={self.data})'\n    \n    def __add__(self, other):\n        \"\"\"\n        Defines the addition operation for `Value` objects.\n\n        Args:\n            other (Value): The other `Value` object to add.\n\n        Returns:\n            Value: A new `Value` object representing the sum of the two values.\n\n        Note:\n            The `_backward` method for the resulting value calculates the gradient contributions\n            to both operands of the addition.\n        \"\"\"\n        other = other if isinstance(other, Value) else Value(other)\n        out = Value(self.data + other.data, (self, other), '+')\n        \n        def _backward():\n            self.grad += 1.0 * out.grad\n            other.grad += 1.0 * out.grad\n        out._backward = _backward\n        \n        return out\n\n    def __mul__(self, other):\n        \"\"\"\n        Defines the multiplication operation for `Value` objects.\n\n        Args:\n            other (Value): The other `Value` object to multiply.\n\n        Returns:\n            Value: A new `Value` object representing the product of the two values.\n\n        Note:\n            The `_backward` method for the resulting value calculates the gradient contributions\n            to both operands of the multiplication.\n        \"\"\"\n        other = other if isinstance(other, Value) else Value(other)\n        out = Value(self.data * other.data, (self, other), '*')\n        \n        def _backward():\n            self.grad += other.data * out.grad\n            other.grad += self.data * out.grad\n        out._backward = _backward\n        \n        return out\n\n    def __pow__(self, other):\n        \"\"\"\n        Defines the power operation for `Value` objects.\n\n        Args:\n            other (int or float): The exponent to which the value is raised.\n\n        Returns:\n            Value: A new `Value` object representing the value raised to the power of `other`.\n\n        Note:\n            The `_backward` method for the resulting value calculates the gradient contribution\n            for the power operation.\n        \"\"\"\n        assert isinstance(other, (int, float)), 'only supporting int/float powers for now'\n        out = Value(self.data**other, (self,), f'**{other}')\n        \n        def _backward():\n            self.grad += other * (self.data ** (other - 1)) * out.grad\n        out._backward = _backward\n        \n        return out\n\n    def __rmul__(self, other):\n        \"\"\"\n        Defines right-side multiplication for `Value` objects.\n\n        Args:\n            other (float or Value): The value to multiply from the right.\n\n        Returns:\n            Value: A new `Value` object representing the product.\n        \"\"\"\n        return self * other\n\n    def __truediv__(self, other):\n        \"\"\"\n        Defines division operation for `Value` objects.\n\n        Args:\n            other (Value): The divisor `Value` object.\n\n        Returns:\n            Value: A new `Value` object representing the division.\n\n        Note:\n            This is implemented as multiplying by the reciprocal.\n        \"\"\"\n        return self * other**-1\n\n    def __neg__(self):\n        \"\"\"\n        Defines the negation operation for a `Value` object.\n\n        Returns:\n            Value: A new `Value` object representing the negated value.\n        \"\"\"\n        return self * -1\n\n    def __sub__(self, other):\n        \"\"\"\n        Defines the subtraction operation for `Value` objects.\n\n        Args:\n            other (Value): The other `Value` object to subtract.\n\n        Returns:\n            Value: A new `Value` object representing the difference.\n        \"\"\"\n        return self + (-other)\n\n    def __radd__(self, other):\n        \"\"\"\n        Defines right-side addition for `Value` objects.\n\n        Args:\n            other (float or Value): The value to add from the right.\n\n        Returns:\n            Value: A new `Value` object representing the sum.\n        \"\"\"\n        return self + other\n\n    def tanh(self):\n        \"\"\"\n        Applies the hyperbolic tangent (tanh) function to the value.\n\n        Returns:\n            Value: A new `Value` object representing the tanh of this value.\n\n        Note:\n            The `_backward` method for the resulting value calculates the gradient contribution\n            for the tanh function: `(1 - tanh(x)^2)`.\n        \"\"\"\n        x = self.data\n        t = (math.exp(2*x) - 1) / (math.exp(2*x) + 1)\n        out = Value(t, (self,), 'tanh')\n        \n        def _backward():\n            self.grad += (1 - t**2) * out.grad\n        out._backward = _backward\n        \n        return out\n\n    def exp(self):\n        \"\"\"\n        Applies the exponential function to the value.\n\n        Returns:\n            Value: A new `Value` object representing the exponential of this value.\n\n        Note:\n            The `_backward` method for the resulting value calculates the gradient contribution\n            for the exponential function: `e^x`.\n        \"\"\"\n        x = self.data\n        out = Value(math.exp(x), (self,), 'exp')\n        \n        def _backward():\n            self.grad += out.data * out.grad\n        out._backward = _backward\n        \n        return out\n\n    def backward(self):\n        \"\"\"\n        Performs backpropagation to calculate gradients for all `Value` objects in the computational graph.\n\n        This method starts from the current `Value` object (usually the loss in a neural network) and\n        propagates gradients to all dependent `Value` objects by traversing the computational graph in\n        reverse topological order.\n\n        Note:\n            Sets the gradient of the starting value to 1.0 (i.e., the gradient of itself).\n        \"\"\"\n        topo = []  # topological order of nodes\n        visited = set()\n\n        def build_topo(v):\n            if v not in visited:\n                visited.add(v)\n                for child in v._prev:\n                    build_topo(child)\n                topo.append(v)\n        build_topo(self)\n        \n        self.grad = 1.0\n        for node in reversed(topo):\n            node._backward()\n```\n\n\n```python\nimport platform\n```\n\n\n```python\n# check if the OS is macOS\nif platform.system() == 'Darwin':\n    !brew install graphviz\n    !pip install graphviz\n```\n\n    \u001b[34m==\u003e\u001b[0m \u001b[1mDownloading https://formulae.brew.sh/api/formula.jws.json\u001b[0m\n    ######################################################################### 100.0%\n    \u001b[34m==\u003e\u001b[0m \u001b[1mDownloading https://formulae.brew.sh/api/cask.jws.json\u001b[0m\n    ######################################################################### 100.0%\n    \u001b[33mWarning:\u001b[0m graphviz 12.2.0 is already installed and up-to-date.\n    To reinstall 12.2.0, run:\n      brew reinstall graphviz\n    Requirement already satisfied: graphviz in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (0.20.3)\n\n\n\n```python\nfrom graphviz import Digraph\n```\n\n\n```python\ndef trace(root):\n    \"\"\"\n    Traces the computational graph starting from the given root node.\n\n    This function builds a set of all nodes and edges in the computational graph\n    by traversing backwards from the root node. It identifies all intermediate\n    `Value` objects (nodes) and their relationships (edges) in the graph.\n\n    Args:\n        root (Value): The root node of the computational graph, typically the final output\n                     (e.g., a loss value in a neural network).\n\n    Returns:\n        tuple:\n            - nodes (set[Value]): A set of all `Value` objects in the computational graph.\n            - edges (set[tuple[Value, Value]]): A set of directed edges representing the\n              parent-child relationships in the graph.\n    \"\"\"\n    nodes, edges = set(), set()\n    \n    def build(v):\n        if v not in nodes:\n            nodes.add(v)\n            for child in v._prev:\n                edges.add((child, v))\n                build(child)\n    \n    build(root)\n    \n    return nodes, edges\n\n\ndef draw_dot(root):\n    \"\"\"\n    Creates a visual representation of the computational graph using the `graphviz` library.\n\n    This function generates a directed graph (in DOT format) of the computational graph\n    starting from the root node. Each node represents a `Value` object, and edges represent\n    the operations connecting them.\n\n    Args:\n        root (Value): The root node of the computational graph, typically the final output\n                     (e.g., a loss value in a neural network).\n\n    Returns:\n        graphviz.Digraph: A directed graph object representing the computational graph.\n\n    Example:\n        \u003e\u003e\u003e from graphviz import Digraph\n        \u003e\u003e\u003e a = Value(2.0, label='a')\n        \u003e\u003e\u003e b = Value(-3.0, label='b')\n        \u003e\u003e\u003e c = Value(10.0, label='c')\n        \u003e\u003e\u003e e = a * b; e.label = 'e'\n        \u003e\u003e\u003e d = e + c; d.label = 'd'\n        \u003e\u003e\u003e f = Value(-2.0, label='f')\n        \u003e\u003e\u003e L = d * f; L.label = 'L'\n        \u003e\u003e\u003e dot = draw_dot(L)\n        \u003e\u003e\u003e dot.render('graph', format='svg', cleanup=True)\n    \"\"\"\n    dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'})  # LR = left to right graph direction\n    \n    nodes, edges = trace(root)  # Trace the computational graph\n    \n    for n in nodes:\n        uid = str(id(n))\n        # for each value in the graph, create a rectangular ('record') node\n        dot.node(\n            name=uid,\n            label=\"{ %s | data %.4f | grad %.4f }\" % (n.label, n.data, n.grad),\n            shape='record'\n        )\n        if n._op:\n            # if the value is a result of an operation, create an operation node\n            dot.node(name=uid + n._op, label=n._op)\n            # connect the operation node to the value node\n            dot.edge(uid + n._op, uid)\n    \n    for n1, n2 in edges:\n        # connect n1 (input) to the operation node of n2 (output)\n        dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n    \n    return dot\n```\n\n## Neural Network\n\n\n```python\nImage(filename = 'nn.jpeg')\n```\n\n\n\n\n    \n![jpeg](README_files/README_12_0.jpg)\n    \n\n\n\n## Manual Feed-Forward (1st Epoch)\n\n\n```python\n# inputs x1,x2\nx1 = Value(2.0, label='x1')\nx2 = Value(0.0, label='x2')\n\n# weights w1,w2\nw1 = Value(-3.0, label='w1')\nw2 = Value(1.0, label='w2')\n\n# bias of the neuron\nb = Value(6.8813735870195432, label='b')\n\n# x1*w1 + x2*w2 + b\nx1w1 = x1*w1; x1w1.label = 'x1*w1'\nx2w2 = x2*w2; x2w2.label = 'x2*w2'\nx1w1x2w2 = x1w1 + x2w2; x1w1x2w2.label = 'x1*w1 + x2*w2'\nn = x1w1x2w2 + b; n.label = 'n'\n\n# activation function\no = n.tanh(); o.label = 'o'\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_15_0.svg)\n    \n\n\n\n## Manual Back Propogation (1st Epoch)\n\n### `o` Gradient\n\n\n```python\no.grad = 1.0  # init back propogation\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_19_0.svg)\n    \n\n\n\n### `n` Gradient\n\n\n```python\n1 - o.data**2  # tanh\n```\n\n\n\n\n    0.4999999999999999\n\n\n\n\n```python\nn.grad = 0.5\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_23_0.svg)\n    \n\n\n\n### `x1*w1 + x2*w2` Gradient\n\n#### We are working with a `+` node so therefore the `n.grad` of 0.5 will simply multiply by 1 or carry through.\n\n\n```python\nx1w1x2w2.grad = 0.5\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_27_0.svg)\n    \n\n\n\n### `b` Gradient\n\n#### We are working with a `+` node so therefore the `n.grad` of 0.5 will simply multiply by 1 or carry through.\n\n\n```python\nb.grad = 0.5\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_31_0.svg)\n    \n\n\n\n### `x1*w1` Gradient\n\n#### We are working with a `+` node so therefore the `x1w1x2w2.grad` of 0.5 will simply multiply by 1 or carry through.\n\n\n```python\nx1w1.grad = 0.5\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_35_0.svg)\n    \n\n\n\n### `x2*w2` Gradient\n\n#### We are working with a `+` node so therefore the `x1w1x2w2.grad` of 0.5 will simply multiply by 1 or carry through.\n\n\n```python\nx2w2.grad = 0.5\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_39_0.svg)\n    \n\n\n\n### `x1` Gradient\n\n#### We are working with a `*` node so therefore `x1.grad` will be `x1w1.grad * w1.data`. \n\n\n```python\nx1.grad = x1w1.grad * w1.data\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_43_0.svg)\n    \n\n\n\n### `w1` Gradient\n\n#### We are working with a `*` node so therefore `w1.grad` will be `x1w1.grad * x1.data`. \n\n\n```python\nw1.grad = x1w1.grad * x1.data\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_47_0.svg)\n    \n\n\n\n### `x2` Gradient\n\n#### We are working with a `*` node so therefore `x2.grad` will be `x2w2.grad * w2.data`. \n\n\n```python\nx2.grad = x2w2.grad * w2.data\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_51_0.svg)\n    \n\n\n\n### `w2` Gradient\n\n#### We are working with a `*` node so therefore `w2.grad` will be `x2w2.grad * x2.data`. \n\n\n```python\nw2.grad = x2w2.grad * x2.data\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_55_0.svg)\n    \n\n\n\n## Manual Feed-Forward (1st Epoch)\n\n\n```python\n# inputs x1,x2\nx1 = Value(2.0, label='x1')\nx2 = Value(0.0, label='x2')\n\n# weights w1,w2\nw1 = Value(-3.0, label='w1')\nw2 = Value(1.0, label='w2')\n\n# bias of the neuron\nb = Value(6.8813735870195432, label='b')\n\n# x1*w1 + x2*w2 + b\nx1w1 = x1*w1; x1w1.label = 'x1*w1'\nx2w2 = x2*w2; x2w2.label = 'x2*w2'\nx1w1x2w2 = x1w1 + x2w2; x1w1x2w2.label = 'x1*w1 + x2*w2'\nn = x1w1x2w2 + b; n.label = 'n'\n\n# activation function\no = n.tanh(); o.label = 'o'\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_58_0.svg)\n    \n\n\n\n## Back Propogation (1st Epoch)\n\n\n```python\no.grad = 1.0  # init back propogation\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_61_0.svg)\n    \n\n\n\n### ALL Gradients\n\n\n```python\no.backward()\n```\n\n\n```python\ndraw_dot(o)\n```\n\n\n\n\n    \n![svg](README_files/README_64_0.svg)\n    \n\n\n\n## PyTorch Implementation\n\n\n```python\n!pip install torch\n```\n\n    Requirement already satisfied: torch in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (2.5.1)\n    Requirement already satisfied: filelock in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (3.13.1)\n    Requirement already satisfied: typing-extensions\u003e=4.8.0 in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (4.11.0)\n    Requirement already satisfied: networkx in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (3.3)\n    Requirement already satisfied: jinja2 in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (3.1.4)\n    Requirement already satisfied: fsspec in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (2024.6.1)\n    Requirement already satisfied: setuptools in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (75.1.0)\n    Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from torch) (1.13.1)\n    Requirement already satisfied: mpmath\u003c1.4,\u003e=1.1.0 in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from sympy==1.13.1-\u003etorch) (1.3.0)\n    Requirement already satisfied: MarkupSafe\u003e=2.0 in /opt/anaconda3/envs/prod/lib/python3.12/site-packages (from jinja2-\u003etorch) (2.1.3)\n\n\n\n```python\nimport torch\n```\n\n\n```python\nx1 = torch.Tensor([2.0]).double()                ; x1.requires_grad = True\nx2 = torch.Tensor([0.0]).double()                ; x2.requires_grad = True\nw1 = torch.Tensor([-3.0]).double()               ; w1.requires_grad = True\nw2 = torch.Tensor([1.0]).double()                ; w2.requires_grad = True\nb = torch.Tensor([6.8813735870195432]).double()  ; b.requires_grad = True\nn = x1*w1 + x2*w2 + b\no = torch.tanh(n)\n\nprint(o.data.item())\n\no.backward()\n\nprint('---')\nprint('x2', x2.grad.item())\nprint('w2', w2.grad.item())\nprint('x1', x1.grad.item())\nprint('w1', w1.grad.item())\n```\n\n    0.7071066904050358\n    ---\n    x2 0.5000001283844369\n    w2 0.0\n    x1 -1.5000003851533106\n    w1 1.0000002567688737\n\n\n### Minimizing Loss\n\n\n```python\n# hyperparameters\nlearning_rate = 0.01  # learning rate for gradient descent\nepochs = 100  # number of epochs\n\n# target value for loss calculation (desired output for binary classifier)\ntarget = torch.tensor([0.0], dtype=torch.double)\n\n# training loop\nfor epoch in range(epochs):\n    # forward pass\n    n = x1 * w1 + x2 * w2 + b  # linear combination\n    o = torch.tanh(n)  # activation function\n    \n    # loss calculation (Mean Squared Error for simplicity)\n    loss = (o - target).pow(2) / 2  # MSE: L = (1/2) * (o - target)^2\n\n    # backward pass\n    loss.backward()  # compute gradients\n    \n    # gradient descent step (manual weight updates)\n    with torch.no_grad():\n        w1 -= learning_rate * w1.grad\n        w2 -= learning_rate * w2.grad\n        b -= learning_rate * b.grad\n\n        # zero gradients for next iteration\n        w1.grad.zero_()\n        w2.grad.zero_()\n        b.grad.zero_()\n        x1.grad.zero_()\n        x2.grad.zero_()\n    \n    # print loss for each epoch\n    print(f'Epoch {epoch + 1}, Loss: {loss.item():.6f}, Output: {o.data.item():.6f}')\n```\n\n    Epoch 1, Loss: 0.250000, Output: 0.707107\n    Epoch 2, Loss: 0.234693, Output: 0.685118\n    Epoch 3, Loss: 0.228050, Output: 0.675352\n    Epoch 4, Loss: 0.221271, Output: 0.665239\n    Epoch 5, Loss: 0.214365, Output: 0.654774\n    Epoch 6, Loss: 0.207342, Output: 0.643959\n    Epoch 7, Loss: 0.200214, Output: 0.632794\n    Epoch 8, Loss: 0.192996, Output: 0.621283\n    Epoch 9, Loss: 0.185704, Output: 0.609433\n    Epoch 10, Loss: 0.178355, Output: 0.597252\n    Epoch 11, Loss: 0.170967, Output: 0.584752\n    Epoch 12, Loss: 0.163562, Output: 0.571948\n    Epoch 13, Loss: 0.156161, Output: 0.558858\n    Epoch 14, Loss: 0.148786, Output: 0.545502\n    Epoch 15, Loss: 0.141461, Output: 0.531904\n    Epoch 16, Loss: 0.134209, Output: 0.518090\n    Epoch 17, Loss: 0.127053, Output: 0.504090\n    Epoch 18, Loss: 0.120018, Output: 0.489935\n    Epoch 19, Loss: 0.113125, Output: 0.475658\n    Epoch 20, Loss: 0.106397, Output: 0.461296\n    Epoch 21, Loss: 0.099852, Output: 0.446883\n    Epoch 22, Loss: 0.093510, Output: 0.432459\n    Epoch 23, Loss: 0.087387, Output: 0.418059\n    Epoch 24, Loss: 0.081496, Output: 0.403723\n    Epoch 25, Loss: 0.075849, Output: 0.389485\n    Epoch 26, Loss: 0.070456, Output: 0.375381\n    Epoch 27, Loss: 0.065322, Output: 0.361446\n    Epoch 28, Loss: 0.060451, Output: 0.347710\n    Epoch 29, Loss: 0.055846, Output: 0.334204\n    Epoch 30, Loss: 0.051506, Output: 0.320953\n    Epoch 31, Loss: 0.047427, Output: 0.307983\n    Epoch 32, Loss: 0.043605, Output: 0.295313\n    Epoch 33, Loss: 0.040034, Output: 0.282962\n    Epoch 34, Loss: 0.036706, Output: 0.270945\n    Epoch 35, Loss: 0.033612, Output: 0.259275\n    Epoch 36, Loss: 0.030742, Output: 0.247960\n    Epoch 37, Loss: 0.028087, Output: 0.237009\n    Epoch 38, Loss: 0.025634, Output: 0.226425\n    Epoch 39, Loss: 0.023373, Output: 0.216210\n    Epoch 40, Loss: 0.021293, Output: 0.206365\n    Epoch 41, Loss: 0.019382, Output: 0.196888\n    Epoch 42, Loss: 0.017630, Output: 0.187775\n    Epoch 43, Loss: 0.016025, Output: 0.179022\n    Epoch 44, Loss: 0.014556, Output: 0.170623\n    Epoch 45, Loss: 0.013215, Output: 0.162570\n    Epoch 46, Loss: 0.011990, Output: 0.154856\n    Epoch 47, Loss: 0.010874, Output: 0.147471\n    Epoch 48, Loss: 0.009857, Output: 0.140408\n    Epoch 49, Loss: 0.008932, Output: 0.133655\n    Epoch 50, Loss: 0.008090, Output: 0.127203\n    Epoch 51, Loss: 0.007326, Output: 0.121042\n    Epoch 52, Loss: 0.006631, Output: 0.115162\n    Epoch 53, Loss: 0.006001, Output: 0.109552\n    Epoch 54, Loss: 0.005429, Output: 0.104202\n    Epoch 55, Loss: 0.004911, Output: 0.099102\n    Epoch 56, Loss: 0.004441, Output: 0.094241\n    Epoch 57, Loss: 0.004015, Output: 0.089611\n    Epoch 58, Loss: 0.003630, Output: 0.085200\n    Epoch 59, Loss: 0.003281, Output: 0.081000\n    Epoch 60, Loss: 0.002965, Output: 0.077002\n    Epoch 61, Loss: 0.002679, Output: 0.073196\n    Epoch 62, Loss: 0.002420, Output: 0.069574\n    Epoch 63, Loss: 0.002186, Output: 0.066129\n    Epoch 64, Loss: 0.001975, Output: 0.062850\n    Epoch 65, Loss: 0.001784, Output: 0.059732\n    Epoch 66, Loss: 0.001611, Output: 0.056766\n    Epoch 67, Loss: 0.001455, Output: 0.053946\n    Epoch 68, Loss: 0.001314, Output: 0.051264\n    Epoch 69, Loss: 0.001187, Output: 0.048714\n    Epoch 70, Loss: 0.001071, Output: 0.046289\n    Epoch 71, Loss: 0.000967, Output: 0.043984\n    Epoch 72, Loss: 0.000873, Output: 0.041793\n    Epoch 73, Loss: 0.000788, Output: 0.039711\n    Epoch 74, Loss: 0.000712, Output: 0.037731\n    Epoch 75, Loss: 0.000643, Output: 0.035850\n    Epoch 76, Loss: 0.000580, Output: 0.034062\n    Epoch 77, Loss: 0.000524, Output: 0.032363\n    Epoch 78, Loss: 0.000473, Output: 0.030748\n    Epoch 79, Loss: 0.000427, Output: 0.029213\n    Epoch 80, Loss: 0.000385, Output: 0.027755\n    Epoch 81, Loss: 0.000348, Output: 0.026370\n    Epoch 82, Loss: 0.000314, Output: 0.025053\n    Epoch 83, Loss: 0.000283, Output: 0.023802\n    Epoch 84, Loss: 0.000256, Output: 0.022613\n    Epoch 85, Loss: 0.000231, Output: 0.021483\n    Epoch 86, Loss: 0.000208, Output: 0.020410\n    Epoch 87, Loss: 0.000188, Output: 0.019391\n    Epoch 88, Loss: 0.000170, Output: 0.018422\n    Epoch 89, Loss: 0.000153, Output: 0.017501\n    Epoch 90, Loss: 0.000138, Output: 0.016627\n    Epoch 91, Loss: 0.000125, Output: 0.015796\n    Epoch 92, Loss: 0.000113, Output: 0.015006\n    Epoch 93, Loss: 0.000102, Output: 0.014256\n    Epoch 94, Loss: 0.000092, Output: 0.013544\n    Epoch 95, Loss: 0.000083, Output: 0.012867\n    Epoch 96, Loss: 0.000075, Output: 0.012224\n    Epoch 97, Loss: 0.000067, Output: 0.011613\n    Epoch 98, Loss: 0.000061, Output: 0.011032\n    Epoch 99, Loss: 0.000055, Output: 0.010481\n    Epoch 100, Loss: 0.000050, Output: 0.009957\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmytechnotalent%2Frenn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmytechnotalent%2Frenn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmytechnotalent%2Frenn/lists"}