https://github.com/dav0dea/pyautodiff
A very small (<90 LOC) pure python library capable of computing gradients of basic arithmetic functions
https://github.com/dav0dea/pyautodiff
automatic-differentiation differentiation gradient python
Last synced: about 1 month ago
JSON representation
A very small (<90 LOC) pure python library capable of computing gradients of basic arithmetic functions
- Host: GitHub
- URL: https://github.com/dav0dea/pyautodiff
- Owner: dav0dea
- License: mit
- Created: 2020-05-16T19:14:55.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-08-05T22:42:46.000Z (almost 6 years ago)
- Last Synced: 2025-05-26T20:28:18.476Z (about 1 year ago)
- Topics: automatic-differentiation, differentiation, gradient, python
- Language: Python
- Homepage:
- Size: 24.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pyautodiff
A very small Python library (<90 LOC) capable of automatic differentiation of addition, subtraction, multiplication, division and exponentiation. The library consists of a Operation class which stores a binary mathematical operation and a small subclass Variable, which only adds functionality for setting a variable's value. It supports having multiple variables in one calculation and computes the gradient using the gradient function. This function accepts either a single Variable object or an iterable of Variable objects. If a single object is passed, the function returns the derivative of the calculation defined in the Operation object with respect to the Variable object passed to the function. If an iterable of Variable objects is passed to gradient, a list of partial derivatives with respect to the variables in the iterable is returned.
The whole library is contained in `pyautodiff.py` and contains two classes. Only the Variable class is necessary for interfacing with the library and differentiating mathematical expressions. It is able to differentiate arbitrarily complex expressions containing addition, subtraction, multiplication, division and exponentiation with multiple variables.
`pyautodiff` is not intended to be used real world problems but as a learning resource. It lacks error handling in many cases and is very inefficient due to the implementation in pure Python code.
## Example
```python
from pyautodiff import Variable
# create variables
x = Variable(5)
y = Variable(2)
# create the computation
computation = (3 * x ** 2) / (y + 2)
# get the numerical value of the computation
result = float(computation)
# get the gradient (partial derivatives with respect to x and y)
gradient = computation.gradient((x, y))
```
More examples can be found in the following files:
- [`example_simple.py`](https://github.com/PhilippThoelke/pyautodiff/blob/master/examples/example_simple.py)
- simple calculations and differentiation with pyautodiff
- [`example_perceptron.py`](https://github.com/PhilippThoelke/pyautodiff/blob/master/examples/example_perceptron.py)
- implmementation of a perceptron with a sigmoid activation function
- iterative training for classification of a logical AND using gradient descent
- gradient computed using pyautodiff
- [`example_mlp.py`](https://github.com/PhilippThoelke/pyautodiff/blob/master/examples/example_mlp.py)
- implementation of a multilayer perceptron (MLP) with variable layer sizes and sigmoid activation function
- iterative training for classification of the non-linear logical XOR
- MLP is trained with gradient descent using pyautodiff