https://github.com/srush/autodiff-puzzles

https://github.com/srush/autodiff-puzzles

Last synced: about 1 year ago
JSON representation

• Host: GitHub
• URL: https://github.com/srush/autodiff-puzzles
• Owner: srush
• License: mit
• Created: 2022-10-30T21:19:57.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-10-18T20:24:33.000Z (over 1 year ago)
• Last Synced: 2025-04-08T16:08:21.097Z (about 1 year ago)
• Language: Jupyter Notebook
• Size: 4.26 MB
• Stars: 425
• Watchers: 6
• Forks: 29
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Autodiff Puzzles

- by [Sasha Rush](http://rush-nlp.com) - [srush_nlp](https://twitter.com/srush_nlp)

**Click here to get started:**

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/srush/Autodiff-Puzzles/blob/main/autodiff_puzzlers.ipynb)

This notebook contains a series of self-contained puzzles for learning about derivatives in tensor libraries. It is the 3rd puzzle set in a series of puzzles about programming for deep learning ([Tensor Puzzles](https://github.com/srush/Tensor-Puzzles), [GPU Puzzles](https://github.com/srush/GPU-Puzzles)).

![image](https://github.com/user-attachments/assets/be04dea0-a15c-4d7d-b2d5-263c171d0f35)

Your goal in these puzzles is to implement the derivatives for each core function. In each case the function takes in a tensor x and returns a tensor f(x), so your job is to compute $\frac{d f(x)_o}{dx_i}$ for all indices $o$ and $i$. If you get discouraged, just remember, you did this in high school (it just had way less indexing).