https://github.com/lnstadrum/sigmoid_like_tf_op

A TensorFlow C++ extension implementing a piecewise-linear approximation of sigmoid activation function used in Beatmup.
https://github.com/lnstadrum/sigmoid_like_tf_op

beatmup sigmoid-activation tensorflow tensorflow-op

Last synced: 7 days ago
JSON representation

A TensorFlow C++ extension implementing a piecewise-linear approximation of sigmoid activation function used in Beatmup.

• Host: GitHub
• URL: https://github.com/lnstadrum/sigmoid_like_tf_op
• Owner: lnstadrum
• Created: 2021-01-23T20:43:17.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2021-04-20T22:30:00.000Z (over 3 years ago)
• Last Synced: 2024-11-07T21:47:23.448Z (about 2 months ago)
• Topics: beatmup, sigmoid-activation, tensorflow, tensorflow-op
• Language: C++
• Homepage:
• Size: 41 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
This repository contains a TensorFlow extension implementing a piecewise-linear approximation of sigmoid activation function used in [Beatmup](https://github.com/lnstadrum/beatmup).

 * This function shrinks the activation signal into 0..1 range, enabling efficient fixed-point storage and reducing the memory bandwidth in inference.

 * Compared to the original sigmoid, this function is very efficient to compute in both forward and backward senses and does not suffer from vanishing gradient issue.

This activation function is featured in a [dog image classifier running on Raspberry Pi GPU](https://lnstadrum.medium.com/towards-gpu-accelerated-image-classification-on-low-end-hardware-ec592e125ad9).

![Activation functions](graph.png)