https://github.com/satyammittal/smai-project-sparsemax
Project Regarding Analysis and Implementation of https://arxiv.org/abs/1602.02068
https://github.com/satyammittal/smai-project-sparsemax
activation-functions machine-learning softmax sparsemax
Last synced: about 1 year ago
JSON representation
Project Regarding Analysis and Implementation of https://arxiv.org/abs/1602.02068
- Host: GitHub
- URL: https://github.com/satyammittal/smai-project-sparsemax
- Owner: satyammittal
- Created: 2018-04-29T13:17:48.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-04-30T16:08:21.000Z (about 8 years ago)
- Last Synced: 2025-02-12T01:54:39.133Z (over 1 year ago)
- Topics: activation-functions, machine-learning, softmax, sparsemax
- Language: Python
- Homepage:
- Size: 13.2 MB
- Stars: 1
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
SMAI Project
=============
Research Paper: https://arxiv.org/abs/1602.02068
The paper focuses on sparsemax, a new activation function which is similar to softmax, but outputs sparse probabilities.
It has properties similar to the softmax and it’s Jacobian can be efficiently computed, enabling its use in a neural network trained with backpropagation.
Then, a new smooth and convex loss function which is the analogue of the logistic loss is defined for sparsemax.
Promising empirical results are obtained in multi-label classification problems and in attention-based neural networks for natural language inference but with a selective, more compact, attention focus.
Implementation
------------
Implementation is available in "Code" Directory
Presentation
------------
It is available in "Presentation" Directory.