https://github.com/wecarsoniv/beta-divergence-metrics
PyTorch implementations of the beta divergence loss.
https://github.com/wecarsoniv/beta-divergence-metrics
beta-divergence distance-measures distance-metric distance-metrics divergence divergences itakura-saito-divergence kl-divergence kullback-leibler-divergence loss loss-functions mean-square-error mean-squared-error nmf nmf-decomposition non-negative-matrix-factorization numpy objective-functions pytorch torch
Last synced: 5 months ago
JSON representation
PyTorch implementations of the beta divergence loss.
- Host: GitHub
- URL: https://github.com/wecarsoniv/beta-divergence-metrics
- Owner: wecarsoniv
- License: bsd-3-clause
- Created: 2022-01-05T22:52:58.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2022-01-31T18:16:21.000Z (over 4 years ago)
- Last Synced: 2025-10-26T15:03:05.856Z (7 months ago)
- Topics: beta-divergence, distance-measures, distance-metric, distance-metrics, divergence, divergences, itakura-saito-divergence, kl-divergence, kullback-leibler-divergence, loss, loss-functions, mean-square-error, mean-squared-error, nmf, nmf-decomposition, non-negative-matrix-factorization, numpy, objective-functions, pytorch, torch
- Language: Python
- Homepage:
- Size: 360 KB
- Stars: 10
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Beta-Divergence Loss Implementations
This repository contains code for Python implementations of the beta-divergence loss, including implementations compatible [NumPy](https://numpy.org/) and [PyTorch](https://pytorch.org/).
## Dependencies
This library is written in Python, and requires Python (with recommended version >= 3.9) to run. In addition to a working PyTorch installation, this library relies on the following libraries and recommended version numbers:
* [Python](https://www.python.org/) >= 3.9
* [NumPy](https://numpy.org/) >= 1.22.0
* [SciPy](https://www.scipy.org/) >= 1.7.3
## Installation
To install the latest stable release, use [pip](https://pip.pypa.io/en/stable/). Use the following command to install:
$ pip install beta-divergence-metrics
## Usage
The [`numpybd.loss`](https://github.com/wecarsoniv/beta-divergence-metrics/blob/main/src/numpybd/loss.py) module contains two beta-divergence function implementations compatible with NumPy and NumPy arrays: one general beta-divergence between two arrays, and a beta-divergence implementation specific to non-negative matrix factorization (NMF). Similarly [`torchbd.loss`](https://github.com/wecarsoniv/beta-divergence-metrics/blob/main/src/torchbd/loss.py) module contains two beta-divergence class implementations compatible with PyTorch and [PyTorch tensors](https://pytorch.org/tutorials/beginner/introyt/tensors_deeper_tutorial.html). Beta-divergence implementations can be imported as follows:
```python
# Import beta-divergence loss implementations
from numpybd.loss import *
from torchbd.loss import *
```
### Beta-divergence between two NumPy arrays
To calculate the beta-divergence between a NumPy array `a` and a target or reference array `b`, use the `beta_div` loss function. The `beta_div` loss function can be used as follows:
```python
# Calculate beta-divergence loss between array a and target array b
loss_val = beta_div(beta=0, reduction='mean')
```
### Beta-divergence between two PyTorch tensors
To calculate the beta-divergence between tensor `a` and a target or reference tensor `b`, use the `BetaDivLoss` loss function. The `BetaDivLoss` loss function can be instantiated and used as follows:
```python
# Instantiate beta-divergence loss object
loss_func = BetaDivLoss(beta=0, reduction='mean')
# Calculate beta-divergence loss between tensor a and target tensor b
loss_val = loss_func(input=a, target=b)
```
### NMF beta-divergence between NumPy array of data and data reconstruction
To calculate the NMF-specific beta-divergence between a NumPy array of data matrix `X` and the product of a scores matrix `H` and a components matrix `W`, use the `nmf_beta_div` loss function. The `nmf_beta_div` loss function can beused as follows:
```python
# Calculate beta-divergence loss between data matrix X (target or
# reference matrix) and matrix product of H and W
loss_val = nmf_beta_div(X=X, H=H, W=W, beta=0, reduction='mean')
```
### NMF beta-divergence between PyTorch tensor of data and data reconstruction
To calculate the NMF-specific beta-divergence between a PyTorch tensor of data matrix `X` and the matrix product of a scores matrix `H` and a components matrix `W`, use the `NMFBetaDivLoss` loss class function. The `NMFBetaDivLoss` loss function can be instantiated and used as follows:
```python
# Instantiate NMF beta-divergence loss object
loss_func = NMFBetaDivLoss(beta=0, reduction='mean')
# Calculate beta-divergence loss between data matrix X (target or
# reference matrix) and matrix product of H and W
loss_val = loss_func(X=X, H=H, W=W)
```
### Choosing beta value
When instantiating beta-divergence loss objects, the value of beta should be chosen depending on data type and application. For NMF applications, a beta value of 0 (Itakura-Saito divergence) is recommemded. Integer values of beta correspond to the following divergences and loss functions:
* beta = 0: [Itakura-Saito divergence](https://en.wikipedia.org/wiki/Itakura-Saito_distance)
* beta = 1: [Kullback-Leibler divergence](https://en.wikipedia.org/wiki/Kullback-Leibler_divergence)
* beta = 2: [mean-squared error](https://en.wikipedia.org/wiki/Mean_squared_error)
## Issue Tracking and Reports
Please use the [GitHub issue tracker](https://github.com/wecarsoniv/beta-divergence-metrics/issues) associated with this repository for issue tracking, filing bug reports, and asking general questions about the package or project.