https://github.com/joshuabillson/mixturedensitynetworks.jl
A simple interface for defining, training, and deploying MDNs.
https://github.com/joshuabillson/mixturedensitynetworks.jl
data-science deep-learning julia machine-learning mdns mixture-density-networks neural-networks
Last synced: 8 months ago
JSON representation
A simple interface for defining, training, and deploying MDNs.
- Host: GitHub
- URL: https://github.com/joshuabillson/mixturedensitynetworks.jl
- Owner: JoshuaBillson
- License: mit
- Created: 2023-03-30T23:26:37.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-07-15T22:44:09.000Z (about 2 years ago)
- Last Synced: 2025-02-20T23:02:57.519Z (8 months ago)
- Topics: data-science, deep-learning, julia, machine-learning, mdns, mixture-density-networks, neural-networks
- Language: Julia
- Homepage: https://joshuabillson.github.io/MixtureDensityNetworks.jl/
- Size: 724 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MixtureDensityNetworks
[](https://JoshuaBillson.github.io/MixtureDensityNetworks.jl/stable/)
[](https://JoshuaBillson.github.io/MixtureDensityNetworks.jl/dev/)
[](https://github.com/JoshuaBillson/MixtureDensityNetworks.jl/actions/workflows/CI.yml?query=branch%3Amain)
[](https://codecov.io/gh/JoshuaBillson/MixtureDensityNetworks.jl)This package provides a simple interface for defining, training, and deploying Mixture Density Networks (MDNs). MDNs were first proposed by [Bishop (1994)](https://publications.aston.ac.uk/id/eprint/373/1/NCRG_94_004.pdf). We can think of an MDN as a specialized type of Artificial Neural Network (ANN), which takes some features `X` and returns a distribution over the labels `Y` under a Gaussian Mixture Model (GMM). Unlike an ANN, MDNs maintain the full conditional distribution P(Y|X). This makes them particularly well-suited for situations where we want to maintain some measure of the uncertainty in our predictions. Moreover, because GMMs can represent multimodal distributions, MDNs are capable of modelling one-to-many relationships, which occurs when each input `X` can be associated with more than one output `Y`.

# MLJ Compatibility
This package implements the interface specified by [MLJModelInterface](https://github.com/JuliaAI/MLJModelInterface.jl) and is thus fully compatible
with the MLJ ecosystem. Below is an example demonstrating the use of this package in conjunction with MLJ.# Example (Native Interface)
```julia
using Flux, MixtureDensityNetworks, Distributions, CairoMakie, Logging, TerminalLoggersconst n_samples = 1000
const epochs = 1000
const batchsize = 128
const mixtures = 8
const layers = [128, 128]function main()
# Generate Data
X, Y = generate_data(n_samples)# Create Model
model = MixtureDensityNetwork(1, 1, layers, mixtures)# Fit Model
model, report = MixtureDensityNetworks.fit!(model, X, Y; epochs=epochs, opt=Flux.Adam(1e-3), batchsize=batchsize)# Plot Learning Curve
fig, _, _ = lines(1:epochs, report.learning_curve, axis=(;xlabel="Epochs", ylabel="Loss"))
save("LearningCurve.png", fig)# Plot Learned Distribution
Ŷ = model(X)
fig, ax, plt = scatter(X[1,:], rand.(Ŷ), markersize=4, label="Predicted Distribution")
scatter!(ax, X[1,:], Y[1,:], markersize=3, label="True Distribution")
axislegend(ax, position=:lt)
save("PredictedDistribution.png", fig)# Plot Conditional Distribution
cond = model(reshape([-2.1], (1,1)))[1]
fig = Figure(resolution=(1000, 500))
density(fig[1,1], rand(cond, 10000), npoints=10000)
save("ConditionalDistribution.png", fig)
endmain()
```# Example (MLJ Interface)
```julia
using MixtureDensityNetworks, Distributions, CairoMakie, MLJconst n_samples = 1000
const epochs = 500
const batchsize = 128
const mixtures = 8
const layers = [128, 128]function main()
# Generate Data
X, Y = generate_data(n_samples)# Create Model
mach = MLJ.machine(MDN(epochs=epochs, mixtures=mixtures, layers=layers, batchsize=batchsize), MLJ.table(X'), Y[1,:])# Fit Model on Training Data, Then Evaluate on Test
@info "Evaluating..."
evaluation = MLJ.evaluate!(
mach,
resampling=Holdout(shuffle=true),
measure=[rsq, rmse, mae, mape],
operation=MLJ.predict_mean,
verbosity=2 # Need to set verbosity=2 to show training progress during evaluation
)
names = ["R²", "RMSE", "MAE", "MAPE"]
metrics = round.(evaluation.measurement, digits=3)
@info "Metrics: " * join(["$name: $metric" for (name, metric) in zip(names, metrics)], ", ")# Fit Model on Entire Dataset
@info "Training..."
MLJ.fit!(mach)# Plot Learning Curve
fig, _, _ = lines(1:epochs, MLJ.training_losses(mach), axis=(;xlabel="Epochs", ylabel="Loss"))
save("LearningCurve.png", fig)# Plot Learned Distribution
Ŷ = MLJ.predict(mach) .|> rand
fig, ax, plt = scatter(X[1,:], Ŷ, markersize=4, label="Predicted Distribution")
scatter!(ax, X[1,:], Y[1,:], markersize=3, label="True Distribution")
axislegend(ax, position=:lt)
save("PredictedDistribution.png", fig)# Plot Conditional Distribution
cond = MLJ.predict(mach, MLJ.table(reshape([-2.1], (1,1))))[1]
fig = Figure(resolution=(1000, 500))
density(fig[1,1], rand(cond, 10000), npoints=10000)
save("ConditionalDistribution.png", fig)
endmain()
```