{"id":34851152,"url":"https://github.com/josemanuel22/isl","last_synced_at":"2026-04-23T06:33:27.487Z","repository":{"id":200338899,"uuid":"705298877","full_name":"josemanuel22/ISL","owner":"josemanuel22","description":"Training Implicit Generative Models via an Invariant statistical loss (ISL)","archived":false,"fork":false,"pushed_at":"2024-03-06T16:48:58.000Z","size":23953,"stargazers_count":0,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-04-16T04:49:45.511Z","etag":null,"topics":["aistats","flux","julia","machine-learning","research","statistics"],"latest_commit_sha":null,"homepage":"https://josemanuel22.github.io/ISL/","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/josemanuel22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["josemanuel22"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"polar":null,"custom":null}},"created_at":"2023-10-15T16:20:17.000Z","updated_at":"2024-05-29T22:42:20.933Z","dependencies_parsed_at":"2024-03-06T18:41:28.327Z","dependency_job_id":null,"html_url":"https://github.com/josemanuel22/ISL","commit_stats":null,"previous_names":["josemanuel22/isl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/josemanuel22/ISL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josemanuel22%2FISL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josemanuel22%2FISL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josemanuel22%2FISL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josemanuel22%2FISL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/josemanuel22","download_url":"https://codeload.github.com/josemanuel22/ISL/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/josemanuel22%2FISL/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28035466,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-25T02:00:05.988Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aistats","flux","julia","machine-learning","research","statistics"],"created_at":"2025-12-25T19:17:39.998Z","updated_at":"2025-12-25T19:17:55.943Z","avatar_url":"https://github.com/josemanuel22.png","language":"Julia","funding_links":["https://github.com/sponsors/josemanuel22"],"categories":[],"sub_categories":[],"readme":"# ISL: Training Implicit Generative Models via an Invariant Statistical Loss\r\n\u003c!--[![arXiv](https://img.shields.io/badge/arXiv-2402.16435-b31b1b.svg)](https://arxiv.org/abs/2402.16435) --\u003e\r\n[![Conference](https://img.shields.io/badge/AISTATS-2024-blue)](https://proceedings.mlr.press/v238/frutos24a/frutos24a.pdf)\r\n[![Build Status](https://github.com/josemanuel22/ISL/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/josemanuel22/ISL/actions/workflows/CI.yml?query=branch%3Amain)\r\n[![Documenter: stable](https://img.shields.io/badge/docs-dev-blue.svg)](https://josemanuel22.github.io/ISL/dev/) [![codecov](https://codecov.io/gh/josemanuel22/ISL/graph/badge.svg?token=ZRtCurBd3z)](https://app.codecov.io/gh/josemanuel22/ISL)\r\n[![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle)\r\n\r\nThis repository contains the Julia Flux implementation of the Invariant Statistical Loss (ISL) proposed in the paper [Training Implicit Generative Models via an Invariant Statistical Loss](https://proceedings.mlr.press/v238/frutos24a/frutos24a.pdf), published in the AISTATS 2024 conference.\r\n\r\nPlease, if you use this code, cite the [article](https://proceedings.mlr.press/v238/frutos24a/frutos24a.pdf):\r\n\r\n```\r\n@inproceedings{de2024training,\r\n  title={Training Implicit Generative Models via an Invariant Statistical Loss},\r\n  author={de Frutos, Jos{\\'e} Manuel and Olmos, Pablo and Lopez, Manuel Alberto Vazquez and M{\\'\\i}guez, Joaqu{\\'\\i}n},\r\n  booktitle={International Conference on Artificial Intelligence and Statistics},\r\n  pages={2026--2034},\r\n  year={2024},\r\n  organization={PMLR}\r\n}\r\n``````\r\n\r\n## Overview of the ISL Repository Structure\r\n\r\nThe `ISL` repository is organized into several directories that encapsulate different aspects of the project, ranging from the core source code and custom functionalities to examples demonstrating the application of the project's capabilities, as well as testing frameworks to ensure reliability.\r\n\r\n### Source Code (`src/`)\r\n\r\n- **`CustomLossFunction.jl`**: This file contains implementations of the ISL custom loss function tailored for the models developed within the repository.\r\n  \r\n- **`ISL.jl`**: Serves as the main module file of the repository, this file aggregates and exports the functionalities developed in `CustomLossFunction.jl`.\r\n\r\n### Examples (`examples/`)\r\n\r\n- **`time_series_predictions/`**: This subdirectory showcases how the ISL project's models can be applied to time series prediction tasks. \r\n\r\n- **`Learning1d_distribution/`**: Focuses on the task of learning 1D distributions with the ISL.\r\n\r\n### Testing Framework (`test/`)\r\n\r\n- **`runtests.jl`**: This script is responsible for running automated tests against the `ISL.jl` module.\r\n\r\n## How to install\r\n\r\nTo install ISL, simply use Julia's package manager. The module is not registered so you need to clone the repository and follow the following steps:\r\n\r\n````\r\njulia\u003e push!(LOAD_PATH,pwd()) # You are in the ISL Repository\r\njulia\u003e using ISL\r\n````\r\n\r\nTo reproduce the enviroment for compiling the repository:\r\n````\r\n(@v1.9) pkg\u003e  activate pathToRepository/ISL\r\n````\r\n\r\nIf you want to use any utility subrepository like GAN or DeepAR, make sure it's within your path.\r\n\r\n# Quick Start Guide\r\n\r\nAfter installing the package, you can immediately start experimenting with the examples provided in the `examples/` directory. These examples are designed to help you understand how to apply the package for different statistical learning tasks, including learning 1-D distributions and time series prediction.\r\n\r\n## Learning 1-D distributions\r\n\r\nThe following example demonstrates how to learn a 1-D distribution from a benchmark dataset. It utilizes a simple neural network architecture for both the generator and the discriminator within the context of an Invariant Statistical Learning (ISL) framework.\r\n\r\n```julia\r\n# Example file: examples/Learning1d_distribution/benchmark_unimodal.jl\r\n\r\nusing ISL # Import the ISL module\r\ninclude(\"../utils.jl\")  # Include necessary utilities\r\n\r\n@test_experiments \"N(0,1) to N(23,1)\" begin\r\n    # Define the generator and discriminator networks with ELU activation\r\n    gen = Chain(Dense(1, 7), elu, Dense(7, 13), elu, Dense(13, 7), elu, Dense(7, 1))\r\n\r\n    # Set up the noise and target models\r\n    noise_model = Normal(0.0f0, 1.0f0)\r\n    target_model = Normal(4.0f0, 2.0f0)\r\n    n_samples = 10000   \r\n\r\n    # Configure the hyperparameters for the ISL\r\n    hparams = AutoISLParams(;\r\n        max_k=10, samples=1000, epochs=1000, η=1e-2, transform=noise_model\r\n    )\r\n\r\n    # Prepare the dataset and the loader for training\r\n    train_set = Float32.(rand(target_model, hparams.samples))\r\n    loader = Flux.DataLoader(train_set; batchsize=-1, shuffle=true, partial=false)\r\n\r\n    # Train the model using the ISL\r\n    auto_invariant_statistical_loss(gen, loader, hparams)\r\n\r\n    # Visualize the learning results\r\n    plot_global(\r\n        x -\u003e quantile.(-target_model, cdf(noise_model, x)),\r\n        noise_model,\r\n        target_model,\r\n        gen,\r\n        n_samples,\r\n        (-3:0.1:3),\r\n        (-2:0.1:10),\r\n    )\r\nend\r\n```\r\n\r\n![Example Image](./docs/src/imgs/readme_images_1.png)\r\n\r\n\r\n## Time Series\r\n\r\nThis section showcases two examples: predicting a univariate time series with an AutoRegressive model and forecasting electricity consumption. These examples illustrate the application of ISL for time series analysis.\r\n\r\n- Example 1: AutoRegressive Model Prediction\r\n\r\n```julia\r\n@test_experiments \"testing AutoRegressive Model 1\" begin\r\n    # --- Model Parameters and Data Generation ---\r\n\r\n    # Define AR model parameters\r\n    ar_hparams = ARParams(;\r\n        ϕ=[0.5f0, 0.3f0, 0.2f0],  # Autoregressive coefficients\r\n        x₁=rand(Normal(0.0f0, 1.0f0)),  # Initial value from a Normal distribution\r\n        proclen=2000,  # Length of the process\r\n        noise=Normal(0.0f0, 0.2f0),  # Noise in the AR process\r\n    )\r\n\r\n    # Define the recurrent and generative models\r\n    recurrent_model = Chain(RNN(1 =\u003e 10, relu), RNN(10 =\u003e 10, relu))\r\n    generative_model = Chain(Dense(11, 16, relu), Dense(16, 1, identity))\r\n\r\n    # Generate training and testing data\r\n    n_series = 200  # Number of series to generate\r\n    loaderXtrain, loaderYtrain, loaderXtest, loaderYtest = generate_batch_train_test_data(\r\n        n_series, ar_hparams\r\n    )\r\n\r\n    # --- Training Configuration ---\r\n\r\n    # Define hyperparameters for time series prediction\r\n    ts_hparams = HyperParamsTS(;\r\n        seed=1234,\r\n        η=1e-3,  # Learning rate\r\n        epochs=n_series,\r\n        window_size=1000,  # Size of the window for prediction\r\n        K=10,  # Hyperparameter K (if it has a specific use, add a comment)\r\n    )\r\n\r\n    # Train model and calculate loss\r\n    loss = ts_invariant_statistical_loss_one_step_prediction(\r\n        recurrent_model, generative_model, loaderXtrain, loaderYtrain, ts_hparams\r\n    )\r\n\r\n    # --- Visualization ---\r\n\r\n    # Plotting the time series prediction\r\n    plot_univariate_ts_prediction(\r\n        recurrent_model,\r\n        generative_model,\r\n        collect(loaderXtrain)[2],  # Extract the first batch for plotting\r\n        collect(loaderXtest)[2],  # Extract the first batch for plotting\r\n        ts_hparams;\r\n        n_average=1000,  # Number of predictions to average\r\n    )\r\nend\r\n```\r\n\r\n![Example Image](./docs/src/imgs/readme_images_2.png)\r\n\r\n- Example 2: Electricity Consumption Forecasting\r\n\r\n```julia\r\n@test_experiments \"testing electricity-c\" begin\r\n     # Data loading and preprocessing\r\n    csv_file_path = \"examples/time_series_predictions/data/LD2011_2014.txt\"\r\n\r\n    # Specify custom types for certain columns to ensure proper data handling\r\n    column_types = Dict(\r\n        \"MT_005\" =\u003e Float32,\r\n        \"MT_006\" =\u003e Float32,\r\n        \"MT_007\" =\u003e Float32,\r\n        \"MT_008\" =\u003e Float32,\r\n        \"MT_168\" =\u003e Float32,\r\n        \"MT_333\" =\u003e Float32,\r\n        \"MT_334\" =\u003e Float32,\r\n        \"MT_335\" =\u003e Float32,\r\n        \"MT_336\" =\u003e Float32,\r\n        \"MT_338\" =\u003e Float32,\r\n    )\r\n\r\n    df = DataFrame(\r\n        CSV.File(csv_file_path; delim=';', header=true, decimal=',', types=column_types)\r\n    )\r\n\r\n    # Hyperparameters setup\r\n    hparams = HyperParamsTS(; seed=1234, η=1e-2, epochs=2000, window_size=1000, K=10)\r\n\r\n    # Model definition\r\n    rec = Chain(RNN(1 =\u003e 3, relu), RNN(3 =\u003e 3, relu))\r\n    gen = Chain(Dense(4, 10, identity), Dense(10, 1, identity))\r\n\r\n    # Training and testing data setup\r\n    start, num_training_data, num_test_data = 35040, 35040, 1000\r\n\r\n    # Aggregate time series data for training and testing\r\n    selected_columns = [\r\n        \"MT_005\",\r\n        \"MT_006\",\r\n        \"MT_007\",\r\n        \"MT_008\",\r\n        \"MT_168\",\r\n        \"MT_333\",\r\n        \"MT_334\",\r\n        \"MT_335\",\r\n        \"MT_336\",\r\n        \"MT_338\",\r\n    ]\r\n\r\n    loader_xtrain, loader_ytrain, loader_xtest = aggregate_time_series_data(\r\n        df, selected_columns, start, num_training_data, num_test_data\r\n    )\r\n\r\n    # Model training\r\n    losses = []\r\n    @showprogress for _ in 1:100\r\n        loss = ts_invariant_statistical_loss(\r\n            rec, gen, loader_xtrain, loader_ytrain, hparams\r\n        )\r\n        append!(losses, loss)\r\n    end\r\n\r\n    # Forecasting\r\n    τ = 24\r\n    xtrain = collect(loaderXtrain)[1]\r\n    xtest = collect(loaderXtest)[1]\r\n    prediction, stds = ts_forecast(\r\n        rec, gen, xtrain, xtest, τ; n_average=1000, noise_model=Normal(0.0f0, 1.0f0)\r\n    )\r\n    plot(prediction[1:τ])\r\n    plot!(xtest[1:τ])\r\nend\r\n```\r\n\r\n- `Prediction τ = 24 (1 day)`\r\n\r\n![Example Image](./docs/src/imgs/user008c-1.png)\r\n\r\n- `Prediction τ = 168 (7 days)`\r\n\r\n![Example Image](./docs/src/imgs/user008long-1.png)\r\n\r\n# Contributors\r\n\r\n[José Manuel de Frutos](https://josemanuel22.github.io/)\r\n\r\nFor further information: jofrutos@ing.uc3m.es\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosemanuel22%2Fisl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjosemanuel22%2Fisl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosemanuel22%2Fisl/lists"}