{"id":32692913,"url":"https://github.com/msmrexe/pytorch-normalizing-flows-2d","last_synced_at":"2026-05-12T23:37:30.518Z","repository":{"id":321569695,"uuid":"1086336586","full_name":"msmrexe/pytorch-normalizing-flows-2d","owner":"msmrexe","description":"A from-scratch implementation of a Normalizing Flow (Real NVP) to model and sample from a 2D 'two-moons' distribution. Allows for GIF visualization of the NF evolution.","archived":false,"fork":false,"pushed_at":"2025-10-30T09:40:28.000Z","size":24,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-30T11:37:26.126Z","etag":null,"topics":["change-of-variables","course-project","density-estimation","generative-models","maximum-likelihood-estimation","normalizing-flows","pytorch","real-nvp","university-project"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/msmrexe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-30T09:27:20.000Z","updated_at":"2025-10-30T09:44:38.000Z","dependencies_parsed_at":"2025-10-30T11:37:27.734Z","dependency_job_id":null,"html_url":"https://github.com/msmrexe/pytorch-normalizing-flows-2d","commit_stats":null,"previous_names":["msmrexe/pytorch-normalizing-flows-2d"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/msmrexe/pytorch-normalizing-flows-2d","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msmrexe%2Fpytorch-normalizing-flows-2d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msmrexe%2Fpytorch-normalizing-flows-2d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msmrexe%2Fpytorch-normalizing-flows-2d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msmrexe%2Fpytorch-normalizing-flows-2d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/msmrexe","download_url":"https://codeload.github.com/msmrexe/pytorch-normalizing-flows-2d/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msmrexe%2Fpytorch-normalizing-flows-2d/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":282166182,"owners_count":26625194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-01T02:00:06.759Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["change-of-variables","course-project","density-estimation","generative-models","maximum-likelihood-estimation","normalizing-flows","pytorch","real-nvp","university-project"],"created_at":"2025-11-01T16:02:10.727Z","updated_at":"2025-11-01T16:04:59.348Z","avatar_url":"https://github.com/msmrexe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Normalizing Flow for 2D Density Estimation\n\nA from-scratch implementation of a Normalizing Flow model, based on Real NVP (Real Non-Volume Preserving) transformations, to learn and sample from a 2D 'two-moons' distribution. This project was developed for the M.S. Generative Models course at Sharif University of Technology.\n\n## Features\n\n* **Real NVP Implementation:** Core logic built using affine coupling layers.\n* **Modular `Flow` Class:** A `nn.Module` that stacks multiple transformations for complex density estimation.\n* **Maximum Likelihood Training:** Optimized by minimizing the negative log-likelihood (NLL) of the data.\n* **Density Visualization:** Generates and saves the learned probability density at each epoch.\n* **GIF Generation:** Automatically creates a GIF visualizing the evolution of the learned density over training.\n* **Data Sampling:** Capable of sampling new data points from the learned distribution.\n\n## Core Concepts \u0026 Techniques\n\n* **Normalizing Flows:** A class of generative models that learn a data distribution $p_X(x)$ by transforming a simple base distribution $p_Z(z)$ (e.g., a Gaussian) through an invertible, differentiable mapping $f: \\mathcal{Z} \\to \\mathcal{X}$.\n* **Change of Variables Formula:** The core principle used for training. The log-likelihood of a data point $x$ is given by the formula:\n\n  $$\\log p_X(x) = \\log p_Z(f(x)) + \\log \\left| \\det J_f(x) \\right|$$\n\n  where $z = f(x)$ is the forward transformation (data to latent) and $J_f(x)$ is its Jacobian.\n  \n* **Coupling Layers (Real NVP):** An efficient invertible transformation $y = f(x)$ where the input $x$ is split into two parts ($x_A, x_B$). One part is left unchanged ($y_A = x_A$), while the other is transformed using a simple affine function (scaling and translation) whose parameters ($s, t$) are generated by a neural network that only takes the unchanged part as input:\n\n  $$y_B = x_B \\odot \\exp(s(x_A)) + t(x_A)$$\n\n  This design results in a triangular Jacobian matrix, making the determinant (and its log) trivial to compute: it is simply the sum of the scaling factors, $\\sum s(x_A)$.\n\n* **Maximum Likelihood Estimation (MLE):** The model is trained by adjusting the parameters of the $s$ and $t$ networks to maximize the log-likelihood of the training data, which is equivalent to minimizing the negative log-likelihood (NLL) loss.\n\n---\n\n## How It Works\n\nThis project learns a complex distribution (two moons) by mapping it to a simple one (a 2D standard Gaussian).\n\n### 1. Core Logic \u0026 Architecture\n\nThe repository implements a `Flow` model composed of a series of `CouplingTransform` layers. Each `CouplingTransform` layer splits the 2D input $x = (x_1, x_2)$ based on an alternating binary mask.\n\n1.  **Transform 1 (Mask `[1, 0]`):**\n    * $x_1$ is **fixed**.\n    * $s_1, t_1 = \\text{NN}_1(x_1)$\n    * $z_1 = x_1$\n    * $z_2 = x_2 \\odot \\exp(s_1) + t_1$\n\n2.  **Transform 2 (Mask `[0, 1]`):**\n    * $z_2$ is **fixed**.\n    * $s_2, t_2 = \\text{NN}_2(z_2)$\n    * $z'_1 = z_1 \\odot \\exp(s_2) + t_2$\n    * $z'_2 = z_2$\n\nThis sequence of invertible transformations ($x \\to z \\to z' \\dots$) maps the complex 'two-moons' data to a 2D standard Gaussian. Training is done by maximizing $\\log p_X(x)$, and sampling is done by drawing $z$ from the Gaussian and applying the inverse of all transforms: $x = f_1^{-1}(f_2^{-1}(\\dots f_N^{-1}(z)))$.\n\n### 2. Key Files \u0026 Functions\n\n* `main.py`: The main executable script. It handles argument parsing, sets up logging, loads the data, initializes the `Flow` model, and runs the training loop. After training, it generates all final visualizations.\n* `src/model.py`: Defines the `Flow` class. This class stacks multiple `CouplingTransform` layers. It implements:\n    * `forward(x)`: Maps data to latent space ($x \\to z$) and returns $z$ and the total $\\log \\det J$.\n    * `inverse(z)`: Maps latent space to data space ($z \\to x$).\n    * `log_prob(x)`: Applies the Change of Variables formula to compute the log-likelihood of the data.\n    * `sample(n)`: Generates $n$ new samples by drawing from the base distribution and applying the `inverse` method.\n* `src/transforms.py`: Defines the `CouplingTransform` class. This class implements the core Real NVP logic, including the $s$ and $t$ neural networks, the `forward` pass, the `inverse` pass, and the log-det-Jacobian calculation.\n* `src/utils.py`: Contains all helper functions for loading the `make_moons` dataset (`load_data`), creating the alternating masks (`create_alternating_masks`), setting up logging (`setup_logging`), and handling all visualizations (`visualize_density`, `plot_comparison`, `create_gif`).\n\n---\n\n## Project Structure\n\n```\npytorch-normalizing-flows-2d/\n├── .gitignore             # Ignores Python caches, envs, and output files\n├── LICENSE                # MIT License file\n├── README.md              # This repository README\n├── main.py                # Main script to train the model and generate outputs\n├── requirements.txt       # Required Python packages\n├── run_flow_model.ipynb   # Jupyter Notebook to run the training and view results\n└── src/\n    ├── __init__.py        # Makes 'src' a Python package\n    ├── model.py           # Contains the main 'Flow' model class\n    ├── transforms.py      # Defines the 'CouplingTransform' layer\n    └── utils.py           # Helper functions for data, logging, and visualization\n```\n\n## How to Use\n\n1.  **Clone the Repository:**\n    ```bash\n    git clone https://github.com/msmrexe/pytorch-normalizing-flows-2d.git\n    cd pytorch-normalizing-flows-2d\n    ```\n\n2.  **Setup the Environment:**\n    It's recommended to use a virtual environment.\n    ```bash\n    python -m venv venv\n    source venv/bin/activate  # On Windows: venv\\Scripts\\activate\n    pip install -r requirements.txt\n    ```\n\n3.  **Run the Training Script:**\n    You can run the model directly from the command line.\n    ```bash\n    python main.py --epochs 150 --lr 0.001 --output_dir outputs\n    ```\n    * `--epochs`: Number of training epochs (e.g., 150).\n    * `--lr`: Learning rate (e.g., 0.001).\n    * `--output_dir`: Directory to save final plots and GIF.\n    * `--skip_frames`: (Optional) Use this flag to disable saving frames every epoch, which speeds up training.\n\n4.  **Example Output / Check Results:**\n    The script will log progress to the console and to `logs/flow.log`. After training, you will find the following files in the `outputs/` directory:\n    * `density_evolution.gif`: An animation of the learned density.\n    * `data_comparison.png`: A scatter plot comparing original and generated data.\n    * `final_density.png`: The final learned probability density.\n\n    Alternatively, you can run the `run_flow_model.ipynb` notebook in Jupyter to execute the training and see the outputs displayed inline.\n\n---\n\n## Author\n\nFeel free to connect or reach out if you have any questions!\n\n* **Maryam Rezaee**\n* **GitHub:** [@msmrexe](https://github.com/msmrexe)\n* **Email:** [ms.maryamrezaee@gmail.com](mailto:ms.maryamrezaee@gmail.com)\n\n---\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for full details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsmrexe%2Fpytorch-normalizing-flows-2d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmsmrexe%2Fpytorch-normalizing-flows-2d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsmrexe%2Fpytorch-normalizing-flows-2d/lists"}