{"id":13717381,"url":"https://github.com/masa-su/pixyz","last_synced_at":"2026-04-02T03:50:45.672Z","repository":{"id":45427329,"uuid":"135975723","full_name":"masa-su/pixyz","owner":"masa-su","description":"A library for developing deep generative models in a more concise, intuitive and extendable way","archived":false,"fork":false,"pushed_at":"2024-07-29T06:41:09.000Z","size":7392,"stargazers_count":491,"open_issues_count":14,"forks_count":41,"subscribers_count":44,"default_branch":"main","last_synced_at":"2024-11-08T03:41:20.184Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pixyz.io","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/masa-su.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-06-04T05:41:09.000Z","updated_at":"2024-10-15T08:42:22.000Z","dependencies_parsed_at":"2024-01-25T19:12:53.543Z","dependency_job_id":null,"html_url":"https://github.com/masa-su/pixyz","commit_stats":{"total_commits":723,"total_committers":12,"mean_commits":60.25,"dds":0.2835408022130014,"last_synced_commit":"a9baf067730035d03351476f5c2e1e43016808ce"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masa-su%2Fpixyz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masa-su%2Fpixyz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masa-su%2Fpixyz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masa-su%2Fpixyz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/masa-su","download_url":"https://codeload.github.com/masa-su/pixyz/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224573515,"owners_count":17333804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T00:01:21.475Z","updated_at":"2025-12-15T17:22:08.512Z","avatar_url":"https://github.com/masa-su.png","language":"Jupyter Notebook","funding_links":[],"categories":["Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Pytorch \u0026 related libraries","Jupyter Notebook"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"readme":"# Pixyz: A library for developing deep generative models\n\n\u003cimg src=\"https://user-images.githubusercontent.com/11865486/58864169-3706a980-86ef-11e9-82f4-18bb0275271b.png\" width=\"800px\"\u003e\n\n[![pypi](https://img.shields.io/pypi/v/pixyz.svg)](https://pypi.python.org/pypi/pixyz)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python Version](https://img.shields.io/badge/python-3.6%20%7C%203.7%20%7C%203.8%20-blue)](https://github.com/masa-su/pixyz)\n[![Pytorch Version](https://img.shields.io/badge/pytorch-1.0-yellow.svg)](https://github.com/masa-su/pixyz)\n[![Read the Docs](https://readthedocs.org/projects/pixyz/badge/?version=latest)](http://docs.pixyz.io)\n[![TravisCI](https://travis-ci.org/masa-su/pixyz.svg?branch=master)](https://travis-ci.org/github/masa-su/pixyz)\n[![Downloads](https://pepy.tech/badge/pixyz)](https://pepy.tech/project/pixyz)\n[![Docker Pulls](https://img.shields.io/docker/pulls/kenoharada/pixyz.svg)](https://hub.docker.com/r/kenoharada/pixyz)\n\n[Docs](https://docs.pixyz.io) | [Examples](https://github.com/masa-su/pixyz/tree/master/examples) | [Pixyzoo](https://github.com/masa-su/pixyzoo)\n\n- [What is Pixyz?](#what-is-pixyz)\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n- [More information](#more-information)\n- [Citation](#citation)\n- [Acknowledgements](#acknowledgements)\n\n## What is Pixyz?\n[**Pixyz**](https://github.com/masa-su/pixyz) is a high-level deep generative modeling library, based on [PyTorch](https://pytorch.org/). It is developed with a focus on enabling easy implementation of various deep generative models.\n\nRecently, many papers about deep generative models have been published. However, its reproduction becomes a hard task, for both specialists and practitioners, because such recent models become more complex and there are no unified tools that bridge mathematical formulation of them and implementation. The vision of our library is to enable both specialists and practitioners to implement such complex deep generative models by **just as if writing the formulas provided in these papers**.\n\nOur library supports the following deep generative models.\n\n* Explicit models (likelihood-based)\n  * Variational autoencoders (variational inference)\n  * Flow-based models\n  * Autoregressive generative models (note: not implemented yet)\n* Implicit models\n  * Generative adversarial networks\n  \nMoreover, Pixyz enables you to implement these different models **in the same framework** and **in combination with each other**.\n\nThe overview of Pixyz is as follows. Each API will be discussed below.\n\u003cimg src=\"https://user-images.githubusercontent.com/11865486/58321994-a3b1b680-7e5a-11e9-89dd-334086a89525.png\" width=\"600px\"\u003e\n\n**Note**: Since this library is under development, there are possibilities to have some bugs.\n\n## Installation\n\nPixyz can be installed by using `pip`.\n```\n$ pip install pixyz\n```\n\nIf installing from source code, execute the following commands.\n```\n$ git clone https://github.com/masa-su/pixyz.git\n$ pip install -e pixyz\n```\n\n\nYou can also install pixyz and PyTorch environment through Docker Hub\n```\n# pull docker image from https://hub.docker.com/r/kenoharada/pixyz\n$ docker pull kenoharada/pixyz:v0.3.0_python_3.7.7_pytorch_1.6.0_cuda_10.1\n\n# Run pixyz environment\n$ docker run --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=0 --rm -it kenoharada/pixyz:v0.3.0_python_3.7.7_pytorch_1.6.0_cuda_10.1\n```\n\n## Quick Start\n\nHere, we consider to implement a variational auto-encoder (VAE) which is one of the most well-known deep generative models. VAE is composed of a inference model\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?q_%7B%5Cphi%7D%28z%7Cx%29\" alt=\"q_{\\phi}(z|x)\"/\u003e\nand a generative model\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p_%7B%5Ctheta%7D%28x%2Cz%29%3Dp_%7B%5Ctheta%7D%28x%7Cz%29p%28z%29\" alt=\"p_{\\theta}(x,z)=p_{\\theta}(x|z)p(z)\"/\u003e\n , each of which is defined by DNN, and this loss function (negative ELBO) is as follows.\n\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?\\mathcal{L}(x;\\phi,\\theta)=-E_{q_{\\phi}(z|x)}\\left[\\log{p_{\\theta}(x|z)}\\right]+D_{KL}\\left[q_{\\phi}(z|x)||p_{prior}(z)\\right]\" alt=\"\\mathcal{L}(x;\\phi,\\theta)=-E_{q_{\\phi}(z|x)}\\left[\\log{p_{\\theta}(x|z)}\\right]+D_{KL}\\left[q_{\\phi}(z|x)||p_{prior}(z)\\right]\"/\u003e (1)\n\nIn Pixyz, deep generative models are implemented in the following three steps:\n1. [Define distributions(Distribution API）](#1-define-distributionsdistribution-api)\n2. [Set the loss function of a model(Loss API)](#2-set-the-loss-function-of-a-modelloss-api)\n3. [Train the model(Model API)](#3-train-the-modelmodel-api)\n\n### 1. Define distributions(Distribution API)\nFirst, we need to define two distributions (\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?q_%7B%5Cphi%7D%28z%7Cx%29\" alt=\"q_{\\phi}(z|x)\"/\u003e\n,\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p_{\\theta}(x|z)\" alt=\"p_{\\theta}(x|z)\"/\u003e\n) with DNNs. In Pixyz, you can do this by building DNN modules just as you do in PyTorch. The main difference is that you should inherit the `pixyz.distributions.*` class (**Distribution API**), instead of `torch.nn.Module` .\n\nFor example, \n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p_{\\theta}(x|z)\" alt=\"p_{\\theta}(x|z)\"/\u003e\n(Bernoulli) \nand\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?q_%7B%5Cphi%7D%28z%7Cx%29\" alt=\"q_{\\phi}(z|x)\"/\u003e\n(normal) are implemented as follows.\n\n```python\n\u003e\u003e\u003e from pixyz.distributions import Bernoulli, Normal\n\u003e\u003e\u003e # inference model (encoder) q(z|x)\n\u003e\u003e\u003e class Inference(Normal):\n...     def __init__(self):\n...         super(Inference, self).__init__(var=[\"z\"],cond_var=[\"x\"],name=\"q\")  # var: variables of this distribution, cond_var: coditional variables.\n...         self.fc1 = nn.Linear(784, 512)\n...         self.fc21 = nn.Linear(512, 64)\n...         self.fc22 = nn.Linear(512, 64)\n... \n...     def forward(self, x):  # the name of this argument should be same as cond_var.\n...         h = F.relu(self.fc1(x))\n...         return {\"loc\": self.fc21(h),\n...                 \"scale\": F.softplus(self.fc22(h))}  # return parameters of the normal distribution\n... \n\u003e\u003e\u003e # generative model (decoder) p(x|z)    \n\u003e\u003e\u003e class Generator(Bernoulli):\n...     def __init__(self):\n...         super(Generator, self).__init__(var=[\"x\"], cond_var=[\"z\"], name=\"p\")\n...         self.fc1 = nn.Linear(64, 512)\n...         self.fc2 = nn.Linear(512, 128)\n... \n...     def forward(self, z):  # the name of this argument should be same as cond_var.\n...         h = F.relu(self.fc1(z))\n...         return {\"probs\": F.sigmoid(self.fc2(h))}    # return a parameter of the Bernoulli distribution\n```\nOnce defined, you can create instances of these classes.\n```python\n\u003e\u003e\u003e p = Generator()\n\u003e\u003e\u003e q = Inference()\n```\n\nIn VAE,\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p(z)\" alt=\"p(z)\"/\u003e\n, a prior of the generative model,  is usually defined as the standard normal distribution, without using DNNs. \nSuch an instance can be created from `pixyz.distributions.*` as\n```python\n\u003e\u003e\u003e prior = Normal(loc=torch.tensor(0.), scale=torch.tensor(1.),\n...                var=[\"z\"], features_shape=[64], name=\"p_prior\")\n```\n\nIf you want to find out what kind of distribution each instance defines and what modules (the network architecture) define it, just `print` them.\n```python\n\u003e\u003e\u003e print(p)\nDistribution:\n  p(x|z)\nNetwork architecture:\n  Generator(\n    name=p, distribution_name=Bernoulli,\n    var=['x'], cond_var=['z'], input_var=['z'], features_shape=torch.Size([])\n    (fc1): Linear(in_features=64, out_features=512, bias=True)\n    (fc2): Linear(in_features=512, out_features=512, bias=True)\n    (fc3): Linear(in_features=512, out_features=784, bias=True)\n  )\n```\nIf you are working on the iPython environment, you can use `print_latex` to display them in the LaTeX compiled format.\n\n![p](https://user-images.githubusercontent.com/11865486/59156055-1c0dae00-8ad0-11e9-9eac-5b9938904a0d.png)\n\nConveniently, each distribution instance can **perform sampling** over given samples, regardless of the form of the internal DNN modules. \n```python\n\u003e\u003e\u003e samples_z = prior.sample(batch_n=1)\n\u003e\u003e\u003e print(samples_z)\n{'z': tensor([[ 0.6084,  1.4716,  0.6413,  1.3184, -0.8930,  0.0603,  1.2254,  0.5910, ..., 0.8389]])}\n\u003e\u003e\u003e samples = p.sample(samples_z)\n\u003e\u003e\u003e print(samples)\n{'z': tensor([[ 1.5377,  0.4713,  0.0354,  0.5013,  1.2584,  0.8908,  0.6323,  1.0844, ..., -0.7603]]),\n 'x': tensor([[0., 1., 0., 1., 0., 0., 1., 1., 0., 0., 1., 1., 1., 1., ..., 0.]])}\n```\nAs in this example, samples are represented in dictionary forms in which the keys correspond to random variable names and the values are their realized values.\n\nMoreover, the instance of joint distribution\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p_{\\theta}(x,z)=p_{\\theta}(x|z)p(z)\" alt=\"p_{\\theta}(x,z)=p_{\\theta}(x|z)p(z)\"/\u003e\ncan be created by **the product of distribution instances**. \n```python\n\u003e\u003e\u003e p_joint = p * prior\n```\n\nThis instance can be checked as\n```python\n\u003e\u003e\u003e print(p_joint)\nDistribution:\n  p(x,z) = p(x|z)p_{prior}(z)\nNetwork architecture:\n  Normal(\n    name=p_{prior}, distribution_name=Normal,\n    var=['z'], cond_var=[], input_var=[], features_shape=torch.Size([64])\n    (loc): torch.Size([1, 64])\n    (scale): torch.Size([1, 64])\n  )\n  Generator(\n    name=p, distribution_name=Bernoulli,\n    var=['x'], cond_var=['z'], input_var=['z'], features_shape=torch.Size([])\n    (fc1): Linear(in_features=64, out_features=512, bias=True)\n    (fc2): Linear(in_features=512, out_features=512, bias=True)\n    (fc3): Linear(in_features=512, out_features=784, bias=True)\n  )\n```\n![p_joint](https://user-images.githubusercontent.com/11865486/59156030-d81aa900-8acf-11e9-8b8a-ef2d944722b2.png)\n\nAlso, it can perform sampling in the same way. \n```python\n\u003e\u003e\u003e p_joint.sample(batch_n=1)\n{'z': tensor([[ 1.5377,  0.4713,  0.0354,  0.5013,  1.2584,  0.8908,  0.6323,  1.0844, ..., -0.7603]]),\n 'x': tensor([[0., 1., 0., 1., 0., 0., 1., 1., 0., 0., 1., 1., 1., 1., ..., 0.]])}\n```\nBy constructing the joint distribution in this way, you can easily implement **more complicated generative models**.\n\n### 2. Set the loss function of a model(Loss API)\nNext, we set the objective (loss) function of the model with defined distributions.\n\n**Loss API** (`pixyz.losses.*`) enables you to define such loss function as if just writing mathematic formulas. The loss function of VAE (Eq.(1)) can easily be converted to the code style  as follows.\n```python\n\u003e\u003e\u003e from pixyz.losses import KullbackLeibler, LogProb, Expectation as E\n\u003e\u003e\u003e reconst = -E(q, LogProb(p)) # the reconstruction loss (it can also be written as `-p.log_prob().expectation()`)\n\u003e\u003e\u003e kl = KullbackLeibler(q,prior) # Kullback-Leibler divergence\n\u003e\u003e\u003e loss_cls = (kl + reconst).mean()\n```\n\nLike Distribution API, you can check the formula of the loss function by printing.\n```python\n\u003e\u003e\u003e print(loss_cls)\nmean \\left(D_{KL} \\left[q(z|x)||p_{prior}(z) \\right] - \\mathbb{E}_{q(z|x)} \\left[\\log p(x|z) \\right] \\right) \n```\n![loss](https://user-images.githubusercontent.com/11865486/59156066-3f385d80-8ad0-11e9-9604-ee78a5dd7407.png)\n\nWhen evaluating this loss function given data, use the `eval` method.\n```python\n\u003e\u003e\u003e loss_tensor = loss_cls.eval({\"x\": x_tensor}) # x_tensor: input data\n\u003e\u003e\u003e print(loss_tensor)\ntensor(1.00000e+05 * 1.2587)\n```\n### 3. Train the model(Model API)\nFinally, Model API (`pixyz.models.Model`) can train the loss function given the optimizer, distributions to train, and training data.\n```python\n\u003e\u003e\u003e from pixyz.models import Model\n\u003e\u003e\u003e from torch import optim\n\u003e\u003e\u003e model = Model(loss_cls, distributions=[p, q],\n...               optimizer=optim.Adam, optimizer_params={\"lr\":1e-3}) # initialize a model\n\u003e\u003e\u003e train_loss = model.train({\"x\": x_tensor}) # train the model given training data (x_tensor) \n```\nAfter training the model, you can perform generation and inference on the model by sampling from\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?p_{\\theta}(x,z)\" alt=\"p_{\\theta}(x,z)\"/\u003e\nand\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?q_%7B%5Cphi%7D%28z%7Cx%29\" alt=\"q_{\\phi}(z|x)\"/\u003e\n, respectively.\n\n## More information\nThese frameworks of Pixyz allow the implementation of more complex deep generative models.\nSee [sample codes](https://github.com/masa-su/pixyz/tree/master/examples) and the [pixyzoo](https://github.com/masa-su/pixyzoo) repository as examples.\n\nFor more detailed usage, please check the [Pixyz documentation](https://docs.pixyz.io).\n\nIf you encounter some problems in using Pixyz, please let us know.\n\n## Citation\n```\n@misc{suzuki2023pixyz,\n      title={Pixyz: a Python library for developing deep generative models}, \n      author={Masahiro Suzuki, Takaaki Kaneko and Yutaka Matsuo},\n      journal = {Advanced Robotics},\n      volume = {0},\n      number = {0},\n      pages = {1-16},\n      year = {2023},\n      publisher = {Taylor \u0026 Francis},\n      doi = {10.1080/01691864.2023.2244568}\n}\n```\n\n## Acknowledgements\nThis library is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO). \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmasa-su%2Fpixyz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmasa-su%2Fpixyz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmasa-su%2Fpixyz/lists"}