{"id":17222996,"url":"https://github.com/cheind/autoregressive","last_synced_at":"2025-04-14T00:18:47.994Z","repository":{"id":47918365,"uuid":"416994785","full_name":"cheind/autoregressive","owner":"cheind","description":":kiwi_fruit: Autoregressive Models in PyTorch.","archived":false,"fork":false,"pushed_at":"2022-04-03T16:09:05.000Z","size":245575,"stargazers_count":80,"open_issues_count":8,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-27T14:21:22.917Z","etag":null,"topics":["autoregressive","deep-learning","generative-model","machine-learning","pytorch","wavenet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cheind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-14T05:03:37.000Z","updated_at":"2025-03-24T21:23:47.000Z","dependencies_parsed_at":"2022-08-12T14:10:30.329Z","dependency_job_id":null,"html_url":"https://github.com/cheind/autoregressive","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fautoregressive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fautoregressive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fautoregressive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fautoregressive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cheind","download_url":"https://codeload.github.com/cheind/autoregressive/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248799984,"owners_count":21163404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoregressive","deep-learning","generative-model","machine-learning","pytorch","wavenet"],"created_at":"2024-10-15T04:07:02.140Z","updated_at":"2025-04-14T00:18:47.973Z","avatar_url":"https://github.com/cheind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://github.com/cheind/autoregressive/actions/workflows/python-package.yml/badge.svg)](https://github.com/cheind/autoregressive/actions/workflows/python-package.yml)\n\n# Autoregressive \n\nThis repository contains all the necessary PyTorch code, tailored to my [presentation](etc/presentation/autoregressive_presentation.pdf), to train and generate data from WaveNet-like autoregressive models.\n\nFor presentation purposes, the WaveNet-like models are applied to randomized Fourier series (1D) and MNIST (2D). In the figure below, two WaveNet-like models with different training settings make an n-step prediction on a periodic time-series from the validation dataset.\n\u003cdiv align=\"center\"\u003e\n \u003cimg src=\"etc/presentation/compare_curves_train_unroll.svg\" width=\"60%\"\u003e\n\u003c/div\u003e\n\nAdvanced functions show how to generate MNIST images and how to estimate the MNIST digit class (progressively) `p(y=class|x)` from observed pixels using a conditional WaveNet `p(x|y=class)` and Bayes rule. Left: sampled MNIST digits, right: progressive class estimates as more pixels are observed.\n\n\u003cdiv align=\"center\"\u003e\n \u003cimg src=\"etc/presentation/wavenet-sample-q256.png\" width=\"40%\"\u003e\n \u003cimg src=\"etc/presentation/progressive_classify_mnist_b000_h757.gif\" width=\"30%\"\u003e\n\u003c/div\u003e\n\n*Note, this library **does not** implement (Gated) PixelCNNs, but unrolls images for the purpose of processing in WaveNet architectures. This works surprisingly well.*\n\n## Features\nCurrently the following features are implemented\n - WaveNet architecture and training as proposed in (*oord2016wavenet*)\n - Conditioning support (*oord2016wavenet*)\n - Fast generation based on (*paine2016fast*)\n - Fully differentiable n-step unrolling in training (*heindl2021autoreg*)\n - 2D image generation, completion, classification, and progressive classification support based on MNIST dataset\n - A randomized Fourier dataset\n\n## Presentation\nA detailed presentation with theoretical background, architectural considerations and experiments can be found below.\n\n\u003cdiv align=\"center\"\u003e\n \u003ca href=\"etc/presentation/autoregressive_presentation.pdf\"\u003e\u003cimg src=\"etc/presentation/title.PNG\" width=\"60%\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\nThe presentation source as well as all generated images are public domain. In case you find them useful, please leave a citation (see References below). All presentation sources can be found in `etc/presentation`. The presentation is written in markdown using [Marp](https://marp.app/), graph diagrams are created using [yEd](https://www.yworks.com/products/yed).\n\n*If you spot errors or if case you have suggestions for improvements, please let me know by opening an issue.*\n\n## Installation\nTo install run,\n\n```\npip install git+https://github.com/cheind/autoregressive.git#egg=autoregressive[dev]\n```\nwhich requires Python 3.9 and a recent PyTorch \u003e 1.9\n\n## Usage\nThe library comes with a set of pre-trained models in [`models/`](models/). The following commands use those models to make various predictions. Many listed commands come with additional parameters; use `--help` to get additional information.\n\n### 1D Fourier series\n\nSample new signals from scratch\n```bash\npython -m autoregressive.scripts.wavenet_signals sample --config \"models/fseries_q127/config.yaml\" --ckpt \"models/fseries_q127/xxxxxx.ckpt\" --condition 4 --horizon 1000\n```\nThe default models conditions on the periodicity of the signal. For the pre-trained model the value range is `int: [0..4]`, corresponding to periods of 5-10secs.\n\n--- \n\nPredict the shape of partially observable curves.\n```bash\npython -m autoregressive.scripts.wavenet_signals predict --config \"models/fseries_q127/config.yaml\" --ckpt \"models/fseries_q127/xxxxxx.ckpt\" --horizon 1500 --num_observed 50 --num_trajectories 20 --num_curves 1 --show_confidence true\n```\n\n### 2D MNIST\n\nTo sample from the class-conditional model\n```bash\npython -m autoregressive.scripts.wavenet_mnist sample --config \"models/mnist_q2/config.yaml\" --ckpt \"models/mnist_q2/xxxxxx.ckpt\"\n```\n---\n\nGenerate images conditioned on the digit class and observed pixels.\n```bash\npython -m autoregressive.scripts.wavenet_mnist predict --config \"models/mnist_q2/config.yaml\" --ckpt \"models/mnist_q2/xxxxxx.ckpt\" \n```\n\n---\nTo perform classification\n```bash\npython -m autoregressive.scripts.wavenet_mnist classify --config \"models/mnist_q2/config.yaml\" --ckpt \"models/mnist_q2/xxxxxx.ckpt\"\n```\n\n## Train\nTo train / reproduce a model\n```bash\npython -m autoregressive.scripts.train fit --config \"models/mnist_q2/config.yaml\"\n```\nProgress is logged to Tensorboard\n```\ntensorboard --logdir lightning_logs\n```\nTo generate a training configuration file for a specific dataset use\n```\npython -m autoregressive.scripts.train fit --data autoregressive.datasets.FSeriesDataModule --print_config \u003e fseries_config.yaml\n```\n\n## Test\nTo run the tests\n```\npytest\n```\n\n## References\n\n```bibtex\n@misc{heindl2021autoreg, \n  title={Autoregressive Models}, \n  journal={PROFACTOR Journal Club}, \n  author={Heindl, Christoph},\n  year={2021},\n  howpublished={\\url{https://github.com/cheind/autoregressive}}\n}\n\n@article{oord2016wavenet,\n  title={Wavenet: A generative model for raw audio},\n  author={Oord, Aaron van den and Dieleman, Sander and Zen, Heiga and Simonyan, Karen and Vinyals, Oriol and Graves, Alex and Kalchbrenner, Nal and Senior, Andrew and Kavukcuoglu, Koray},\n  journal={arXiv preprint arXiv:1609.03499},\n  year={2016}\n}\n\n@article{paine2016fast,\n  title={Fast wavenet generation algorithm},\n  author={Paine, Tom Le and Khorrami, Pooya and Chang, Shiyu and Zhang, Yang and Ramachandran, Prajit and Hasegawa-Johnson, Mark A and Huang, Thomas S},\n  journal={arXiv preprint arXiv:1611.09482},\n  year={2016}\n}\n\n@article{oord2016conditional,\n  title={Conditional image generation with pixelcnn decoders},\n  author={Oord, Aaron van den and Kalchbrenner, Nal and Vinyals, Oriol and Espeholt, Lasse and Graves, Alex and Kavukcuoglu, Koray},\n  journal={arXiv preprint arXiv:1606.05328},\n  year={2016}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheind%2Fautoregressive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcheind%2Fautoregressive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheind%2Fautoregressive/lists"}