{"id":20663702,"url":"https://github.com/vita-group/alista","last_synced_at":"2026-03-14T08:01:57.507Z","repository":{"id":52077332,"uuid":"171522349","full_name":"VITA-Group/ALISTA","owner":"VITA-Group","description":"[ICLR 2019] \"ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA\", by Jialin Liu*, Xiaohan Chen*, Zhangyang Wang and Wotao Yin.","archived":false,"fork":false,"pushed_at":"2021-12-31T05:22:52.000Z","size":7383,"stargazers_count":70,"open_issues_count":1,"forks_count":21,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T09:41:59.645Z","etag":null,"topics":["compressive-sensing","deep-learning","lista","machine-learning","sparse-coding","sparse-reconstruction"],"latest_commit_sha":null,"homepage":"https://openreview.net/forum?id=B1lnzn0ctQ","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VITA-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-19T17:55:34.000Z","updated_at":"2025-03-24T16:01:06.000Z","dependencies_parsed_at":"2022-09-06T13:11:18.451Z","dependency_job_id":null,"html_url":"https://github.com/VITA-Group/ALISTA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FALISTA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FALISTA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FALISTA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FALISTA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VITA-Group","download_url":"https://codeload.github.com/VITA-Group/ALISTA/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249731218,"owners_count":21317341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compressive-sensing","deep-learning","lista","machine-learning","sparse-coding","sparse-reconstruction"],"created_at":"2024-11-16T19:19:25.607Z","updated_at":"2026-03-14T08:01:57.458Z","avatar_url":"https://github.com/VITA-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA\nThis repository is for Analytic-LISTA networks proposed in the following paper:\n\n[Jialin Liu\\*](http://www.math.ucla.edu/~liujl11/),\n[Xiaohan Chen\\*](http://people.tamu.edu/~chernxh),\n[Zhangyang Wang](http://www.atlaswang.com/) and\n[Wotao Yin](http://www.math.ucla.edu/~wotaoyin/)\n\"ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA\", accepted at\nICLR 2019. The pdf can be found [here](https://openreview.net/pdf?id=B1lnzn0ctQ)..\n\n\\*: These authors contributed equally and are listed alphabetically.\n\nThe code is tested in Linux environment (Python: 3.5.2, Tensorflow: 1.12.0,\nCUDA9.0) with Nvidia GTX 1080Ti GPU.\n\n\n\u003c!-- vim-markdown-toc GFM --\u003e\n\n* [Introduction](#introduction)\n* [Run the codes](#run-the-codes)\n    * [Generate problem files](#generate-problem-files)\n    * [Baseline: LISTA-CPSS](#baseline-lista-cpss)\n    * [TiLISTA](#tilista)\n    * [ALISTA](#alista)\n        * [Solve analytic weight from measurement matrix](#solve-analytic-weight-from-measurement-matrix)\n        * [ALISTA with analytic weight](#alista-with-analytic-weight)\n    * [Robust ALISTA](#robust-alista)\n        * [Pre-train encoders](#pre-train-encoders)\n        * [Jointly train encoder and decoder](#jointly-train-encoder-and-decoder)\n        * [Testing](#testing)\n    * [Data augmented decoders](#data-augmented-decoders)\n    * [Convolutional LISTA for natural image denoising](#convolutional-lista-for-natural-image-denoising)\n* [Cite this work](#cite-this-work)\n\n\u003c!-- vim-markdown-toc --\u003e\n\n## Introduction\nDeep neural networks based on unfolding an iterative algorithm, for example,\nLISTA (learned iterative shrinkage thresholding algorithm), have been an\nempirical success for sparse signal recovery. The weights of these neural\nnetworks are currently determined by data-driven “black-box” training. In this\nwork, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is\ncomputed as the solution to a data-free optimization problem, leaving only the\nstepsize and threshold parameters to data-driven learning. This significantly\nsimplifies the training. Specifically, the data-free optimization problem is based\non coherence minimization. We show our ALISTA retains the optimal linear\nconvergence proved in (Chen et al., 2018) and has a performance comparable to\nLISTA. Furthermore, we extend ALISTA to convolutional linear operators, again\ndetermined in a data-free manner. We also propose a feed-forward framework that\ncombines the data-free optimization and ALISTA networks from end to end, one\nthat can be jointly trained to gain robustness to small perturbations in the\nencoding model.\n\n## Run the codes\n\n### Generate problem files\nTo run most of experiments in this repository, you need to first generate an\ninstance of `Problem` or `ProblemConv` class, which you can find in\n`utils/prob.py` or `utils/prob_conv.py` file.\n\nRun the following command to generate a random measurement matrix:\n\n```\npython3 utils/prob.py --M 250 --N 500 \\\n    --pnz 0.1 --SNR inf --con_num 0.0 --column_normalized True\n```\n\nExplation for the options:\n* `--M`: the dimension of measurements.\n* `--N`: the dimension of sparse signals.\n* `--pnz`: the approximate of non-zero elements in sparse signals.\n* `--SNR`: the *signal-to-noise* ratio in dB unit in the measurements. `inf`\n  means noiseless setting.\n* `--con_num`: the condition number. 0.0 (default) means the condition number\n  will not be changed.\n* `--column_normalized`: whether normalize the columns of the measurement matrix\n  to unit l-2 norm.\n\nThe generated will be saved to the `experiments/m250_n500_k0.0_p0.1_s40/prob.npz`.\nIf you want to generate a problem from an existing measurement matrix, which\nshould be saved in Numpy `npy` file format, use `--load_A` option with the path\nto the matrix file. In this case, options `--M` and `--N` will be overwriiten by\nthe shape of loaded matrix.\n\n### Baseline: LISTA-CPSS\nUse the baseline model *LISTA-CPSS* in [this paper](http://papers.nips.cc/paper/8120-theoretical-linear-convergence-of-unfolded-ista-and-its-practical-weights-and-thresholds)\nto basicly explain how to train and test models. To train or test a LISTA-CPSS\nmodel, use the following command:\n```\npython3 main.py --task_type sc -g 0 [-t] \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net LISTA_cpss -T 16 -p 1.2 -maxp 13 \\\n    --scope LISTA_cpss --exp_id 0\n```\n\nExplanation for the options (all optinos are parsed in `config.py`):\n* `--task_type`: the task on which you will train/test your model. Possible\n  values are:\n  * `sc` standing for normal simulated sparse coding algorithm;\n  * `cs` for natural image compressive sensing;\n  * `denoise` for natural image denoising using convolutional LISTA;\n  * `encoder` for encoder pre-training; and\n  * `robust` for robustness training.\n* `-g/--gpu`: the id of GPU used. GPU 0 will be used by default.\n* `-t/--test` option indicates training or testing mode. Use this option for\n  testing.\n* `-n/--net`: specifies the network to use.\n* `-T`: the number of layers.\n* `-p/--percent`: the percentage of entries to be added to the support in each\n  layer.\n* `-maxp/--max_percent`: maximum percentage of entries to be selected.\n* `--scope`: the name of variable scope of model variables in TensorFlow.\n* `--exp_id`: experiment id, used to differentiate experiments with the same\n  setting.\n\n### TiLISTA\n\nTo train or test a TiLISTA (Tied-LISTA) network, run the following command:\n```\npython3 main.py --task_type sc -g 0 [-t] \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net TiLISTA -T 16 -p 1.2 -maxp 13 \\\n    --scope TiLISTA --exp_id 0\n```\n\n### ALISTA\n#### Solve analytic weight from measurement matrix\nUse MatLab script `matlabs/CalculateW.m` to solve an analytic weight matrix from\nan existing measurement matrix, which should be saved as a MatLab `mat` file\nwith key word `D` for the matrix. We provide an example in `data/D.mat`, which\nis the same matrix as in `experiments/m250_n500_k0.0_p0.1_sinf/prob.npz`. Use\n`matlabs/CalculateW_conv.m` for convolutional dictionaries. We provide a\npre-solved weight saved as `data/W.npy`.\n\n#### ALISTA with analytic weight\n```\npython3 main.py --task_type sc -g 0 [-t] \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net ALISTA -T 16 -p 1.2 -maxp 13 -W ./data/W.npy \\\n    --better_wait 2000 \\\n    --scope ALISTA --exp_id 0\n```\n\nExplanation for options:\n* `-W`: path to the specified weight matrix.\n* `--better_wait`: maximum waiting time for a better validation accuracy before\n  going to the next training stage. ALISTA model has 2T parameters, thus having\n  a very stabilized training process. Therefore, we can use a smaller waiting\n  time than LISTA-CPSS (use `--better_wait 5000` by default).\n\n### Robust ALISTA\nTo train a robust ALISTA model, you need 3 steps:\n1. Pre-train a encoder.\n2. Pre-train a ALISTA decoder. We can use the ALISTA modeled trained in the above\n   section.\n3. Jointly train the encoder and the decoder.\n#### Pre-train encoders\n```\npython3 main.py --task_type encoder -g 0 \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net AtoW_grad --eT 4 --Binit default --eta 1e-3 --loss rel2 \\\n    --Abs 16 --encoder_psigma 1e-2 --encoder_pre_lr 1e-4 \\\n    --scope AtoW --exp_id 0\n```\n\nExplanation for the options:\n* `--net AtoW_grad`: the encoding model unfoled from projected gradient descent.\n* `--eT`: the number of layers in the encoder.\n* `--Binit`: use the default method to initialize weights in the encoder. You\n  can use random initialization by specifying `normal` or `uniform` here.\n* `--eta`: the initial step size in the projected gradient descent.\n* `--loss`: the objective function in the original optimization, and the cost\n  function used to train the encoder. `rel2` means *reweighted l2*.\n* `--Abs`: the batch size to sample perturbed matrices.\n* `--encoder_psigma`: the noise level to perturb the measurement matrix.\n* `--encoder_pre_lr`: the initial learning rate for pre-training the encoder.\n\n#### Jointly train encoder and decoder\n\n```\npython3 main.py --task_type robust -g 0 [-t] \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net robust_ALISTA \\\n    --eT 4 --Binit default --eta 1e-3 --loss rel2 --encoder_scope AtoW \\\n    --encoder_psigma 1e-2 --encoder_pre_lr 1e-4 --encoder_id 0 \\\n    --dT 16 --lam 0.4 -p 1.2 -maxp 13 -W .data/W.npy \\\n    --decoder_scope ALISTA --decoder_id 0 \\\n    --psigma_max 2e-2 --psteps 5 --msigma 0.0 \\\n    --encoder_lr 1e-9 --decoder_lr 1e-4 \\\n    --Abs 4 --xbs 16 --maxit 50000 --exp_id 0\n```\n\nExplanation for the options:\n* `--dT`: the number of layers in the decoder.\n* `--psigma_max`: the maximum level of perturbations during the joint training.\n* `--psteps`: the number of steps of the curriculum training where we gradually\n  increase the level of perturbations till `psigma_max`.\n* `--msigma`: the level of measurement noises during the joint training.\n* `--xbs`: the batch size used to generate sparse signals for each perturbed\n  measurement matrix. The total number of measurement-signal pairs in a batch is\n  `Abs` times `xbs`.\n* `--maxit`: the maximum number of training steps for each curriculum training\n  stage in the whole training process.\n\n#### Testing\nUse the above command with `-t/--t` option to test. For testing we genereate a\nsample of perturbed measurement matrices, which you can download using this\n[Dropbox link](https://www.dropbox.com/s/9m7s1g1u4apy4wx/robust_test_A.npz?dl=0).\nThe original measurement matrix used to generate this file is the same as in\n`experiments/m250_n500_k0.0_p0.1_sinf/prob.npz`.\n\n### Data augmented decoders\nTo train or test a data-augmented decoding model, run the following command (use\nTiLISTA for example):\n```\npython3 main.py --task_type robust -g 0 [-t] \\\n    --M 250 --N 500 --pnz 0.1 --SNR inf --con_num 0 --column_normalized True \\\n    --net TiLISTA -T 16 --lam 0.4 -p 1.2 -maxp 13 \\\n    --psigma_max 2e-2 --psteps 5 --msigma 0.0 \\\n    --decoder_lr 1e-4 --Abs 4 --xbs 16 --maxit 50000 \\\n    --scope TiLISTA --exp_id 0\n```\n\n### Convolutional LISTA for natural image denoising\n1. Download BSD500 dataset. Split into train, validation and test sets as you\n   wish.\n2. Genereate the tfrecords using:\n   ```\n   python3 utils/data.py --task_type denoise \\\n       --dataset_dir /path/to/your/[train,val,test]/folder \\\n       --out_dir path/to/the/folder/to/store/tfrecords \\\n       --out_file [train,val,test].tfrecords \\\n       --suffix jpg\n   ```\n3. Learn a convolutional dictionary from the BSD500 dataset using the algorithm\n   in the paper [First- and Second-Order Methods for Online Convolutional\n   Dictionary Learning](https://arxiv.org/abs/1709.00106). Or use the dictionary\n   proveided in `data/D3_M100_lam0.05.mat`.\n4. Generate a problem file using the learned dictionary and the following\n   command:\n   ```\n   python3 utils/prob_conv.py --conv_d 3 --conv_m 100 --conv_lam 0.05 \\\n       --load_dict ./data/D3_M100_lam0.05.mat\n   ```\n   where `--conv_d` is the size of filters in the dictionary, `--conv_m` is the\n   number of filters, `--conv_lam` is the parameter used in convolutional\n   dictionary learning algorithm, and `--load_dict` specifies the dictionary to\n   be loaded and saved. The generated problem file will be saved to\n   `experiments/denoise_d3_m100_lam0.05/prob.npz`.\n5. Train and test the convolutional denoising model (use Conv-TiLISTA as an\n   example):\n   ```\n   python3 main.py --task_type denoise -g 0 [-t] \\\n       --net TiLISTA -T 5 --lam 0.1 --conv_alpha 0.1 \\\n       --sigma 20 --height_crop 321 --width_crop 321 \\\n       --num_epochs -1 --tbs 4 --vbs 16 \\\n       --data_folder data/denoise_tfrecords\n       --train_file training_tfrecords_filename \\\n       --val_file validation_tfrecords_filename\n   ```\n   Explanation for the options:\n   * `--conv_alpha`: the initial step size in learned convolutional model.\n   * `--sigma`: the noise level in the images.\n   * `--height_crop` and `--width_crop`: size of cropped images in training.\n   * `--num_epochs`: the number of epochs to train over the BSD500 training set.\n     The default `-1` value means infinite nubmer of opochs. The training will\n     be ended as in `sc` task.\n   * `--tbs` and `--vbs`: training and validation batch sizes.\n   * `--data_folder`: the path to the folder that holds the tfrecords files.\n\n## Cite this work\nIf you find our code helpful in your resarch or work, please cite our paper.\n```\n@inproceedings{\nliu2018alista,\ntitle={{ALISTA}: Analytic Weights Are As Good As Learned Weights in {LISTA}},\nauthor={Jialin Liu and Xiaohan Chen and Zhangyang Wang and Wotao Yin},\nbooktitle={International Conference on Learning Representations},\nyear={2019},\nurl={https://openreview.net/forum?id=B1lnzn0ctQ},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Falista","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvita-group%2Falista","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Falista/lists"}