{"id":20663857,"url":"https://github.com/vita-group/insp","last_synced_at":"2025-07-09T10:35:45.057Z","repository":{"id":61620990,"uuid":"548574188","full_name":"VITA-Group/INSP","owner":"VITA-Group","description":"[NeurIPS 2022] \"Signal Processing for Implicit Neural Representations\" by Dejia Xu*, Peihao Wang*, Yifan Jiang, Zhiwen Fan, Zhangyang Wang","archived":false,"fork":false,"pushed_at":"2022-12-01T16:25:03.000Z","size":51301,"stargazers_count":52,"open_issues_count":4,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T09:51:12.068Z","etag":null,"topics":["audio-processing","geometry-processing","image-processing","implicit-neural-representation","signal-processing"],"latest_commit_sha":null,"homepage":"https://vita-group.github.io/INSP","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VITA-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-09T20:59:49.000Z","updated_at":"2024-12-29T15:17:58.000Z","dependencies_parsed_at":"2023-01-22T20:15:13.030Z","dependency_job_id":null,"html_url":"https://github.com/VITA-Group/INSP","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FINSP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FINSP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FINSP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FINSP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VITA-Group","download_url":"https://codeload.github.com/VITA-Group/INSP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249731742,"owners_count":21317343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-processing","geometry-processing","image-processing","implicit-neural-representation","signal-processing"],"created_at":"2024-11-16T19:20:17.925Z","updated_at":"2025-04-19T15:56:59.123Z","avatar_url":"https://github.com/VITA-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Signal Processing for Implicit Neural Representations\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n\nThe official implementation of NeurIPS 2022 paper [\"Signal Processing for Implicit Neural Representations\"]().\n\nDejia Xu*, Peihao Wang*, Yifan Jiang, Zhiwen Fan, Zhangyang (Atlas) Wang\n\n[[Paper]](https://arxiv.org/abs/2210.08772) [[Website]](https://vita-group.github.io/INSP)\n\n## Method Overview\n\n![](./docs/static/media/overview.e47f8ec0149b9912e940.png)\n\n![](./docs/static/media/framework.0c59d0c8b8386b9f7f45.png)\n\n## Environment\n\nYou can then set up a conda environment with all dependencies like so:\n\n```\nconda env create -f environment.yml\nconda activate INSP\n```\n\n## High-Level structure\n\n- Fit INR\n- Export gradients for INR\n- Train INSP-Net\n- Inference INSP-Net\n\n## Image Processing\n\nFor image processing, we experiment on div2k dataset.\n\n```bash\nwget http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_HR.zip \u0026\nunzip DIV2K_train_HR.zip\n```\n\n- Fit multiple INR \n\n    Use `--type ` to specify the type of images you want to train on.\n\n    ```python gen_div2k.py | zsh```\n\n- Export gradients for INR \n\n    `--load` is used for `glob` to filter out corresponding INRs.\n\n    ```\n    python export_colorray.py --save_dir grad/train_color_noise/ --load 'div2k*.png_color_noise_'\n    ```\n\n    Then, manually divide `grad/train_color_noise` and put a few of them into `grad/test_color_noise` (in our case we used first 100 images in DIV2K for training and the following 100 images for testing)\n\n- Train INSP-Net\n\n    `--img_num` changes the number of images that are used for training.\n    \n    The training should converge in a couple of minutes.\n\n    ```\n    python experiment_scripts/train_img_grad_offline.py --model_type=sine --experiment_name denoise --noise_level 0 --target denoise --img_num 100 --overwrite --sigma 7 --sz 256 --batch_size 10240 --lr 1e-4\n    ```\n\n- Inference INSP-Net\n\n    ```\n    python eval_insp.py --save_path output/denoise --target denoise --ckpt_path logs/denoise/checkpoints/model_current.pth\n    ```\n\nThe INRs used in our experiments can be found [here](https://drive.google.com/drive/folders/1VaEgKiWIGpQhIw5uxPJGWL0OdTTM-cuo?usp=sharing).\n\n## Audio Denoising\n\n- Fit INR\n\n    ```\n    python experiment_scripts/train_audio.py --model_type=sine --wav_path=data/gt_bach.wav --experiment_name audio_noisy_10\n    ```\n\n- Export gradients for INR \n\n    ```\n    python export_audio.py\n    ```\n\n- Train INSP-Net\n\n    ```\n    python experiment_scripts/train_audio_insp.py --experiment_name audio_denoise --batch_size 10240\n    ```\n\n- Inference INSP-Net\n\n    ```\n    python eval_audio_insp.py\n    ```\n\n## SDF Smoothing\n\n\n- Fit INR\n\n    ```\n    \n    ```\n\n- Export gradients for INR \n\n    ```\n    python export_sdf_ray.py\n    ```\n\n- Train INSP-Net\n\n    ```\n    python experiment_scripts/train_sdf_insp.py --experiment_name smooth_armadillo --sz 256 --ti 10 --batch_size 1\n    ```\n\n- Inference INSP-Net\n\n    ```\n    python eval_sdf_insp.py\n    ```\n\n\n## Image Classification\n\nDue to the large size of MNIST and CIFAR INRs, we can't provide all of the checkpoints. However, we share the scripts to generate the INRs.\n\n## Citation\n\n```\n@inproceedings{Xu_2022_INSP,\n    title={Signal Processing for Implicit Neural Representations},\n    author={Xu, Dejia and Wang, Peihao and Jiang, Yifan and Fan, Zhiwen and Wang, Zhangyang},\n    booktitle={Advances in Neural Information Processing Systems (NeurIPS)},\n    year={2022}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Finsp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvita-group%2Finsp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Finsp/lists"}