{"id":28988737,"url":"https://github.com/eidoslab/alice","last_synced_at":"2025-06-24T22:06:43.001Z","repository":{"id":265050512,"uuid":"870724782","full_name":"EIDOSLAB/ALICE","owner":"EIDOSLAB","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-27T10:51:23.000Z","size":608,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-27T11:33:44.691Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EIDOSLAB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-10T14:54:19.000Z","updated_at":"2024-11-27T10:51:26.000Z","dependencies_parsed_at":"2024-11-27T14:34:31.913Z","dependency_job_id":null,"html_url":"https://github.com/EIDOSLAB/ALICE","commit_stats":null,"previous_names":["eidoslab/alice"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/EIDOSLAB/ALICE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EIDOSLAB%2FALICE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EIDOSLAB%2FALICE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EIDOSLAB%2FALICE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EIDOSLAB%2FALICE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EIDOSLAB","download_url":"https://codeload.github.com/EIDOSLAB/ALICE/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EIDOSLAB%2FALICE/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261764391,"owners_count":23206256,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-24T22:06:42.289Z","updated_at":"2025-06-24T22:06:42.978Z","avatar_url":"https://github.com/EIDOSLAB.png","language":"Python","readme":"# ALICE: Adapt your Learnable Image Compression modEl for variable bitrates \n\nPytorch implementation of the paper \"**ALICE: Adapt your Learnable Image Compression modEl for variable bitrates**\", published at VCIP 2024. This repository is based on [CompressAI](https://github.com/InterDigitalInc/CompressAI) and [STF](https://github.com/Googolxx/STF).\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"imgs/alice_arch.png\" alt=\"teaser\" width=\"1000\"/\u003e\n\u003c/div\u003e\n\n## Abstract\nWhen training a Learned Image Compression\nmodel, the loss function is minimized such that the encoder and\nthe decoder attain a target Rate-Distorsion trade-off. Therefore, a\ndistinct model shall be trained and stored at the transmitter and\nreceiver for each target rate, fostering the quest for efficient vari-\nable bitrate compression schemes. This paper proposes plugging\nLow-Rank Adapters into a transformer-based pre-trained LIC\nmodel and training them to meet different target rates. With our\nmethod, encoding an image at a variable rate is as simple as\ntraining the corresponding adapters and plugging them into the\nfrozen pre-trained model. Our experiments show performance\ncomparable with state-of-the-art fixed-rate LIC models at a\nfraction of the training and deployment cost.\n\n## Usage\n\n- First download openimages dataset using `src/downloader_openimages.py`.\n- For evaluation and validation during training also download KODAK dataset.\n\nThe script for downloading [OpenImages](https://github.com/openimages) is provided in `downloader_openimages.py`. Please install [fiftyone](https://github.com/voxel51/fiftyone) first.\n\n## Environment\n- conda env create -f environment.yml\n- conda activate alice\n\n## Evaluation\n\n- Download our pretrained model in the ALICE directory from [here](https://drive.google.com/file/d/1Hz-5ODbXkSUOei807ThGSqL63Ow8ZanV/view?usp=sharing).\n- Extract pretrained.zip\n```\nALICE\n│   README.md\n│  \n└───pretrained\n│   └───adapt_0483_seed_42_conf_lora_8_8_opt_adam_sched_cosine_lr_0_0001\n│        │   inference.json\n|        |   inference_merge.json\n│        │   013_checkpoint_best.pth.tar\n|        |   025_checkpoint_best.pth.tar\n│        │   ...\n|  \n│       adapt_0483_seed_42_conf_vanilla_adapt_opt_adam_sched_cosine_lr_0_0001\n│        │   inference.json\n│        │   013_checkpoint_best.pth.tar\n|        |   025_checkpoint_best.pth.tar\n│        │   ...\n|  \n│       stf\n│        │   inference.json\n│        │   stf_013_best.pth.tar\n|        |   stf_025_best.pth.tar\n│        │   ...   \n|        |\n└───src\n    │   train.py\n    │   ...\n```\n\n- Finally, run the following command:\n```\ncd src\n\npython -m evaluate.eval --test-dir /test/to/kodak/ --file-name results_kodak --save-path /path/to/save/results\n```\n\n**To evaluate your own model** follow the same structure proposed above:\n- First create a folder named `pretrained/your_model`.\n- In this folder collect all of the trained models making sure that each one ends with best.pth.tar, for example:\n  - `pretrained/your_model/your_model_013_best.pth.tar`\n  - `pretrained/your_model/your_model_0018_best.pth.tar`\n  - `pretrained/your_model/your_model_025_best.pth.tar`\n  - ...\n- Create a file named `pretrained/your_model/inference.json` in the same folder in this way:\n\n```\n{\n    \"model\":\"your_model\",\n    \"checkpoints_path\":\"pretrained/your_model\"\n}\n```\n\n- You can compare multiple models at the same time creating a folder with an inference.json file and saving the chekpoints.\n- List all of the model that you want to compare in the variable `configs` in the main function of `src/evaluate/eval.py`\n\n### MALICE\nYou can evaluate *MALICE* including in the `configs` variable defined in the main function of `src/evaluate/eval.py` the following configuration: \n```\n../pretrained/adapt_0483_seed_42_conf_lora_8_8_opt_adam_sched_cosine_lr_0_0001/inference_merge.json\n```\n\n\n\n## Train\nThe training script is provided in\n`src/train.py`. \n\n### Training with LoRA:\n```\ncd src\n\npython train.py --batch-size=16 --checkpoint=../pretrained/stf/stf_0483_best.pth.tar --cuda=1 --dataset=../../../data/openimages/ --epochs=15 --lambda=0.013 --learning-rate=0.0001 --lora=1 --lora-config=../configs/lora_8_8.yml --lora-opt=adam --lora-sched=cosine --model=stf --save=1 --save-dir=../results/adapt_models_lora/adapt_0483 --test-dir=../../../data/kodak/\n```\n\n### Finetuning Only the FC layer (Vanilla):\n```\ncd src\n\npython train.py --batch-size=16 --checkpoint=../pretrained/stf/stf_0483_best.pth.tar --cuda=1 --dataset=../../../data/openimages/ --epochs=15 --lambda=0.013 --learning-rate=0.0001 --lora=1 --lora-opt=adam --lora-sched=cosine --model=stf --save=1 --save-dir=../results/adapt_models_vanilla/adapt_0483 --test-dir=../../../data/kodak/ --vanilla-adapt=1\n```\n\nSweep files to adapt the model for all lambda values are saved in: \n- [sweeps/lora_adam_all_points.yaml](sweeps/lora_adam_all_points.yaml)\n- [sweeps/vanilla_adam_all_points.yaml](sweeps/vanilla_adam_all_points.yaml)\n\n\n## Results\n\n- Rate-Distortion\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"imgs/rd-alice.png\" alt=\"rd\" width=800\"/\u003e\n\u003c/div\u003e\n\n- Complexity Considerations\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"imgs/memory_alice.png\" alt=\"rd\" width=800\"/\u003e\n\u003c/div\u003e\n\n\n\n\n\n## Pretrained Models (from STF repo)\nPretrained models (optimized for MSE) trained from scratch using randomly chose 300k images from the OpenImages dataset.\n\n| Method | Lambda | Link                                                                                           |\n| ---- |--------|---------------------------------------------------------------------------------------------------|\n| STF | 0.0018 | [stf_0018](https://drive.google.com/file/d/15ujpSjif628iwVEay3mAWN-Vyqls3r23/view?usp=sharing) |\n| STF | 0.0035 | [stf_0035](https://drive.google.com/file/d/1OFzZoEaofNgsimBuOPHtgOJiGsR_RS-M/view?usp=sharing)    |\n| STF | 0.0067 | [stf_0067](https://drive.google.com/file/d/1SjhqcKyP3SqVm4yhJQslJ6HgY1E8FcBL/view?usp=share_link) |\n| STF | 0.013  | [stf_013](https://drive.google.com/file/d/1mupv4vcs8wpNdXCPclXghliikJyYjgj-/view?usp=share_link)  |\n| STF | 0.025  | [stf_025](https://drive.google.com/file/d/1rsYgEYuqSYBIA4rfvAjXtVSrjXOzkJlB/view?usp=sharing)     |\n| STF | 0.0483 | [stf_0483](https://drive.google.com/file/d/1cH5cR-0VdsQqCchyN3DO62Sx0WGjv1h8/view?usp=share_link) |\n\n\n\n## Related links\n * STF: https://github.com/Googolxx/STF\n * CompressAI: https://github.com/InterDigitalInc/CompressAI\n * Swin-Transformer: https://github.com/microsoft/Swin-Transformer\n * Tensorflow compression library by Ballé et al.: https://github.com/tensorflow/compression\n * Range Asymmetric Numeral System code from Fabian 'ryg' Giesen: https://github.com/rygorous/ryg_rans\n * Kodak Images Dataset: http://r0k.us/graphics/kodak/\n * Open Images Dataset: https://github.com/openimages\n * fiftyone: https://github.com/voxel51/fiftyone\n * CLIC: https://www.compression.cc/\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feidoslab%2Falice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feidoslab%2Falice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feidoslab%2Falice/lists"}