{"id":20881913,"url":"https://github.com/aiff22/sqad","last_synced_at":"2025-05-12T17:30:56.174Z","repository":{"id":192962198,"uuid":"679879329","full_name":"aiff22/SQAD","owner":"aiff22","description":"SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking ","archived":false,"fork":false,"pushed_at":"2023-10-02T16:04:36.000Z","size":47,"stargazers_count":24,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-01T08:44:50.434Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aiff22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-17T20:33:09.000Z","updated_at":"2024-11-11T16:44:42.000Z","dependencies_parsed_at":"2024-11-18T07:41:39.658Z","dependency_job_id":null,"html_url":"https://github.com/aiff22/SQAD","commit_stats":null,"previous_names":["aiff22/sqad"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2FSQAD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2FSQAD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2FSQAD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2FSQAD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aiff22","download_url":"https://codeload.github.com/aiff22/SQAD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253787187,"owners_count":21964293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T07:26:59.168Z","updated_at":"2025-05-12T17:30:55.822Z","avatar_url":"https://github.com/aiff22.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking\nOfficial webpage for ICCV'23 paper \"SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking\"\u003cbr\u003e\n\nDataset download: [SQAD](http://data.vision.ee.ethz.ch/ihnatova/public/sqad/sqad.zip)\n* There are three folders contained: *train*, *test*, and *sample_crop16*. Follow the instructions below to prepare the data for training and testing.\n\nYou can download our pre-trained models from this link: [ResNet-based](http://data.vision.ee.ethz.ch/ihnatova/public/sqad/resnet_models.zip) / [MANIQA-based](http://data.vision.ee.ethz.ch/ihnatova/public/sqad/maniqa_models.zip)\n* Here are MANIQA-based models with all 6 quality factors, and ResNet-based models for resolution on cross-camera prediction task. Please refer to Table 4 in the paper for camera set division.\n\n* Feel free to explore on cross-camera prediction task since there is still high potential to improve further.\n\n\n## Baselines\nHere we provide some baselines from sota **MANIQA** model for testing on `sample_crop16` subset and some randomly generated test set from `test` folder. Within one image, the information levels (i.e. edge details, color vividness) for different parts usually vary, there would be fluctuations for the evaluation metrics if you apply your own cropping.\n\nResults evaluated with `sample_crop16` subset:\n\n Metrics      | Resolution | Color Accuracy |   Noise  |   DR   |   PSF   | Aliasing\n------------- |:----------:|:--------------:|:--------:|:------:|:-------:|:---------:\n *SROCC*      |   0.9292   |     0.9475     |  0.9336  | 0.9546 |  0.9424 |  0.9172\n *PLCC*       |   0.9460   |     0.9592     |  0.9784  | 0.9694 |  0.9823 |  0.9812\n\n Results evaluated with randomly cropped subset from `test` folder:\n\n Metrics      | Resolution | Color Accuracy |   Noise  |   DR   |   PSF   | Aliasing\n------------- |:----------:|:--------------:|:--------:|:------:|:-------:|:---------:\n *SROCC*      |   0.9475   |     0.9242     |  0.8756  | 0.9328 |  0.9117 |  0.8940\n *PLCC*       |   0.9479   |     0.9414     |  0.9373  | 0.9332 |  0.9704 |  0.9688\n\n* For cross-camera prediction, please refer to *Ablations* section in our paper for more details. Here we provide results for **multi-crop (16×)** predictions .\n\n Models   |   Metrics   |   SET 1   |   SET 2   |   SET 3  |   SET 4  |   SET 5  |  Overall\n--------- |:-----------:|:---------:|:---------:|:--------:|:--------:|:--------:|:--------:\n ResNet50 |  *SROCC*    |  0.5747   |   0.5590  |  0.6700  |  0.4017  |  0.8481  |  0.6212\n ResNet50 |  *PLCC*     |  0.8161   |   0.4244  |  0.6837  |  0.6539  |  0.6160  |  0.6075\n MANIQA   |  *SROCC*    |     -     |      -    |     -    |    -     |    -     |    -\n MANIQA   |  *PLCC*     |  0.6585   |   0.4087  |  0.8354  |  0.6324  |  0.7529  |  0.4486\n\n\n## Model training\n\nFor the quality factor regression with MANIQA model, please follow the guidelines provided by [CVPRW 2022: MANIQA](https://github.com/IIGROUP/MANIQA) to set up the environment. Thanks for their great works on image quality assessment.\u003cbr\u003e\n\nFor ResNet50-based backbone, there is **no** specific requirement for the environment settings. Just make sure you have installed *pytorch* and *Pillow* properly. If there are some package related issues, follow the error instructions.\n\n### Dataset Preparation\n* Please download the SQAD dataset, unzip and put all files into the `data` folder in the following structure:\n```\nSQAD\n|—— data\n|\t|—— processed\n|\t|—— train\n|\t|\t|—— 01_ASUS_Z00AD\n|\t|\t|\t|—— P_20000102_085109.jpg \u0026 ...\n|\t|\t|—— 02_...\n|\t|—— test\n|\t|\t|—— 01_ASUS_Z00AD\n|\t|\t|—— 02_...\n|\t|—— sample_crop16\n|—— prepare_data.py\n|—— camera_ground_truth.json\n|—— maniqa_plugin\n|—— resnet_\n```\nTo get the image crops for training, simply run:\n```\npython prepare_data.py\n```\nOr use the following command to specify the crop number and crop size (based on backbone network) per-image. You can also add `--crop_test_dataset` flag to create a randomly cropped test set for model evaluations.\n```\npython prepare_data.py --crop_num 32 --crop_size 240 --crop_test_dataset\n```\nThen, the cropped images and corresponding label files for training are stored in `./data/processed`.\u003cbr\u003e\n\nFor model testing, we have provided a pre-cropped image sets in the folder `sample_crop16`, which is extracted from the `test` folder. But it is free to use `--crop_test_dataset` flag to create your own test test as well.\n\n\n### Train with MANIQA\nPlease clone the MANIQA project and **replace** the original `train_maniqa.py` and `inference.py` by the files with the same name in `maniqa_plugin` folder.\n```\ngit clone https://github.com/IIGROUP/MANIQA.git\n```\nThen, add `camera29.py` in `maniqa_plugin/data` into the original MANIQA `data` folder. Now you can modify the configurations in `train_maniqa.py` Line136-141 and `inference.py` Line75-78 to play with the model training and quality factor inference.\n\n* The \"factor\" element in *config* uses the same key name with `camera_ground_truth.json`, they are: resolution, color_acc, noise, dr, psf, aliasing.\n\n* Download the pre-trained models and put them into the save path you specified in `inference.py` for direct evaluation.\n\n\n### Train with ResNet backbone\n\nTo train the regression model with ResNet50 backbone, run `train_regress.py` file in the `resnet_` folder with different quality factors:\n```\npython train_regress.py --factor dr(resolution/color_acc/...) --batch_size 64 --train_epochs 200 --save_eval_every 1\n```\n\n* To utilize the entire test set, one can also apply the random cropping during *DataLoader* and specify different image crop sizes and desired crop numbers. We provide a *FullImageDataset* class in `CameraDataset.py` for exploring. Note that the running speed for this test would be much slower than the pre-cropped setting, since the dataloader needs to load the full-size image first, which is slightly time-consuming for some high quality photos captured by high-end mobile phones.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiff22%2Fsqad","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faiff22%2Fsqad","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiff22%2Fsqad/lists"}