{"id":13586473,"url":"https://github.com/idealo/image-quality-assessment","last_synced_at":"2025-08-07T16:32:01.840Z","repository":{"id":40320008,"uuid":"137086172","full_name":"idealo/image-quality-assessment","owner":"idealo","description":"Convolutional Neural Networks to predict the aesthetic and technical quality of images.","archived":false,"fork":false,"pushed_at":"2024-07-12T19:16:31.000Z","size":81882,"stargazers_count":2093,"open_issues_count":44,"forks_count":448,"subscribers_count":51,"default_branch":"master","last_synced_at":"2024-11-26T11:09:18.887Z","etag":null,"topics":["aws","computer-vision","convolutional-neural-networks","deep-learning","e-commerce","idealo","image-quality-assessment","keras","machine-learning","mobilenet","neural-network","nima","tensorflow"],"latest_commit_sha":null,"homepage":"https://idealo.github.io/image-quality-assessment/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/idealo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-12T14:46:09.000Z","updated_at":"2024-11-26T02:42:51.000Z","dependencies_parsed_at":"2022-07-21T12:03:07.204Z","dependency_job_id":"72a9fba5-0d40-45a3-9e98-645334520875","html_url":"https://github.com/idealo/image-quality-assessment","commit_stats":{"total_commits":66,"total_committers":8,"mean_commits":8.25,"dds":"0.19696969696969702","last_synced_commit":"dceaf7c2d218bc6e80b21d6e147e3b56a21b7f31"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idealo%2Fimage-quality-assessment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idealo%2Fimage-quality-assessment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idealo%2Fimage-quality-assessment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idealo%2Fimage-quality-assessment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/idealo","download_url":"https://codeload.github.com/idealo/image-quality-assessment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227914155,"owners_count":17839212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","computer-vision","convolutional-neural-networks","deep-learning","e-commerce","idealo","image-quality-assessment","keras","machine-learning","mobilenet","neural-network","nima","tensorflow"],"created_at":"2024-08-01T15:05:35.562Z","updated_at":"2024-12-03T12:02:02.018Z","avatar_url":"https://github.com/idealo.png","language":"Python","funding_links":[],"categories":["Toolbox","Python","Repos","📦 Legacy \u0026 Inactive Projects"],"sub_categories":["Libraries"],"readme":"# Image Quality Assessment\n\n[![Build Status](https://travis-ci.org/idealo/image-quality-assessment.svg?branch=master)](https://travis-ci.org/idealo/image-quality-assessment)\n[![Docs](https://img.shields.io/badge/docs-online-brightgreen)](https://idealo.github.io/image-quality-assessment/)\n[![License](https://img.shields.io/badge/License-Apache%202.0-orange.svg)](https://github.com/idealo/image-quality-assessment/blob/master/LICENSE)\n\nThis repository provides an implementation of an aesthetic and technical image quality model based on Google's research paper [\"NIMA: Neural Image Assessment\"](https://arxiv.org/pdf/1709.05424.pdf). You can find a quick introduction on their [Research Blog](https://research.googleblog.com/2017/12/introducing-nima-neural-image-assessment.html).\n\nNIMA consists of two models that aim to predict the aesthetic and technical quality of images, respectively. The models are trained via transfer learning, where ImageNet pre-trained CNNs are used and fine-tuned for the classification task.\n\nFor more information on how we used NIMA for our specifc problem, we did a write-up on two blog posts:\n\n* NVIDIA Developer Blog: [Deep Learning for Classifying Hotel Aesthetics Photos](https://devblogs.nvidia.com/deep-learning-hotel-aesthetics-photos/)\n* Medium: [Using Deep Learning to automatically rank millions of hotel images](https://medium.com/idealo-tech-blog/using-deep-learning-to-automatically-rank-millions-of-hotel-images-c7e2d2e5cae2)\n\nThe provided code allows to use any of the pre-trained models in [Keras](https://keras.io/applications/). We further provide Docker images for local CPU training and remote GPU training on AWS EC2, as well as pre-trained models on the [AVA](https://github.com/ylogx/aesthetics/tree/master/data/ava) and [TID2013](http://www.ponomarenko.info/tid2013.htm) datasets.\n\nRead the full documentation at: [https://idealo.github.io/image-quality-assessment/](https://idealo.github.io/image-quality-assessment/).\n\nImage quality assessment is compatible with Python 3.6 and is distributed under the Apache 2.0 license. We welcome all kinds of contributions, especially new model architectures and/or hyperparameter combinations that improve the performance of the currently published models (see [Contribute](#contribute)).\n\n\n## Trained models\n| \u003csub\u003ePredictions from aesthetic model\u003c/sub\u003e\n| :--:\n| ![](readme_figures/images_aesthetic/aesthetic1.jpg_aesthetic.svg)\n\n\n| \u003csub\u003ePredictions from technical model\u003c/sub\u003e\n| :--:\n| ![](readme_figures/images_technical/techncial3.jpgtechnical.svg)\n\n\n\n\nWe provide trained models, for both aesthetic and technical classifications, that use MobileNet as the base CNN. The models and their respective config files are stored under `models/MobileNet`. They achieve the following performance\n\nModel      | Dataset | EMD  | LCC | SRCC  \n----- |  ------- | ---  | --- | ----  \nMobileNet aesthetic | AVA | 0.071 |0.626|0.609\nMobileNet technical | TID2013 | 0.107 |0.652|0.675\n\n\n\n## Getting started\n\n1. Install [jq](https://stedolan.github.io/jq/download/)\n\n2. Install [Docker](https://docs.docker.com/install/)\n\n3. Build docker image `docker build -t nima-cpu . -f Dockerfile.cpu`\n\nIn order to train remotely on **AWS EC2**\n\n4. Install [Docker Machine](https://docs.docker.com/machine/install-machine/)\n\n5. Install [AWS Command Line Interface](https://docs.aws.amazon.com/cli/latest/userguide/installing.html)\n\n\n## Predict\nIn order to run predictions on an image or batch of images you can run the prediction script\n\n1. Single image file\n    ```bash\n    ./predict  \\\n    --docker-image nima-cpu \\\n    --base-model-name MobileNet \\\n    --weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \\\n    --image-source $(pwd)/src/tests/test_images/42039.jpg\n    ```\n\n2. All image files in a directory\n    ```bash\n    ./predict  \\\n    --docker-image nima-cpu \\\n    --base-model-name MobileNet \\\n    --weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \\\n    --image-source $(pwd)/src/tests/test_images\n    ```\n\n\n## Train locally on CPU\n\n1. Download dataset (see instructions under [Datasets](#datasets))\n\n2. Run the local training script (e.g. for TID2013 dataset)\n    ```bash\n    ./train-local \\\n    --config-file $(pwd)/models/MobileNet/config_technical_cpu.json \\\n    --samples-file $(pwd)/data/TID2013/tid_labels_train.json \\\n    --image-dir /path/to/image/dir/local\n    ```\nThis will start a training container from the Docker image `nima-cpu` and create a timestamp train job folder under `train_jobs`, where the trained model weights and logs will be stored. The `--image-dir` argument requires the path of the image directory on your local machine.\n\nIn order to stop the last launched container run\n    ```bash\n    CONTAINER_ID=$(docker ps -l -q)\n    docker container stop $CONTAINER_ID\n    ```\n\nIn order to stream logs from last launched container run\n    ```bash\n    CONTAINER_ID=$(docker ps -l -q)\n    docker logs $CONTAINER_ID --follow\n    ```\n\n## Train remotely on AWS EC2\n\n1. Configure your AWS CLI. Ensure that your account has limits for GPU instances and read/write access to the S3 bucket specified in config file [[link](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html)]\n    ```bash\n    aws configure\n    ```\n\n2. Launch EC2 instance with Docker Machine. Choose an Ubuntu AMI based on your region (https://cloud-images.ubuntu.com/locator/ec2/).\nFor example, to launch a `p2.xlarge` EC2 instance named `ec2-p2` run\n(NB: change region, VPC ID and AMI ID as per your setup)\n    ```bash\n    docker-machine create --driver amazonec2 \\\n                          --amazonec2-region eu-west-1 \\\n                          --amazonec2-ami ami-58d7e821 \\\n                          --amazonec2-instance-type p2.xlarge \\\n                          --amazonec2-vpc-id vpc-abc \\\n                          ec2-p2\n    ```\n\n3. ssh into EC2 instance\n    ```bash\n    docker-machine ssh ec2-p2\n    ```\n\n4. Update NVIDIA drivers and install **nvidia-docker** (see this [blog post](https://towardsdatascience.com/using-docker-to-set-up-a-deep-learning-environment-on-aws-6af37a78c551) for more details)\n    ```bash\n    # update NVIDIA drivers\n    sudo add-apt-repository ppa:graphics-drivers/ppa -y\n    sudo apt-get update\n    sudo apt-get install -y nvidia-375 nvidia-settings nvidia-modprobe\n\n    # install nvidia-docker\n    wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb\n    sudo dpkg -i /tmp/nvidia-docker_1.0.1-1_amd64.deb \u0026\u0026 rm /tmp/nvidia-docker_1.0.1-1_amd64.deb\n    ```\n\n5. Download dataset to EC2 instance (see instructions under [Datasets](#datasets)). We recommend to save the AMI with the downloaded data for future use.\n\n6. Run the remote EC2 training script (e.g. for AVA dataset)\n    ```bash\n    ./train-ec2 \\\n    --docker-machine ec2-p2 \\\n    --config-file $(pwd)/models/MobileNet/config_aesthetic_gpu.json \\\n    --samples-file $(pwd)/data/AVA/ava_labels_train.json \\\n    --image-dir /path/to/image/dir/remote\n    ```\nThe training progress will be streamed to your terminal. After the training has finished, the train outputs (logs and best model weights) will be stored on S3 in a timestamped folder. The S3 output bucket can be specified in the **config file**. The `--image-dir` argument requires the path of the image directory on your remote instance.\n\n\n## Contribute\nWe welcome all kinds of contributions and will publish the performances from new models in the performance table under [Trained models](#trained-models).\n\nFor example, to train a new aesthetic NIMA model based on InceptionV3 ImageNet weights, you just have to change the `base_model_name` parameter in the config file `models/MobileNet/config_aesthetic_gpu.json` to \"InceptionV3\". You can also control all major hyperparameters in the config file, like learning rate, batch size, or dropout rate.\n\nSee the [Contribution](CONTRIBUTING.md) guide for more details.\n\n## Datasets\nThis project uses two datasets to train the NIMA model:\n\n1. [**AVA**](https://github.com/ylogx/aesthetics/tree/master/data/ava) used for aesthetic ratings ([data](http://academictorrents.com/details/71631f83b11d3d79d8f84efe0a7e12f0ac001460))\n2. [**TID2013**](http://www.ponomarenko.info/tid2013.htm) used for technical ratings\n\nFor training on AWS EC2 we recommend to build a custom AMI with the AVA images stored on it. This has proven much more viable than copying the entire dataset from S3 to the instance for each training job.\n\n\n## Label files\nThe **train** script requires JSON label files in the format\n```\n[\n  {\n    \"image_id\": \"231893\",\n    \"label\": [2,8,19,36,76,52,16,9,3,2]\n  },\n  {\n    \"image_id\": \"746672\",\n    \"label\": [1,2,7,20,38,52,20,11,1,3]\n  },\n  ...\n]\n```\n\nThe label for each image is the normalized or un-normalized frequency distribution of ratings 1-10.\n\nFor the AVA dataset these frequency distributions are given in the raw data files. For the TID2013 dataset we inferred the normalized frequency distribution, i.e. probability distribution, by finding the maximum entropy distribution that satisfies the mean score. The code to generate the TID2013 labels can be found under `data/TID2013/get_labels.py`.\n\nFor both datasets we provide train and test set label files stored under\n```\ndata/AVA/ava_labels_train.json\ndata/AVA/ava_labels_test.json\n```\nand\n```\ndata/TID2013/tid2013_labels_train.json\ndata/TID2013/tid2013_labels_test.json\n```\n\nFor the AVA dataset we randomly assigned 90% of samples to the train set, and 10% to the test set, and throughout training a 5% validation set will be split from the training set to evaluate the training performance after each epoch. For the TID2013 dataset we split the train/test sets by reference images, to ensure that no reference image, and any of its distortions, enters both the train and test set.\n\n## Serving NIMA with TensorFlow Serving\nTensorFlow versions of both the technical and aesthetic MobileNet models are provided,\nalong with the script to generate them from the original Keras files, under the `contrib/tf_serving` directory.\n\nThere is also an already configured TFS `Dockerfile` that you can use.\n\nTo get predictions from the aesthetic or technical model:\n1. Build the NIMA TFS Docker image `docker build -t tfs_nima contrib/tf_serving`\n2. Run a NIMA TFS container with `docker run -d --name tfs_nima -p 8500:8500 tfs_nima`\n3. Install python dependencies to run TF serving sample client\n    ```\n    virtualenv -p python3 contrib/tf_serving/venv_tfs_nima\n    source contrib/tf_serving/venv_tfs_nima/bin/activate\n    pip install -r contrib/tf_serving/requirements.txt\n    ```\n4. Get predictions from aesthetic or technical model by running the sample client\n    ```\n    python -m contrib.tf_serving.tfs_sample_client --image-path src/tests/test_images/42039.jpg --model-name mobilenet_aesthetic\n    python -m contrib.tf_serving.tfs_sample_client --image-path src/tests/test_images/42039.jpg --model-name mobilenet_technical\n    ```\n\n## Cite this work\nPlease cite Image Quality Assessment in your publications if this is useful for your research. Here is an example BibTeX entry:\n```BibTeX\n@misc{idealods2018imagequalityassessment,\n  title={Image Quality Assessment},\n  author={Christopher Lennan and Hao Nguyen and Dat Tran},\n  year={2018},\n  howpublished={\\url{https://github.com/idealo/image-quality-assessment}},\n}\n```\n\n## Maintainers\n* Christopher Lennan, github: [clennan](https://github.com/clennan)\n* Hao Nguyen, github: [MrBanhBao](https://github.com/MrBanhBao)\n* Dat Tran, github: [datitran](https://github.com/datitran)\n\n## Copyright\n\nSee [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidealo%2Fimage-quality-assessment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fidealo%2Fimage-quality-assessment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidealo%2Fimage-quality-assessment/lists"}