{"id":20881923,"url":"https://github.com/aiff22/ai-challenge","last_synced_at":"2025-07-18T03:06:17.471Z","repository":{"id":94500382,"uuid":"134177799","full_name":"aiff22/ai-challenge","owner":"aiff22","description":"A code for converting and testing Tensorflow models for AI Challenge","archived":false,"fork":false,"pushed_at":"2020-07-16T16:10:53.000Z","size":62010,"stargazers_count":29,"open_issues_count":0,"forks_count":11,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-12T17:36:35.572Z","etag":null,"topics":["ai-challenges","computer-vision","deep-learning","image-enhancement","image-processing","machine-learning","super-resolution"],"latest_commit_sha":null,"homepage":"http://ai-benchmark.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aiff22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-20T18:55:22.000Z","updated_at":"2023-03-09T06:13:55.000Z","dependencies_parsed_at":"2023-06-19T16:35:44.910Z","dependency_job_id":null,"html_url":"https://github.com/aiff22/ai-challenge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aiff22/ai-challenge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2Fai-challenge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2Fai-challenge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2Fai-challenge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2Fai-challenge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aiff22","download_url":"https://codeload.github.com/aiff22/ai-challenge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aiff22%2Fai-challenge/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265694963,"owners_count":23812486,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-challenges","computer-vision","deep-learning","image-enhancement","image-processing","machine-learning","super-resolution"],"created_at":"2024-11-18T07:27:01.452Z","updated_at":"2025-07-18T03:06:17.457Z","avatar_url":"https://github.com/aiff22.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Perceptual Image Enhancement on Smartphones\r\n\r\n### [[Challenge Website]](http://ai-benchmark.org), [[PIRM 2018]](https://www.pirm2018.org/)\r\n\r\n\u003cbr/\u003e\r\n\r\nThis repository provides a guideline and code to convert your pre-trained models into an appropriate submission format for [AI Challenge](http://ai-benchmark.org) organized in conjunction with ECCV 2018 Conference.\u003cbr/\u003e\u003cbr/\u003e\r\n\r\n\r\n#### 1. Prerequisites\r\n\r\n- [TensorFlow (\u003e=1.8)](https://www.tensorflow.org/install/)\r\n- Python (3.5+) packages: [scipy](https://pypi.org/project/scipy/), [numpy](https://pypi.org/project/numpy/), [psutil](https://pypi.org/project/psutil/)\u003cbr/\u003e\u003cbr/\u003e\r\n\r\n\r\n#### 2. General model requirements\r\n\r\n- Your model should be able to process images of \u003cb\u003earbitrary resolution\u003c/b\u003e\r\n- It should require no more than \u003cb\u003e3.5GB of RAM\u003c/b\u003e while processing HD-resolution ```[1280x720px]``` photos\r\n- Maximum model size: \u003cb\u003e100MB\u003c/b\u003e\r\n- Should be saved as \u003cb\u003eTensorflow .pb file\u003c/b\u003e\u003c/br\u003e\u003c/br\u003e\r\n\r\n\r\n#### 3. Model conversion and validation\r\n\r\nYour model should be implemented as a function that takes one single input and produces one output:\r\n\r\n- \u003cb\u003eInput:\u003c/b\u003e\u0026nbsp; Tensorflow 4-dimensional tensor of shape ```[batch_size, image_height, image_width, 3]```\u003c/br\u003e\u003csub\u003eIn the Super-Resolution task, input images are already \u003cb\u003ebicubically interpolated (x4)\u003c/b\u003e and have the same size as target high-resolution photos\u003c/br\u003e\u003c/sub\u003e\r\n- \u003cb\u003eOutput:\u003c/b\u003e\u0026nbsp; Same as input\r\n- \u003cb\u003eValues:\u003c/b\u003e\u0026nbsp; The values of both original and processed images should lie in the \u003cb\u003e```interval [0, 1]```\u003c/b\u003e\r\n\r\nHere is a valid SRCNN function provided for you as a reference:\u003c/br\u003e\r\n\r\n\u003csub\u003e\r\n\r\n```bash\r\ndef srcnn(images):\r\n\r\n    with tf.variable_scope(\"generator\"):\r\n\r\n        weights = {\r\n          'w1': tf.Variable(tf.random_normal([9, 9, 3, 64], stddev=1e-3), name='w1'),\r\n          'w2': tf.Variable(tf.random_normal([5, 5, 64, 32], stddev=1e-3), name='w2'),\r\n          'w3': tf.Variable(tf.random_normal([5, 5, 32, 3], stddev=1e-3), name='w3')\r\n        }\r\n\r\n        biases = {\r\n          'b1': tf.Variable(tf.zeros([64]), name='b1'),\r\n          'b2': tf.Variable(tf.zeros([32]), name='b2'),\r\n          'b3': tf.Variable(tf.zeros([1]), name='b3')\r\n        }\r\n\r\n        conv1 = tf.nn.relu(tf.nn.conv2d(images, weights['w1'], strides=[1,1,1,1], padding='SAME') + biases['b1'])\r\n        conv2 = tf.nn.relu(tf.nn.conv2d(conv1, weights['w2'], strides=[1,1,1,1], padding='SAME') + biases['b2'])\r\n        conv3 = tf.nn.conv2d(conv2, weights['w3'], strides=[1,1,1,1], padding='SAME') + biases['b3']\r\n\r\n    return tf.nn.tanh(conv3) * 0.58 + 0.5\r\n```\r\n\u003c/sub\u003e\r\n\r\nTo test and convert your pre-trained models, run the following scripts:\r\n\r\n- \u003cb\u003eTrack A,\u003c/b\u003e Image Super-Resolution:\u0026nbsp; \u003cb\u003e```evaluate_super_resolution.py```\u003c/b\u003e\r\n- \u003cb\u003eTrack B,\u003c/b\u003e Image Enhancement:\u0026nbsp; \u003cb\u003e```evaluate_enhancement.py```\u003c/b\u003e\r\n\r\nYou need to modify two lines in the headers of the above scripts:\r\n\r\n```bash\r\nfrom \u003cmodel_file\u003e import \u003cyour_model\u003e as test_model\r\nmodel_location = \"path/to/your/saved/pre-trained/model\"\r\n```\r\n\r\nHere, \u003cb\u003e```model_file.py```\u003c/b\u003e should be a python file containing your model definition, \u003cb\u003e```your_model```\u003c/b\u003e is the actual function that defines your model, and \u003cb\u003e```model_location```\u003c/b\u003e points to your saved pre-trained model file.\u003c/br\u003e\r\n\r\nAfter running these scripts, they will:\r\n\r\n1. save your model as ```model.pb``` file stored in ```models_pretrained/``` folder\r\n2. compute PSNR/SSIM scores on a subset of validation images/patches\r\n3. compute running time and \u003cb\u003eestimated\u003c/b\u003e RAM consumption for HD-resolution images\u003c/br\u003e\u003c/br\u003e\r\n\r\n#### 4. Provided pre-trained models\r\n\r\nApart from the validation scripts, we also provide you several pre-trained models that can be restored and validated using the same scripts. In all cases, model architectures are defined in the ```models.py``` file.\r\n\r\n\u003cb\u003eSuper-resolution task:\u003c/b\u003e\r\n\r\n1. SRCNN, function: ```srcnn```, pre-trained model: ```models_pretrained/div2k_srcnn```\r\n2. ResNet with one residual block, function: ```resnet_6_16```, pre-trained model: ```models_pretrained/div2k_resnet_6_16```\r\n3. VGG-19, function: ```vgg_19```, pre-trained model: ```models_pretrained/div2k_vgg19_vdsr.ckpt```\r\n\r\n\u003cb\u003eImage Enhancement task:\u003c/b\u003e\r\n\r\n1. SRCNN, function: ```srcnn```, pre-trained model: ```models_pretrained/dped_srcnn```\r\n2. ResNet with 4 residual blocks, function: ```resnet_12_64```, pre-trained model: ```models_pretrained/dped_resnet_12_64```\r\n3. ResNet with 2 residual blocks, function: ```resnet_8_32```, pre-trained model: ```models_pretrained/dped_resnet_8_32```\u003c/br\u003e\u003c/br\u003e\r\n\r\n#### 5. Team registration and model submission\r\n\r\nTo register your team, send an email to \u003cb\u003e```ai.mobile.challenge@gmail.com```\u003c/b\u003e with the following information:\r\n\r\n```bash\r\nEmail Subject:  AI Mobile Challenge Registration\r\n\r\nEmail Text:     Team Name\r\n                Team Member 1 (Name, Surname, Affiliation)\r\n                Team Member 2 (Name, Surname, Affiliation)\r\n                ....\r\n```\r\n\r\nTo validate your model, send an email indicating the ```track```, ```team id``` and the corresponding ```model.pb``` file:\r\n\r\n```bash\r\nEmail Subject:  [Track X] [Team ID] [Team Name] Submission\r\n\r\nEmail Text:     Link to model.pb file\r\n```\r\nYou are allowed to send up to ```2 submissions per day``` for each track. The leaderboard will show the results of your last successful submission. Please make sure that the results provided by our validation scripts are meaningful before sending your submission files.\u003c/br\u003e\u003c/br\u003e\r\n\r\n\r\n#### 6. Scoring formulas\r\n\r\nThe performance of your solution will be assessed based on three metrics: its speed compared to a baseline network, its fidelity score measured by PSNR, and its perceptual score computed based on MS-SSIM metric. Since PSNR and SSIM scores do not always objectively reflect image quality, during the test phase we will conduct a user study where your final submissions will be evaluated by a large number of people, and the resulting MOS Scores will replace MS-SSIM results. The total score of your solution will be calculated as a weighted average of the previous scores:\r\n\r\n```bash\r\nTotalScore = α * (PSNR_solution - PSNR_baseline) + β * (SSIM_solution - SSIM_baseline) + γ * min(Time_baseline / Time_solution, 4) \r\n```\r\n\r\nWe will use three different validation tracks for evaluating your results. Score A is giving preference to solution with the highest fidelity (PSNR) score, score B is aimed at the solution providing the best visual results (MS-SSIM/MOS scores), and score C is targeted at the best balance between the speed and perceptual/quantitative performance. For each track, we will use the above scoring formula but with different coefficients:\r\n\r\n\u003cb\u003eTrack A (Super-Resolution):\u003c/b\u003e\r\n\r\n- ```PSNR_baseline``` = 26.5, ```SSIM_baseline``` = 0.94\r\n- (```α```, ```β```, ```γ```): \u0026nbsp; score A - (4, 100, 1); \u0026nbsp; score B - (1, 400, 1); \u0026nbsp; score C - (2, 200, 1.5)\r\n\r\n\u003cb\u003eTrack B (Image Enhancement):\u003c/b\u003e\r\n\r\n- ```PSNR_baseline``` = 21, ```SSIM_baseline``` = 0.9\r\n- (```α```, ```β```, ```γ```): \u0026nbsp; score A - (4, 100, 2); \u0026nbsp; score B - (1, 400, 2); \u0026nbsp; score C - (2, 200, 2.9)\u003c/br\u003e\u003c/br\u003e\r\n\r\n\r\n#### 7. Other remarks\r\n\r\n- Note that the provided code is used only for preliminary model validation, while all final numbers will be obtained by us by testing all submissions on the test parts of the datasets (accuracy) and on the same hardware (speed)\r\n\r\n- To check the above RAM requirements, we will run your submissions on a GPU with 3.5GB of RAM.\u003cbr/\u003e In case this won't be enough for your model, it will be disqualified from the final validation stage\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiff22%2Fai-challenge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faiff22%2Fai-challenge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faiff22%2Fai-challenge/lists"}