{"id":16098880,"url":"https://github.com/defasium/expertglasses","last_synced_at":"2025-04-12T16:12:40.658Z","repository":{"id":126671368,"uuid":"288726570","full_name":"Defasium/expertglasses","owner":"Defasium","description":"Expert eyeglasses recommendation system with Generative Adversarial Networks written in Python, 2020.","archived":false,"fork":false,"pushed_at":"2020-08-27T21:01:59.000Z","size":6563,"stargazers_count":10,"open_issues_count":1,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-26T10:36:28.672Z","etag":null,"topics":["expert","expert-eyeglasses-recommendation","eyeglasses","gan","keras","performance-benchmarks","recommender-system","resolution-gan"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Defasium.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-19T12:30:45.000Z","updated_at":"2025-02-10T13:40:56.000Z","dependencies_parsed_at":"2023-03-09T02:31:00.974Z","dependency_job_id":null,"html_url":"https://github.com/Defasium/expertglasses","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defasium%2Fexpertglasses","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defasium%2Fexpertglasses/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defasium%2Fexpertglasses/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defasium%2Fexpertglasses/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Defasium","download_url":"https://codeload.github.com/Defasium/expertglasses/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248594140,"owners_count":21130313,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expert","expert-eyeglasses-recommendation","eyeglasses","gan","keras","performance-benchmarks","recommender-system","resolution-gan"],"created_at":"2024-10-09T18:24:57.731Z","updated_at":"2025-04-12T16:12:40.639Z","avatar_url":"https://github.com/Defasium.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ExpertGlasses\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/schema.jpg\" alt=\"title\" width=\"100%\"/\u003e    \n\u003c/p\u003e\n\nExpert eyeglasses recommendation system with Generative Adversarial Networks written in Python, 2020.\n\nLive Demo: [Glasses:eyes:Expert](http://glasses-expert.herokuapp.com/)\n____\n\n## Table of Contents\n  * [Description](#description)\n  * [Installation](#installation)\n  * [Reminder about API-services](#reminder-about-api-services)\n  * [Additional features](#additional-features)\n    + [Download pretrained GAN models](#download-pretrained-gan-models)\n    + [Download images for A/B testing](#download-images-for-ab-testing)\n  * [How to use ExpertGlasses](#how-to-use-expertglasses)\n    + [In code](#in-code)\n    + [GUI](#gui)\n  * [Performance benchmarks](#performance-benchmarks)\n  * [Sources](#sources)\n____\n## Description\n\nThe task of recommending eyeframes based on people's appearance is quite hard and there are no similar projects available at the moment. This system takes into account over 20 facial attributes, maps them into eyeglasses features with the help of expert module (which consists with over 40 branches). Expert module apply written beforehand rules to get necessary mappings. High interpretability of such approach guaranties user's understanding and loyality towards the system.\n\nTo get necessary attributes from the photo this system uses bunch of machine learning algorithms and existing free services, i.e [BetaFaceAPI](https://www.betafaceapi.com/wpa/) and [Face++](https://www.faceplusplus.com/). Face++ allows to locate bounding boxes of faces in images, while BetaFace uses various classifiers to get most of the secondary-importance features. So to use this repository, you should have access to the internet.\n\nTo detect face shape probabilities, iris color, forehead size, jawtype type, skintone the system uses own pretrained convolutional neural networks (CNNs). These models run on your local machine on CPU.\n\nTo get all necessary eyeframes attributes, the large dataset (\u003e8k records) of eyeframes was parsed and processed. Because in real life there are not so many eyeframe models in the local shop available, the generation of unique eyewear by given features was implemented. The system use a conditional GAN followed by Super Resolution GAN to create non-existing high-definition images of the eyeframes.\n\n____\n\n## Installation\nRequirements:\n* Python 3.6 or greater\n* requests\n* numpy\n* pandas\n* opencv_python_headless\n* scikit_image\n* tensorflow\n* Keras_Applications\n* PySimpleGUI\n\nFor the full list of requirements consider to check requirements.txt\n\nTo install repository, use the following commands:\n\n__`Windows`__:\n\n    $ git clone https://github.com/Defasium/expertglasses\n    $ cd expertglasses/\n    $ pip3 install -r requirements.txt\n\n__`Linux`__:\n\n    $ git clone https://github.com/Defasium/expertglasses\n    $ cd expertglasses/\n    $ sudo pip3 install -r requirements.txt\n\n__`Linux`__ inside virtualenv:\n\n    $ git clone https://github.com/Defasium/expertglasses\n    $ cd expertglasses/\n    $ pip3 install -r requirements.txt\n\n\n## Reminder about API-services\n\n__In current version without Face++ api key you can't use Expert Recommender System!__\n\nWhile BetaFace api has the same free api key for everyone with some limitations, allowing you to upload up to 500 images from your IP per day. Face++ utilizes a different strategy, requiring you to create an account on their site. In this account you can generate your own free api key and api secret in the Apps -\u003e API key section:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/api_key.png\" alt=\"title\"/\u003e    \n\u003c/p\u003e\n\nAfter that consider changing values of API_KEY and API_SECRET in the [faceplusplus.py](faceplusplus.py) module:\n```Python\n\n...\nAPI_KEY = \"api_key\"\nAPI_SECRET = \"api_secret\"\nRETURN_ATTRIBUTES = \"gender,beauty,age,smiling,headpose,ethnicity,skinstatus\"\nRETURN_LANDMARK = 1\n...\n```\n\n## Additional features\n\nIn the github repository there are no currently some functionality, e.g. pretrained GAN models and best selling eyeglasses. If you want to include this features in the system, you need to download necessary files. Links are listed bellow.  \n\n### Download pretrained GAN models\n| Storage Service | Link |\n| ---- | ----------- |\n| \u003cp align=\"center\"\u003e\u003cimg title=\"GDrive\" src=\"http://ssl.gstatic.com/docs/doclist/images/infinite_arrow_favicon_5.ico\" width=\"50px\"/\u003e\u003c/p\u003e|[GDrive](https://drive.google.com/drive/folders/12KfsfZH25Ov7HvFVeIFVpANjvcjRthCf)|\n| \u003cp align=\"center\"\u003e\u003cimg title=\"DropBox\" src=\"https://cfl.dropboxstatic.com/static/images/logo_catalog/logo_m1.png\" width=\"50px\"/\u003e\u003c/p\u003e|[DropBox](https://www.dropbox.com/sh/iwolbxkr90rkuic/AABifNPYEQVO_t9wfycTy9upa?dl=0)|\n| \u003cp align=\"center\"\u003e\u003cimg title=\"Mega\" src=\"https://mega.nz/favicon.ico?v=3\" width=\"50px\"/\u003e\u003c/p\u003e|[Mega](https://mega.nz/folder/JpdDmA4C#ou62DrQCzo4uhs_qt8iACw)|\n\n\nDownload 2 files, __cgan.h5__ and __srgan.h5__, and place them in __utils/__ directory. \n\n### Download images for A/B testing\n| Storage Service | Link |\n| ---- | ----------- |\n| \u003cp align=\"center\"\u003e\u003cimg title=\"GDrive\" src=\"http://ssl.gstatic.com/docs/doclist/images/infinite_arrow_favicon_5.ico\" width=\"50px\"/\u003e\u003c/p\u003e|[GDrive](https://drive.google.com/file/d/1aL60K5BX1kXQuM5xJC_jg9VMorPgotpL)|\n| \u003cp align=\"center\"\u003e\u003cimg title=\"DropBox\" src=\"https://cfl.dropboxstatic.com/static/images/logo_catalog/logo_m1.png\" width=\"50px\"/\u003e\u003c/p\u003e|[DropBox](https://www.dropbox.com/s/s00vugkaau9twcs/abtest.zip?dl=0)|\n| \u003cp align=\"center\"\u003e\u003cimg title=\"Mega\" src=\"https://mega.nz/favicon.ico?v=3\" width=\"50px\"/\u003e\u003c/p\u003e|[Mega](https://mega.nz/folder/Bt9SRKqI#sjmJPOKF2VWgNxgt80ItmQ)|\n\nDownload files, create a directory named __abtest/__ and extract __man/__ and __woman/__ directories inside of it.\n\n## How to use ExpertGlasses\n\nAt the current state there are only 2 ways of using expert eyeglasses recommender system: in your python code and via GUI.\n\n### In code\n\nTo use this system, simply import class in your python code:\n```Python\nfrom expert_backend import ExpertEyeglassesRecommender\n```\n\nAfter that create an instance of this class with a specified path to the image:\n```Python\nins = ExpertEyeglassesRecommender('test.jpg')\n\n# by passing a `lang` parameter you can specify language, which will be used at explanation step\nins = ExpertEyeglassesRecommender('test.jpg', lang='en')\n```\nInitialization of class may take quite a long time (from 30 second up to 2 minutes). \n____\n\nAfter initialization recomendations will be completed and to get the top 6 best images of eyeglasses use:\n```Python\nins.plot_recommendations()\n```\nBy changing strategy value you can obtain different results:\n```Python\n# in standart strategy feature with the biggest value impacts the results the most\nins.plot_recommendations(strategy='standart')\n\n# in factorized strategy (default value) features are divided into three main groups:\n# shape of eyewear, its rim and other features, thus making results more various\nins.plot_recommendations(strategy='factorized')\n\n# in factorized plus strategy every feature greatly influence the final result\n# thus providing many varios nonsimilar eyeframes\nins.plot_recommendations(strategy='factorized_plus')\n\n# in color only strategy the resulting images will have the same color as in color vector\nins.plot_recommendations(strategy='color_only')\n\n# shape only strategy's results are similar to standart's one but doesn't take into account color\nins.plot_recommendations(strategy='shape_only')\n```\nIf you have [__downloaded abtest directory__](#download-images-for-ab-testing), you can also try recommending random most popular eyeglasses (so the system don't use any information about your appearance):\n```Python\n# will output 6 randomly chosen eyeglasses \nins.plot_recommendations(strategy='most_popular')\n```\n____\n\nTo get explanation of the system try:\n```Python\nprint(ins.description)\n```\n____\n\nTo work with new image use:\n```Python\nins.update_image('test2.jpg')\nins.expert_module()\n```\n\nYou can alse use url as an argument:\n```Python\nins.update_image('https://github.com/Defasium/expertglasses/blob/master/assets/gui.png?raw=true')\nins.expert_module()\n```\n\n____\n\nTo generate unique image with GANs (implying that you have downloaded pretrained models, look [__Download pretrained GAN models__](#download-pretrained-gan-models) section) use:\n```Python\nimage = ins.generate_unique(show=True)\n```\nWill generate eyeglasses according to features, plot it with matplotlib if show is True and returns an numpy.ndarray\n____\n\n### GUI\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/gui.png\"\u003e\n\u003c/p\u003e\n\nGraphical user interface is powered by PySimpleGui framework, which allows fast and simple prototyping for cross-platform systems.\n\nTo launch GUI, use the following command from terminal:\n\n    $ python gui.py\n    \nIn the first popup window you should load an image to process, i.e. the fast start, module would automatically calculate all the necessary stuff in background. After that all machine learning models will be loaded in RAM (loading usually takes from 30 seconds up to 2 minutes). When progressbar will be filled, the main interface will appear.\n\n____\n\n## Performance benchmarks\n\nIn this section you can find information about current performance of the system, speed benchmarks and so on.\n\nSpeed benchmarks were tested on 2 different machines:\n * Windows 10 with 4 cores 2.3 GHz Intel® Core™ i5-4200U and 4 GB of RAM (low spec)\n * Linux Ubuntu 16.04 LTS with 4 cores 3.4 GHz Intel® Core™ i7-4770 and 20 GB of RAM (high spec)\n\nThe results are listed below:\n \n\n|Action|Time, low spec|Time, high spec|Standart deviation, low spec|Standart deviation, high spec|\n|:--------------|:--:|:--:|---:|---:|\n|Image alignment|4.7s|__3.9s__|2.4s|__0.5s__|\n|Cached image alignment|62.0ms|__39.1ms__|55.0ms|__5.7ms__|\n|BetaFace api request|__1.5s__|1.6s|0.4s|__0.4s__|\n|Cached betaFace api request|2.0μs|__0.7μs__|__0.2μs__|0.3μs|\n|Extraction of facial attributes|2.7s|__2.3s__|1.6s|__1.2s__|\n|Cached extraction of facial attributes|3.0μs|__2.4μs__|__0.1μs__|1.8μs|\n|Features translation|0.78s|__0.11s__|0.62s|__0.54s__|\n|Cached features translation|15.0μs|__3.2μs__|12.1μs|__1.6μs__|\n|Database search with cosine distances|1.1s|__0.8s__|0.4s|__0.4s__|\n|Unique eyeglasses generation|4.2s|__4.0s__|__0.2s__|1.4s|\n|Initialization of class instance|44.7s|__27.2s__|__1.9s__|2.7s|\n\n____\n\nTo measure quality of recommendations A/B testing was made with ~50 participants. After that normalized discounted cumulative gain at top 5 items was calculated. The results are shown below:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/abtest.png\"\u003e\n\u003c/p\u003e\n\nAs you can see, expert recommender system outperforms the results of randomly chosen best selling eyeframes. \n____\n\n## Sources\n\nFor classifiers this system uses modified version of ShuffleNetV2, implemented by [@opconty](https://github.com/opconty/keras-shufflenetV2).\n\nFor upscaling generated images this system uses modified version of [@krasserm](https://github.com/krasserm/super-resolution) Super Resolution GAN implementation.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefasium%2Fexpertglasses","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdefasium%2Fexpertglasses","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefasium%2Fexpertglasses/lists"}