{"id":16392128,"url":"https://github.com/imdeepmind/processed-imdb-wiki-dataset","last_synced_at":"2025-03-21T02:32:16.528Z","repository":{"id":45153095,"uuid":"165409603","full_name":"imdeepmind/processed-imdb-wiki-dataset","owner":"imdeepmind","description":"Processes IMDB WIKI dataset ready to be used in any projects ","archived":false,"fork":false,"pushed_at":"2019-03-14T10:27:49.000Z","size":50209,"stargazers_count":62,"open_issues_count":2,"forks_count":12,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-17T19:52:29.322Z","etag":null,"topics":["age-classification","computer-vision","dataset","deep-learning","gender-classification","imdb-wiki-dataset","machine-learning"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/imdeepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-12T16:27:45.000Z","updated_at":"2025-02-26T16:38:38.000Z","dependencies_parsed_at":"2022-07-13T14:44:10.388Z","dependency_job_id":null,"html_url":"https://github.com/imdeepmind/processed-imdb-wiki-dataset","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imdeepmind%2Fprocessed-imdb-wiki-dataset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imdeepmind%2Fprocessed-imdb-wiki-dataset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imdeepmind%2Fprocessed-imdb-wiki-dataset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imdeepmind%2Fprocessed-imdb-wiki-dataset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/imdeepmind","download_url":"https://codeload.github.com/imdeepmind/processed-imdb-wiki-dataset/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244725582,"owners_count":20499632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["age-classification","computer-vision","dataset","deep-learning","gender-classification","imdb-wiki-dataset","machine-learning"],"created_at":"2024-10-11T04:48:44.538Z","updated_at":"2025-03-21T02:32:16.200Z","avatar_url":"https://github.com/imdeepmind.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Processed IMDB WIKI Dataset\n\nThis GitHub repository contains a preprocessed IMDB WIKI dataset.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/34741145/51108233-75bac680-1817-11e9-8b79-6a1ee05d8aa4.png\" /\u003e\n\u003c/p\u003e\n\n## Table of contents:\n- [Introduction](#introduction)\n- [IMDB WIKI Dataset](#imdb-wiki-dataset)\n- [The Problem](#the-problem)\n- [The Solution](#the-solution)\n- [File Structure](#file-structure)\n- [How to Run Locally](#how-to-run-locally)\n- [Dependencies](#dependencies)\n- [Acknowledgments](#acknowledgments)\n\n## Introduction\nIMDB WIKI dataset is the largest dataset of human faces with gender, name and age information. In this project, I preprocessed the entire dataset so that it can be used easily without any problems.\n\n\n## IMDB WIKI Dataset\nIMDB WIKI dataset is the largest publically available dataset of human faces with gender, age, and name. It contains more than `500 thousand+` images with all the meta information. All the images are in `.jpg` format. \n\nFor more information about the dataset please visit [this website](https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/).\n\n## The Problem\nThe dataset is great for research purposes. It contains more than `500 thousand+` images of faces. But the dataset is not ready for any Machine Learning algorithm. There are some problems with the dataset. \n\n  - All the images are of different size\n  - Some of the images are completely corrupted\n  - Some images don't have any faces\n  - Some of the ages are invalid\n  - The distribution between the gender is not equal(there are more male faces than female faces)\n  - Also, the meta information is in `.mat` format. Reading `.mat` files in python is a tedious process.\n\n## The Solution\nIn this project, I filter all the images, resized them all to `128x128`, remove all the images with invalid age, fix the gender distribution problem, and save them in the proper format. Along with that, I’ve also processed the `.mat` files and converted them in `.csv` files also.\n\n## File Structure\nThis repository contains 3 files\n - `mat.py`\n - `gender.py`\n - `age.py`\n\nThe first `mat.py` file converts the mat files IMDB and WIKI dataset to `.csv` format and merge them into one file.  \n\nThe last two file process the images for gender and age classification.\n\n**As the size of the dataset is huge, I can not upload it here on GitHub**\n\n\n## How to Run Locally\nFollowing are the steps for running it locally\n  - Download the dataset from [this](https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/) link and unzip it\n  - Extract the dataset and save it in the project directory\n  - After that, you should have the following folders\n    - `imdb_crop`\n    - `wiki_crop`\n  - Run the `mat.py` file\n  - Run `age.py` and `gender.py` file\n  - Now the dataset in preprocessed and ready for your project\n\n## Dependencies\n  - `Numpy=1.15.4`\n  - `Scipy=1.2.0`\n  - `pandas=0.23.4`\n  - `cv2=4.0.0`\n\n## Acknowledgments\nI really thankful to these peoples for providing this amazing dataset\n  - [IMDB-WIKI – 500k+ face images with age and gender labels](https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/)\n  - [yu4u/age-gender-estimation](https://github.com/yu4u/age-gender-estimation)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimdeepmind%2Fprocessed-imdb-wiki-dataset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimdeepmind%2Fprocessed-imdb-wiki-dataset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimdeepmind%2Fprocessed-imdb-wiki-dataset/lists"}