{"id":29605600,"url":"https://github.com/freedomofkeima/transfer-learning-anime","last_synced_at":"2025-07-20T16:06:44.688Z","repository":{"id":69442108,"uuid":"110232558","full_name":"freedomofkeima/transfer-learning-anime","owner":"freedomofkeima","description":"Transfer Learning for Anime Characters Recognition","archived":false,"fork":false,"pushed_at":"2017-11-18T11:32:27.000Z","size":65278,"stargazers_count":187,"open_issues_count":2,"forks_count":24,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-04-17T21:47:42.447Z","etag":null,"topics":["anime","classification","computer-vision","opencv","python","tensorflow","transfer-learning","transferlearning"],"latest_commit_sha":null,"homepage":"https://freedomofkeima.com/moeflow/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/freedomofkeima.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-11-10T10:03:25.000Z","updated_at":"2024-03-20T23:47:07.000Z","dependencies_parsed_at":"2023-04-18T02:45:53.361Z","dependency_job_id":null,"html_url":"https://github.com/freedomofkeima/transfer-learning-anime","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/freedomofkeima/transfer-learning-anime","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freedomofkeima%2Ftransfer-learning-anime","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freedomofkeima%2Ftransfer-learning-anime/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freedomofkeima%2Ftransfer-learning-anime/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freedomofkeima%2Ftransfer-learning-anime/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/freedomofkeima","download_url":"https://codeload.github.com/freedomofkeima/transfer-learning-anime/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freedomofkeima%2Ftransfer-learning-anime/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266152703,"owners_count":23884563,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anime","classification","computer-vision","opencv","python","tensorflow","transfer-learning","transferlearning"],"created_at":"2025-07-20T16:06:44.223Z","updated_at":"2025-07-20T16:06:44.682Z","avatar_url":"https://github.com/freedomofkeima.png","language":"Python","funding_links":[],"categories":["Image Recognition"],"sub_categories":[],"readme":"# Transfer Learning for Anime Characters\n\n**Warning**: This repository size is quite big (approx. 100 MB) since it includes training and test images.\n\n## Introduction\n\nThis repository is the continuation of [Flag #15 - Image Recognition for Anime Characters](http://freedomofkeima.com/blog/posts/flag-15-image-recognition-for-anime-characters).\n\nIn Flag #15, we can see that Transfer Learning works really well with 3 different anime characters: Nishikino Maki, Kotori Minami, and Ayase Eli.\n\n![](screenshots/first_3_characters.png)\n\nIn this experiment, we will try to push Transfer Learning further, by using 3 different anime characters which have hair color similarity: Nishikino Maki, Takimoto Hifumi, and Sakurauchi Riko.\n\n![](screenshots/current_3_characters.png)\n\nThis experiment has 3 main steps:\n1. Utilize `lbpcascade_animeface` to recognize character face from each images\n2. Resize each images to 96 x 96 pixels\n3. Split images into training \u0026 test before creating the final model\n\n![](screenshots/schema.png)\n\n`raw` directory contains 36 images for each characters (JPG \u0026 PNG format). The first 30 images are used for training while the last 6 images are used for test.\n\n![](screenshots/takimoto_hifumi_raw.png)\n\nAs an example, we got the following result after applying Step 1 (`cropped` directory is shown at the right side):\n\n![](screenshots/process.png)\n\n`lbpcascade_animeface` can detect character faces with an accuracy of around **83%**. Failed images are stored in `raw (unrecognized)` for future improvements.\n\nSince we have 3 characters and 6 test images for each which are not part of training, `resized_for_test` contains 18 images in total. Surprisingly, almost **all characters** are detected properly!\n\n**Update (Nov 13, 2017)**: See `animeface-2009` section below, which push face detection accuracy to **93**%.\n\n## Requirements\n\n- OpenCV (https://github.com/opencv/opencv)\n- TensorFlow (https://github.com/tensorflow/tensorflow)\n\n## Steps\n\n1. The following command is used to populate `cropped` directory.\n\n```\n$ python bulk_convert.py raw/[character_name] cropped\n```\n\n2. The following command is used to populate `resized_for_training` \u0026 `resized_for_test` directory.\n\n```\n$ python bulk_resize.py cropped/[character_name] resized\n```\n\nAfter running the step above, you can decide how many images will be used in `resized_for_training` and how many images will be used in `resized_for_test`.\n\n3. Re-train the Inception model by using transfer learning:\n\n```\n$ bazel-bin/tensorflow/examples/image_retraining/retrain --image_dir ~/transfer-learning-anime/resized_for_traning/\n$ bazel build tensorflow/examples/image_retraining:label_image\n```\n\n4. At this point, the model is ready to use. We can run the following command to get the classification result:\n\n```\n$ bazel-bin/tensorflow/examples/image_retraining/label_image --graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt --output_layer=final_result:0 --image=$HOME/transfer-learning-anime/resized_for_test/[character name]/[image name]\n```\n\nIf everything works properly, you will get the classification result.  See [TensorFlow Documentation](https://www.tensorflow.org/tutorials/image_retraining) for more options.\n\nOptionally, sample model can be downloaded by running `download_model.sh` script inside  `models (example)` directory.\n\n## Result Analysis\n\nInitially, we run the experiment with 2 characters: Nishikino Maki and Takimoto Hifumi.\n\n```\nINFO:tensorflow:2017-11-10 08:50:36.151387: Step 3999: Train accuracy = 100.0%\nINFO:tensorflow:2017-11-10 08:50:36.151592: Step 3999: Cross entropy = 0.002191\nINFO:tensorflow:2017-11-10 08:50:36.210147: Step 3999: Validation accuracy = 100.0% (N=100)\nINFO:tensorflow:Final test accuracy = 92.9% (N=14)\n```\n\nThe result is as the following:\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test/nishikino_maki/31.jpg) | nishikino maki (score = 0.99874)\u003cbr\u003etakimoto hifumi (score = 0.00126) | OK |\n| ![](resized_for_test/nishikino_maki/32.jpg) | nishikino maki (score = 0.75519)\u003cbr\u003etakimoto hifumi (score = 0.24481) | OK |\n| ![](resized_for_test/nishikino_maki/33.jpg) | nishikino maki (score = 0.99513)\u003cbr\u003etakimoto hifumi (score = 0.00487) | OK |\n| ![](resized_for_test/nishikino_maki/34.jpg) | nishikino maki (score = 0.98629)\u003cbr\u003etakimoto hifumi (score = 0.01371) | OK |\n| ![](resized_for_test/nishikino_maki/35.jpg) | nishikino maki (score = 0.99723)\u003cbr\u003etakimoto hifumi (score = 0.00277) | OK |\n| ![](resized_for_test/nishikino_maki/36.jpg) | nishikino maki (score = 0.99695)\u003cbr\u003etakimoto hifumi (score = 0.00305) | OK |\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test/takimoto_hifumi/31.jpg) | takimoto hifumi (score = 0.63084)\u003cbr\u003enishikino maki (score = 0.36916) | OK|\n| ![](resized_for_test/takimoto_hifumi/32.jpg) | takimoto hifumi (score = 0.99728)\u003cbr\u003enishikino maki (score = 0.00272) | OK |\n| ![](resized_for_test/takimoto_hifumi/33.jpg) | takimoto hifumi (score = 0.99972)\u003cbr\u003enishikino maki (score = 0.00028) | OK |\n| ![](resized_for_test/takimoto_hifumi/34.jpg) | takimoto hifumi (score = 0.98852)\u003cbr\u003enishikino maki (score = 0.01148) | OK |\n| ![](resized_for_test/takimoto_hifumi/35.jpg) | takimoto hifumi (score = 0.99456)\u003cbr\u003enishikino maki (score = 0.00544) | OK |\n| ![](resized_for_test/takimoto_hifumi/36.jpg) | takimoto hifumi (score = 0.96630)\u003cbr\u003enishikino maki (score = 0.03370) | OK |\n\nFrom the result above, 10 out of 12 have threshold \u003e 0.95, while the lowest threshold is 0.63.\n\nAt this point, I decided to add Sakurauchi Riko, which is known for its similarity to Nishikino Maki.\n\n```\nINFO:tensorflow:2017-11-10 13:13:59.270717: Step 3999: Train accuracy = 100.0%\nINFO:tensorflow:2017-11-10 13:13:59.270912: Step 3999: Cross entropy = 0.005526\nINFO:tensorflow:2017-11-10 13:13:59.328139: Step 3999: Validation accuracy = 100.0% (N=100)\nINFO:tensorflow:Final test accuracy = 80.0% (N=15)\n```\n\nWith 3 similar characters, the result is as the following:\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test/nishikino_maki/31.jpg) | nishikino maki (score = 0.99352)\u003cbr\u003esakurauchi riko (score = 0.00612)\u003cbr\u003etakimoto hifumi (score = 0.00036) | OK |\n| ![](resized_for_test/nishikino_maki/32.jpg) | nishikino maki (score = 0.47391)\u003cbr\u003esakurauchi riko (score = 0.37913)\u003cbr\u003etakimoto hifumi (score = 0.14696) | OK |\n| ![](resized_for_test/nishikino_maki/33.jpg) | nishikino maki (score = 0.95976)\u003cbr\u003esakurauchi riko (score = 0.02797)\u003cbr\u003etakimoto hifumi (score = 0.01227) | OK |\n| ![](resized_for_test/nishikino_maki/34.jpg) | nishikino maki (score = 0.88851)\u003cbr\u003esakurauchi riko (score = 0.07526)\u003cbr\u003etakimoto hifumi (score = 0.03623) | OK |\n| ![](resized_for_test/nishikino_maki/35.jpg) | nishikino maki (score = 0.99025)\u003cbr\u003esakurauchi riko (score = 0.00766)\u003cbr\u003etakimoto hifumi (score = 0.00209) | OK |\n| ![](resized_for_test/nishikino_maki/36.jpg) | nishikino maki (score = 0.96782)\u003cbr\u003esakurauchi riko (score = 0.02783)\u003cbr\u003etakimoto hifumi (score = 0.00435) | OK |\n\nAs you can see above, the similarity between Nishikino Maki and Sakurauchi Miko starts to lower down the confidence level of the resulted model. Nevertheless, all classifications are still correct, where 4 out of 6 maintain the threshold of \u003e 0.95.\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test/takimoto_hifumi/31.jpg) | takimoto hifumi (score = 0.86266)\u003cbr\u003enishikino maki (score = 0.13632)\u003cbr\u003esakurauchi riko (score = 0.00102) | OK |\n| ![](resized_for_test/takimoto_hifumi/32.jpg) | takimoto hifumi (score = 0.87614)\u003cbr\u003esakurauchi riko (score = 0.12334)\u003cbr\u003enishikino maki (score = 0.00051) | OK |\n| ![](resized_for_test/takimoto_hifumi/33.jpg) | takimoto hifumi (score = 0.99964)\u003cbr\u003esakurauchi riko (score = 0.00023)\u003cbr\u003enishikino maki (score = 0.00013) | OK |\n| ![](resized_for_test/takimoto_hifumi/34.jpg) | takimoto hifumi (score = 0.99417)\u003cbr\u003enishikino maki (score = 0.00472)\u003cbr\u003esakurauchi riko (score = 0.00110) | OK |\n| ![](resized_for_test/takimoto_hifumi/35.jpg) | takimoto hifumi (score = 0.94923)\u003cbr\u003esakurauchi riko (score = 0.04842)\u003cbr\u003enishikino maki (score = 0.00235) | OK |\n| ![](resized_for_test/takimoto_hifumi/36.jpg) | takimoto hifumi (score = 0.96029)\u003cbr\u003esakurauchi riko (score = 0.02822)\u003cbr\u003enishikino maki (score = 0.01150) | OK |\n\nInterestingly, the addition of 3rd character increases the confidence level of several Takimoto Hifumi testcases (see 1st and 4th result). Overall, this character can be easily differentiated compared to the other two.\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test/sakurauchi_riko/31.jpg) | sakurauchi riko (score = 0.98747)\u003cbr\u003etakimoto hifumi (score = 0.01054)\u003cbr\u003enishikino maki (score = 0.00199) | OK |\n| ![](resized_for_test/sakurauchi_riko/32.jpg) | sakurauchi riko (score = 0.96840)\u003cbr\u003etakimoto hifumi (score = 0.02895)\u003cbr\u003enishikino maki (score = 0.00265) | OK |\n| ![](resized_for_test/sakurauchi_riko/33.jpg) | sakurauchi riko (score = 0.97713)\u003cbr\u003enishikino maki (score = 0.02167)\u003cbr\u003etakimoto hifumi (score = 0.00119) | OK |\n| ![](resized_for_test/sakurauchi_riko/34.jpg) | sakurauchi riko (score = 0.90159)\u003cbr\u003enishikino maki (score = 0.06989)\u003cbr\u003etakimoto hifumi (score = 0.02852) | OK |\n| ![](resized_for_test/sakurauchi_riko/35.jpg) | sakurauchi riko (score = 0.99713)\u003cbr\u003etakimoto hifumi (score = 0.00184)\u003cbr\u003enishikino maki (score = 0.00103) | OK |\n| ![](resized_for_test/sakurauchi_riko/36.jpg) | sakurauchi riko (score = 0.79957)\u003cbr\u003enishikino maki (score = 0.19310)\u003cbr\u003etakimoto hifumi (score = 0.00733) | OK |\n\nFrom this experiment, it seems that the current bottleneck is located at Step 1 (face detection), which have the overall accuracy of 83% in face detection.\n\n## animeface-2009\n\n[nagadomi/animeface-2009](https://github.com/nagadomi/animeface-2009) provides another method of face detection. 13 out of 21 unrecognized images are now recognized in `cropped (unrecognized)` directory.\n\n**Current found limitations**: it seems the script requires more memory and slower to run compared to `lbpcascade_animeface.xml`.\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test_unrecognized/nishikino_maki/1.jpg) | nishikino maki (score = 0.99296)\u003cbr\u003esakurauchi riko (score = 0.00694)\u003cbr\u003etakimoto hifumi (score = 0.00010) | OK |\n| ![](resized_for_test_unrecognized/nishikino_maki/3.jpg) | nishikino maki (score = 0.93702)\u003cbr\u003esakurauchi riko (score = 0.04017)\u003cbr\u003etakimoto hifumi (score = 0.02281) | OK |\n| ![](resized_for_test_unrecognized/nishikino_maki/4.jpg) | nishikino maki (score = 0.99406)\u003cbr\u003esakurauchi riko (score = 0.00565)\u003cbr\u003etakimoto hifumi (score = 0.00030) | OK |\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/2.jpg) | takimoto hifumi (score = 0.99242)\u003cbr\u003enishikino maki (score = 0.00431)\u003cbr\u003esakurauchi riko (score = 0.00327) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/3.jpg) | takimoto hifumi (score = 0.99596)\u003cbr\u003esakurauchi riko (score = 0.00403)\u003cbr\u003enishikino maki (score = 0.00001) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/4.jpg) | takimoto hifumi (score = 0.98369)\u003cbr\u003esakurauchi riko (score = 0.01498)\u003cbr\u003enishikino maki (score = 0.00133) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/6.jpg) | takimoto hifumi (score = 0.99796)\u003cbr\u003esakurauchi riko (score = 0.00189)\u003cbr\u003enishikino maki (score = 0.00015) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/8.jpg) | takimoto hifumi (score = 0.99601)\u003cbr\u003enishikino maki (score = 0.00335)\u003cbr\u003esakurauchi riko (score = 0.00064) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/9.jpg) | takimoto hifumi (score = 0.99960)\u003cbr\u003esakurauchi riko (score = 0.00029)\u003cbr\u003enishikino maki (score = 0.00011) | OK |\n| ![](resized_for_test_unrecognized/takimoto_hifumi/10.jpg) | takimoto hifumi (score = 0.99995)\u003cbr\u003enishikino maki (score = 0.00004)\u003cbr\u003esakurauchi riko (score = 0.00001) | OK |\n\n\n|Image|Classification| OK/NG |\n| --- | --- | --- |\n| ![](resized_for_test_unrecognized/sakurauchi_riko/2.jpg) | sakurauchi riko (score = 0.84480)\u003cbr\u003enishikino maki (score = 0.12101)\u003cbr\u003etakimoto hifumi (score = 0.03419) | OK |\n| ![](resized_for_test_unrecognized/sakurauchi_riko/5.jpg) | sakurauchi riko (score = 0.94310)\u003cbr\u003enishikino maki (score = 0.04296)\u003cbr\u003etakimoto hifumi (score = 0.01393) | OK |\n| ![](resized_for_test_unrecognized/sakurauchi_riko/7.jpg) | sakurauchi riko (score = 0.96176)\u003cbr\u003etakimoto hifumi (score = 0.03217)\u003cbr\u003enishikino maki (score = 0.00607) | OK |\n\nSince this method gives better result in detecting anime character face and classification still works with almost the same result, the overall face detection accuracy is now around **93%**.\n\n## License\n\n![lbpcascade_animeface.xml](lbpcascade_animeface.xml) is created by [nagadomi/lbpcascade_animeface](https://github.com/nagadomi/lbpcascade_animeface).\n\nCopyright for all images are owned by their respective creators.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomofkeima%2Ftransfer-learning-anime","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreedomofkeima%2Ftransfer-learning-anime","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomofkeima%2Ftransfer-learning-anime/lists"}