{"id":15576221,"url":"https://github.com/ct83/rumaro","last_synced_at":"2025-09-16T18:31:46.941Z","repository":{"id":40979938,"uuid":"204114351","full_name":"CT83/Rumaro","owner":"CT83","description":"[Failed] AI to recognize emotions, apparel, body pose and activity in images,  and measure how they affect audience engagement. ie. likes and comments.","archived":false,"fork":false,"pushed_at":"2022-12-08T06:04:37.000Z","size":11082,"stargazers_count":12,"open_issues_count":5,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-08T16:38:07.485Z","etag":null,"topics":["apparel","audience-engagement","deep-learning","deepfashion","engagement","insights","instagram","instagram-analytics","instagram-post","instagram-scraper","photos","python","recognize-emotions"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CT83.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-24T05:44:34.000Z","updated_at":"2023-09-02T10:43:02.000Z","dependencies_parsed_at":"2023-01-24T15:00:42.079Z","dependency_job_id":null,"html_url":"https://github.com/CT83/Rumaro","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CT83/Rumaro","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CT83%2FRumaro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CT83%2FRumaro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CT83%2FRumaro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CT83%2FRumaro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CT83","download_url":"https://codeload.github.com/CT83/Rumaro/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CT83%2FRumaro/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275464781,"owners_count":25469881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apparel","audience-engagement","deep-learning","deepfashion","engagement","insights","instagram","instagram-analytics","instagram-post","instagram-scraper","photos","python","recognize-emotions"],"created_at":"2024-10-02T18:43:53.472Z","updated_at":"2025-09-16T18:31:45.907Z","avatar_url":"https://github.com/CT83.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rumaro - AI to recognize emotions, apparel, body pose and activity in images [![Build Status](https://travis-ci.com/CT83/SmoothStream.svg?branch=master)](https://travis-ci.com/CT83/SmoothStream)\n\n![Drag Racing](images/rumaro-insta.png)\n\nRumaro uses AI to recognize emotions, apparel, body pose and activity in images, \nand measures how they affect audience engagement. ie. likes and comments.\n\n\u003eThis started out as one of my weekend side hustles and was initially meant to be launched as a paid SAAS. But, on \nresearching more about the privacy concerns, Instagrams restrictive APIs and mixed results from the initial tests I \ndecided to scrape the idea and just Open Source it instead.\n\n## Setup\n0. Setup MS Cognitive Services API Keys, DeepFashion, and Instagram Credentials\n1. Install the dependencies \n`pip install -r requirements.txt`\n2. Run the actual analysis `python run_rumaro.py --instagram_id \u003cINSTA_ID\u003e`\n\nHere comes the tricky part, now grab your pair of debugging goggles and start digging around the various property of \nthe `data_groomer` object for something useful. I am just waaay to burnt out to add graphs and UI \nto this now.  \n\n### The why?\n\n\u003e With Rumaro, I tried to identify *what* really made photos special! 🦄\n\nI hypothesised that - Photos which the user liked, i.e photos which were more desirable could be identified using AI \nmodels and the key _features_ which made them special could be thus identified. \n\nI thought the Racy-ness, Body Pose and \nApparel could be easily identified using Deep Learning and these would be the *only* features which would be driving the\nsocial media engagement. \n\n### The how?\n##### Outline \n\n![Drag Racing](images/block_db.png)\n\nAll the instagram posts of a user to be analyzed would be downloaded, then these would be sent to several deep learning models\nto generate insights from them, to know what makes them special.\n\n##### Steps\n\n![Drag Racing](images/algo_dg.png)\n\n\n1. An Instagram post is fetched and the public url of the image is stored\n2. It is sent to Microsoft Vision API to be categorized into a scenery/other image, full body/ pose image or a portrait or selfie.\n3. If the image is a scenery it is not sent to any further analysis\n4. Body Pose images are sent to a Pose Detection Model running on OpenCV and then to DeepFashion API from Algorithmia for apparel detection. \n5. The Portraits or Selfie posts are sent to Microsoft's Emotion API.\n6. All of the insights are then stored into the DB and the user is redirect to the payment page\n7. Show the user the curated analysis of his profile.\n\n\u003e Yes! I even started a Instagram Page and tried to find a target audience - influencers, who were actually willing \n\u003eto pay to use the product\n\n### Vision\n\nThe vision was to create a massive SAAS Company who charged people ~10$ for reviewing a profile. \nI imagined it would have been possible to create a sustainable business out of this. \n\nI had a vision that the insights generated would be so useful that it would allow users to boost their social media profiles\nsignificantly, and I imagined that the Deep Learning Models would be super-accurate and would predict everything as \nreal humans perceive it.  \n\n#### How far ahead did I actually get?\n\n* I successfully managed to create an okay looking Frontend using MaterializeCSS, with Flask on the Backend.\n* I managed to create the core AI components of the project - talking to all the APIs, the Deep Learning part\n* I managed to create add authentication and databases to it.\n  \nI decided that it only made sense to publish the stripped down version of the entire codebase, without the Web Framework\ncode and Flask packages. \n\n##### Screenshots\n\n![](images/full_ui.png)\n\u003e The generated Report after analysing a random instagram account. \n\u003e Yes, I had to censor all of the images, yeah, that kinda' defeats the whole purpose of the screenshots 🤷🏽‍♂️\n\n\n![](images/graphs.png)\n\u003e Statistics generated after analysing a random instagram profile\n\n### ⚓ Jumping off the sinking ship...  \n\nUnfortunately as I moved forward with the project I discovered that the coorelation between the different features\nand the engagement was very weak. Also, this project had nightmarish privacy concerns which made it extremely difficult\nto monetise and market. \n\nIn retrospect it seems really obvious that the plan was a bit flawed from the square one. \n🤷🏽‍♂️\n\nEhh... I did get to play around with a lot of Deep Learning APIs, so all was good.\n\n\n### Reach out to me!\n\nIf you are interested to build something similar yourself, or would like help getting this up and running or if you are \n_planning to turn this into a movie!!!_ Hit me up on Google Hangouts at `rohansawantct83@gmail.com`   ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fct83%2Frumaro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fct83%2Frumaro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fct83%2Frumaro/lists"}