{"id":20415463,"url":"https://github.com/nomomon/oramava","last_synced_at":"2025-06-30T06:03:56.705Z","repository":{"id":105034761,"uuid":"344662853","full_name":"nomomon/OramaVA","owner":"nomomon","description":":eye_speech_bubble: PWA for visually impaired people that announces objects detected with user's phone camera.","archived":false,"fork":false,"pushed_at":"2024-05-09T06:33:23.000Z","size":77917,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-05T03:32:55.109Z","etag":null,"topics":["blind-people","computer-vision","machine-learning","mobilenet","mobilenetssd","object-detection","teachablemachine","technovation","tensorflowjs","tfjs"],"latest_commit_sha":null,"homepage":"https://nomomon.github.io/OramaVA","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nomomon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-05T01:48:52.000Z","updated_at":"2024-05-09T06:33:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"406f3bf2-6da4-445c-a6ec-0bd6be24a6b3","html_url":"https://github.com/nomomon/OramaVA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nomomon/OramaVA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomomon%2FOramaVA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomomon%2FOramaVA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomomon%2FOramaVA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomomon%2FOramaVA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nomomon","download_url":"https://codeload.github.com/nomomon/OramaVA/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomomon%2FOramaVA/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262720642,"owners_count":23353447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blind-people","computer-vision","machine-learning","mobilenet","mobilenetssd","object-detection","teachablemachine","technovation","tensorflowjs","tfjs"],"created_at":"2024-11-15T06:16:18.832Z","updated_at":"2025-06-30T06:03:56.653Z","avatar_url":"https://github.com/nomomon.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"*Currently the project is work in progress. See an older version of the website \u0026 PWA [here](https://oramava.nomomon.repl.co/).*\n\n---\n\n![Orama Visual Assistant banner](https://nomomon.github.io/projects/orama-visual-assistant/assets/oramava-preview.jpeg)\n# Orama Visual Assistant\n\n[Pitch video](./README.md#pitch-video) \u003cbr\u003e\n[Story behind the project](./README.md#story-behind-the-project) \u003cbr\u003e\n[Project time line](./README.md#project-time-line) \u003cbr\u003e\n[WIP / For future](./README.md#wip--for-future) \u003cbr\u003e\n[License](./README.md#license) \u003cbr\u003e\n\n\n**Orama Visual Assistant** (from Greek: *Όραμα*, meaning *\"vision\"*) is an app for visually impaired people that announces objects detected using user's phone camera.\n\nOur goal is to help visually impaired people to become more independent.\n\n## Pitch video\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.youtube.com/watch?v=a6ABuAaqgfA\"\u003e\n    \u003cimg width=\"450\" src=\"assets/video_cover.jpg\"/\u003e\n    \u003cbr\u003e\n    Link to the pitch video on YouTube\n  \u003c/a\u003e\n\u003cp\u003e\n  \n## Story behind the project\nWe are the Nurmukhambetov family. There are seven of us in the family: mom, dad, four sons and a daughter. Three of the sons seen in the photograph developed this project.\n  \n\u003e “Our youngest daughter, Sofia, drove us to create this project. Sofia, was born prematurely, from birth we fought for her life, they tried as much as possible to save her eyesight. In consequence of 12 operations, we were able to save Sofia's light perception. But this is not enough, in order to live a full-fledged life on her own, she needs outside help.”\n\u003e \n\u003e ‒ Damira Nurmukhambetova\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"450\" src=\"./assets/family.jpg\"/\u003e\n\u003cp\u003e\n  \n## Project time line\n  \n\u003cdl\u003e\n  \u003cdt\u003eNov 2019\u003c/dt\u003e\n  \u003cdd\u003eWe began brainstorming on the ideas.\u003c/dd\u003e\n  \n  \u003cdt\u003eDec 2019\u003c/dt\u003e\n  \u003cdd\u003eFirst demo made using KNN algorithm on images.\u003c/dd\u003e\n  \n  \u003cdt\u003eApr 2020\u003c/dt\u003e\n  \u003cdd\u003eClassification MobileNetv2 model trained with transfer learning (\u003ca href='https://teachablemachine.withgoogle.com/'\u003eTeachable Machine\u003c/a\u003e).\u003c/dd\u003e\n  \n  \u003cdt\u003eMay 2020\u003c/dt\u003e\n  \u003cdd\u003eThird place finalists at HackDay 2020. First funding of the project.\u003c/dd\u003e\n  \n  \u003cdt\u003eJul 2020\u003c/dt\u003e\n  \u003cdd\u003eFinalists across the Aisa continent at \u003ca href='https://www.curiositymachine.org/about/'\u003eTechnovation Families\u003c/a\u003e.\u003c/dd\u003e\n\n  \u003cdt\u003eJan 2022\u003c/dt\u003e\n  \u003cdd\u003eChange from image classification to object detection (\u003ca href='https://tfhub.dev/tensorflow/tfjs-model/ssd_mobilenet_v2/1/default/1'\u003eSSD Mobile Netv2\u003c/a\u003e). Change from naming everything to search for an object task. \u003c/dd\u003e\n  \n  \u003cdd\u003eOCR functionality with \u003ca href='https://tesseract.projectnaptha.com/'\u003eTesseract\u003c/a\u003e. It is there, but it has bad performance.\u003c/dd\u003e\n\n  \u003cdt\u003eMar 2022\u003c/dt\u003e\n  \u003cdd\u003eBegan working on a react native version of the app.\u003c/dd\u003e\n\u003c/dl\u003e\n\n## WIP / For future\n\n- [ ] Improve OCR reader.\n- [ ] Change to React Native.\n- [ ] Transfer to cloud processing rather than on-device.\n- [ ] Currency classification. Name what banknote that is seen in the camera.\n- [ ] Facial recognition. Each user will have their own pool of people saved in the app for future facial recognition tasks.\n- [ ] Image description. Describe what's in the image.\n- [ ] _'Open with OramaVA'_ on images to do OCR/ facial recognition/ image description.\n \n## License\nAll of the codebase is **MIT Licensed** unless otherwise stated.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnomomon%2Foramava","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnomomon%2Foramava","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnomomon%2Foramava/lists"}