{"id":15407221,"url":"https://github.com/ltfschoen/aind-recognizer","last_synced_at":"2025-07-01T07:04:41.152Z","repository":{"id":76619920,"uuid":"88940577","full_name":"ltfschoen/AIND-Recognizer","owner":"ltfschoen","description":"Term 1 Project 3 Design a Sign Language Recognition System by Luke Schoen for Udacity Artificial Intelligence Nanodegree (AIND)","archived":false,"fork":false,"pushed_at":"2017-05-02T00:42:03.000Z","size":2249,"stargazers_count":13,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-01T07:02:49.023Z","etag":null,"topics":["algorithms","artificial-intelligence","baum-welch","bayesian-information-criterion","cross-platform","discriminative-criterion","feature-selection","hidden-markov-models","jupyter-notebook","model-selection","model-training","n-grams","natural-language-processing","pandas-dataframes","probabilistic-models","python36","sign-language-recognition-system"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ltfschoen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-21T04:34:46.000Z","updated_at":"2024-07-12T18:35:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"730e697b-2578-4b00-b9df-ec45226fe312","html_url":"https://github.com/ltfschoen/AIND-Recognizer","commit_stats":{"total_commits":48,"total_committers":5,"mean_commits":9.6,"dds":"0.45833333333333337","last_synced_commit":"f255a8fefa6582a1016bde0c34b13c88669b9751"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ltfschoen/AIND-Recognizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ltfschoen%2FAIND-Recognizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ltfschoen%2FAIND-Recognizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ltfschoen%2FAIND-Recognizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ltfschoen%2FAIND-Recognizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ltfschoen","download_url":"https://codeload.github.com/ltfschoen/AIND-Recognizer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ltfschoen%2FAIND-Recognizer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262916574,"owners_count":23383883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","artificial-intelligence","baum-welch","bayesian-information-criterion","cross-platform","discriminative-criterion","feature-selection","hidden-markov-models","jupyter-notebook","model-selection","model-training","n-grams","natural-language-processing","pandas-dataframes","probabilistic-models","python36","sign-language-recognition-system"],"created_at":"2024-10-01T16:27:48.285Z","updated_at":"2025-07-01T07:04:41.128Z","avatar_url":"https://github.com/ltfschoen.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Artificial Intelligence Engineer Nanodegree\n## Probabilistic Models\n## Project: Build a Sign Language Recognition System\n\n# Table of Contents\n\n  * [Chapter 0 - Key Results](#chapter-0)\n  * [Chapter 1 - My Setup](#chapter-1)\n  * [Chapter 2 - Rubrics Checklist](#chapter-2)\n  * [Chapter 3 - Info from Udacity](#chapter-3)\n\n  * [Git Commit Style Guide](https://udacity.github.io/git-styleguide/)\n\n# Chapter 0 - Key Results \u003ca id=\"chapter-0\"\u003e\u003c/a\u003e\n\n* Part 3 Screenshots\n\n![alt tag](https://raw.githubusercontent.com/ltfschoen/AIND-Recognizer/master/screenshots/screenshot_part3.png)\n\n* Part 2 Screenshots\n\n![alt tag](https://raw.githubusercontent.com/ltfschoen/AIND-Recognizer/master/screenshots/screenshot_part2_1.png)\n![alt tag](https://raw.githubusercontent.com/ltfschoen/AIND-Recognizer/master/screenshots/screenshot_part2_2.png)\n\n* Part 3 Terminal Outputs\n    * Terminal log output from running `main.py` for Model Selectors including Constant, CV, BIC,\n    and DIC against ground, norm, polar, and delta Feature Sets are included in:\n    `part3_results/log_features_originals.md`\n    * Terminal log output from running `main.py` for Model Selectors including Constant, CV, BIC,\n    and DIC against All Features Sets (i.e. ground, norm, polar, and delta),\n    All Features Sets (excluding Norm and Polar),\n    All Features Sets (excluding Delta), and\n    All Features Sets (excluding Rescaled), are included in:\n    `part3_results/log_features_all_and_withouts.md`\n\n# Chapter 1 - My Setup \u003ca id=\"chapter-1\"\u003e\u003c/a\u003e\n\n### Instructions and Setup Environment\n\n* Use Python 3.4 or higher\n\n* IntelliJ: File \u003e Project Structure \u003e Project Settings \u003e Project \u003e Project SDK \u003e Python 3.6.0 (~/miniconda3/bin/python)\n* IntelliJ: Preferences \u003e Editor \u003e File Types \u003e Python\n    * Add *.py\n\n* Switch to Miniconda env\n    * `source activate aind` (same steps as in https://github.com/ltfschoen/aind/blob/master/README.md)\n\n* Install dependencies\n```\npython3 -m pip3 install mypy typing\npython3 -m pip3 install numpy scipy scikit-learn pandas matplotlib jupyter\npython3 -m pip3 install git+https://github.com/hmmlearn/hmmlearn.git\n```\n\n* Run with `jupyter notebook asl_recognizer.ipynb` and wait for browser to open\n\n* Select a snippet of code and go to menu\n    * \"Kernel\" \u003e \"Restart \u0026 Clear Output\"\n    * \"Kernel\" \u003e \"Restart \u0026 Run All\"\n\n* Debug using IntelliJ breakpoints (instead of time-consuming iPython)\nby setting up configurating that runs `main.py`\n\n* Troubleshooting\n    * If get errors in Jupyter Notebook, it may be becaue\n    compiled .pyc files are out of sync with .py files. Try resolving by\n    deleting .pyc files `rm *.pyc` and restarting the Jupyter notebook\n\n# Chapter 2 - Rubrics Checklist \u003ca id=\"chapter-2\"\u003e\u003c/a\u003e\n\nBuild system that recognizes words communicated using American Sign Language (ASL).\nGiven a preprocessed dataset of tracked hand and nose positions extracted from video.\nGoal is to train a set of Hidden Markov Models (HMMs) using part of the given dataset to try\nand identify individual words from test sequences.\n\nOptional challenge is to incorporate Statistical Language Models (SLMs) that capture the\nconditional probability of particular sequences of words occurring, in order to help you improve\nthe recognition accuracy of the system.\n\n### Project Specification Checklist https://review.udacity.com/#!/rubrics/749/view\n\n## CRITERIA / MEETS SPECIFICATIONS\n\nComplete each of these parts by implementing segments of code as instructed,\nand filling out any written responses to questions in markdown cells.\nMake sure you run each cell and include the output.\nThe code must be free of errors and needs to meet the specified requirements.\n\n* PART 1: Data\n\n    * Prepare data for modeling\n\n        * [X] - Student provides correct alternate feature sets:\n        delta, polar, normalized, and custom.\n\n        * [X] - Student passes unit tests.\n\n        * [X] - Student provides a reasonable explanation for what\n        custom set was chosen and why (Q1).\n\n* PART 2: Model Selection\n\n    * Implement model selection techniques\n\n        * [X] - Student correctly implements BIC model selection\n        technique in \"my_model_selectors.py\".\n\n        * [X] - Student correctly implements DIC model selection\n        technique in \"my_model_selectors.py\".\n\n        * [X] - Student correctly implements CV model selection\n        technique in \"my_model_selectors.py\".\n\n        * [X] - Student code runs error-free in notebook, passes unit tests\n        and code review of the algorithms.\n\n        * [X] - Student provides a brief but thoughtful comparison of the selectors (Q2).\n\n* PART 3: Recognizer\n\n    * Recognize ASL words\n\n        * [X] - Student implements a recognizer in \"my_recognizer.py\" which runs\n        error-free in the notebook and passes all unit tests\n\n        * [X] - Student provides three examples of feature/selector combinations\n        in the submission cells of the notebook.\n\n        * [X] - Student code provides the correct words within \u003c60% WER for at least one\n        of the three examples student provided.\n\n        * [X] - Student provides a summary of results and speculates on how to improve the WER.\n\n* PART 4: (OPTIONAL)\n\n    * Improve the WER with Language Models\n\n        * [ ] - The recognizer you implemented in Part 3 is equivalent to a \"0-gram\" SLM.\n        Improve the WER with the SLM data provided with the data set in the link above\n        using \"1-gram\", \"2-gram\", and/or \"3-gram\" statistics.\n\n* Submission:\n\n    * Once you have completed the project and met all the requirements set in the rubric,\n    save the notebook as an HTML file by going to the File menu in the notebook and choosing\n    \"Download as\" \u003e HTML. Submit the following files (and only these files) in a .zip archive:\n\n        * asl_recognizer.ipynb\n        * asl_recognizer.html\n        * my_model_selectors.py\n        * my_recognizer.py\n\n    * Note: Do not include the data directory as it will lead to a huge .zip file, and may be\n    rejected by our reviews system.\n\n# Chapter 3 - Info from Udacity \u003ca id=\"chapter-3\"\u003e\u003c/a\u003e\n\n### Install\n\nThis project requires **Python 3** and the following Python libraries installed:\n\n- [NumPy](http://www.numpy.org/)\n- [SciPy](https://www.scipy.org/)\n- [scikit-learn](http://scikit-learn.org/0.17/install.html)\n- [pandas](http://pandas.pydata.org/)\n- [matplotlib](http://matplotlib.org/)\n- [jupyter](http://ipython.org/notebook.html)\n- [hmmlearn](http://hmmlearn.readthedocs.io/en/latest/)\n\nNotes: \n1. It is highly recommended that you install the\n[Anaconda](http://continuum.io/downloads) distribution of Python and load the\nenvironment included in the \"Your conda env for AI ND\" lesson.\n2. The most recent development version of hmmlearn, 0.2.1, contains a bugfix\nrelated to the log function, which is used in this project.  In order to install\nthis version of hmmearn, install it directly from its repo with the following\ncommand from within your activated Anaconda environment:\n```sh\npip install git+https://github.com/hmmlearn/hmmlearn.git\n```\n\n### Code\n\nTemplate notebook provided is `asl_recognizer.ipynb`  (combined tutorial/submission doc)\nSome of the codebase and some\nof your implementation will be external to the notebook. For submission,\ncomplete the **Submission** sections of each part.  This will include running\nyour implementations in code notebook cells, answering analysis questions, and\npassing provided unit tests provided in the codebase and called out in the notebook.\n\n### Run\n\n* Go to project directory `cd AIND_recognizer/` and run\n`jupyter notebook asl_recognizer.ipynb` to open the Jupyter Notebook in browser.\n* Follow the instructions in notebook to complete the project.\n\n\n### Additional Information\n##### Provided Raw Data\n\nThe data in the `asl_recognizer/data/` directory was derived from \nthe [RWTH-BOSTON-104 Database](http://www-i6.informatik.rwth-aachen.de/~dreuw/database-rwth-boston-104.php). \nThe handpositions (`hand_condensed.csv`) are pulled directly from \nthe database [boston104.handpositions.rybach-forster-dreuw-2009-09-25.full.xml](boston104.handpositions.rybach-forster-dreuw-2009-09-25.full.xml). The three markers are:\n\n*   0  speaker's left hand\n*   1  speaker's right hand\n*   2  speaker's nose\n*   X and Y values of the video frame increase left to right and top to bottom.\n\nTake a look at the sample [ASL recognizer video](http://www-i6.informatik.rwth-aachen.de/~dreuw/download/021.avi)\nto see how the hand locations are tracked.\n\nThe videos are sentences with translations provided in the database.  \nFor purposes of this project, the sentences have been pre-segmented into words \nbased on slow motion examination of the files.  \nThese segments are provided in the `train_words.csv` and `test_words.csv` files\nin the form of start and end frames (inclusive).\n\nThe videos in the corpus include recordings from three different ASL speakers.\nThe mappings for the three speakers to video are included in the `speaker.csv` \nfile.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fltfschoen%2Faind-recognizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fltfschoen%2Faind-recognizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fltfschoen%2Faind-recognizer/lists"}