{"id":24717883,"url":"https://github.com/ashithapallath/ner-using-hmm","last_synced_at":"2025-03-22T10:29:05.427Z","repository":{"id":272414742,"uuid":"916517602","full_name":"ashithapallath/NER-Using-HMM","owner":"ashithapallath","description":" This project leverages a statistical model approach to implement a Hidden Markov Model (HMM) for Named Entity Recognition (NER). As part of a Computational Linguistics initiative, it uses the `ai4bharat/naamapadam` dataset to identify and classify named entities such as persons, organizations, and locations within text. ","archived":false,"fork":false,"pushed_at":"2025-01-14T08:56:36.000Z","size":184,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-27T10:12:40.867Z","etag":null,"topics":["computational-linguistics","hmm-model","named-entity-recognition","statistical-models","viterbi-algorithm"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashithapallath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-14T08:53:24.000Z","updated_at":"2025-01-14T08:59:02.000Z","dependencies_parsed_at":"2025-01-14T09:51:03.818Z","dependency_job_id":"af64e592-aeb3-41ac-a448-80295b9b6ec8","html_url":"https://github.com/ashithapallath/NER-Using-HMM","commit_stats":null,"previous_names":["ashithapallath/ner-using-hmm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashithapallath%2FNER-Using-HMM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashithapallath%2FNER-Using-HMM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashithapallath%2FNER-Using-HMM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashithapallath%2FNER-Using-HMM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashithapallath","download_url":"https://codeload.github.com/ashithapallath/NER-Using-HMM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244942749,"owners_count":20536068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-linguistics","hmm-model","named-entity-recognition","statistical-models","viterbi-algorithm"],"created_at":"2025-01-27T10:12:42.860Z","updated_at":"2025-03-22T10:29:05.404Z","avatar_url":"https://github.com/ashithapallath.png","language":"Jupyter Notebook","readme":"# HMM for Named Entity Recognition (NER)\n\nThis repository contains an implementation of a Hidden Markov Model (HMM) for Named Entity Recognition (NER) using Python. The project demonstrates how to train and evaluate an HMM for recognizing named entities in text data, such as names of persons, organizations, and locations. This work is a part of a Computational Linguistics project based on statistical modeling.\n\n## Features\n- **Custom Implementation of HMM**: Includes methods for calculating transition and emission probabilities.\n- **NER Task**: Focused on tagging entities in a dataset with predefined labels.\n- **Evaluation Metrics**: Provides accuracy and other evaluation metrics to assess the model's performance.\n- **Modular Code**: Designed to be easily extensible for other sequence tagging tasks.\n\n## Requirements\nEnsure you have the following installed:\n- Python 3.8+\n- Jupyter Notebook\n- NumPy\n- Pandas\n- scikit-learn\n\nInstall the dependencies using pip:\n```bash\npip install -r requirements.txt\n```\n\n## Project Structure\n- `HMMModelNER.ipynb`: The main Jupyter Notebook containing the implementation, training, and evaluation of the HMM for NER.\n- `data/`: Folder for storing datasets used for training and testing.\n- `utils.py`: Utility functions for preprocessing data and calculating probabilities (if applicable).\n\n## Usage\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/yourusername/hmm-ner.git\n   cd hmm-ner\n   ```\n\n2. Open the Jupyter Notebook:\n   ```bash\n   jupyter notebook HMMModelNER.ipynb\n   ```\n\n3. Follow the instructions in the notebook to:\n   - Load and preprocess your dataset.\n   - Train the HMM.\n   - Evaluate its performance.\n\n## Dataset\nThis project uses the `ai4bharat/naamapadam` dataset for training and evaluating the HMM. You can also use other datasets such as CoNLL 2003 for NER tasks. Ensure the data is formatted appropriately with token-label pairs for training.\n\n## Example\nThe notebook provides a step-by-step guide, including:\n- Loading a dataset.\n- Computing transition and emission probabilities.\n- Performing inference to tag new sequences.\n- Evaluating the results.\n\n## Contributing\nContributions are welcome! Feel free to submit issues or pull requests for enhancements, bug fixes, or additional features.\n\n## License\nThis project is licensed under the MIT License. See the LICENSE file for details.\n\n## Acknowledgments\n- Inspired by foundational concepts in HMM and its applications to NLP tasks.\n- Thanks to the open-source community for providing datasets and tools.\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashithapallath%2Fner-using-hmm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashithapallath%2Fner-using-hmm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashithapallath%2Fner-using-hmm/lists"}