{"id":22890221,"url":"https://github.com/jessicasaikia/multilingual-bert-mbert","last_synced_at":"2025-03-31T20:54:59.434Z","repository":{"id":267163108,"uuid":"900427103","full_name":"jessicasaikia/multilingual-BERT-mBERT","owner":"jessicasaikia","description":"This repository implements a Multilingual BERT (mBERT) model for performing Parts-of-Speech (POS) Tagging on Assamese-English code-mixed texts.","archived":false,"fork":false,"pushed_at":"2024-12-08T19:24:29.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-07T00:29:30.782Z","etag":null,"topics":["assamese","assamese-text","code-mixed","code-mixing","english","english-language","mbert","multilingual-bert","nlp","nlp-machine-learning","parts-of-speech","parts-of-speech-tagging","pos-tagger","pos-tagging"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessicasaikia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-08T18:40:52.000Z","updated_at":"2024-12-08T19:24:33.000Z","dependencies_parsed_at":"2024-12-08T19:36:29.595Z","dependency_job_id":null,"html_url":"https://github.com/jessicasaikia/multilingual-BERT-mBERT","commit_stats":null,"previous_names":["jessicasaikia/multilingual-bert-mbert"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessicasaikia%2Fmultilingual-BERT-mBERT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessicasaikia%2Fmultilingual-BERT-mBERT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessicasaikia%2Fmultilingual-BERT-mBERT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessicasaikia%2Fmultilingual-BERT-mBERT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessicasaikia","download_url":"https://codeload.github.com/jessicasaikia/multilingual-BERT-mBERT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246540518,"owners_count":20793930,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assamese","assamese-text","code-mixed","code-mixing","english","english-language","mbert","multilingual-bert","nlp","nlp-machine-learning","parts-of-speech","parts-of-speech-tagging","pos-tagger","pos-tagging"],"created_at":"2024-12-13T22:00:00.181Z","updated_at":"2025-03-31T20:54:59.411Z","avatar_url":"https://github.com/jessicasaikia.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multilingual BERT (mBERT)\nThis repository implements a Multilingual BERT (mBERT) model for performing Parts-of-Speech (POS) Tagging on Assamese-English code-mixed texts.\n\n## Introduction to Parts-of-Speech (PoS) Tagging\nPoS tagging is the process of identifying and labeling grammatical roles of words in texts, supporting applications like machine translation and sentiment analysis. While different languages may have their own PoS tags, I have used my own custom PoS tags for this model. The Table below defines the custom PoS tags used in this model-\n\n![Table](https://github.com/jessicasaikia/hidden-markov-model-HMM/blob/main/Custom%20PoS%20tags%20Table.png)\n\n## About Multilingual BERT (mBERT)\nIt is a pre-trained language model by Google, based on the BERT (Bidirectional Encoder Representations from Transformers) architecture, designed for multilingual tasks. It is trained on text from 104 languages using a masked language model (MLM) objective, where some words in the input are randomly masked, and the model learns to predict them based on context. This training allows mBERT to capture cross-lingual patterns and relationships, making it highly effective for tasks involving multilingual or code-mixed text without the need for explicit translation.\n\n**Algorithm**:\n1.\tThe model imports the required libraries and loads the dataset.\n2.\tThe model tokenises the input sentences using the mBERT WordPiece tokeniser\n3.\tTokenised sentences are adjusted to a fixed length by adding [PAD] tokens and generate masks to differentiate valid tokens (1) from padded ones (0).\n4.\tEach token is aligned with its corresponding POS tag. Subwords inherit the same tag as the original word.\n5.\tThe tokens are mapped to numerical IDs using mBERT’s vocabulary.\n6.\tThe token IDs and attention masks are passed into mBERT to compute contextualised embeddings for each token. These embeddings capture both the meaning of the token and its context within the sentence.\n7.\tThe embeddings are fed into a dense (fully connected) layer to map each token’s embedding to probabilities for all POS tags.\n8.\tSoftmax is applied to convert the output into probabilities for each POS tag.\n9.\tFor each token, the POS tag with the highest probability is selected.\n10.\tNumerical predictions are mapped back to their original tag format\n11.\tThe POS tags for all tokens in the sentence are returned.\n\n## Where should you run this code?\nI used Google Colab for this Model.\n1. Create a new notebook (or file) on Google Colab.\n2. Paste the code.\n3. Upload your CSV dataset file to Google Colab.\n4. Please make sure that you update the \"path for the CSV\" part of the code based on your CSV file name and file path.\n5. Run the code.\n6. The output will be displayed and saved as a different CSV file.\n\nYou can also VScode or any other platform (this code is just a python code)\n1. In this case, you will have to make sure you have the necessary libraries installed and datasets loaded correctly.\n2. Run the program for the output.\n   \n## Additional Notes from me\nIf you need any help or questions, feel free to reach out to me in the comments or via my socials. My socials are:\n- Discord: jessicasaikia\n- Instagram: jessicasaikiaa\n- LinkedIn: jessicasaikia (www.linkedin.com/in/jessicasaikia-787a771b2)\n\nAdditionally, you can find the custom dictionaries that I have used in this project and the dataset in their respective repositories on my profile. Have fun coding and good luck! :D\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessicasaikia%2Fmultilingual-bert-mbert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessicasaikia%2Fmultilingual-bert-mbert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessicasaikia%2Fmultilingual-bert-mbert/lists"}