{"id":21285258,"url":"https://github.com/pritamgouda11/modelling-indian-names","last_synced_at":"2025-03-15T15:12:26.907Z","repository":{"id":273639576,"uuid":"772899346","full_name":"pritamgouda11/Modelling-Indian-Names","owner":"pritamgouda11","description":"Used patterns in Indian names that models could learn, modelling those using n-gram models, then moved to neural n-gram and RNN models.","archived":false,"fork":false,"pushed_at":"2024-07-08T04:47:17.000Z","size":2481,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T05:15:24.364Z","etag":null,"topics":["fnn","machine-learning","natural-language-processing","ngrams","rnn","smoothing"],"latest_commit_sha":null,"homepage":"https://colab.research.google.com/github/pritamgouda11/Modelling-Indian_Names/blob/main/Gouda_Pritam_Trilochan_Savita_assignment2.ipynb","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pritamgouda11.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-16T07:16:56.000Z","updated_at":"2024-12-05T13:36:46.000Z","dependencies_parsed_at":"2025-01-22T05:15:25.580Z","dependency_job_id":null,"html_url":"https://github.com/pritamgouda11/Modelling-Indian-Names","commit_stats":null,"previous_names":["pritamgouda11/modelling-indian-names"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pritamgouda11%2FModelling-Indian-Names","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pritamgouda11%2FModelling-Indian-Names/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pritamgouda11%2FModelling-Indian-Names/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pritamgouda11%2FModelling-Indian-Names/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pritamgouda11","download_url":"https://codeload.github.com/pritamgouda11/Modelling-Indian-Names/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243746199,"owners_count":20341203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fnn","machine-learning","natural-language-processing","ngrams","rnn","smoothing"],"created_at":"2024-11-21T11:19:14.344Z","updated_at":"2025-03-15T15:12:26.886Z","avatar_url":"https://github.com/pritamgouda11.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Modelling-Indian-Names\nImplemented different types of language models for modelling Indian names. There are clearly patterns in Indian names that models could learn, and we start modelling those using n-gram models, then move to neural n-gram and RNN models.\n\n```\nModelling-Indian-Names\n├── Modelling-Indian-Names.ipynb\n├── main.py\n├── fnn\n│   ├── loss.json\n│   ├── model.pt\n│   └── vocab.pt\n├── rnn\n│   ├── loss.json\n│   ├── model.pt\n│   └── vocab.pt\n└── README.md\n```\n\n## N-gram language models\nIn natural language processing, an n-gram is a sequence of n words. For example, “statistics” is a unigram (n = 1), “machine learning” is a bigram (n = 2), “natural language processing” is a trigram (n = 3). For longer n-grams, people just use their lengths to identify them, such as 4-gram, 5-gram, and so on.\n\n![image](https://github.com/pritamgouda11/Modelling-Indian-Names/assets/46958858/5a1398d0-9029-48fb-be66-9d7dbc63365f)\n\n## N-Grams Smoothing\nThe standard N-gram models are trained from some corpus. The finiteness of the training corpus leads to the absence of some perfectly acceptable N-grams. This results in sparse bigram matrices. This method tend to underestimate the probability of strings that do not occur in their training corpus.\n\nThere are some techniques that can be used for assigning a non-zero probabilty to these 'zero probability bigrams'. This task of reevaluating some of the zero-probability and low-probabilty N-grams, and assigning them non-zero values, is called smoothing. Some of the techniques are: Add-One Smoothing, in Add-One smooting, we add one to all the bigram counts before normalizing them into probabilities. This is called add-one smoothing.\n\n\u003cimg width=\"1238\" alt=\"Screenshot 2024-06-29 at 12 02 27 AM\" src=\"https://github.com/pritamgouda11/Modelling-Indian-Names/assets/46958858/47c525d1-dc50-402e-97fe-702147bc4590\"\u003e\n\n## Results: \n\n### **Unigram:**\n```\nunigram train perplexity: 16.623900007096303\nEVALUATION of 1-gram on validation perplexity: 30.863827225768976\nEVALUATION 1-gram top most likely chars after aa are a, i, n\n```\n### **Smoothed Unigram:**\n```\nsmoothed unigram train perplexity: 16.623900007096303\nEVALUATION of 1-gram on validation perplexity: 11.14730834795356\nEVALUATION 1-gram top most likely chars after aa are a, r, i, n\n```\n### **Bigram:**\n```\n\nbigram train perplexity: 7.658283554851139\nEVALUATION of 2-gram on validation perplexity: 5.594377295480409\nEVALUATION 2-gram generated names are prana, shana, shaa, sh, bha\nEVALUATION 2-gram top most likely chars after aa are , n, r, m, l\n```\n### **Smoothed Bigram:**\n```\n\nsmoothed bigram train perplexity: 7.658283554851139\nEVALUATION of 2-gram on validation perplexity: 4.715609379404076\nEVALUATION 2-gram top most likely chars after aa are , n, r, m, l\n```\n### **Smoothed Trigram:**\n```\n\ntrigram train perplexity: 4.298966505162752\nEVALUATION of 3-gram on validation perplexity: 5.178486767038496\nEVALUATION 3-gram generated names are aam, sam, sahi, san, sandeen\nEVALUATION 3-gram top most likely chars after aa are n, s, r, l, m\n```\n## **Neural N-gram Language Model:**\n\n**Feed-Forward Neural Networks**\nThe feedforward neural network is one of the most basic artificial neural networks. In this ANN, the data or the input provided travels in a single direction. It enters into the ANN through the input layer and exits through the output layer while hidden layers may or may not exist. So the feedforward neural network has a front-propagated wave only and usually does not have backpropagation\n\n**Recurrent Neural Networks**\nThe Recurrent Neural Network saves the output of a layer and feeds this output back to the input to better predict the outcome of the layer. The first layer in the RNN is quite similar to the feed-forward neural network and the recurrent neural network starts once the output of the first layer is computed. After this layer, each unit will remember some information from the previous step so that it can act as a memory cell in performing computation\n\n![image](https://github.com/pritamgouda11/Modelling-Indian-Names/assets/46958858/80a8e2ba-47f6-48d2-8b09-0138c0f63809)\n\n### **FNN**\n```\n\nEVALUATION of FNN on valid perplexity: 6.7446325639391134\nEVALUATION RNN generated names are nanda, shrewq, sumandana, ahas\nEVALUATION RNN the top most likely chars after aa are v, r, m, n, l\n```\n### **RNN**\n```\n\nEVALUATION of RNN on valid perplexity: 5.4496259689331055\nEVALUATION RNN generated names are nanda, anaki, nabtil, sumandana, mytri\nEVALUATION RNN the top most likely chars after aa are n, r, m, s, l\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpritamgouda11%2Fmodelling-indian-names","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpritamgouda11%2Fmodelling-indian-names","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpritamgouda11%2Fmodelling-indian-names/lists"}