{"id":20700863,"url":"https://github.com/vidyasagarmsc/shingling","last_synced_at":"2025-03-11T03:53:29.202Z","repository":{"id":241589708,"uuid":"807162250","full_name":"VidyasagarMSC/shingling","owner":"VidyasagarMSC","description":"Code for Shingling","archived":false,"fork":false,"pushed_at":"2024-06-12T15:17:48.000Z","size":12,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-17T19:42:03.865Z","etag":null,"topics":["generative-ai","nlp-machine-learning","python3","shingles","shingling"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VidyasagarMSC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-28T15:33:37.000Z","updated_at":"2024-06-13T12:17:36.000Z","dependencies_parsed_at":"2025-01-17T19:41:34.837Z","dependency_job_id":"0de3628f-babb-457c-af13-9522f9718c75","html_url":"https://github.com/VidyasagarMSC/shingling","commit_stats":null,"previous_names":["vidyasagarmsc/shingling"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VidyasagarMSC%2Fshingling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VidyasagarMSC%2Fshingling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VidyasagarMSC%2Fshingling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VidyasagarMSC%2Fshingling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VidyasagarMSC","download_url":"https://codeload.github.com/VidyasagarMSC/shingling/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242967741,"owners_count":20214281,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-ai","nlp-machine-learning","python3","shingles","shingling"],"created_at":"2024-11-17T00:37:30.916Z","updated_at":"2025-03-11T03:53:29.169Z","avatar_url":"https://github.com/VidyasagarMSC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Code for Shingling\n\nThe repository contains code for creating Shingles (set of words) and to calculate Jaccard Similarity between two generated Shingles.\n\n## Usage \n\n### Jaccard Similarity\n```sh\nusage: shingling.py [-h] [-k K] test_a test_b\n\nPass two sentences...\n\npositional arguments:\n  text_a      text 1 to create shingles\n  text_b      text 2 to create shingles and compare for similarity.\n\noptional arguments:\n  -h, --help  show this help message and exit\n  -k K        the size of shingles\n```\n\n### Weighted Shingling\n\n```commandline\nusage: weighted-shingling.py [-h] [-k K] text weights\n\nPass a text sentence with size and weights...\n\npositional arguments:\n  text        The input text.\n  weights     A dictionary mapping words to their weights in string format.\n\noptional arguments:\n  -h, --help  show this help message and exit\n  -k K        The length of shingles.\n\n```\n\n## Example\n\n### Jaccard Similarity\n```sh\n$ python3 shingling.py \"Generates a set of shingles for given text.\" \"Generates a set of shingles.\" -k 5\n\nShingles for text_a is  {'es a ', 'rates', 'iven ', ' of s', 'given', 'a set', ' set ', 'n tex', ' for ', 'les f', 'gener', 's a s', 't of ', ' shin', 'en te', 'nerat', 'shing', 'ingle', 'of sh', 'or gi', 'tes a', 'for g', 'ven t', 'set o', 'enera', 'text.', 's for', ' give', 'ngles', 'ates ', 'r giv', ' a se', 'es fo', 'erate', 'et of', 'hingl', ' text', 'gles ', 'f shi'}\nShingles for text_b is  {'es a ', 'rates', ' of s', 'gles.', 'a set', ' set ', 'gener', 's a s', 't of ', ' shin', 'nerat', 'shing', 'ingle', 'of sh', 'tes a', 'set o', 'enera', 'ngles', 'ates ', ' a se', 'erate', 'et of', 'hingl', 'f shi'}\nIntersection - text_a ∩ text_b 23\nUnion - text_a ∪ text_b 40\nJaccard Similarity: 0.5750\n```\n\n### Weighted shingling \n\n```commandline\n$ python3 weighted-shingling.py \"This is a sample text with some important words like neural networks and deep learning\" '{\"neural\": 2, \"deep\": 2, \"networks\": 1.5, \"learning\": 1.5}' -k 5\n{'neural': 2, 'deep': 2, 'networks': 1.5, 'learning': 1.5}\n\n\nList of Shingles with corresponding weights:  [('this is a sample text', 5), ('is a sample text with', 5), ('a sample text with some', 5), ('sample text with some important', 5), ('text with some important words', 5), ('with some important words like', 5), ('some important words like neural', 6), ('important words like neural networks', 6.5), ('words like neural networks and', 6.5), ('like neural networks and deep', 7.5), ('neural networks and deep learning', 8.0)]\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvidyasagarmsc%2Fshingling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvidyasagarmsc%2Fshingling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvidyasagarmsc%2Fshingling/lists"}