{"id":26158739,"url":"https://github.com/dselivanov/lshr","last_synced_at":"2025-04-14T10:13:21.065Z","repository":{"id":33576529,"uuid":"37222908","full_name":"dselivanov/LSHR","owner":"dselivanov","description":"Locality Sensitive Hashing In R","archived":false,"fork":false,"pushed_at":"2019-01-11T10:53:48.000Z","size":101,"stargazers_count":40,"open_issues_count":5,"forks_count":13,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-14T10:12:59.436Z","etag":null,"topics":["approximate-nearest-neighbor-search","locality-sensitive-hashing","minhash","random-projections"],"latest_commit_sha":null,"homepage":null,"language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dselivanov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-06-10T21:12:36.000Z","updated_at":"2025-01-29T21:25:03.000Z","dependencies_parsed_at":"2022-09-12T17:25:27.508Z","dependency_job_id":null,"html_url":"https://github.com/dselivanov/LSHR","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dselivanov%2FLSHR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dselivanov%2FLSHR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dselivanov%2FLSHR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dselivanov%2FLSHR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dselivanov","download_url":"https://codeload.github.com/dselivanov/LSHR/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860168,"owners_count":21173342,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["approximate-nearest-neighbor-search","locality-sensitive-hashing","minhash","random-projections"],"created_at":"2025-03-11T10:59:57.057Z","updated_at":"2025-04-14T10:13:21.036Z","avatar_url":"https://github.com/dselivanov.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Locality Sensitive Hashing in R \nLSHR - fast and memory efficient package for near-neighbor search in high-dimensional data. Two LSH schemes implemented at the moment:\n\n1. Minhashing for jaccard similarity\n2. Sketching (or random projections) for cosine similarity.\nMost of ideas are based on brilliant [Mining of Massive Datasets](http://www.mmds.org) book. \n\n# Materials\n\n* [Slides](http://www.slideshare.net/MailRuGroup/okru-finding-similar-items-in-highdimensional-spaces-locality-sensitive-hashing) (in english) and [video](https://youtu.be/ko0a0Z75oZQ?list=PLcJ8pdaABCSk1dNtpgaHvuV5y2gWItuUO) (in russian) from my talk at Moscow Data Science meetup.\n\n# Quick reference\n```R\n# devtools::install_github('dselivanov/text2vec')\nlibrary(text2vec)\nlibrary(LSHR)\ndata(\"movie_review\")\nit \u003c- itoken(movie_review$review, preprocess_function = tolower, tokenizer = word_tokenizer)\ndtm \u003c- create_dtm(it, hash_vectorizer())\ndtm = as(dtm, \"RsparseMatrix\")\n\nhashfun_number = 120\ns_curve \u003c- get_s_curve(hashfun_number, n_bands_min = 5, n_rows_per_band_min = 5)\n# Examine S-curve.\n# Find tradeoff between accuracy and false-positive rate.\n```\n![S-curves](https://cloud.githubusercontent.com/assets/5123805/13917531/82d5162a-ef72-11e5-8f5a-59a8d1f1f729.png)\n```R\nseed = 1\npairs = get_similar_pairs(dtm, bands_number = 10, rows_per_band = 32, distance = 'cosine', seed = seed)\n\npairs[order(-N)]\n\n#        id1  id2  N\n#    1: 1054 1417 10\n#    2: 1084 3462 10\n#    3: 1291 1356 10\n#    4: 1615 3846 10\n#    5: 2805 4763  4\n#   ---             \n# 2304: 4767 4961  1\n# 2305: 4772 4776  1\n# 2306: 4810 4859  1\n# 2307: 4854 4945  1\n# 2308: 4905 4918  1\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdselivanov%2Flshr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdselivanov%2Flshr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdselivanov%2Flshr/lists"}