{"id":13747606,"url":"https://github.com/asmuth/recommendify","last_synced_at":"2025-05-16T13:06:20.884Z","repository":{"id":2333454,"uuid":"3295176","full_name":"asmuth/recommendify","owner":"asmuth","description":"Generate recommendations using collaborative filtering","archived":false,"fork":false,"pushed_at":"2014-05-08T21:18:47.000Z","size":1910,"stargazers_count":1660,"open_issues_count":18,"forks_count":147,"subscribers_count":71,"default_branch":"master","last_synced_at":"2025-04-03T08:12:03.327Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"Daemonite/material","license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asmuth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-01-29T03:56:10.000Z","updated_at":"2025-03-10T05:41:24.000Z","dependencies_parsed_at":"2022-08-20T08:50:12.418Z","dependency_job_id":null,"html_url":"https://github.com/asmuth/recommendify","commit_stats":null,"previous_names":["paulasmuth/recommendify"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asmuth%2Frecommendify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asmuth%2Frecommendify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asmuth%2Frecommendify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asmuth%2Frecommendify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asmuth","download_url":"https://codeload.github.com/asmuth/recommendify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248531017,"owners_count":21119680,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T06:01:34.946Z","updated_at":"2025-04-12T07:22:13.273Z","avatar_url":"https://github.com/asmuth.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"recommendify\n============\n\n_Recommendify is a ruby/redis based recommendation engine_  - The recommendations can be updated/processed incrementally and on multiple hosts. The worker is implemented in plain ruby and native C. \n\n[ ![Build status - Travis-ci](https://secure.travis-ci.org/paulasmuth/recommendify.png) ](http://travis-ci.org/paulasmuth/recommendify)\n\n---\n\n#### usecases\n\n+ __\"Users that bought this product also bought...\"__ from `user_id--bought--\u003eproduct_id` pairs\n+ __\"Users that viewed this video also viewed...\"__ from `user_id--viewed--\u003evideo_id` pairs\n+ __\"Users that like this venue also like...\"__ from `user_id--likes--\u003evenue_id` pairs\n\n\n\nsynopsis\n--------\n\nYour input data (the so called interaction-sets) should look like this:\n\n```\n# FORMAT A: user bought products (select buyerid, productid from sales group_by buyerid)\n[user23] product5 produt42 product17\n[user42] product8 produt16 product5\n\n# FORMAT B: user watched video (this can be transformed to the upper representation with a map/reduce)\nuser3 -\u003e video3\nuser6 -\u003e video19\nuser3 -\u003e video6\nuser1 -\u003e video42\n```\n\nThe output data will look like this:\n\n```\n# similar products based on co-concurrent buys\nproduct5 =\u003e product17 (0.78), product8 (0.43), product42 (0.31)\nproduct17 =\u003e product5 (0.36), product8 (0.21), product42 (0.18)\n\n# similar videos based on co-concurrent views\nvideo19 =\u003e video3 (0.93), video6 (0.56), video42 (0.34)\nvideo42 =\u003e video19 (0.32), video3 (0.21), video6 (0.08)\n```\n\nYou can add new interaction-sets to the processor incrementally, but the similarities for changed items have to be re-processed after new interactions were added. You can either re-process all items (recommender.process!) from time to time or keep track of the updates and only process the changed items (recommender.process_item!)\n\n\nusage\n-----\n\n```ruby\n\n# Our similarity matrix, we calculate the similarity via co-concurrence \n# of products in \"orders\" using the jaccard similarity measure.\nclass MyRecommender \u003c Recommendify::Base\n\n  # store only the top fifty neighbors per item\n  max_neighbors 50\n\n  # define an input data set \"order_items\". we'll add \"order_id-\u003eproduct_id\"\n  # pairs to this input and use the jaccard coefficient to retrieve a \n  # \"customers that ordered item i1 also ordered item i2\" statement and apply\n  # the result to the item\u003c-\u003eitem similarity matrix with a weight of 5.0\n  input_matrix :order_items,  \n    # :native =\u003e true,\n    :similarity_func =\u003e :jaccard,    \n    :weight =\u003e 5.0\n\nend\n\nrecommender = MyRecommender.new\n\n# add `order_id-\u003eproduct_id` interactions to the order_item_sim input\n# you can add data incrementally and call RecommendedItem.process! to update\n# the similarity matrix at any time.\nrecommender.order_items.add_set(\"order1\", [\"product23\", \"product65\", \"productm23\"])\nrecommender.order_items.add_set(\"order2\", [\"product14\", \"product23\"])\n\n# Calculate all elements of the similarity matrix\nrecommender.process!\n\n# ...or calculate a specific row of the similarity matrix (a specific item)\n# use this to avoid re-processing the whole matrix after incremental updates\nrecommender.process_item!(\"product65\")\n\n# retrieve similar products to \"product23\"\nrecommender.for(\"item23\") \n  =\u003e [ \u003cRecommendify::Neighbor item_id:\"product65\" similarity:0.23\u003e, (...) ]\n\n# remove \"product23\" from the similarity matrix and the input matrices. you should \n# do this if your items 'expire', since it will speed up the calculation\nrecommender.delete_item!(\"product23\") \n```\n\n### how it works\n\nRecommendify keeps an incrementally updated `item x item` matrix, the \"co-concurrency matrix\". This matrix stores the number of times that a combination of two items has appeared in an interaction/preferrence set. The co-concurrence counts are processed with a jaccard similarity measure to retrieve another `item x item` similarity matrix, which is used to find the N most similar items for each item. This is also called \"Item-based Collaborative Filtering with binary ratings\" (see Miranda, Alipio et al. [1])\n\n1. Group the input user-\u003eitem pairs by user-id and store them into interaction sets\n2. For each item\u003c-\u003eitem combination in the interaction set increment the respective element in the co-concurrence matrix\n3. For each item\u003c-\u003eitem combination in the co-concurrence matrix calculate the item\u003c-\u003eitem similarity\n3. For each item store the N most similar items in the respective output set.\n\n\n### does it scale?\n\nThe maximum number of entries in the co-concurrence and similarity matrix is k(n) = (n^2)-(n/2), it grows O(n^2). However, in a real scenario it is very unlikely that all item\u003c-\u003eitem combinations appear in a interaction set and we use a sparse matrix which will only use memory for elemtens with a value \u003e 0. The size of the similarity grows O(n). \n\n### native/fast worker\n\nAfter you have compiled the native worker, you can pass the `:native =\u003e true` option to the input_matrix. This speeds up processing by at least 10x.\n\n```\ncd ~/.rvm/gems/ruby-1.9.3-p0/gems/recommendify-0.2.2/\nbundle exec rake build_native\n```\n\nexample\n-------\n\nThese recommendations were calculated from 2,3mb \"profile visit\"-data (taken from www.talentsuche.de) - keep in mind that the recommender uses only visitor-\u003evisited data, it __doesn't know the gender__  of a user. \n\n[ ![Example Results](https://raw.github.com/paulasmuth/recommendify/master/doc/example.png) ](http://falbala.23loc.com/~paul/recommendify_out_1.html)\n\nfull snippet: http://falbala.23loc.com/~paul/recommendify_out_1.html \n\nInitially processing the 120.047 `visitor_id-\u003eprofile_id` pairs currently takes around half an hour with the ruby-only implementation and ~130 seconds with the native/c implementation on a single core. It creates a 24.1mb hashtable in redis (with truncated user_rows a' max 100 items). In another real data set with very short user rows (purchase/payment data) it used only 3.4mb for 90k items with very good results. You can try this for yourself; the complete data and code is in `doc/example.rb` and `doc/example_data.csv`. \n\n\n\n\nSources / References\n--------------------\n\n[1] Miranda C. and Alipio J. (2008). Incremental collaborative filtering for binary ratings (LIAAD - INESC Porto, University of Porto)\n\n[2] George Karypis (2000) Evaluation of Item-Based Top-N Recommendation Algorithms (University of Minnesota, Department of Computer Science / Army HPC Research Center)\n\n[3] Shiwei Z., Junjie W. Hui X. and Guoping X. (2011) Scaling up top-K cosine similarity search (Data \u0026 Knowledge Engineering 70)\n\n\n\nLicense\n-------\n\nCopyright (c) 2011 Paul Asmuth\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to use, copy and modify copies of the Software, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasmuth%2Frecommendify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasmuth%2Frecommendify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasmuth%2Frecommendify/lists"}