{"id":25147076,"url":"https://github.com/dse-capstone-sharknado/models-legacy","last_synced_at":"2025-04-28T13:08:52.069Z","repository":{"id":275257112,"uuid":"79676224","full_name":"DSE-capstone-sharknado/models-legacy","owner":"DSE-capstone-sharknado","description":"BPR-Based Recommender Systems for Amazon Dataset","archived":false,"fork":false,"pushed_at":"2017-05-25T20:27:40.000Z","size":144,"stargazers_count":3,"open_issues_count":0,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-28T13:08:46.557Z","etag":null,"topics":["bpr","machine-learning","ranking","recommendations","sgd"],"latest_commit_sha":null,"homepage":"sharknado.eggie5.com","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DSE-capstone-sharknado.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-01-21T22:15:01.000Z","updated_at":"2019-11-22T01:36:18.000Z","dependencies_parsed_at":"2025-02-01T07:44:26.035Z","dependency_job_id":"fbb0ce28-240b-49a6-8bca-757d9dbef6e4","html_url":"https://github.com/DSE-capstone-sharknado/models-legacy","commit_stats":null,"previous_names":["dse-capstone-sharknado/models-legacy"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSE-capstone-sharknado%2Fmodels-legacy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSE-capstone-sharknado%2Fmodels-legacy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSE-capstone-sharknado%2Fmodels-legacy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSE-capstone-sharknado%2Fmodels-legacy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DSE-capstone-sharknado","download_url":"https://codeload.github.com/DSE-capstone-sharknado/models-legacy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251319607,"owners_count":21570427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bpr","machine-learning","ranking","recommendations","sgd"],"created_at":"2025-02-08T20:28:24.925Z","updated_at":"2025-04-28T13:08:51.955Z","avatar_url":"https://github.com/DSE-capstone-sharknado.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Amazon Dataset RecSys Models\n\nIncludes reference implementations for:\n\n* BMR: https://arxiv.org/pdf/1205.2618.pdf\n* VBMR: https://arxiv.org/pdf/1510.01784.pdf\n* TVBMP: https://arxiv.org/pdf/1602.01585.pdf\n\nrecommender systems trained on an Amazon Reviews dataset:\n\nDataset: http://jmcauley.ucsd.edu/data/amazon/\n\nOriginal Code: https://sites.google.com/a/eng.ucsd.edu/ruining-he/ ( Ruining He)\n\n## Requirements\n\nTraditional gcc tool-chain setup, i.e. not a mac (LLVM).  * See Docker notes below *\n\n* compiler support for openmp (ubuntu: `sudo apt install libomp-dev`)\n* armidillo lib (ubuntu: `sudo apt-get install libarmadillo-dev`)\n\n\n## Preprocessing\n\n### Transform\n\nAll the models need the reviews dataset in a simplified format (CSV), use the `convert_to_simple.py` util to do this:\n\n```\npython convert_to_simple.py reviews_Clothing_Shoes_and_Jewelry.json.gz reviews_simple.gz\n```\n\nIt is now possible to train w/ this output file which is all reviews across all categories of clothing. It is often useful to segment across categories: \n\n### Segmentation\n\nIt is useful for evaluation to segment the clothing dataset into:\n\n* women\n* men\n* boys\n* girls\n* baby \n\nusing the `getClothingSubReviews` script.\n\nFor input it takes the gzipped simplified output from above and a meta data file which has some category mappings:\n\n https://storage.googleapis.com/sharknado-recsys-public/productMeta_simple.txt.gz\n\nThe command takes the simplified dataset file and the metadata as input:\n\n```\n ./getClothingSubReviews ../data/clothing_full.gz ../data/productMeta_simple.txt\n```\n\n This will generate these segmented review files (which you need to gzip to use in the training program):\n\n```\nreviews_Baby.txt\nreviews_Boys.txt\nreviews_Girls.txt\nreviews_Men.txt\nreviews_Women.txt\n```\n\n\n\n## Build Suite\n\nTo build the suite, run the `Makefile`, by typing `make` in the project directory. This will build a binary named `train`.\n\n\n### Training\n\nTo train all the models, pass in the following args:\n\n1. Review file path — the simplified dataset\n2. Img feature path — the image features\n3. Latent Feature Dim. (K) — hyperparameter\n4. Visual Feature Dim. (K') — hyperparameter\n5. alpha (for WRMF only) — hyperparameter\n6. biasReg (regularizer for bias terms) — hyperparameter\n7. lambda  (regularizer for general terms) — hyperparameter\n8. lambda2 (regularizer for \\\"sparse\\\" terms) — hyper-parameter\n9. Epochs (number of epochs)\n10. Max iterations\n11. Corpus/Category name under \\\"Clothing Shoes \u0026 Jewelry\\\" (e.g. Women)\n\nAlthough some parameters don't apply to some of the models, they are just passed in bulk.\n\nTo start the training routine:\n\n```\n./train simple_out.gz image_feat.gz 20 k2 alpha 10 10 lambda2 epoch 10 \"Clothing/women\"\n```\n\nExample Output:\n\n```\n{\n  \"corpus\": \"simple_out.gz\",\n  Loading votes from simple_out.gz, userMin = 5, itemMin = 0  ....\n\n  Generating votes data\n  \"nUsers\": 39387, \"nItems\": 23033, \"nVotes\": 278677\n\n\u003c\u003c\u003c BPR-MF__K_20_lambda_10.00_biasReg_10.00 \u003e\u003e\u003e\n\nIter: 1, took 0.266870\nIter: 2, took 0.247799\nIter: 3, took 0.250260\nIter: 4, took 0.248863\nIter: 5, took 0.262415\n[Valid AUC = 0.358993], Test AUC = 0.360745, Test Std = 0.305036\nIter: 6, took 0.245542\nIter: 7, took 0.245433\nIter: 8, took 0.236978\nIter: 9, took 0.236842\nIter: 10, took 0.234738\n[Valid AUC = 0.610926], Test AUC = 0.611459, Test Std = 0.294283\n\n\n \u003c\u003c\u003c BPR-MF \u003e\u003e\u003e Test AUC = 0.611459, Test Std = 0.294283\n\n\n \u003c\u003c\u003c BPR-MF \u003e\u003e\u003e Cold Start: #Item = 11453, Test AUC = 0.554613, Test Std = 0.300705\n\nModel saved to Clothing__BPR-MF__K_20_lambda_10.00_biasReg_10.00.txt.\n}\n```\n\n### Computational Note\n\nThe training routine heavy make use of threading. In experimentation we have been able to utilize 15 cores during training. This is especially important during the expensive AUC calculation.\n\n##Docker\n\nTo run the docker image:\n\n```\ndocker run -v ~/Development/DSE/capstone/UpsDowns:/mnt/mac  -ti updowns /bin/bash\n```\n\nthe `v` flag will mount the source repot to `/mnt/mac` in the container.\n\n\n## Models\n\n### BPR\n\nOn Amazon, regularization hyper-parameter lambda=10 works the best for BPR-MF, MM-MF and VBPR in most cases. \n\n\n### VBPR\n\n```\n../tools/getClothingSubImgFeatures _image_features_Clothing_Shoes_and_Jewelry.b productMeta_simple.txt.gz\n```\n\n```\n./train data/reviews_Women.txt data/image_features_Women.b 10 10 na 1 1 1 na 10 \"women\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdse-capstone-sharknado%2Fmodels-legacy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdse-capstone-sharknado%2Fmodels-legacy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdse-capstone-sharknado%2Fmodels-legacy/lists"}