{"id":22974044,"url":"https://github.com/bolcom/serenade","last_synced_at":"2025-10-28T15:22:41.590Z","repository":{"id":38396171,"uuid":"459574864","full_name":"bolcom/serenade","owner":"bolcom","description":"Session-based recommender system: Serenade","archived":false,"fork":false,"pushed_at":"2024-11-06T12:47:01.000Z","size":2700,"stargazers_count":82,"open_issues_count":7,"forks_count":7,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-08-21T04:02:36.006Z","etag":null,"topics":["recommendation","recommendation-system","recommender-system","rust"],"latest_commit_sha":null,"homepage":"https://ssc.io/pdf/modds003.pdf","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bolcom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-15T12:45:37.000Z","updated_at":"2025-07-20T09:18:11.000Z","dependencies_parsed_at":"2024-06-13T11:12:18.093Z","dependency_job_id":"7831aadc-614c-4e78-bc6d-df42443415d9","html_url":"https://github.com/bolcom/serenade","commit_stats":{"total_commits":102,"total_committers":3,"mean_commits":34.0,"dds":0.5098039215686274,"last_synced_commit":"6ce2a420ed4e1ba77a085aec03c19174b5ec0b8c"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/bolcom/serenade","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bolcom%2Fserenade","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bolcom%2Fserenade/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bolcom%2Fserenade/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bolcom%2Fserenade/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bolcom","download_url":"https://codeload.github.com/bolcom/serenade/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bolcom%2Fserenade/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272839660,"owners_count":25001862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-30T02:00:09.474Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["recommendation","recommendation-system","recommender-system","rust"],"created_at":"2024-12-14T23:59:57.142Z","updated_at":"2025-10-28T15:22:41.508Z","avatar_url":"https://github.com/bolcom.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Serenade: Low-Latency Session-Based Recommendations](https://github.com/bolcom/serenade)\n\n[![Crates.io](https://img.shields.io/crates/v/serenade?style=flat-square)](https://crates.io/crates/serenade)\n[![Crates.io](https://img.shields.io/crates/d/serenade?style=flat-square)](https://crates.io/crates/serenade)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square)](https://github.com/bolcom/serenade/LICENSE.md)\n[![Build Status](https://img.shields.io/github/workflow/status/bolcom/serenade/Release/v0.2.6?style=flat-square)](https://github.com/bolcom/serenade/actions/workflows/release.yml?query=branch%3Av0.2.6)\n[![API Docs](https://docs.rs/serenade/badge.svg)](https://docs.rs/serenade)\n[![Contributors](https://img.shields.io/github/contributors/bolcom/serenade?style=flat-square)](https://github.com/bolcom/serenade/graphs/contributors)\n\n[\u003cimg src=\"https://raw.githubusercontent.com/bolcom/serenade/main/docs/airlab.png\" width=\"250\" align=\"right\"\u003e](https://icai.ai/airlab/)\n[\u003cimg src=\"https://raw.githubusercontent.com/bolcom/serenade/main/docs/uva.png\" width=\"250\" align=\"right\"\u003e](https://irlab.science.uva.nl/)\nThis repository contains the official code for session-based recommender system Serenade, which employs VMIS-kNN.\nIt learns users' preferences by capturing the short-term and sequential patterns from the evolution of user behaviors and predicts interesting next items with low latency with support for millions of distinct items.\nVMIS-kNN is an index-based variant of a state-of-the-art nearest neighbor algorithm to session-based recommendation, which scales to use cases with hundreds of millions of clicks to search through.\n\nThe VMIS-kNN implementation has a p90 prediction latency of \u003c1.7ms in our micro benchmarks on private and public trainingsets up to 60M user-item interactions with 1.76 million distinct items. \nThe Serenade recommender service using the VMIS-kNN algorithm easily handles 1000 predictions per second using only two vCPU's in total. The p90 prediction latency of the deployed system with kubernetes is \u003c 7ms end-to-end, measured from a different node using a http client, including http overhead, network traffic, istio loadbalancers, fetching session information from RocksDb and filtering for product availablity and intimacy, the serializing of the results etc. Training data is 2.3 billion records of which 582 million training records are used after index pruning and contains about 6.5 million distinct products. The index requires about 11GB of memory per serving node.\n\n# Quick guide: getting started with Serenade.\n\n## Table of contents\n1. [Downloads](#downloads)\n2. [Find the best hyperparameter values](#find-hyperparams)\n3. [Configure Serenade to use the hyperparameter values](#update-config)\n4. [Start the Serenade service](#start-service)\n5. [Retrieve recommendations using python](#retrieve-recommendations)\n6. [Evaluate the testset](#evaluator)\n7. [Using your own train- and testset](#dataset)\n8. [Citation](#citation)\n9. [Join us](#join-us)\n10. [License](#license)\n\n\n### Downloads \u003ca name=\"downloads\"\u003e\u003c/a\u003e\nSerenade can be downloaded [here](https://github.com/bolcom/serenade/releases). Binary executables are available for Windows, Linux and MacOS.\nDownload the [toy example project](https://github.com/bolcom/serenade/raw/main/assets/example/example.zip) which contains toy datasets and a preconfigured configuration file.\n\nExtract both downloaded files in the same directoy. You now have the following files:\n```\nserving\ntpe_hyperparameter_optm\nevaluator\ntrain.txt\ntest.txt\nvalid.txt\nexample.toml\n```\n\n### Find the best hyperparameter values \u003ca name=\"find-hyperparams\"\u003e\u003c/a\u003e\nThe next step is finding the hyperparameters using the train and test-datasets.\nSerenade uses Tree-Structured Parzen Estimator (TPE) for finding the hyperparameters. TPE achieves low validation errors compared to Exhaustive Grid Search ([Bergstra et al](http://proceedings.mlr.press/v28/bergstra13.pdf)).\nThe section `[hyperparam]` in the `example.toml` contains the ranges of hyperparameter values that will be explored.\n\n* The hyperparameter search can be started using:\n```bash\n./tpe_hyperparameter_optm example.toml\n```\n\nThe results will be printed out in the terminal, for example:\n```\n===============================================================\n===          HYPER PARAMETER OPTIMIZATION RESULTS          ====\n===============================================================\nMRR@20 for validation data: 0.3197\nMRR@20 for test data: 0.3401\nenabled business_logic for evaluation:false\nbest hyperparameter values:\nn_most_recent_sessions:1502\nneighborhood_size_k:288\nidf_weighting:2\nlast_items_in_session:4\nHPO done\n```\nand also in the output file defined in the config file, for example:\n```\nout_path = \"results.csv\"\n```\n\n### Configure Serenade to use the hyperparameter values \u003ca name=\"update-config\"\u003e\u003c/a\u003e\nWe now update the `[model]` values in configuration file `example.toml` to use the hyperparameter values and set the training_data_path with the location of the ```train.txt```.\nThis is the content of the example configuration file with the new `[model]` paramer values.\n```\nconfig_type = \"toml\"\n\n[server]\nhost = \"0.0.0.0\"\nport = 8080\nnum_workers = 4\n\n[log]\nlevel = \"info\" # not implemented\n\n[data]\ntraining_data_path=\"train.txt\"\n\n[model]\nm_most_recent_sessions = 1502\nneighborhood_size_k = 288\nmax_items_in_session = 4\nnum_items_to_recommend = 21\nidf_weighting = 1\n\n[logic]\nenable_business_logic = \"false\"\n\n[hyperparam]\ntraining_data_path = \"train.txt\"\ntest_data_path = \"test.txt\"\nvalidation_data_path = \"valid.txt\"\nnum_iterations = 15\nsave_records = true\nout_path = \"results.csv\"\nenable_business_logic = false\nn_most_recent_sessions_range = [100, 2500]\nneighborhood_size_k_range = [50, 1500]\nlast_items_in_session_range = [1, 20]\nidf_weighting_range = [0, 5]\n```\n\n### Start the Serenade service \u003ca name=\"start-service\"\u003e\u003c/a\u003e\nStart the `serving` binary for your platform with the location of the configuration file as argument\n```bash\n./serving example.toml\n```\n\n\nYou can open your webbrowser and goto http://localhost:8080/ you should see an internal page of Serenade.\n\nSerenade exposes [Prometheus metrics](http://localhost:8080/internal/prometheus) out-of-the-box for monitoring.\n\n### Retrieve recommendations using python \u003ca name=\"retrieve-recommendations\"\u003e\u003c/a\u003e\n\n```python\nimport requests\nfrom requests.exceptions import HTTPError\ntry:\n    myurl = 'http://localhost:8080/v1/recommend'\n    params = dict(\n        session_id='144',\n        user_consent='true',\n        item_id='13598',\n    )\n    response = requests.get(url=myurl, params=params)\n    response.raise_for_status()\n    # access json content\n    jsonResponse = response.json()\n    print(jsonResponse)\nexcept HTTPError as http_err:\n    print(f'HTTP error occurred: {http_err}')\nexcept Exception as err:\n    print(f'Other error occurred: {err}')\n```\n```\n[2835,10,12068,4313,3097,8028,3545,7812,17519,1164,17935,1277,13335,8655,14664,14556,6868,13509,9248,2498,11724]\n```\nThe returned json object is a list with recommended items.\n\n\n### Evaluate the testset \u003ca name=\"evaluator\"\u003e\u003c/a\u003e\nThe `evaluator` application can be used to evaluate a test dataset. It reports on several metrics.\n* The evaluation can be started using:\n```bash\n./evaluator example.toml\n```\n\n```\n===============================================================\n===               START EVALUATING TEST FILE               ====\n===============================================================\nMrr@20,Ndcg@20,HitRate@20,Popularity@20,Precision@20,Coverage@20,Recall@20,F1score@20\n0.3277,0.3553,0.6402,0.0499,0.0680,0.2765,0.4456,0.1180\nQty test evaluations: 931\nPrediction latency\np90 (microseconds): 66\np95 (microseconds): 66\np99.5 (microseconds): 66\n```\n\n\n### Using your own train- and testset \u003ca name=\"dataset\"\u003e\u003c/a\u003e\nA train- and testset must be created from historical user-item click data, outside of Serenade. Each row in the training- or test set should contain an historical user-item interaction event with the following fields:\n* ```SessionId``` the ID of the session. Format: 64 bit Integer\n* ```ItemId``` the ID of the interacted item. Format: 64 bit Integer\n* ```Time``` the time when the user-item interaction occurred. In epoch seconds: 32 bit Floating point.\n\nThe last 24 hours in the historical data can be used as test-set while the rest of the sessions can be used as the training-set and written as plain text using a ```'\\t'``` as field separator.\nThis is an example of a training data CSV file train.txt:\n```\nSessionId       ItemId  Time\n10036   14957   1592337718.0\n10036   14713   1592337765.0\n10036   2625    1592338184.0\n10037   7267    1591979344.0\n10037   13892   1591979380.0\n10037   7267    1591979504.0\n10037   3595    1591979784.0\n10038   6424    1591008704.0\n```\n\n\n# Citation  \u003ca name=\"citation\"\u003e\u003c/a\u003e\n\u003e [Serenade - Low-Latency Session-Based Recommendation in e-Commerce at Scale](https://ssc.io/pdf/modds003.pdf)\n\n    @inproceedings{Kersbergen2022SerenadeScale,\n        title = {{Serenade - Low-Latency Session-Based Recommendation in e-Commerce at Scale}},\n        year = {2022},\n        series = {SIGMOD '22}\n        author = {Kersbergen, Barrie and Sprangers, Olivier and Schelter, Sebastian},\n        booktitle = {Proceedings of the 2022 International Conference on Management of Data},\n        pages = {150–159},\n        numpages = {10},\n        location = {Philadelphia, PA, USA},\n        isbn = {9781450392495},\n        publisher = {Association for Computing Machinery},\n        address = {New York, NY, USA},\n        doi = {10.1145/3514221.3517901},\n    }\n\n# Join us \u003ca name=\"join-us\"\u003e\u003c/a\u003e\nAnyone who interested in our project is welcome to join us. Let us build a wonderful open source community for this session-based recommender system!\nWe are always looking for the developers that contribute core code (Rust and python) or specific algorithms to the Serenade Project.\nExample ideas for improvement are:\nadd specific algorithms such as:\n* (STAN)[https://github.com/rn5l/session-rec/blob/master/algorithms/knn/stan.py] \n* (V-STAN)[https://github.com/rn5l/session-rec/blob/master/algorithms/knn/vstan.py] \nmake hardcoded design choices configurable\n* let the user define the metric (MRR, Hitrate, NDCG etc) and its length for the evaluator.rs via the config file instead of hardcoded\n* improve the duration for the hyperparameter.rs for larger files. Right now it uses just one thread which takes \n* let the user define the percentiles in the configuration file that should be printed when using the evaluator.rs\nand more!\nPlease contact us.\n\n# License \u003ca name=\"license\"\u003e\u003c/a\u003e\nThis project is licensed under the terms of the [Apache 2.0 license](LICENSE.md).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbolcom%2Fserenade","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbolcom%2Fserenade","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbolcom%2Fserenade/lists"}