{"id":13958364,"url":"https://github.com/awarebayes/RecNN","last_synced_at":"2025-07-20T23:31:18.064Z","repository":{"id":44796860,"uuid":"166170681","full_name":"awarebayes/RecNN","owner":"awarebayes","description":"Reinforced Recommendation toolkit built around pytorch 1.7","archived":false,"fork":false,"pushed_at":"2020-12-08T11:37:49.000Z","size":145297,"stargazers_count":575,"open_issues_count":1,"forks_count":114,"subscribers_count":29,"default_branch":"master","last_synced_at":"2024-08-09T13:18:36.638Z","etag":null,"topics":["library","news-recommendation","pytorch","pytorch-rl","recommendation-system","recommender-system","reinforcement-learning","toolkit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awarebayes.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-17T06:12:03.000Z","updated_at":"2024-08-06T07:50:17.000Z","dependencies_parsed_at":"2022-08-21T06:10:43.414Z","dependency_job_id":null,"html_url":"https://github.com/awarebayes/RecNN","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awarebayes%2FRecNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awarebayes%2FRecNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awarebayes%2FRecNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awarebayes%2FRecNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awarebayes","download_url":"https://codeload.github.com/awarebayes/RecNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226845000,"owners_count":17691139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["library","news-recommendation","pytorch","pytorch-rl","recommendation-system","recommender-system","reinforcement-learning","toolkit"],"created_at":"2024-08-08T13:01:30.558Z","updated_at":"2024-11-28T01:31:52.361Z","avatar_url":"https://github.com/awarebayes.png","language":"Python","funding_links":[],"categories":["5. GitHub Repositories","其他_推荐系统","Recommendation"],"sub_categories":["网络服务_其他","Tools"],"readme":"\u003cp align=\"center\"\u003e \n\u003cimg src=\"./res/logo big.png\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e \n\n\u003ca href='https://recnn.readthedocs.io/en/latest/?badge=latest'\u003e\n    \u003cimg src='https://readthedocs.org/projects/recnn/badge/?version=latest' alt='Documentation Status' /\u003e\n\u003c/a\u003e\n\n\u003ca href='https://circleci.com/gh/awarebayes/RecNN'\u003e\n    \u003cimg src='https://circleci.com/gh/awarebayes/RecNN.svg?style=svg' alt='Documentation Status' /\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://codeclimate.com/github/awarebayes/RecNN/maintainability\"\u003e\n    \u003cimg src=\"https://api.codeclimate.com/v1/badges/d3a06ffe45906969239d/maintainability\" /\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://github.com/awarebayes/RecNN\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/stars/awarebayes/RecNN?style=social\" /\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://colab.research.google.com/github/awarebayes/RecNN/\"\u003e\n    \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" /\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://github.com/psf/black\"\u003e\n    \u003cimg alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e \nThis is my school project. It focuses on Reinforcement Learning for personalized news recommendation. The main distinction is that it tries to solve online off-policy learning with dynamically generated item embeddings. I want to create a library with SOTA algorithms for reinforcement learning recommendation, providing the level of abstraction you like.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://recnn.readthedocs.io\"\u003erecnn.readthedocs.io\u003c/a\u003e\n\u003c/p\u003e\n\n### 📊 The features can be summed up to\n\n- Abstract as you decide: you can import the entire algorithm (say DDPG) and tell it to ddpg.learn(batch), you can import networks and the learning function separately, create a custom loader for your task, or can define everything by yourself.\n\n- Examples do not contain any of the junk code or workarounds: pure model definition and the algorithm itself in one file. I wrote a couple of articles explaining how it functions.\n\n- The learning is built around sequential or frame environment that supports ML20M and like. Seq and Frame determine the length type of sequential data, seq is fully sequential dynamic size (WIP), while the frame is just a static frame.\n\n- State Representation module with various methods. For sequential state representation, you can use LSTM/RNN/GRU (WIP) \n\n- Parallel data loading with Modin (Dask / Ray) and caching\n\n- Pytorch 1.7 support with Tensorboard visualization.\n\n- New datasets will be added in the future.\n\n## 📚 Medium Articles\n\nThe repo consists of two parts: the library (./recnn), and the playground (./examples) where I explain how to work with certain things. \n\n -  Pretty much what you need to get started with this library if you know recommenders but don't know much about\n  reinforcement learning:\n\n\u003cp align=\"center\"\u003e \n   \u003ca href=\"https://towardsdatascience.com/reinforcement-learning-ddpg-and-td3-for-news-recommendation-d3cddec26011\"\u003e\n        \u003cimg src=\"./res/article_1.png\"  width=\"70%\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n -  Top-K Off-Policy Correction for a REINFORCE Recommender System:\n\u003cp align=\"center\"\u003e \n   \u003ca href=\"https://towardsdatascience.com/top-k-off-policy-correction-for-a-reinforce-recommender-system-e34381dceef8\"\u003e\n        \u003cimg src=\"./res/article_2.png\" width=\"70%\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n\n## Algorithms that are/will be added\n\n\u003cp align=\"center\"\u003e \n    \n| Algorithm                             | Paper                            | Code                       |\n|---------------------------------------|----------------------------------|----------------------------|\n| Deep Q Learning (PoC)                 | https://arxiv.org/abs/1312.5602  | examples/0. Embeddings/ 1.DQN |\n| Deep Deterministic Policy Gradients   | https://arxiv.org/abs/1509.02971 | examples/1.Vanilla RL/DDPG |\n| Twin Delayed DDPG (TD3)               | https://arxiv.org/abs/1802.09477 | examples/1.Vanilla RL/TD3  |\n| Soft Actor-Critic                     | https://arxiv.org/abs/1801.01290 | examples/1.Vanilla RL/SAC  |\n| Batch Constrained Q-Learning          | https://arxiv.org/abs/1812.02900 | examples/99.To be released/BCQ |\n| REINFORCE Top-K Off-Policy Correction | https://arxiv.org/abs/1812.02353 | examples/2. REINFORCE TopK |\n\n\u003c/p\u003e\n\n### ‍Repos I used code from\n\n -  Sfujim's [BCQ](https://github.com/sfujim/BCQ) (not implemented yet)\n -  Higgsfield's [RL Adventure 2](https://github.com/higgsfield/RL-Adventure-2) (great inspiration)\n\n### 🤔 What is this\n\n\u003cp align=\"center\"\u003e \nThis is my school project. It focuses on Reinforcement Learning for personalized news recommendation. The main distinction is that it tries to solve online off-policy learning with dynamically generated item embeddings. Also, there is no exploration, since we are working with a dataset. In the example section, I use Google's BERT on the ML20M dataset to extract contextual information from the movie description to form the latent vector representations. Later, you can use the same transformation on new, previously unseen items (hence, the embeddings are dynamically generated). If you don't want to bother with embeddings pipeline, I have a DQN embeddings generator as a proof of concept.\n\u003c/p\u003e\n\n\n## ✋ Getting Started\n\u003cp align=\"center\"\u003e \n\u003ca href=\"https://colab.research.google.com/drive/1xWX4JQvlcx3mizwL4gB0THEyxw6LsXTL\"\u003e\u003cimg src=\"https://i.postimg.cc/mDzKRc1K/code.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e \n\u003ca href=\"https://colab.research.google.com/drive/1xWX4JQvlcx3mizwL4gB0THEyxw6LsXTL\"\u003e\u003cimg src=\"https://i.postimg.cc/D0Qjy1vp/get-started.png\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\np.s. Image is clickable. here is direct link:\n\u003ca href=\"https://colab.research.google.com/drive/1xWX4JQvlcx3mizwL4gB0THEyxw6LsXTL\"\u003e\n    \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" /\u003e\n\u003c/a\u003e\n\nTo learn more about recnn, read the docs: \u003ca href=\"https://recnn.readthedocs.io\"\u003erecnn.readthedocs.io\u003c/a\u003e \n\n### ⚙️ Installing\n\n```\npip install git+git://github.com/awarebayes/RecNN.git\n```\n\nPyPi is on its way...\n\n### 🚀 Try demo\n\nI built a [Streamlit](https://www.streamlit.io/) demo to showcase its features.\nIt has 'recommend me a movie' feature! Note how the score changes when you **rate** the movies. When you start\nand the movies aren't rated (5/10 by default) the score is about ~40 (euc), but as you rate them it drops to \u003c10,\nindicating more personalized and precise predictions. You can also test diversity, check out the correlation of\nrecommendations, pairwise distances, and pinpoint accuracy.\n\nRun it:\n```\ngit clone git@github.com:awarebayes/RecNN.git \ncd RecNN \u0026\u0026 streamlit run examples/streamlit_demo.py\n```\n\n[Docker image is available here](https://github.com/awarebayes/recnn-demo)\n\n## 📁 Downloads\n -  [MovieLens 20M](https://grouplens.org/datasets/movielens/20m/)\n -  [Movie Embeddings](https://drive.google.com/open?id=1EQ_zXBR3DKpmJR3jBgLvt-xoOvArGMsL)\n -  [Misc Data](https://drive.google.com/open?id=1TclEmCnZN_Xkl3TfUXL5ivPYmLnIjQSu)\n -  [Parsed (omdb,tmdb)](https://drive.google.com/open?id=1t0LNCbqLjiLkAMFwtP8OIYU-zPUCNAjK)\n\n## 📁 [Download the Models](https://drive.google.com/file/d/1goGa15XZmDAp2msZvRi2v_1h9xfmnhz7/view?usp=sharing)\n\n## 📄 Citing\nIf you find RecNN useful for an academic publication, then please use the following BibTeX to cite it:\n\n```\n@misc{RecNN,\n  author = {M Scherbina},\n  title = {RecNN: RL Recommendation with PyTorch},\n  year = {2019},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/awarebayes/RecNN}},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawarebayes%2FRecNN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawarebayes%2FRecNN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawarebayes%2FRecNN/lists"}