{"id":13653066,"url":"https://github.com/williamliujl/LLMRec","last_synced_at":"2025-04-23T06:31:04.714Z","repository":{"id":175306952,"uuid":"650085709","full_name":"williamliujl/LLMRec","owner":"williamliujl","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-19T06:01:07.000Z","size":1221,"stargazers_count":52,"open_issues_count":2,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-10T04:36:20.011Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/williamliujl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-06-06T09:58:14.000Z","updated_at":"2024-11-06T10:57:35.000Z","dependencies_parsed_at":"2024-01-14T13:13:25.735Z","dependency_job_id":"d955973c-0284-4099-babe-0cc09be9f2c6","html_url":"https://github.com/williamliujl/LLMRec","commit_stats":null,"previous_names":["williamliujl/llmrec"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamliujl%2FLLMRec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamliujl%2FLLMRec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamliujl%2FLLMRec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamliujl%2FLLMRec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/williamliujl","download_url":"https://codeload.github.com/williamliujl/LLMRec/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250384809,"owners_count":21421796,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:01:05.433Z","updated_at":"2025-04-23T06:31:01.459Z","avatar_url":"https://github.com/williamliujl.png","language":"Python","readme":"# LLMRec\n\n## Abstract\n\nLLMRec, a LLM-based recommender system designed for benchmarking LLMs on various recommendation tasks. Specifically, we construct a set of well-designed templates to first transform the recommendation data into textual prompts and then benchmark several popular LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. \n\n\u003cimg src=\"https://github.com/williamliujl/LLMRec/blob/main/docs/workflow.png\" width=\"860\" /\u003e\n\n## Prompt Construction\n\nWe have used various prompts for off-the-shelf LLMs evaluation, as shown in the following figure.\n\n\u003cimg src=\"https://github.com/williamliujl/LLMRec/blob/main/docs/off-the-shelf_test-prompts.png\" width=\"860\" /\u003e\n\nTo fully assess the recommendation capabilities of LLMs, we conducted task-specific finetuning on various open-source LLMs. We utilized LLMRec's prompt construction module to produce training data for finetuning. The prompt templates are presented below.\n\n\u003cimg src=\"https://github.com/williamliujl/LLMRec/blob/main/docs/finetuing_prompts.png\" width=\"860\" /\u003e\n\n## Data Acquire\nOur prompt data is provided on Google Drive and Baidu Yun.\n\n- Google Drive: https://drive.google.com/drive/folders/1Euz7DAbiWKAwiTErLBnebbUNkCY0Vo4C?usp=sharing\n\n- Baidu Yun: https://pan.baidu.com/s/12mxZWih6D0yJCdw3Jo15Qg 提取码: 9ks9 \n\n\n## To-Do\n\n- [ ] add off-the-shelf LLMs evaluation scripts\n\n- [ ] add LLMs finetuing scripts\n\n\n\n## Citation\nIs ChatGPT a Good Recommender? A Preliminary Study\nhttps://arxiv.org/abs/2304.10149\n\n```\n@article{liu2023chatgpt,\n  title={Is ChatGPT a Good Recommender? A Preliminary Study},\n  author={Liu, Junling and Liu, Chao and Lv, Renjie and Zhou, Kang and Zhang, Yan},\n  journal={arXiv preprint arXiv:2304.10149},\n  year={2023}\n}\n```\n","funding_links":[],"categories":["Datasets-or-Benchmark","2. Datasets \u0026 Benchmarks"],"sub_categories":["垂直领域","2.2 Benchmarks"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilliamliujl%2FLLMRec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwilliamliujl%2FLLMRec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilliamliujl%2FLLMRec/lists"}