https://github.com/williamliujl/LLMRec

https://github.com/williamliujl/LLMRec

Last synced: 6 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/williamliujl/LLMRec
• Owner: williamliujl
• License: apache-2.0
• Created: 2023-06-06T09:58:14.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-08-19T06:01:07.000Z (about 1 year ago)
• Last Synced: 2024-08-03T02:09:25.403Z (4 months ago)
• Language: Python
• Size: 1.16 MB
• Stars: 45
• Watchers: 3
• Forks: 3
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-llm-eval - LLMRec - 10-08)| (Datasets-or-Benchmark / 垂直领域)
• Awesome-LLM-for-RecSys - [Link

README

        
# LLMRec

## Abstract

LLMRec, a LLM-based recommender system designed for benchmarking LLMs on various recommendation tasks. Specifically, we construct a set of well-designed templates to first transform the recommendation data into textual prompts and then benchmark several popular LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. 



## Prompt Construction

We have used various prompts for off-the-shelf LLMs evaluation, as shown in the following figure.



To fully assess the recommendation capabilities of LLMs, we conducted task-specific finetuning on various open-source LLMs. We utilized LLMRec's prompt construction module to produce training data for finetuning. The prompt templates are presented below.



## Data Acquire

Our prompt data is provided on Google Drive and Baidu Yun.

- Google Drive: https://drive.google.com/drive/folders/1Euz7DAbiWKAwiTErLBnebbUNkCY0Vo4C?usp=sharing

- Baidu Yun: https://pan.baidu.com/s/12mxZWih6D0yJCdw3Jo15Qg 提取码: 9ks9 

## To-Do

- [ ] add off-the-shelf LLMs evaluation scripts

- [ ] add LLMs finetuing scripts

## Citation

Is ChatGPT a Good Recommender? A Preliminary Study

https://arxiv.org/abs/2304.10149

```

@article{liu2023chatgpt,

  title={Is ChatGPT a Good Recommender? A Preliminary Study},

  author={Liu, Junling and Liu, Chao and Lv, Renjie and Zhou, Kang and Zhang, Yan},

  journal={arXiv preprint arXiv:2304.10149},

  year={2023}

}

```