![[Chat-REC] Proposal For LLM-based Recommendation System](https://aisholar.s3.ap-northeast-1.amazonaws.com/media/July2024/chat-rec.png)
[Chat-REC] Proposal For LLM-based Recommendation System
3 main points
✔️ Propose Chat-REC, an LLM-based recommendation system with interactivity and explainability
✔️ Build a conversational recommendation system based on in-context learning with user profiles and past interactions as prompts
✔️ Experiments have shown that it is possible to learn user preferences from conversational interactions Experiments have shown that it is possible to learn user preferences from conversational interactions and to make recommendations across different product domains.
Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System
written by Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, Jiawei Zhang
(Submitted on 25 Mar 2023 (v1), last revised 4 Apr 2023 (this version, v2))
Comments: Publised on arxiv.
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)code：
The images used in this article are from the paper, the introductory slides, or were created based on them.
Introduction
In today's era of big data, recommendation systems are widely used when we search for or select a product.
Since it is impossible to manually search for a product from a myriad of product groups, there is a need for a system that automatically extracts and infers user preferences and makes high-quality product recommendations accordingly.
However, various problems remain in model design and data distribution bias.
For example, there is a lack of user-friendliness, such as interactivity and explainability, a cold-start problem that reduces the accuracy of recommendations for new users and new items, anddifficulties in making recommendations across product domains.
Large Language Models (LLMs), which have recently become very popular, are expected to solve these problems.
LLM provides a clear description and is capable of cross-domain information processing between product domains, making full use of the vast amount of information it holds internally.
In addition, LLMs are adept at user interaction and can enhance the user experience.
This paper presents a method for extending LLM as a recommendation system using in-context learning without requiring training.
Technique
Bridging Recommendation Systems and LLM
Conventional recommendation systems make recommendations by extracting user preferences from user data such as click history and purchase history.
On the other hand, it has become clear that natural language processing technology has the potential to extend the recommendation system by extracting user preferences from text-based content such as reviews and SNS posts, and by enabling natural responses in text.
In this paper, we propose Chat-REC, a ChatGPT-based recommendation system that follows both trends.
Chat-REC is a recommendation system that uses the interaction history between a user and an item (=history of how the user responded to which item. Chat-REC is connected to $R$, a recommendation system thattakes user profiles ( i . e., age, gender, location, interests, etc. ),user queries (i.e., input text describing requests) $Q$, and interaction history $H$ as input and presents candidates for recommended items.
The prompt construction module $C$ summarizes this input information and generates a prompt.
When $Q$ determines that the task specified in the query is a recommended task, it calls the recommendation system $R$ to generate a set of candidate recommended items as intermediate answers, which are entered into the prompt to generate the answer text, and when the task is not a recommended task, ChatGPT directly When it determines that the task is not a recommended task, ChatGPT responds directly.
Narrowing down the candidate set
In conventional recommendation systems, products are sorted from a myriad of products and presented to the user based on the results of scoring based on factors such as the reliability of the recommendation.
In this paper, we propose a method for narrowing down a set of candidate products using the in-context learning capability of LLMs.
First, the user's profile and past interactions are entered into the prompt, asking the LLM to summarize the user's preferences.
With this knowledge of the relationship between product attributes and user preferences, the recommendation system can take the set of candidate recommendations generated by the recommendation system, filter and reorder them according to user preferences, and present a more optimized set of candidates. The user can then filter and reorder them according to his or her preferences and present a more optimized set of candidates.
Response to new items
The datasets used by LLM for training have time limitations and usually do not contain the most recent information.
Therefore, if a user asks for a recommendation of the latest item, the above approach is not sufficient.
A possible solution to this problem is to store data on new items in a database in advance and refer to it when a recommendation for the latest item is requested.
Specifically, the text encoding of the recommended items is performed, the embedded expressions are stored in a database, and information on highly relevant items is brought to the prompt based on the similarity of the embedded expressions in the request text.
Recommendation across product domains
LLMs are trained on the vast amount of data available on the Internet and are considered to function as a knowledge base with many perspectives.
Thus, as shown in the example below, a recommendation for a movie can be followed by a recommendation for a product in another domain, such as a book or a TV program.
Experiment
Dataset and experimental setup
MovieLens 100K, a movie review dataset, is used.
This dataset contains data from 943 users who rated 1682 films on a 5-point scale from 1 to 5.
It contains statistical information about each user (age, gender, occupation, etc.) and movie attributes (title, year of release, genre, etc.).
In this experiment, 200 users are randomly selected.
The performance evaluation of recommendations uses the Precision, Recall, and Normalized Discounted Cumulative Gain (NDCG) indices, while the performance evaluation of the evaluation forecast task uses the Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) are used to evaluate performance on the evaluation forecasting task.
Baseline
For comparison, we use the following four classical recommendation system models
LightFM: A method that combines collaborative filtering (a recommendation method based on item and user relationships) and content-based methods (a recommendation method based on item information)
LightGCN: A graph-based collaborative filtering method. Approximates the relationship between users and items with a graph convolutional network.
Item-KNN: A KNN-based collaborative filtering method.
Matrix Factorization (MF): A collaborative filtering method that compresses the relationship between users and items to a lower dimension.
In addition, gpt-3.5-turbo, text-davinci-003, and text-davinci-002 are used as backbone models for ChatGPT.
Result
Top 5 Recommendations
Use prompts such as
The accuracy in the task of recommending the top five was as follows
While the conventional method outperforms the proposed method for Recall, the proposed method Chat-REC is the best for Precision and NDCG.
Rated Value Prediction
Use prompts such as
For the task of predicting the valuation values, the results were as follows
It can be seen that the proposed method Chat-REC showed the best performance.
The results show that LLM is able to extract user preferences from in-context learning alone and accurately predict ratings.
On the other hand, we can see that the accuracy is low when gpt-3.5-turbo is used as the backbone model for ChatGPT.
This may be due to gpt-3.5-turbo's emphasis on the ability to interact with humans and the trade-off in in-context learning ability, which is consistent with the results of other studies.
Summary
The paper presented in this issueproposed Chat-REC, a model thatbridges LLM and recommendation systems by inputting user information and user-item interaction information into prompts to make effective recommendations.
LLM seems to have the potential to improve recommendation performance in terms of interactivity, explainability, and product domain-to-domain recommendations.
It is expected that the performance of recommendation systems will be further improved to provide a smoother and more comfortable user experience when we select products in our daily lives.
Categories related to this article
