The Impact Of AI Assistants Used For Writing On Users Is Revealed!

3 main points
✔️ Study of the impact of a writing assistant with a language model on users' opinions and writingcontent
✔️ Large-scale online experiment with 1506 participants and 500 judges
✔️ confirms that the use of a language model subconsciously influences users' opinions and writing.

Co-Writing with Opinionated Language Models Affects Users' Views
written by Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, Mor Naaman
(Submitted on 1 Feb 2023)
Comments: Published on arxiv.
Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

code：  

The images used in this article are from the paper, the introductory slides, or were created based on them.

Introduction.

In recent years, large-scale language models such as GPT-3 have become a part of human life, and this aspect has led to an increasing number of studies investigating the impact on people of using large-scale language models for everyday communication.

In particular, research on generative AI has uncovered issues such as stereotypes and biases in language models that automate and optimize the generation of content such as advertisements, which favor certain cultural values over others, and has revealed the dangers of using AI for such tasks.

Similarly, while these language models are increasingly being used as writing assistants, there has been little examination of how the text generated by language models affects users.

In this paper, we describe a paper in which we conducted a large-scale online experiment with 1506 participants and 500 judges to investigate the impact of a writing assistant with a language model on users' opinions and writing content, and conducted a detailed analysis based on a logistic regression model. Explanation.

Methods

This paper asks 1506 participants to respond to a social media post in a simulated online discussion to determine whether writing tasks using a writing assistant equipped with a language model can change users' writing and influence their personal opinions An online experiment was conducted.

The discussion topic for this experiment was "Is social Media Good for Society? " to which users responded either positively or negatively.

The language model used in the writing assistant was set to generate sentences that supported one side of the argument or the other. As a comparison, a group without a writing assistant (= control group) was asked to perform the experiment under the same conditions.

Experimental Design

To conduct this experiment, the authors created a custom platform that combines a social media discussion page mockup, text editor, and writing assistant, as shown in the figure below.

Participants were randomly divided into the following three groups and asked to write at least five sentences of their thoughts in the form of the above platform entry.

1. Control group: Groups that responded without a writing assistant
2. Techno-optimist language model treatment: groups responded using a language model set up to argue that social media is good for society
3. Techno-pessimist language model treatment: groups responded using a language model set up to argue that social media is bad for society

GPT-3 was used as the language model for the writing assistant used in this experiment, and was set to produce diverse and creative writing by setting sampling temperature = 0.85.

Evaluation Method

To evaluate the submitted responses, we divided the participants' written texts and asked the crowd workers to rate the opinions asserted in each text.

Each crowdworker was responsible for 25 sentences and was evaluated on whether each sentence argued thatsocial media is good for society,bad for society, orboth good and bad for society.

In addition, to assess changes in opinion due to the writing task using the writing assistant, participants answered the question after the writing task, "Do you think social media is good for society? " after the writing task.

Results

In this experiment, we first analyzed the content of participants' social media posts and then analyzed whether the language model influenced their opinions based on a logistic regression model.

Did the interaction with the language model influence the participants' writing?

The figure below shows the frequency with which participants used the language model of supportive (top), critical (bottom), or neutral (middle) opinions about social media posts, and the horizontal axis shows the frequency with which participants asserted the opinion that social media is good for society (orange), bad for society (blue), or neutral (white). The horizontal axis shows the frequency with which participants asserted the opinion that social media is good for society (orange), bad for society (blue), or neutral (white).

Notably, participants who responded using the model in favor of social media tended to assert in their posts that social media is good for society. (and vice versa).

Specifically, participants who responded using a model supporting social media were 2.04 times more likely than the control group to claim that social media is good, and participants who responded using a model criticizing social media were 2.0 times more likely than the control group to were 2.0 times more likely than the control group to claim that social media is bad.

From these results, it can be inferred that the language model used influenced the participants so that their responses were in line with the opinions dictated by the model.

Did the interaction with the language model influence the participants' opinions?

Next, we analyzed whether the interaction with the language model influenced participants' opinions expressed in the post-task question, "Do you think social media is good for society?" We then analyzed whether the interaction with the language model influenced the participants' opinions expressed in the post-task question, "Do you think social media is good for society?

The results of the analysis are shown in the figure below, where the vertical axis is the same as in the previous graph, and the horizontal axis is the result of responses to the question, "Do you think social media is good for society?" The horizontal axis is the result of the answer to the question "Do you think social media is good for society?

In this experiment, as before, participants who responded using the model favoring social media tended to give positive social media responses to the questions. (And vice versa.)

Thus, we found that the interaction with the language model influenced not only the participants' writing but also their thinking.

Were participants able to recognize that their writing was influenced by the language model?

Finally, participants were asked questions to ascertain whether they themselves were able to recognize that their writing and thinking were influenced by the language model.

The results of the responses are shown in the figure below, with the vertical axis indicating whether the participant used a language model that supported his or her own opinion, and the horizontal axis indicating the response to the question, "Was your opinion influenced by the language model?" The horizontal axis shows the results of the responses to the question, "Was your opinion influenced by the language model?

The results indicate that participants who used a language model that supported their opinion were more likely to indicate that the model had influenced their opinion.

On the other hand, overall, few participants were aware that the language model influenced their opinions, and the results suggest that participants were unknowingly influenced by the language model and that their sentences and ideas became what the model supported.

summary

How was it? In this article, we described a paper in which we conducted a large-scale online experiment with 1506 participants and 500 judges to investigate the impact of a writing assistant equipped with a language model on users' opinions and writing content, and conducted a detailed analysis based on a logistic regression model.

The author warns from the experimental results of this paper that interactions with language models that are biased toward a particular way of thinking can influence users' opinions, even if unintended, and that we should be even more cautious about using large-scale language models.

On the other hand, the current experiment tested whether the language model affects participants' views on a single topic, and further research is needed to investigate whether these phenomena generalize to other topics and to what extent they persist, so future progress will be interesting.

The details of the analysis of the experiments presented here can be found in this paper for those who are interested.

• 
• 
• 

Categories related to this article

• Natural Language Processing
• Large Language Models