Image by kp yamu Jayanath from Pixabay
It will probably be well into 2027 before we know how much generative AI influences this November’s election. Or perhaps we will never know, but we should be wary of how these technologies can persuade us.
I previously talked about a study published in Nature last December examined the ability of AI chatbots to influence political attitudes in the United States, Canada and Poland. If you remember, in that study researchers paired people with chatbots that had been instructed to advocate for a particular candidate or ballot measure, measuring the “favorability” of that candidate or measure before and after the interaction with the chatbot. They found that the interaction with the AI chatbot changed voters’ attitudes more than a traditional TV ad, a well-refined form of political persuasion.
Unlike static political ads, robocalls, or a human manning a candidate’s phone bank, a chatbot can talk forever and quickly adjust to your questions or objections with explanations that fit your viewpoint.
Another paper published in Science at about the same time found similar results in a much broader study that surveyed changes in people’s attitudes about 700+ political issues and involved almost 77,000 participants. This study also reassessed attitudes a month later and found that roughly 36 to 42 percent of the chatbots’ persuasive effect remained.
New Research released this June as a “preprint” (meaning it has not yet passed the review process and been officially published) took the comparison further, pitting AI chatbots against human persuaders in the UK, from ordinary people and professional canvassers to competitive debaters, including four world champions. Across four different experiments, they analyzed nearly 19,000 conversations with 6,923 participants.
The expert human persuaders were given opportunities to prepare and the chance to earn £1,000 bonuses. AI still came out ahead. In the first experiment, AI shifted agreement with policy positions by about 13 points on a 100-point scale relative to a control group. Elite debaters managed about eight points.
The chatbot’s advantage appeared to come from how much information it could provide and how fast it could deliver that information. When researchers restricted the length and speed of the chatbot’s response so it resembled human pacing and length, the result was a statistical tie. So it might not be that a chatbot understands people better than people do, but that it talks longer and faster.
The Nature and Science studies found that while the AI chatbots were effective at changing perceptions, their arguments were not always accurate. Chatbots were given instructions to advocate for a candidate or position and often generated a long list of supporting claims, many of which contained factual errors and omissions of fact. David Rand, a co-author on both papers, explained that “even arguments built on accurate claims can still mislead by omission.”
In both studies, the participants were informed that they were interacting with a chatbot, and I wonder how much of a factor that disclosure was in the persuasion. We think of computers as being logical and unbiased machines, and perhaps that makes many people believe the output from a chatbot more than they should. It is easy to forget how weird, unpredictable, inaccurate and biased generative AI can be.
These studies have implications outside of elections, for product sales and even nonprofit fundraising. These findings might encourage nonprofits to use chatbots to seek donations or assist their fundraising staff. Perhaps salespeople will use headphones for live coaching from a store’s chatbot.
These studies cannot tell us how many votes AI might change this November. They do show that it can shift people’s opinions, even when they know they are talking to a machine. Whether a pitch concerns a candidate, a product or a donation, a convincing answer might get us to take action while omitting important information. That could leave us with severe buyer’s remorse after we leave the store or voting booth.
Let me know your thoughts in the comments.
