Jason Burton on LLMs and collective intelligence, algorithmic amplification, AI in deliberative processes, and decentralized networks (AC Ep68)

“When you get a response from a language model, it’s a bit like a response from a crowd of people, shaped by the preferences of countless individuals.”

– Jason Burton

About Jason Burton

Jason Burton is an assistant professor at Copenhagen Business School and an Alexander von Humboldt Research fellow at the Max Planck Institute for Human Development. His research applies computational methods to studying human behavior in a digital society, including reasoning in online information environments and collective intelligence.

LinkedIn: Jason William Burton

Google Scholar page: Jason Burton

University Profile (Copenhagen Business School): Jason Burton

What you will learn

• Exploring AI’s role in collective intelligence
• How large language models simulate crowd wisdom
• Benefits and risks of AI-driven decision-making
• Using language models to streamline collaboration
• Addressing the homogenization of thought in AI
• Civic tech and AI’s potential in public discourse
• Future visions for AI in enhancing group intelligence

Episode Resources

• Nature Human Behavior
• How Large Language Models Can Reshape Collective Intelligence
• ChatGPT
• Max Planck Institute for Human Development
• Reinforcement learning from human feedback
• DeepMind
• Digital twin
• Wikipedia
• Algorithmic Amplification and Society
• Wisdom of the crowd
• Recommender system
• Decentralized autonomous organizations
• Civic technology
• Collective intelligence
• Deliberative democracy
• Echo chambers
• Post-truth

People

• Jürgen Habermas
• Dave Rand
• Ulrika Hahn
• Helena Landemore

Transcript

Ross: Ross, Jason, it is wonderful to have you on the show.

Jason Burton: Hi, Ross. Thanks for having me.

Ross: So you and 27 co-authors recently published in Nature Human Behavior a wonderful article called How Large Language Models Can Reshape Collective Intelligence. I’d love to hear the backstory of how this paper came into being with 28 co-authors.

Jason: It started in May 2023. There was a research retreat at the Max Planck Institute for Human Development in Berlin, about six months or so after ChatGPT had really come into the world, at least for the average person. We convened a sort of working group around this idea of the intersection between language models and collective intelligence, something interesting that we thought was worth discussing.

At that time, there were just about five or six of us thinking about the different ways to view language models intersecting with collective intelligence: one where language models are a manifestation of collective intelligence, another where they can be a tool to help collective intelligence, and another where they could potentially threaten collective intelligence in some ways. On the back of that working group, we thought, well, there are lots of smart people out there working on similar things. Let’s try to get in touch with them and bring it all together into one paper. That’s how we arrived at the paper we have today.

Ross: So, a paper being the manifestation of collective intelligence itself?

Jason: Yes, absolutely.

Ross: You mentioned an interesting part of the paper—that LLMs themselves are an expression of collective intelligence, which I think not everyone realizes. How does that work? In what way are LLMs a type of collective intelligence?

Jason: Sure, yeah. The most obvious way to think about it is these are machine learning systems trained on massive amounts of text. Where are the companies developing language models getting this text? They’re looking to the internet, scraping the open web. And what’s on the open web? Natural language that encapsulates the collective knowledge of countless individuals.

By training a machine learning system to predict text based on this collective knowledge they’ve scraped from the internet, querying a language model becomes a kind of distilled form of crowdsourcing. When you get a response from a language model, you’re not necessarily getting a direct answer from a relational database. Instead, you’re getting a response that resembles the answer many people have given to similar queries.

On top of that, once you have the pre-trained language model, a common next step is training through a process called reinforcement learning from human feedback. This involves presenting different responses and asking users, “Did you like this response or that one better?” Over time, this system learns the preferences of many individuals. So, when you get a response from a language model, it’s shaped by the preferences of countless individuals, almost like a response from a crowd of people.

Ross: This speaks to the mechanisms of collective intelligence that you write about in the paper, like the mechanisms of aggregation. We have things like markets, voting, and other fairly crude mechanisms for aggregating human intelligence, insight, or perspective. This seems like a more complex and higher-order aggregation mechanism.

Jason: Yeah. I think at its core, language models are performing a form of compression, taking vast amounts of text and forming a statistical representation that can generate human-like text. So, in a way, a language model is just a new aggregation mechanism.

In an analog sense, maybe taking a vote or deliberating as a group leads to a decision. You could use a language model to summarize text and compress knowledge down into something more digestible.

Ross: One core part of your article discusses how LLMs help collective intelligence. We’ve had several mechanisms before, and LLMs can assist in existing aggregation structures. What are the primary ways that LLMs assist collective intelligence?

Jason: A lot of it boils down to the realization of how easy it is to query and generate text with a language model. It’s fast and frictionless. What can we do with that? One straightforward use is that, if you think of a language model as a kind of crowd in itself, you can use it to replace traditional crowdsourcing.

If you’re crowdsourcing ideas for a new product or marketing campaign, you could instead query a language model and get results almost instantaneously. Crowdsourcing taps into crowd diversity, producing high-quality, diverse responses. However, it requires setting up a crowd and a mechanism for querying, which can be time and resource-intensive. Now, we have these models at our fingertips, making it much quicker.

Another potential use that excites me is using language models to mediate deliberative processes. Deliberation is beneficial because individuals exchange information, allowing them to become more knowledgeable about a task. I have some knowledge, and you have some knowledge. By communicating, we learn from each other.

Ross: Yeah, and there have been some other researchers looking at nudges for encouraging participation or useful contributions. I think another point in your paper is around aggregating group discussions so that other groups or individuals can effectively take those in, allowing for scaled participation and discussion.

Jason: Yeah, absolutely. There’s a well-documented trade-off. Ideally, in a democratic sense, you want to involve everybody in every discussion, as everyone has knowledge to share. By bringing more people into the conversation, you establish a shared responsibility in the outcome. But as you add more people to the room, it becomes louder and noisier, making progress challenging.

If we can use technological tools, whether through tradition