The Signal

Prolific

The Signal is Prolific's podcast on research evidence - not tools, not trends, but findings from the field about how data collection actually works, and what that means for anyone whose decisions depend on it. Hosted by Prolific's Research Sciences team alongside academic and industry guests, each episode is a peer-level conversation on data quality, panel methodology, reproducibility, and the questions researchers actually debate about, grounded in specific findings.

Episodes

  1. 11 Nov

    The Science Behind Polling with Andrew Gelman | The Signal

    How can a survey of a few thousand people tell us what millions think? In this episode of The Signal, Andrew Gelman, Professor of Statistics and Political Science at Columbia University, joins Oriol Bosch, Head of Polling at Prolific, to explore the science behind polling and what makes survey results reliable. A few thousand survey responses can say something about millions of people, but only if the inference holds up. Andrew and Oriol discuss why no sample is ever perfectly representative, why the distinction between probability and non-probability surveys is blurrier than it looks, and why statistical adjustment is so important, regardless of how a sample was collected. They also explore how MRP (multilevel regression and post-stratification) works, how a survey of Xbox users produced surprisingly accurate election estimates, how to check whether statistical models are going wrong, and what to consider when commissioning survey research. Topics covered: What makes a survey reliable, and why a "representative sample" isn't necessarily a guarantee of qualityWhy the distinction between probability and non-probability sampling is more complicated than it seemsWhy statistical adjustments matter, even for carefully designed samplesHow MRP helps researchers produce small-area estimates and adjust for samples that don't match the populationThe Xbox study and what it showed about drawing conclusions from heavily skewed samplesModel validation and adjusting for variables the census doesn't haveWhy a 50/50 election forecast can still be informativeAndrew's advice for anyone commissioning survey research About the speakers: Oriol Bosch is Head of Polling at Prolific, leading work on what shapes public opinion and how to measure it more effectively. Andrew Gelman is a professor of statistics and political science at Columbia University who developed MRP (multilevel regression and poststratification), a method now widely used to estimate public opinion from survey data. He is the author of Bayesian Data Analysis and Red State, Blue State, Rich State, Poor State. References: Prolific Polling: https://www.prolific.com/polling Andrew Gelman’s blog, Statistical Modeling, Causal Inference, and Social Science https://statmodeling.stat.columbia.edu/  Reporting on Opinion Polls in the Mass Media: Statement by the 2025-2027 Committee of the European Survey Research Association (ESRA Committee, 2026) https://doi.org/10.18148/srm/2026.v20i2.8974 The Signal is Prolific's podcast on research evidence - not tools, not trends. Real findings from the field, peer-level conversation, no vendor pitch. Subscribe for new episodes monthly.

  2. 6 Oct

    Can Digital Twins Really Replace Human Participants? | The Signal

    Can an AI trained to mimic you really predict how you'd respond to a survey? In this episode of The Signal, Prolific's Oriol Bosch sits down with Olivier Toubia (Columbia Business School, co-head of the Digital Twins Lab) and Beca Almeida (Founder, Doppel Market) to unpack what digital twins and synthetic respondents can and can't do in research today. The conversation covers the defining differences between digital twins, synthetic personas, and silicon samples, how Doppel Market keeps real people in the loop rather than replacing them, where synthetic data genuinely complements traditional surveys versus where it falls short, and methods for deciding whether a question is actually suited to synthetic prediction at all. Topics covered: What digital twins, synthetic personas, and silicon samples actually mean Why Doppel Market's approach keeps humans in the loop rather than removing them The four ways AI is reshaping research, from question creation to respondent simulation A practical heuristic for deciding when synthetic data is (and isn't) appropriate Why neither guest believes full replacement of human respondents is realistic About the speakers: Oriol Bosch is Head of Polling at Prolific, leading work on what shapes public opinion and how to measure it more effectively. Olivier Toubia is Professor of Business at Columbia Business School and co-head of the Digital Twins Lab. Beca Almeida is the founder of Doppel Market, a consumer-insights platform built around digital twins. References: "Digital Twins are Funhouse Mirrors: Five Systematic Distortions" https://www.science.org/doi/10.1126/sciadv.aeh8260 "Synthetic Data and Digital Twins: The Hype, The Facts, the Promise" https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7065919 Doppel Market https://doppelmarket.com/ The Signal is Prolific's podcast on research evidence - not tools, not trends. Real findings from the field, peer-level conversation, no vendor pitch. Subscribe for new episodes monthly.

  3. 8 Sept

    Rethinking Survey Research in the Age of AI with Microsoft Research’s David Rothschild | The Signal

    AI is changing survey research, bringing benefits alongside real risks. In this first episode of The Signal, Prolific's Andrew Gordon sits down with Dr. David Rothschild, co-chair of AAPOR's Task Force on Responsible AI Integration in Survey Research, to unpack what's actually changing in survey research as AI reshapes the field - and what isn't. David walks through the different roles AI can play across the research process, and where he believes the greatest risks actually lie. The conversation covers whether synthetic respondents can ever truly substitute for real humans, Prolific and David's joint research measuring AI pollution across major online sample providers, and practical guidance for any researcher being pitched a synthetic sample product tomorrow. They also get into disclosure standards, the limits of self-declaration checkboxes, and where David comes down between the optimism versus pessimism about AI's role in research. Topics covered: What AAPOR's new report gets right about AI in research methodology AI agent prevalence: measuring contamination across major online sample providers Synthetic data's real limits: why it gets broad signals right but falls apart under scrutiny AAPOR's disclosure framework: what's required vs. what's 'enhanced'  Where AI genuinely helps vs. where it undermines data quality Why vendor opacity is the practical risk researchers should actually be watching About the speakers: Dr. Andrew Gordon is Head of Research Sciences at Prolific, driving improvements in data quality, online research methodology, and AI evaluation. LinkedIn: https://www.linkedin.com/in/andrew-gordon-03879919a/ X: @andrew_j_gordonBluesky: @andrewgordon.bsky.social Dr. David Rothschild is an Economist at Microsoft Research and Co-chair of AAPOR's Task Force on Responsible AI Integration in Survey Research. Website: https://researchdmr.com/ LinkedIn: https://www.linkedin.com/in/davmicrot/ Bluesky: @davmicrot.bsky.social References: Responsible AI Integration in Survey Research, AAPOR https://aapor.org/announcements/task-force-on-responsible-ai-integration-in-survey-research-report/  AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers https://osf.io/preprints/psyarxiv/pvdjr The Signal is Prolific's podcast on research evidence - not tools, not trends. Real findings from the field, peer-level conversation, no vendor pitch. Subscribe for new episodes monthly.

About

The Signal is Prolific's podcast on research evidence - not tools, not trends, but findings from the field about how data collection actually works, and what that means for anyone whose decisions depend on it. Hosted by Prolific's Research Sciences team alongside academic and industry guests, each episode is a peer-level conversation on data quality, panel methodology, reproducibility, and the questions researchers actually debate about, grounded in specific findings.

You Might Also Like