Social Science Bites SAGE Publishing

Do policies built around social and behavioral science research actually work? That’s a big, and contentious, question. It’s also almost an existential question for the disciplines involved. It’s also a question that Megan Stevenson, a professor of law and of economics at the University of Virginia School of Law, grapples with as she explores how well randomized control trials can predict the real-world efficacy of interventions in criminal justice. What she’s found so far in that particular niche has echoed across the research establishment.
As she writes in the abstract of an article she saw published in the Boston University Law Review:
"This Essay is built around a central empirical claim: that most reforms and interventions in the criminal legal space are shown to have little lasting effect when evaluated with gold standard methods. While this might be disappointing from the perspective of someone hoping to learn what levers to pull to achieve change, I argue that this teaches us something valuable about the structure of the social world. When it comes to the type of limited-scope interventions that lend themselves to high-quality evaluation, social change is hard to engineer. Stabilizing forces push people back towards the path they would have been on absent the intervention. Cascades—small interventions that lead to large and lasting changes—are rare. And causal processes are complex and context-dependent, meaning that a success achieved in one setting may not port well to another."

In this Social Science Bites podcast, Stevenson tells interviewer David Edmonds that “the paper is not saying ‘nothing works ever.’ It’s saying nothing works among this subset of interventions, and interventions, as we talked about, are the type of interventions that get studied by randomized control trials tend to be pretty limited in scope. You can randomly allocate money, but you can’t randomly allocate class or socioeconomic status.”
Despite this cautionary finding in her research. Stevenson hasn’t despaired about her career choice or that of other social and behavioral scientists. “Many of us are in this line of work because we care about the world,” she notes. “We want to make the world a better place. We want to think about the best way to do it. And this is valuable information along that path. It’s valuable information in that it shuts some doors. … So keep trying other doors, keep experimenting.”

Here's a thought experiment: You want to spend a reasonably large sum of money providing assistance to a group of people with limited means. There's a lot of ways you might do that with a lot of strings and safeguards involved, but what about just giving them money -- "get cash directly into the hands of the poor in the cheapest, most efficient way possible." You and I might prefer that, since we, of course, are reputable people and good stewards and understand our own particular needs. But what about, well, others?

Economist Tavneet Suri has done more than just think about that; her fieldwork includes handing out money across villages in two rural areas in Kenya to see what happens. Her experiments include giving out a lump sum of cash and also spreading out that same amount over time. The results she details for host David Edmonds in this Social Science Bites podcast are, to be frank, heartening, although the mechanisms of disbursement definitely affect the outcomes.

Despite the good news, the idea of a universal basic income is by no means a settled remedy for helping the poor. For one thing, Suri says, "it's super, super expensive. It’s really expensive. And so, the question is, “Is that expense worth it?” And to understand that I think we need a few more years of understanding the benefits, understanding what people do with the incomes, understanding whether this can really kickstart these households out of poverty."

And perhaps the biggest question is whether the results of fieldwork in Kenya is generalizable. "I would love to do a study that replicates this in the West," she says. "The one thing about the West that I think is worth saying that's different is you wouldn't add it on top of existing programs. The idea is you would substitute existing programs with this. And that to me is the question: if you substituted it, what would happen?"

Suri is the Louis E. Seley Professor of Applied Economics and at the Massachusetts Institute of Technology's Sloan School of Management. She is an editor at the Review of Economics and Statistics; co-chair of the Agricultural Technology Adoption Initiative at the Abdul Latif Jameel Poverty Action Lab, known as J-PAL, at MIT; co-chair of the Digital Identification and Finance Initiative at J-PAL Africa; a member of the executive committee at J-PAL; and a faculty research fellow at the National Bureau of Economic Research.

How hard do we fight against information that runs counter to what we already think? While quantifying that may be difficult, Alex Edmans notes that the part of the brain that activates when something contradictory is encountered in the amygdala - “that is the fight-or-flight part of the brain, which lights up when you are attacked by a tiger. This is why confirmation can be so strong, it's so hardwired within us, we see evidence we don't like as being like attacked by a tiger.” 

In this Social Science Bites podcast, Edmans, a professor of finance at London Business School and author of the just-released May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – And What We Can Do About It, reviews the persistence of confirmation bias -- even among professors of finance. 

“So, what is confirmation bias?” he asks host David Edmonds. “This is the temptation to accept something uncritically because we'd like it to be true. On the flip side, to reject a study, even if it's really careful, because we don't like the conclusions.” 

Edmans made his professional name studying social responsibility in corporations; his 2020 book Grow the Pie: How Great Companies Deliver Both Purpose and Profit was a Financial Times Book of the Year. Yet he himself encountered the temptation to both quickly embrace findings, even flimsy ones, that support our thesis and to reject or even tear apart research, even robust results, that doesn’t. 

While that might seem like an obviously critical thinking pitfall, surely knowing that it’s likely makes it easier to avoid. You might think so, but not necessarily. “So smart people can find things to nitpick with, even if the study is completely watertight,” Edmans details. “But then the same critical thinking facilities are suddenly switched off when they see something they like. So intelligence is, unfortunately, something deployed only selectively.” 

Meanwhile, he views the glut of information and the accompanying glut of polarization as only making confirmation bias more prevalent, and not less. 

Edmans, a fellow of the Academy of Social Sciences and former Fulbright Scholar, was previously a tenured professor at the Wharton Business School and an investment banker at Morgan Stanley. He has spoken to policymakers at the World Economic Forum and UK Parliament, and given the TED talk “What to Trust in a Post-Truth World." He was named Professor of the Year by Poets & Quants in 2021.  

Consider some of the conflicts bubbling or boiling in the world today, and then plot where education – both schooling and less formal means of learning – fits in. Is it a victim, suffering from the conflict or perhaps a target of violence or repression? Maybe you see it as complicit in the violence, a perpetrator, so to speak. Or perhaps you see it as a liberator, offering a way out a system that is unjust in your opinion. Or just maybe, its role is as a peacebuilder.

Those scenarios are the framework in which Tejendra Pherali, a professor of education, conflict and peace at University College London, researches the intersection of education and conflict. In this Social Science Bites podcast, Pherali discusses the various roles education takes in a world of violence.

“We tend to think about education as teaching and learning in mathematics and so forth,” he tells interviewer David Edmonds. “But numeracy and literacy are always about something, so when we talk about the content, then we begin to talk about power, who decides what content is relevant and important, and for what purpose?”

Pherali walks us through various cases outlining the above from locales as varied as Gaza, Northern Ireland and his native Nepal, and while seeing education as a perpetrator might seem a sad job, his overall work endorses the value and need for education in peace and in war.

He closes with a nod to the real heroes of education in these scenarios.

“No matter where you go to, teachers are the most inspirational actors in educational systems. Yet, when we talk about education in conflict and crisis, teachers are not prioritized. Their issues, their lack of incentives, their lack of career progression, their stability in their lives, all of those issues do not feature as the important priorities in these programs. This is my conviction that if we really want to mitigate the adverse effects of conflict and crisis on education of millions of children, we need to invest in teachers.”

A fellow of the Academy of Social Sciences and of the Higher Education Academy, he is a co-research director of Education Research in Conflict and Crisis and chair of the British Association for International and Comparative Education.