Volts podcast: how the left can suck less at messaging, with Anat Shenker-Osorio

In this episode, messaging expert Anat Shenker-Osorio — a researcher, campaigner, author, and speaker — discusses the elements of an effective message, what’s required to spread messages, and the right way to test whether they’re working. We also get into the best way to craft climate messages and the current debate over “popularism.”

Full transcript of Volts podcast featuring Anat Shenker-Osorio, December 20, 2021

(PDF version)

David Roberts:

People involved with politics are obsessed with messaging: what to say, and how to say it, to sway voters or politicians to their side.

Everyone has strong opinions about messaging, but almost everyone’s opinions are drawn from their personal experiences, preferences, and priors, which are rarely reliable guides to what works in practice.

There are, however, people down in the trenches doing real message testing in the field, as part of real grassroots campaigns, like Anat Shenker-Osorio, head of ASO Communications and author of the book Don't Buy It: The Trouble with Talking Nonsense about the Economy. She helps campaigns communicate for a living, and she discusses the lessons learned from successful campaigns on her podcast Words to Win By.

Shenker-Osorio is a co-founder of the Race-Class Narrative project, which is developing a coherent response to America’s familiar racial dog-whistle politics. She has advised several environmental campaigns and done a lot of thinking about the right way to message around climate change, as well as its place in the race-class narrative.

As long-time readers know, I have a love-hate relationship with the subject of messaging, so I’m happy to dig in with Anat to figure out what we really know about good and bad message testing, the elements of a good message, how to actually get messages to voters, and how to talk about climate change in a compelling way.

Without further ado, welcome, Anat, to Volts. Thanks for coming on.

Anat Shenker-Osorio:  

Thanks for having me.

David Roberts:   

I'm excited to talk about messaging. I want to start with a distinction. The side of messaging that people think about most often is word selection: choosing your words, slogans, catchphrases, and verbiage for your ads. But the other side of messaging is about the infrastructure that allows you to get the messages you've developed to voters: the spokespeople, institutions, media outlets, social media pages, civic groups, all the mechanisms that allow those messages to reach their intended audience. 

It's always seemed to me that it is on this latter side of messaging where the left is really getting its ass kicked. It seems like the right has a robust ecosystem that's very coordinated and capable. If they have a new message — you know, “Critical race theory is taking over schools” — they can get that to the ears and eyes of every single conservative in the country basically at will. The left, it seems to me, lacks that ability. What does it need to do to build that kind of infrastructure? 

Anat Shenker-Osorio:  

There are so many ways into this question.

First, of course, I agree with you. That's something that I have remarked upon myself, frequently: A message nobody hears is, by definition, not persuasive. It doesn't matter how fancy your survey or RCT or field test, everything that you did to create that thing: If nobody hears it, it didn't persuade them. 

I think it is too simple a distinction to put those things in two buckets, and here's why. Part of the problem we have is, if your base won't carry the message, then the middle isn't going to hear it.

Yes, it would be amazing to have an actual functional media that would properly do its job. Yes, it would be amazing to have a left-wing specialized media infrastructure of the size and capability of Fox News and OAN and conservative talk radio and all the rest of it. Yes, those would all be great things to have, and we would be much, much better off.

But we do have the knowledge that a message is like a baton that needs to be passed from person to person to person, and if it gets dropped anywhere along the way, it is, by definition, not persuasive. 

Why was it possible for the left to spread the message “love is love” and “love makes a family” and with it shift culture, shift perception of gay and lesbian unions (what used to be called gay marriage and is now properly called marriage equality)? Why was it possible in city after city and then state after state to spread a message of Fight for $15?

Why was it possible in the post-election for us to create content, with a crackerjack team of designers and artists, that said Count Every Vote? Those memes were viewed more than a billion times, and that's just a domestic US audience. There are times when we have broken a signal through the noise, despite all of the disadvantages that you point to — and those have been the times when we have properly attended to that wording question.

So again, I don't disagree with your diagnosis, I just think that the way that we resolve this issue actually has to do with the messages that we're putting out, at least partially.

David Roberts:   

Let's talk about how we figure those messages out, then. Another one of my longstanding beefs with the endless messaging talk that I hear — and I'm mostly coming from a climate perspective — is: I frequently read studies and survey groups telling me how people react to messages when they see them in isolation, one at a time, in the calm of a focus group, or assembled by an academic. Then they take the different ways that people react to these messages in that context and vastly over-interpret them regarding what kind of messages work out in the world. 

It's pointing out the obvious, but the way people encounter messages in the wild bears no resemblance to that whatsoever. When people encounter messages in the wild, it's in the midst of the noise and chaos of our modern information system. They get partial messages, and the messages are surrounded, often, by counter-messages from the other side. So the way people encounter and absorb messages in the real world seems to me so distant and different from the way these focus groups are done that there's just not a lot to learn from the latter about the former. 

So how do you messaging experts, or testers, figure out how a message will perform not just in isolation, but in the scrum of an actual political fight in the actual world?

Anat Shenker-Osorio:  

I feel like you are an audience plant for me, raising up all of my core beefs and things I yell and scream and write and tweet and bang my head against the wall about.

Yes, you're absolutely right. In-channel testing is any kind of empirical test where you are providing stimuli to the respondents and asking for their feedback about that stimuli in the same moment at which they are receiving it. So that's a telephone survey, that’s an online dial test; even more sophisticated processes like using a randomized control trial (RCT) and not a sequential survey is still in-channel testing. Same with focus groups. 

First of all, you are literally paying them for their attention. That is what you are doing: providing them a financial incentive to listen to your thing, watch your thing, and tell you about your thing. You have their undivided attention — or at least you have their somewhat undivided attention, because remember, a lot of this testing is happening digitally, which means that, like in the way when people are on Zoom calls, they also have seven tabs open. So you're still getting some level of distraction, because people are not just listening to you. The same goes for when they're taking a phone survey; they're also making dinner and yelling at the kids or whatever's going on. 

But yes, it is what you say. So how do we deal with that?

We understand that each tool is useful for its purpose and not for another. Things like in-channel testing — qualitative and, more importantly, quantitative — can be used in order to understand whether one frame is more effective than another, or whether one frame is more comprehensible, logical, clear than another.

What it can't be used to do is determine effect size. You can't see an effect size in an in-channel test, say, “This moved people 8 percentage points,” and believe that that's actually what's going to happen in the field. That's not true, for the reasons that you say.

Number two, you can design those tests to be closer to the real world by making them legitimate combat tests, having people in the survey exposed to more opposition messaging than our own — which, of course, is what is happening in the real world — and testing our messages against what the other side is saying. This is one of my 5,700 beefs with a lot of academic research, that they do this test-tube experiment where they don't expose folks to opposition messages. 

Next thing you can do, you can be a lot smarter about what you are rating the message to do. That's when we're doing message testing, which, by the way, is not what's happening most of the time. What's happening most of the time is that people are doing polling; they are doing research to take the temperature, not doin