Tackling More Management Myths: Deming in Schools Case Study with John Dues (Part 9)

How do we motivate employees? Traditionally, we offer merit pay, focus on accountability, and use other extrinsic motivators tied to performance. The ideas sound good on the surface, but John and Andrew discuss the many pitfalls and unintended consequences - and what to do instead.

0:00:02.4 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. The topic for today is...well, in fact, we are continuing our discussions about management myths that keep fooling us. And today, we're talking about merit pay, accountability, and extrinsic motivators. John, take it away.

0:00:37.7 John Dues: Yeah, thanks for having me back, Andrew. We've sort of been on this sort of mini-series talking about some of these myths that Dr. Deming talked about. So two episodes ago we sort of introduced the idea that Deming said we're living in this sort of age of mythology. We talked about two myths: the myth of best practices, and the myth of the hero educator. And then last time we talked about the myth of performance appraisal, and really talked about sort of this failure to consider the role of the system on individual performance. And really what we're gonna do is kind of wrap up those myths with the three you mentioned today. I think when I think of the myth, I'm thinking about sort of management dos and don'ts, and the myths are the don'ts. And then sort of the idea would be after we cover the myths, we can turn to some guiding principles, and those would be sort of the dos, the things that management leaders should do, that sort of Dr. Deming talked about.

0:01:38.7 JD: So we can dive into the first one, which is sort of a continuation of last time, this idea of rating and ranking. Last time we talked about performance appraisal, and now it's sort of the merit pay side of rating and ranking. And it something...merit pay is a practice that has been sort of tried over time in education. What I can tell, it goes...the idea in education goes back at least to the Reagan administration. So at least to the '80s. So since that time, this sort of merit pay idea for teachers or other educators in the system has been taken up by various governors and presidents in the United States. I think most recently during President Obama's administration, there was the Race to the Top program. And sort of as a part of that program, there were teacher and principal evaluations where merit pay was sort of a key part of those evaluation systems.

0:02:36.1 JD: And so the basic theory is that if you pay people based on results, that motivator to make money will drive improvement of outcomes in schools. And I think one sort of key differentiation, because pay in all sectors, but especially in education, is a hot button topic. I'm not talking about sort of the core salary, whether or not teachers are paid enough or not. Those sort of base salary levels are sort of a separate discussion topic altogether. I'm just talking about sort of merit pay, bonus pay, performance pay, that type of thing. And when I think of merit pay, I mean a lot of these ideas, I think, sort of sound good before you really dig in and start to think about them.

0:03:34.0 AS: And that's, Dr. Deming would say pay for performance makes sense. Seems like you're gonna get a good outcome, but in fact it's a little different or a little bit more complex than that.

0:03:44.9 JD: It's a lot, yeah. A lot more complex, I think that's exactly right. When I was thinking about merit pay and the theory behind it, what are the problems, especially in education? So for me, problem number one is paramount, and that's how do you define a meaningful measure of performance by which to judge individual educators? That's a pretty thorny problem. I think problem two is that the basic theory suggests that additional money will incentivize these...improves teaching and in turn improve student outcomes. But for that to be true, that means that teachers were previously withholding their best efforts and if you just paid them this bonus, that they would then sort of unleash this previously withheld power. And then another really thorny problem is this idea, if you sort of create this environment where you have a merit pay system, it sort of disincentivizes the behaviors that are important to improving any complex system.

0:05:00.2 JD: Things like, cooperation and teamwork. And so, especially in a merit pay system, the ones that I've sort of been aware of, the merit pay calculation is often sort of viewed as sort of opaque. How is this calculated? Often sort of the algorithms are proprietary, they're viewed as unfair, and then they can lead to these undesirable behaviors like unwillingness to share ideas or, just as problematic, unwillingness to take on certain teaching assignments, the tougher assignments. The very kids that you want to have the best teachers are often the toughest to get results with. And so you're sort of disincentivizing people taking those assignments because of the differential pay or the poor rating.

0:05:57.9 AS: Yeah, it's interesting too - that the point that people are withholding their best work. If we just give them merit pay, then...and I just had a vision in my mind. Imagine that we had a peaceful cage of tigers, and they're all chilling out and these tigers consume, at every meal, they consume, let's say all of them consume a hundred kilos of meat. [laughter] And we end up putting in 50 kilos in there and say, "okay, you gotta fight for this."

0:06:44.9 JD: Yeah.

0:06:47.4 AS: What's it gonna look like?

0:06:48.8 JD: What's it gonna look like? Yeah.

0:06:50.2 AS: And in a way, like what we're doing with merit pay is saying there's a limited pool. We've allocated a pool that's available to you. Yes, you've got your survival pay, but here is this pool of additional merit pay. And then...yeah, some people, some of them may just some people may sit back and go, "I'm not doing the work for that. I'll stick with my monthly pay." Whereas others will be extremely competitive to get that pay.

0:07:19.8 JD: Yeah. Yeah. But all of it assumes that, let's say, today I don't have the merit pay system. I get certain results. Tomorrow, I have this merit pay system. It assumes I'd know what to do tomorrow to get the outcomes, right? And I think that's a huge problematic assumption.

0:07:42.7 AS: So is that's part... If you think about what you're saying, that's part of the myth that, you know, you just think...

0:07:47.8 JD: It's part of the the myth. Yeah.

0:07:49.3 AS: It's like "just put this in place." I mean, come on. It's internal competition. We want everybody to...well, wait a minute. Internal competition in the company?

0:08:00.4 JD: Yeah. I mean, I could work, potentially, work harder for some period of time, but if I don't have any different methods for bringing about these outcomes, then the merit pay itself is...assuming everything else went well, assuming it was seen as fair and transparent, the fact that you don't have any new methods with which to bring about this improvement is a serious impediment to just thinking that merit pay in and of itself is gonna be an effective system. Putting aside all the other issues like disincentivizing the very behaviors or taking on the various assignments that you want the sort of top-notch teachers to do.

0:08:36.5 AS: Yeah. I guess that's another way of thinking about it too, is that it's like you have a certain set of tools on the tool floor, on the factory floor or in the classroom or whatever. There's a certain set, and you're not adding to any of that. You're just saying, "we're gonna pay you to get more out of that," and it's just...there's a limit as to what you can get on that. All right.

0:09:01.9 JD: Yeah, and I think the most typical way this has showed up in, say, the last 15 years or so, are these value-added models, that instead of just focusing on the absolute test scores of individual students, what percentage are hitting that proficiency standard? The value-added models did allow you to sort of attempt to measure the progress of individual students. So even if they didn't hit proficiency, you could look at, well, did they grow a lot? And a lot of the sort of merit pay schemes, during the Race to the Top era, were based on these value-added models. But I was...just as an example, I was reading a working paper from a Cal Berkeley economics professor who looked at some of these models. And one of the things that he said really stood out, he said, Teachers gain... "Value-added model scores are evidently inflated or depressed, in part due to the students who they teach, who differ in unobserved ways that are stable over time.

0:10:04.5 JD: This bias accounts for as much as one third of the variation in teachers value-added scores, enough to create a great deal of misclassification in value-added model based evaluations of teacher effectiveness." So I think that that type of finding is exactly the thing that I'm talking about. Creating these models is very, very difficult. And according to at least that one research study, up to a third of the variance in the results of teachers value-added gains for their students was not attributable to the teacher themselves. It was for...to these other things.

0:10:41.5 AS: And so... Go ahead.

0:10:43.7 JD: Oh, go ahead. Yeah.

0:10:45.1 AS: I was just gonna say that one of the takeaways that people take from this type of discussion is, oh, I see, but okay, so merit pay needs to be better implemented.

[laughter]

0:10:56.0 AS: Right? Like, and I had...there was a LinkedIn discussion where someone po