The Problems with Standardized Tests: Deming in Education with David P. Langford

In this episode, David and Andrew discuss the dreaded standardized tests, including how they evolved, how they're used (and not used) now, and what Deming said about them. David also offers practical tips for educators who want to move away from standardized tests in their classrooms.

TRANSCRIPTION:

Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today, I'm continuing my discussion with David P. Langford, who has devoted his life to applying Dr. Deming's philosophy to education, and he offers us his practical advice for implementation. Today's topic is, what would Dr. Deming say about today's standardized testing? David, take it away.

Langford: So one of the things when I started learning about the Dr. Deming's field is, there's a lot of people that know about manufacturing, a lot of people know about management, etcetera, a lot of people don't, but everybody knows about education, everybody went to school somewhere, someway, homeschooling, charter schools, whatever it might be, so everybody went to school, so everybody's sort of knowledgeable on that. And one of the things that's pretty common through that process is the emphasis over the last 30 years is of standardized test, as the way to sort of measure, not only the performance of a system, or a teacher or a student individual kind of thing, but also the success of how a nation is doing, right? Can we compare the US to Japan and Australia and Thailand? And where do we rank? And who's 14th? And all these kinds of things. So I just wanna go back a little bit about, where did this evolve? Hundreds of years ago, we didn't have standardized tests, and amazingly people survived, people got jobs, became entrepreneurs, they did all kinds of things, and they didn't have a standardized test scores to be able to rank them, to be able to do stuff.

Langford: So one of the reasons these things did evolve, was a quick and easy way that we could rank individuals or rank systems, and that was one of the things that Deming was most opposed to, ranking people, either through performance appraisals, grades, standardized tests, whatever it might be. So a lot of the purpose of why we do it, is a simple way to rank schools and try to understand, "Well, who's number one? And who's not?" And that kind of a thing. And then the second one...

Stotz: And let me add something there just 'cause I wanna make sure that this comes out clear, is that obviously Dr. Deming was against the idea of ranking, but from a sincere perspective of some leaders out there, they may say, "Well, I'm not using it to rank so much as I'm just using it to figure out where is the trouble spots that I should be focusing on."

Langford: Yeah, maybe you're not doing that, but people that are getting your scores are doing that, I can guarantee it. And then now it's gone to parents looking up scores online and comparing schools even across the street to try to see, "Oh wow, that school has higher test scores than this school, so it must be a better school." Deming had a lot of comments about standardized testing, but one of the ones I remember was, he said our education system would be significantly improved if we could get rid of standardized tests and grading as a performance measure. And it goes back to... We keep talking about the purpose of the system and the constancy of purpose of the system, and so is your constancy of purpose to get good test scores? Is that really the only thing you're trying to accomplish in a school? And if that's the case, you're gonna do all kinds of really, very dysfunctional things in that school to get test scores. Like for instance, well, one of the first things we're gonna cut out is music programs and sports programs, right? 'Cause we don't test us for those, and so we don't need those, we can cut that out, and then we spend more time on this.

Langford: And I know the schools that are now mandating 90 minutes of math a day, whether you need it or not, you're gonna get 90 minutes of math a day and just waste and all kinds of stuff, and then it causes all kinds of crazy stuff within the system. And the tests themselves have slowly started to change, and one of the things that, especially that people used to say when you'd say, "Well, we should get rid of standardized tests." People say, "Well, how do we get people into colleges then? 'Cause they won't have an SAT score and they won't be able to get into college." And I think interesting thing is that there are numbers of universities, they're now throwing those things out and just saying... I know that in California, there's universities that do not require SAT scores any longer to go to the university as a measure of how well you're gonna do.

Stotz: And would just abandoning that and say, "Well, college is open for everyone." Would that be the smart thing to do? Or is it to say, "Well, let's create a system that tries to optimize for something else."? Or does that add a huge amount of complexity for a school that doesn't have a lot of resources, let's say?

Langford: Well, you want my method or do you want... Stotz: I'm only here for you.

Langford: So I don't mind that there should be some kind of a base level, you need to have graduated from a high school, you need to have done certain things, and then all the people that meet that certain baseline that wanna go into college or wanna go on to a particular college, it just goes into a lottery. And now, let's say you have a thousand spots open, so you draw out a thousand names, all those people get to come to your university or your college for that year. Now, if you weren't chosen, you know why you weren't chosen.

Stotz: Pretty clear.

Langford: Yeah, it's pretty clear. And you could put your name in for multiple universities, and if you somehow got drawn for several universities, you could make a choice, "Which one do I really wanna go to?" And you would stop all this crazy stuff about people paying to get their kids in certain universities, and some universities are so expensive that only very rich people can get their kids in those universities and just on and on and on.

Stotz: Yeah, but it could be abused. The legislators or the people in the university would say, "Well, the good news about our student body, David, is that they're all lucky."

Langford: Yeah, that's right. But if you had randomly chosen students at your university and you were still churning out some of the top students in the world in these professions, you're really doing something. You've really optimized the system in some way. On the other hand, if all you're doing is choosing the very best people that are probably gonna be successful at any place they go, you can actually get away with a lot of dysfunctional stuff, and these students are still gonna be successful because they're just gonna overcome any barriers that are put in front of them. So there's no incentive to actually optimize a system moving forward with that.

Stotz: I guess unless you're... If your method... What you're optimizing for is "I just want the top, top, top smartest, smartest as measured by score, young people in the world or in my state, or my city, or my country that's it. That's all I'm up to, man. I'm not trying to bring education to everyone," but that's not what Cal State Long Beach was doing, when I went there, they were trying to bring education to everybody.

Langford: Yeah, so it's much different constancy of purpose of what you're trying to do. The other thing about standardized tests, and we all grew up with multiple choice tests, and I remember one time somebody asking Deming at a conference about that and he paused for a long time and he said, "What do you think about multiple choice questions?" And he said, "It's ridiculous." He said, "I would much rather you have a multiple choice questions which students have to answer: in what circumstances would A and B be correct? And in what circumstances would A and D be correct?

And what would be the system if none of these were correct?" Because that would require thinking, and responses, and deep level understanding of something. But this game is still being played, especially in K-12 to get test scores, teachers are taught to teach children to guess. If you don't know the answer, just guess, because if there's four things there, you got 25% chance that you guessed right. Well, this is ridiculous, because all trying just to get your scores up. So if you really wanna find out if somebody knows something or doesn't, you certainly don't want them just guessing, that's not gonna tell you anything. And maybe 25% of the time they guess correctly, and so you're getting scores that's not telling you anything about the system.

Stotz: I was in a senior level class in my final year in university, taught by an amazing woman who had actually lived in pre-Nazi Germany, and she had just seen communism, fascism and democracy in action, and that was the name of the course. And the final exam was a essay exam, and we went in and it was, I don't know, three hours, and when I got that exam back, it said A plus, and I just felt like I just really, really understood the material, I enjoyed writing that exam and I was proud of what I accomplished. And I can say there's no multiple choice exam that I took that I'm like, "I'm really proud of what I did there."

Langford: Yeah, the same thing. We could all look back at experiences that we had, and usually they don't involve Draconian methods of testing, and standardized tests and things like that. One of the best classes I ever took in college, and I still vividly can recall or remember the name of the class was "The Assassination of American Presidents." And we went through every attempted assassination, everything, all the way up through Kennedy, and on thr