Automated Testing, Everything That Can Be Automated, Should Be Automated Agile Weekly Podcast

Jade Meskill: Hello. Welcome to another episode of the Agile Weekly Podcast. I’m Jade Meskill.


Roy vandeWater: I’m Roy vandeWater.


Clayton Lengel‑Zigich: I’m Clayton Lengel‑Zigich.


Derek Neighbors: And I’m Derek Neighbors.


[laughter]


Automated Testing Struggling to Go Fast

Jade: Very nice. Derek, you had something you wanted to talk about. What was it?


Derek: Yeah. I’ve been doing the work with a number of teams, and one thing that I’m starting to see emerge as a pattern is the team’s struggle to go fast. They struggle to work particularly within the Scrum framework a lot of times. Not only are they siloed where they got developers and QA that are separate, but where they don’t have a lot of automation. This could be automation around deployment.


This could be automation around testing, so maybe they don’t have an automated testing suite. Or if they’ve got some automated testing suite, there’s still some manual regression happening. Even if they have a fully automated testing suite, they’re not running it automatically.


They’re not having continuous integration, or it takes an enormous amount of long time to build. What I’m finding is it’s very, very hard for them to see what performance looks like, because they can’t get over the concept of their current reality. The current reality is, “Hey, it takes five days to run the test suite. There’s no way we can have a one‑week iteration. That’s just impossible. It takes us one week just to test, once the developer’s done with a feature.”


Have you guys seen instances, where maybe the current reality or the lack of automation makes it so teams just find…I had somebody specifically today, tell me, “That’s really nice that you stand up there and talk about a 10 minute build. Frankly, there’s no way you’ve ever done that before, because I’ve worked for five different companies, and none of them have ever been able to do that. I find what you’re saying impossible to believe.”


Jade: [inaudible 02:06] we have a 10 second build?


Roy: Yeah. I agree. I’ve struggled so hard to get a build to take 10 minutes. I don’t believe it’s possible.


Derek: In their current reality, I understand that. If you’re a manual regression tester, and you’ve got a fairly complicated suite, it’s taking two or three days to run a single test plan. I can understand how it feels impossible that you could, with any level of comfort, run a test suite under 10 minutes that made you feel like you weren’t shipping crap.


Roy: We’re working with the team right now, but we’re not working with working microphones.


[laughter]

Letting Technology Get In The Way

Jade: That’s right. We are working with the team right now that a single test takes two weeks of constant computation, a single test.


Clayton: I’ve seen that in a lot of instances where, the technology gets in the way. If you’ve got a java applet that loads a Swf, how do you automate testing of that content? That seems like this impossible thing. I think a lot of people say, “It is what it is, throw my hands up in the air. What else are we going to do but, let those hourly regression people slave away at typing in commands and looking at the screen”?


Roy: Along the idea of trying to test a flash out, I think that’s one of the first mistakes that immediately causes your test suite to get larger, and take longer, regardless of whether it’s manual regression or automated testing. That’s when you write the test case last, after everything is done.


Jade: Yeah.


Test Driven Development Helps Reduce Some Problems

Derek: When you write the test case first, you end up using solutions that are easier to test. You don’t run into those things.


Clayton: That’s one of my favorite things about doing, not even just TDD, but just test first. Where you have to take into account what easy to test and what isn’t. I’m working with