Testers Shine on Agile Projects
Testers Shine on Agile Projects
Agile projects draw testers out of the background and into the spotlight. Testers can play a distinctive role and drive product development by creating acceptance tests before any code is even written. In this column, Johanna Rothman sets the stage and explains the benefits of giving testers their chance to shine.
A Project Manager Described his Recent Agile Project in this Way:
Agile practices helped us know where we were the whole way through the project, but we had a side benefit I hadn't anticipated. The testers drove the project. In every other project I've managed, the testers were downtrodden, always complaining about the time they had left to test. But here, the testers defined the acceptance tests and helped the customer define her requirements. A couple of times the testers even explained risks to the customer. In one case, the testers told the customer that postponing a specific user story was tantamount to not putting the user story in this release.
I wasn't surprised. When I've seen testers fully participate on any project, they shine. Agile projects push testers into leadership roles. Projects that don't require tests before the code exists can easily push the testers into a background role. If you've ever been on a project where the developers didn't quite meet their deadlines but the schedule didn't slip to meet the testers' needs, you were on a project where the testers were pushed into the background. On those projects, it doesn't really matter what the testers provide for information; there isn't enough time in the schedule for anyone to hear or act on the information anyway.
Contrast those projects with agile projects. On an ideal agile project, the project team discusses the user stories to roughly estimate the time it will take to implement each one. The customer ranks the user stories (1,2,3,4, and so on) to determine which user stories the project team will approach in this iteration. (If you don't have access to real users, use a surrogate, such as a product manager.) Then, before a single line of code is written, the testers write acceptance tests with the customer.
The developers can discuss how they might implement the user story, they may even prototype to discover any of the system-level "gotchas" before implementation, but they can't write any production code until the acceptance tests are complete. Testers write system level acceptance tests to drive the product development. The developers implement only enough code to pass the tests.
Some people get stuck here. "What do you mean we write tests before the product exists? How the heck do we do that?" One technique is to think of concrete examples of how the system will work. Say you're testing an elevator for a four-story building. The user story says the elevator comes when the user presses the button. Rather than "elevator response time shall be such-and-so," you think of one or several worst cases for response time (everyone presses a button with the elevator in a particular position).
Then define the response time for that worst case scenario. This scenario leads to questions about other scenarios, and therefore more tests. Those tests are easy to write before the implementation, and they tend to get written in essential terminology, which clarifies the business domain in people's minds [see Marick's blog]. The tests help the developers start on their design, so that they create a product that meets your specifications.
One major side effect of this write-the-test-first doctrine is that the tests are as easy to automate and maintain as the code. The project staff and the customer start discussing the domain model in the code. The testers (with the customer) write the tests in domain terms, the developers write the code, draw mockups as GUI specifications, write the GUI code to the mockups, then test the GUIs manually. (Some people do develop GUIs test-first, though, instead of mockup-GUI-manual.) Because the domain model is the common terminology, testers can write automation from the command level or the published API. In any case, the testers define the contract between the product code and the test code, not the norm in traditional projects.
Another side effect is that performance, reliability, and other system attributes (what many people call non-functional requirements) can be more easily defined with a user story at the beginning of the project. The testers and the developers can both keep measuring those system attributes, making sure new code doesn't slow down the system or make the system less reliable. If you've ever been on a project that tried to improve performance or reliability late in the project, you know how difficult late improvements are.
To fully participate in agile projects, the testers need to be able to translate user stories into acceptance tests. That may mean that the testers need help writing automated tests, or that the testers write the automation themselves. If you can't write code or develop automated tests in some fashion, it's worth your time to invest in training so that you can be a fully participating member of an agile project. Even if your project doesn't become fully agile, the idea of ranking requirements and working on the most important ones first, of always having a fully functioning system, is too attractive to ignore.
If you're a tester, review your testing skills. Make sure you know how to test in a variety of ways, so that you can test requirements, performance, reliability, security, and any other system attributes a user story might contain. Think about how you could create automated tests that help the developers know when they have implemented only what they need for the user stories.
As you're reviewing your skills, take a look at your communication skills. Agile practices don't usually demand a lot of written documentation, although your adaptation of agile might. If you communicate verbally, make sure your verbal skills are up to the potential debates with customers and developers. When you're working on an agile project, you're all working for the good of the eventual user, not for any one group.
Agile projects require testers who have adaptable testing skills and excellent communication skills. Those who do will shine on…
Acknowledgements
I thank Dwayne Phillips and Brian Marick for their review.
Reference
Brian Marick's blog



Comments
#1 Submitted by AFrazier on Tue, 11/18/2003 - 9:13am.
Hi Johanna,I really enjoyed your article. It helped me to understand what Agile Testing means. My question is in relation to analysis. After reading your article it sounds like with Agile Testing the only analysis takes place between the Tester and Customer. Is there any other resources involved with this (other than possibly the developer to let them know if there is a system limitation)?Also if you try to reduce the documentation on a system by using the Agile Testing method, do you feel that over time, as resources move from system to system, you will also lose knowledge of important processing that may be taking place?
#2 Submitted by Joshua Barnes on Mon, 11/17/2003 - 11:59am.
Johanna,I would like to inquire about the sentence "The customer ranks the user stories (1,2,3,4, and so on) to determine which user stories the project team will approach in this iteration". Customer priority is a huge part of planning an iteration, but I feel that it is an input to the prioritization of the use cases (or stories in agile speak) that the software architect does. Meeting the customer's needs is what we are all here for, however, the customer generally knows very little (nothing) about architecture (as it should be) which drives much of software development risk. Were you working from more of a construction stand point in that an executable that met a subset of the requirements and that proved the architecture could support the requirements (Elaboration milestone) had already been reached and the project was in construction building the system?
#3 Submitted by Lisa Crispin on Thu, 11/20/2003 - 1:47pm.
Hi Johanna, great column! This is a great explanation of the role testers play on agile teams.I would add that testers who don't have a programming or test automation background shouldn't be discouraged from joining an agile team. I have talked to several people who made valuable contributions as testers on XP and other agile teams, but the programmers on the team did all the test automation. Working on an agile team that uses the pair programming practice is a great way to develop test automation skills. There are many other ways testers add value, though.Even if the tester is skilled at automation, she's likely to need support from programmers. If the team is using a tool such as Fit or FitNesse, the programmers may need to write test fixtures. My experience has been that on a sizeable project, acceptance test automation is too much for one or two testers. A benefit of the whole team taking responsibility for automating the acceptance tests is that it makes the programmers even more conscious of designing for testability!Thanks again for a super column. -- Lisa
#4 Submitted by Anko Tijman on Thu, 11/20/2003 - 11:36am.
Thank you Johanna for this supporting article! Next month I will be presenting at Eurostar, introducing testers to agile software development, such as Extreme Programming, Scrum and Crystal. It is very nice and good to hear that so many of the well known "testguru's" embrace agile development. I think it's a major breakthrough in software development for delivering quality software on time to the customer. Gone are the techies that make decisions for the customer, it's all about business value. Thanks again! Anko Tijman
#5 Submitted by Denis Yurkin on Fri, 05/14/2004 - 4:32am.
"If you've ever been on a project that tried to improve performance or reliability late in the project, you know how difficult late improvements are." Johanna, do you mean agile projects here? How does this correspond to these two?http://c2.com/cgi/wiki?MakeItWorkMakeItRightMakeItFasthttp://c2.com/cgi/wiki?RulesOfOptimization
#6 Submitted by Gunnar Skogsholm on Wed, 10/20/2010 - 10:25am.
I think the essential attribute of so called "agile" or extreme programming is much less planning. Conversely, the essential of so called "waterfall" is more planning. Based on these premises, waterfall is superior.What you describe as a so called "agile" process includes significant discussion with the customer as to what the product should do before coding begins. This is actually waterfall by another name. This is reinventing traditional software engineering, and simply using a new name. It's not true that traditional SWE de-emphasizes testers and customer interaction. The way most people actually practice "agile" development ends up de-emphasizing testing and planning.
#7 Submitted by Don Mills on Wed, 04/06/2005 - 3:36am.
Good article, Johanna, and I have only one quibble (which has gotta be good for a tester). You wrote, "On an ideal agile project ...", which suggests that not all agile projects are "ideal". Converely, I could write, "On an ideal waterfall project ...". The majority of my major customers (government, banks, finance houses, etc.) are still committed to waterfall for core systems. Some of them, at least partly due to my personal influence in consulting and training, now use the V- (or W-) model, practising what I've called for twelve years now, "test-first development". In "TFD" projects, accurate and unambiguous business requirements are pinned down very early, product design and construction proceed much faster than normal, test execution finds very few defects, and the delivered software typically runs for weeks or months without experiencing a failure. This has been well-established but inadequately known for decades -- well before the invention of agile methods.Agile development and agile testing are important "schools" of software development, but they don't yet represent "main-stream" development, as far as I can see. Project managers need to understand the importance of TFD in any environment, not just "agile".But undoubtedly, testers faced with agile development environments need the information in your article. Well done!