Agile Testing: Methods and Models

The presentation covers several established agile based testing methods and models that testers may be able to use to further enhance their testing approach.

1. Agile Testing: Methods & Models Rad Butalid , ICP-ACC, CKC May 31, 2019

2. I BM Recognized Agile Thought Leader and Agile Champion International Consortium for Agile - Agile Certified Coach (ICP-ACC) International Business and Quality Management Institute – Certified Kanban Coach (CKC) Project Management Institute – “Incoming” Agile Certified Professional (PMI-ACP) Hello!

3. Agenda What is Agile Testing? Testing Methods and Models What’s next for Agile Testing? Q&A

4. What is Agile Testing? 4

5. “ Agile development has always centered around testing and quality. That characteristic drew us to agile: Test first, test all the time, build in quality.  ” – Lisa Crispin & Janet Gregory

6. 6 What is Agile Testing? Agile testing is a software testing practice that follows the principles of agile software development . Agile testing not only facilitates the early detection of defects but also  reduces the cost of bugs  by fixing them early. This approach also yields a customer-centric approach by delivering a high-quality product as early as possible. Both development and testing activities are concurrent unlike the Waterfall model.

7. Agile Testing Principles Test Driven: Test First vs. Test Last Continuous Testing = Continuous Progress Tests performed by the whole team Continuous feedback Decrease time of feedback response Bug fix within iteration Lightweight documentation 7

8. Advantages of Agile Testing It saves time and money Agile testing reduces documentation It is flexible and highly adaptable to changes It provides an easier way for receiving regular feedback from the end user Better determination of issues through daily standups/meetings 8

9. 9

10. Agile Testing: Methods and Models 10

11. Agile Test Plan The scope of the testing Consolidating new functionalities to be tested Types of testing/Levels of testing Performance & load testing In agile testing, the test plan is written as well as updated for every release. A test plan in agile includes: 11 Consideration of infrastructure Risks Plan Planning of resources Deliverables & Milestones

12. Agile Testing Lifecycle The agile testing lifecycle includes the following 5 phases: Impact assessment Agile Testing Planning Release Readiness Daily Scrums Test Agility Review 12

13. 15 Behavior Driven Development (BDD)

14. Behavior Driven Development (BDD) 16 Who is it for?  The BDD methodology is ideal for teams that work on feature-focused software and/or teams that put the user experience first. BDD starts with a functional specification using the Gherkin Given/When/Then syntax. This specification then guides developers, testers and product owners who move across features. As they do, they use automated test functions to determine completeness, refining code until it passes the test, much like in the TDD approach except at the team level.

15. How is it different from standard Waterfall testing? 17 Waterfall testing requires test cases to be written early against requirements and calls for those tests to be executed toward the end of the development cycle. With BDD, tests are not based on requirements (but on “end user behavior”) and testing happens in conjunction with development of the features. Additionally, within a Waterfall methodology, testers are the ones writing the test cases. A BDD approach, on the other hand, lends itself to the business owners writing the tests. This switch reduces communication (or miscommunication) between the business analysts, developers and testers.

16. What does adoption entail? 18 Changing to a BDD methodology can be challenging when the team is used to a traditional style of testing. It requires a BA to write tests up front and for the developers to write the test specification in code to match. It’s a new type of coordination within the team, but it’s extremely positive in that the team works together as one unit, business users included.

17. Best Practices 19 Streamlining documentation in order to keep the entire process lean Embracing a “three amigos” model where the product owner, developer and tester form a cohesive team Using a test framework like Cucumber to define criteria Building automated tests in a way that make them easy to reuse as much as possible Having business analysts learn the Gherkin syntax and write test cases directly

18. 20 Acceptance Test Driven Development (ATDD)

19. 21 Who is it for?  Due to its emphasis on user perception, it’s best suited for teams that are focused on the user experience, have goals around high rates of adoption and want to minimize the number of feature changes in future releases.  The idea behind ATDD is that user perception of the product is just as important as functionality, and should drive product performance in order to help increase adoption. ATDD collects and uses input from customers to develop acceptance criteria, translates that criteria into manual or automated acceptance tests and then develops code against those tests.  Acceptance Test Driven Development (ATDD)

20. 22 ATDD is different because it is a test-first methodology. Standard Waterfall testing calls for test cases to be written upfront based on requirements, whereas ATDD is not a requirements driven testing process.  How is it different from standard Waterfall testing?

21. 23 Because ATDD represents such a departure from traditional methods, going from one to the other isn’t easy for teams to do. To be in the best position to adopt an ATDD methodology, teams need to get stakeholder buy-in, which can prove challenging at times. What does adoption entail?

22. 24 Interacting closely with customers, for example through focus groups, in order to determine expectations Leaning on customer-facing team members, such as sales representative, customer service agents and account managers, to understand customer expectations Developing acceptance criteria based on customer expectations Prioritizing two questions: Will customers use the system if it does X? How can we validate if the system does X? Best Practices

23. Exploratory Testing

24. 26 Who is it for?  Best suited for teams that are under time constraints, teams that need help identifying the best types of tests to run (especially in cases where there are no specifications from developers) and those who want to make sure they didn’t miss anything from previous tests. Exploratory testing can help reduce time spent testing, find more defects and improve code coverage. Because of its unscripted approach, exploratory testing often mimics how users will interact with the software in real life. Exploratory Testing

25. 27 Exploratory testing can actually be done in both Waterfall and Agile environments, but the tight integration between testers and developers in an Agile environment helps ease any bottlenecks that might surface while running exploratory testing in a Waterfall environment. How is it different from standard Waterfall testing?

26. 28 Embracing exploratory testing is relatively easy, as it is quick to launch (and scale), simple to learn and provides benefits for the entire team. That said, it’s important to keep in mind that it should not be the only form of testing (rather, it should inform what type of testing happens next). Additionally, even though it’s unscripted, exploratory testing should not be unstructured (testers still need to set a goal, log your activities and take on the mindset of a specific user persona). What does adoption entail?

27. 29 Organizing the functionality in the application using something like a Mindmap or a spreadsheet Focusing on certain areas or certain scenarios Tracking what gets tested to help reproduce any errors Documenting results for accountability on what got tested Best Practices

28. Session Based Testing

29. 31 Session based testing builds on exploratory testing by providing more structure. It provides this structure by conducting testing during time-boxed, uninterrupted sessions, testing against a charter and requiring testers to report on the testing that took place during each session. Additionally, session based testing should be capped off with a “debrief” between the tester(s) and the manager that covers the five PROOF points: What happened (Past), what was achieved (Results), what got in the way (Obstacles), what still needs to be done (Outlook) and how does the tester feel about it (Feelings). Session Based Testing

30. 32 The same as exploratory testing, session based testing can be run in both Agile and Waterfall environments, but it is more conducive to the tight collaboration between testers and developers that is typically found in Agile environments. How is it different from standard Waterfall testing?

31. 33 Much like with exploratory testing, adopting session based testing proves relatively easy because it is easy to pick up and launch quickly. For testers already accustomed to exploratory testing, the biggest hurdle is embracing the additional structure for which session based testing calls. Also like with exploratory testing, teams running session based testing should remember that it is not a final stop, but rather a method to help determine the best type of testing to conduct next. What does adoption entail?

32. 34 Outlining a mission so that testers are clear about the software they’re testing Developing a clear charter that indicates the mission, the areas of the software to test, which tester(s) will run the session, when the session will take place and for how long, the tests designed and executed, the bugs found and overall notes. Running the testing session without any interruptions Clearly documenting activities and notes during the session in a session report Holding a debrief between the tester(s) and the manager to review findings from the session and discuss next steps for testing Best Practices

33. Marick Test Matrix 35 Back in 2003, Brian Marick published an influential series of articles on agile testing. He was attempting to point the way forward for agile testers. But, in the process, he came up with an elegant method of cataloguing testing methods that has become known as the “ Marick Test Matrix.” Marick’s work categorized tests by asking two questions: Is the test business facing or technology facing? Does the test support programming or critique a product?

34. 37 The Agile test (automation) pyramid was introduced by Mike Cohn in his book ” Succeeding with Agile”, back in 2009 .  The model splits types of testing into three layers based on the return on investment (ROI) offered by automating that particular type: Unit tests involve testing small units of code. They are the least expensive to write and maintain, and they provide value to the team multiple times per day. Integration or Acceptance tests, provide the next greatest benefit with User Interface testing having the least. The Agile Test (Automation) Pyramid

35. 39 DevOps shift-left testing is an approach to software and system testing in which testing is performed earlier in the software lifecycle, with the goal of increasing quality, shortening long test cycles and reducing the possibility of software defects making their way into production code. One way that DevOps teams do Shift Left testing is by shifting integration testing to the left of its usual position in the delivery pipeline so that it occurs as close as possible to the build process.  This allows teams to receive feedback on code quality faster, with more accurate results. Shift Left Testing

36. What’s next for Agile Testing? 41

37. 42 In the future, three key tenets will become particularly important for testers working in Agile environments: 1) Communication Agile requires a tight collaboration between testers and developers, and that collaboration makes communication a top priority for testers. Additionally, in a world where quality becomes everyone’s responsibility, testers will become “quality champions” that serve as internal experts, which will put their ability to clearly communicate testing needs and reasoning under the spotlight. 2) Skill Diversity In an Agile environment, everything can change on a dime, and that requires testers to be adaptable. Part of this adaptability is having a diverse skillset so that testers can change course as needed. For instance, functional testers need to expand their skills beyond manual scripted execution. This diverse skillset will be a must as different sprints require different types of testing to be executed in a short timeframe. 3) Business Mindset Finally, Agile takes on a very customer-centric approach in order to ensure customers receive as much value as possible as quickly and as early on as possible. Testers have a big role to play in delivering this value, but it requires them to take on a business mindset so that they can understand customer expectations, desires and concerns and develop their testing strategies accordingly.

38. 43 Although Agile has already made significant inroads into the software development lifecycle, there’s still a long way to go, especially among testing teams.. Going forward, more widespread adoption and greater maturity of Agile methodologies will require testers to go beyond test creation and execution and begin to focus on code delivery and integration too. At the same time, testers will need to hone their automation skills, become more involved in the entire software development process and continue to develop a collaborative relationship with developers. Ultimately, these changes will also require testers to become experts at development and product usage in order to provide more holistic testing strategies and take on the role of “quality champions.”

39. “Lots of methodologies have come and gone, paradigms have changed but the requirements are always the same; Make it good and make it fast.” - Anonymous 44