Software Engineering and Project Management - Software Testing + Agile Methodology

Software Testing: A Strategic Approach to Software Testing, Strategic Issues, Test Strategies for Conventional Software, Test Strategies for Object -Oriented Software, Validation Testing, System Testing, The Art of Debugging.
Agile Methodology: Before Agile – Waterfall, Agile Development.

1. Software Engineering & Project Management Module 3 Software Testing: A Strategic Approach to Software Testing, Strategic Issues, Test Strategies for Conventional Software, Test Strategies for Object -Oriented Software, Validation Testing, System Testing, The Art of Debugging. Agile Methodology: Before Agile – Waterfall, Agile Development. Presented By: Dr. Prakhyath Rai

2. Software Testing Software testing is the process of evaluating and verifying that a software application or system meets specified requirements and works as intended. Software testing strategy characteristics, To perform effective testing, you should conduct effective technical - many errors will be eliminated before testing commences. Testing begins at the component level and works “outward” toward the integration of the entire computer-based system. Different testing techniques are appropriate for different software engineering approaches and at different points in time. Testing is conducted by the developer of the software and (for large projects) an independent test group. Testing and debugging are different activities, but debugging must be accommodated in any testing strategy.

3. Software Testing Strategic approach to Software testing, Verification and Validation Organizing for Software Testing – ITG (Independent Test Groups) Software Testing Strategy—The Big Picture Criteria for Completion of Testing

4. Verification and Validation Verification: Ensuring that the software meets the specified requirements during the development process. Validation: Confirming that the software fulfills the intended use and meets the user's needs and expectations. Verification: “Are we building the product right?” Validation: “Are we building the right product?” Verification and Validation includes a wide array of SQA activities: technical reviews, quality and configuration audits, performance monitoring, simulation, feasibility study, documentation review, database review, algorithm analysis, development, testing, usability testing, qualification testing, acceptance testing, and installation testing Software Quality Assurance

5. Organizing for Software Testing Developers testing their own software have a vested interest in showing it's error-free. This can lead to insufficient testing and overlooked errors. Independent Test Group (ITG): Provides unbiased testing, removing the conflict of interest. Works closely with developers throughout the project. Involved from analysis and design stages to ensure thorough testing. Quotes William Howden: “ Testing is the unavoidable part of any responsible effort to develop a software system.” Kent Beck: “Optimism is the occupational hazard of programming; testing is the treatment.”

6. Software Testing Strategy Software Testing Steps Testing Strategy

7. Software Testing Strategy Cont.. Procedural Steps in Software Testing Unit Testing: Ensures each component functions properly. Uses techniques to exercise specific paths for maximum error detection. Integration Testing: Assembles components into a complete software package. Focuses on verification and program construction. Validation Testing: Evaluates validation criteria established during requirements analysis. Ensures software meets all requirements. System Testing: Combines software with other system elements (hardware, people, databases). Verifies overall system function and performance.

8. Criteria for Completion of Testing "When are we done testing—how do we know that we’ve tested enough?“ Cleanroom Software Engineering: Uses statistical techniques to execute a series of tests. Tests derived from a statistical sample of all possible executions by targeted users. Statistical Modelling and Software Reliability Theory: Predicts the completeness of testing. Advocates for empirical approaches over raw intuition. By collecting metrics during software testing and making use of existing software reliability models, it is possible to develop meaningful guidelines for answering the question: “When are we done testing?”

9. Strategic Issues Quantifiable Requirements: Specify product requirements in a quantifiable manner long before testing commences Specify product requirements in measurable terms. Assess quality characteristics like portability, maintainability, and usability. Explicit Testing Objectives: State testing objectives explicitly State objectives in measurable terms. Include metrics such as test effectiveness, coverage, mean-time-to-failure, cost to fix defects, remaining defect density, and test work-hours. Understand Users: Understand the users of the software and develop a profile for each user category Develop profiles for each user category. Use cases to focus testing on actual use. Rapid Cycle Testing Plan: Develop a testing plan that emphasizes “rapid cycle testing.” Emphasize "rapid cycle testing" (2% of project effort). Use feedback to control quality levels and test strategies.

10. Strategic Issues Cont.. Build Robust Software: for Self-Testing Design software to test itself (antibugging techniques). Accommodate automated and regression testing. Technical Reviews: Use effective technical reviews as a filter prior to testing Use reviews to uncover errors before testing. Reduce testing effort and improve quality. Continuous Improvement: Develop a continuous improvement approach for the testing process. Measure the test strategy. Use metrics for statistical process control and continuous improvement .

11. Testing Strategies for Conventional Software Unit Testing + Integration Testing + Regression Testing + Smoke Testing Unit Testing Unit testing focuses verification effort on the smallest unit of software design—the software component or module. Unit Test Considerations The module interface is tested to ensure that information properly flows into and out of the program unit under test. Local data structures are examined to ensure that data stored temporarily maintains its integrity during all steps in an algorithm’s execution. Boundary conditions are tested to ensure that the module operates properly at boundaries established to limit or restrict processing. Selective testing of execution paths. A good design anticipates error conditions and establishes error-handling paths to reroute or cleanly terminate processing when an error does occur.

12. Unit Testing Cont.. Unit Test Procedures Unit Test Environment Driver is nothing more than a “main program” that accepts test case data, passes such data to the component (to be tested), and prints relevant results. Stubs serve to replace modules that are subordinate (invoked by) the component to be tested.

13. Integration Testing Top-down integration The integration process is performed in a series of five steps: 1. The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module. 2. Depending on the integration approach selected (i.e., depth or breadth first), subordinate stubs are replaced one at a time with actual components. 3. Tests are conducted as each component is integrated. 4. On completion of each set of tests, another stub is replaced with the real component. 5. Regression testing may be conducted to ensure that new errors have not been introduced. Top-down integrtion

14. Integration Testing Bottom-up integration A bottom-up integration strategy may be implemented with the following steps: 1. Low-level components are combined into clusters (sometimes called builds) that perform a specific software subfunction. 2. A driver (a control program for testing) is written to coordinate test case input and output. 3. The cluster is tested. 4. Drivers are removed, and clusters are combined moving upward in the program structure. Bottom-up integration

15. Regression Testing With Integration Testing a new driver/module gets added which brings along, New data flow paths are established New I/O may occur New control logic is invoked The changes can cause some flaws with previously run successful problems, so Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects. The regression test suite (the subset of tests to be executed) contains three different classes of test cases: • A representative sample of tests that will exercise all software functions. • Additional tests that focus on software functions that are likely to be affected by the change. • Tests that focus on the software components that have been changed.

16. Smoke Testing Smoke testing is an integration testing approach that is commonly used when product software is developed. The smoke-testing approach encompasses the following activities: 1. Software components that have been translated into code are integrated into a build. A build includes all data files, libraries, reusable modules, and engineered components that are required to implement one or more product functions. 2. A series of tests is designed to expose errors that will keep the build from properly performing its function. The intent should be to uncover “showstopper” errors that have the highest likelihood of throwing the software project behind schedule. 3. The build is integrated with other builds, and the entire product (in its current form) is smoke tested daily. The integration approach may be top down or bottom up.

17. Testing Strategies for OO Software Unit Testing in the OO Context Encapsulation drives the definition of classes and objects An encapsulated class is usually the focus of unit testing Operations (methods) within the class are the smallest testable units Class testing for OO software is the equivalent of unit testing for conventional software, class testing for OO software is driven by the operations encapsulated by the class and the state behaviour of the class Integration Testing in the OO Context Thread based testing - Integrates the set of classes required to respond to one input or event for the system Use-based testing - Begins the construction of the system by testing those classes (called independent classes) that use very few (if any) server classes. First independent and then dependent classes are tested.

18. Validation Testing Validation testing begins at the culmination of integration testing. Validation testing focuses on user-visible actions and user-recognizable output from the system. Validation-Test Criteria Software validation is achieved through a series of tests that demonstrate conformity with requirements. Validation Test Results in two possibilities, The function or performance characteristic conforms to specification and is accepted or a deviation from specification is uncovered and a deficiency list is created. Configuration Review To ensure that all elements of the software configuration have been properly developed, are catalogued, and have the necessary detail to bolster the support activities. Termed as audit

19. Validation Testing Cont.. Alpha and Beta Testing The alpha test is conducted at the developer’s site by a representative group of end users. The software is used in a natural setting with the developer “looking over the shoulder” of the users and recording errors and usage problems. Alpha tests are conducted in a controlled environment. The beta test is conducted at one or more end-user sites, usually in the absence of developer. The beta test is a “live” application of the software in an environment that cannot be controlled by the developer. A variation on beta testing, called customer Acceptance Testing , is sometimes performed when custom software is delivered to a customer under contract.

20. System Testing System testing is a series of different tests whose primary purpose is to fully exercise the computer-based system Recovery Testing Recovery testing is a system test that forces the software to fail in a variety of ways and verifies that recovery is properly performed If recovery is automatic (performed by the system itself), reinitialization, checkpointing mechanisms, data recovery, and restart are evaluated for correctness. If recovery requires human intervention, the mean-time-to-repair (MTTR) is evaluated to determine whether it is within acceptable limits. Security Testing Security testing attempts to verify that protection mechanisms built into a system will, in fact, protect it from improper penetration. The role of the system designer is to make penetration cost more than the value of the information that will be obtained.

21. System Testing Cont.. Stress Testing Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. For example, (1) special tests may be designed that generate ten interrupts per second, when one or two is the average rate, (2) input data rates may be increased by an order of magnitude to determine how input functions will respond, (3) test cases that require maximum memory or other resources are executed, (4) test cases that may cause thrashing in a virtual operating system are designed, (5) test cases that may cause excessive hunting for disk-resident data are created. Essentially, the tester attempts to break the program.

22. System Testing Cont.. Performance Testing To measure resource utilization (e.g., processor cycles) in an exacting fashion. External instrumentation can monitor execution intervals, log events (e.g., interrupts) as they occur, and sample machine states on a regular basis. By instrumenting a system, the tester can uncover situations that lead to degradation and possible system failure. Deployment Testing Deployment testing, sometimes called configuration testing , exercises the software in each environment in which it is to operate. Deployment testing examines all installation procedures and specialized installation software (e.g., “installers”) that will be used by customers, and all documentation that will be used to introduce the software to end users.

23. The Art of Debugging Debugging occurs as a consequence of successful testing. That is, when a test case uncovers an error, debugging is the process that results in the removal of the error. The debugging process will usually have one of two outcomes: (1) the cause will be found and corrected or (2) the cause will not be found – design additional tests + look out for causes The Debugging Process

24. The Art of Debugging Cont.. Characteristics of bugs, The symptom may disappear (temporarily) when another error is corrected. The symptom may actually be caused by non-errors (e.g., round-off inaccuracies). The symptom may be caused by human error that is not easily traced. The symptom may be a result of timing problems, rather than processing problems. It may be difficult to accurately reproduce input conditions (e.g., a real-time application in which input ordering is indeterminate). The symptom may be intermittent. This is particularly common in embedded systems that couple hardware and software inextricably. The symptom may be due to causes that are distributed across a number of tasks running on different processors The symptom and the cause may be geographically remote, Highly coupled components exacerbate this situation.

25. The Art of Debugging Cont.. Debugging Strategies Brute Force – From memory dumps or run time traces (logs) Backtracking - Beginning at the site where a symptom has been uncovered, the source code is traced backward (manually) until the cause is found Cause Elimination – Introduces the concept of binary partitioning Automated Debugging Debugging compilers + Dynamic debugging aids (“tracers”) + Automatic test-case generators + Cross-reference mapping tools Correcting the Errors Is the cause of the bug reproduced in another part of the program? What “next bug” might be introduced by the fix I’m about to make? What could we have done to prevent this bug in the first place?

26. Agile Methodology Agile is a mindset, a culture, and a series of practices, but simply put, it's about working together in a way that emphasizes iteration, co-creation, and flexibility. Agile methodologies types Scrum, Kanban and others.

27. Agile Manifesto

28. Waterfall Process: Requirements defined upfront. Budget allocated per project. Value released at project end. Challenges of Waterfall Methodology: Fixed Budget, Scope, and Schedule: Rigidity in planning. Difficulty in accommodating changes. Pressure on Development Teams: Death marches: Working nights and weekends. Employee burnout. Lack of End-User Feedback: Little to no feedback during development. Product may not meet current user needs. Impact of Major Requirement Changes: Restarting the Process: Need to restart the waterfall process. Project Abandonment: Potential to drop projects completely. Waste of Resources: Time and money wasted. Negative impact on employee morale.

29. Agile Framework: Scrum Breaks Down Complex Projects: Smaller, manageable pieces. Continuous delivery of value. Collaborative and Flexible: Responds to client’s evolving needs. Adapts to market changes. Flexible Scope: Continuous refining of functionality (Product Backlog). Budget based on product performance. Time frame extends until the end of the product life cycle.

30. The Agile Mindset Focus on Individuals: Emphasizes team members and end-users. Values great teams to create great products. Cultural Shift: Changes thinking and approaches. Agile is a mindset, not just a process. Benefits: Makes work more enjoyable and rewarding. Enhances team dynamics and productivity.

31. Waterfall vs Agile

32. Waterfall vs Agile - 

33. What is Scrum? Scrum (n): A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value. Scrum is: Lightweight Simple to understand Difficult to master The Scrum framework consists of Scrum Teams and their associated roles, events, artifacts, and rules. Each component within the framework serves a specific purpose and is essential to Scrum’s success and usage.

34. Agile Scrum Team

35. Agile – Iterative approach to engage Customer

36. Sprint Ceremonies

37. Sprint Ceremonies

38. How does Scrum work? A product owner creates a prioritized wish list called a product backlog . During sprint planning , the team pulls a small chunk from the items towards the top of the list. That chunk becomes the sprint backlog. The team decides how to implement the sprint backlog within the time frame of the sprint . The team has the given sprint (usually one month or less) to complete its work, but it meets each day to assess its progress (in the daily scrum ). Along the way, the scrum master keeps the team focused on its goal. At the end of the sprint, the work should be potentially shippable : ready to hand to a customer , put on a store shelf, or show to a stakeholder. The sprint ends with a sprint review and retrospective . As the next sprint begins, the team chooses another chunk of the product backlog and begins working again.

39. Backlog Refinement