Purpose of testing, Dichotomies, taxonomy of bugs: Flow graphs and Path testing

Purpose of testing, Dichotomies, taxonomy of bugs: Flow graphs and Path testing, Basics concepts of path testing, predicates, path predicates and achievable paths, path sensitizing

1. Software Testing Methodologies Presented by: Dr. B.Rajalingam Associate Professor & HOD Department of Artificial Intelligence and Data Science (AI&DS), St. Martin's Engineering College (UGC Autonomous) B.Tech AI&DS III Year - I Semester Unit-I

2. Syllabus UNIT - I Introduction: Purpose of testing, Dichotomies, model for testing, consequences of bugs, taxonomy of bugs: Flow graphs and Path testing: Basics concepts of path testing, predicates, path predicates and achievable paths, path sensitizing, path instrumentation, application of path testing. UNIT - II Transaction Flow Testing: transaction flows, transaction flow testing techniques. Dataflow testing: Basics of dataflow testing, strategies in dataflow testing, application of dataflow testing. : domains and paths, Nice & ugly domains, domain testing, domains and interfaces testing, domain and interface testing, domains and testability. STM(Unit 1) - Dr. B.Rajalingam 2 10 October 2023

3. Syllabus UNIT - III Paths, Path products and Regular expressions: path products & path expression, reduction procedure, applications, regular expressions & flow anomaly detection. Logic Based Testing: overview, decision tables, path expressions, kv charts, specifications. UNIT - IV State, State Graphs and Transition testing: state graphs, good & bad state graphs, state testing, Testability tips. UNIT - V Graph Matrices and Application: Motivational overview, matrix of graph, relations, power of a matrix, node reduction algorithm, building tools. (Student should be given an exposure to a tool like JMeter or Win-runner). STM(Unit 1) - Dr. B.Rajalingam 3 10 October 2023

4. TEXT BOOKS: 1. Software Testing techniques – Baris Beizer , Dreamtech , second edition. 2. Software Testing Tools – Dr. K. V. K. K. Prasad, Dreamtech . REFERENCE BOOKS: 1. The craft of software testing – Brian Marick , Pearson Education. 2. Software Testing Techniques – SPD(Oreille) 3. Software Testing in the Real World – Edward Kit, Pearson. 4. Effective methods of Software Testing, Perry, John Wiley. 5. Art of Software Testing – Meyers, John Wiley. STM(Unit 1) - Dr. B.Rajalingam 4 10 October 2023

5. SOFTWARE TESTING METHODOLOGIES Prerequisites 1. A course on “Software Engineering” Course Objectives To provide knowledge of the concepts in software testing such as testing process, criteria, strategies, and methodologies. To develop skills in software test automation and management using latest tools. Course Outcomes : Design and develop the best test strategies in accordance to the development model. STM(Unit 1) - Dr. B.Rajalingam 5 10 October 2023

6. Software Testing Software testing is a process of identifying the correctness of software by considering its all attributes (Reliability, Scalability, Portability, Re-usability, Usability) and evaluating the execution of software components to find the software bugs or errors or defects. Software testing is widely used technology because it is compulsory to test each and every software before deployment. Our Software testing tutorial includes all topics of Software testing such as Methods such as Black Box Testing, White Box Testing, Visual Box Testing and Gray Box Testing. Levels such as Unit Testing, Integration Testing, Regression Testing, Functional Testing. System Testing, Acceptance Testing, Alpha Testing, Beta Testing, Non-Functional testing, Security Testing, Portability Testing. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 6

7. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 7

8. ST( Cont …) Software testing provides an independent view and objective of the software and gives surety of fitness of the software. It involves testing of all components under the required services to confirm that whether it is satisfying the specified requirements or not. The process is also providing the client with information about the quality of the software. Testing is mandatory because it will be a dangerous situation if the software fails any of time due to lack of testing. So, without testing software cannot be deployed to the end user . Uncover as many as errors (or bugs) as possible in a given product. Demonstrate a given software product matching its requirement specifications. Validate the quality of a software using the minimum cost and efforts. Generate high-quality test cases, perform effective tests, and issue correct and helpful problem reports. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 8

9. Software testing is often divided into 2 main processes: Verification and Validation. Verification  in software testing is the process when your team just need to check whether the software, system or framework consistent, aligned with the requirements of a documentation. Validation  is the process that your team needs to verify the accuracy of the system. In this process, you will look back to the product, system and think about what users actually want and what has been done . Software testing provides an independent view and objective of the software and gives surety of fitness of the software. It involves testing of all components under the required services to confirm that whether it is satisfying the specified requirements or not. The process is also providing the client with information about the quality of the software. Testing is mandatory because it will be a dangerous situation if the software fails any of time due to lack of testing. So, without testing software cannot be deployed to the end user. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 9

10. Cont … In Software Testing, there is the difference between  Errors ,  Defects , and  Bugs  that we should distinguish clearly to avoid misunderstanding problem. Error  is a deviation from the actual and the expected result. It represents the mistakes made by people. Bug  is an error found BEFORE the application goes into production. A programming error that causes a program to work poorly, produce incorrect results, or crash. An error in software or hardware that causes a program to malfunction. Defect  happens once the error is identified during testing, it is logged as a ‘Defect’ in the tracking system. Failure  is the incapacity of a system to conduct its required functions within clarified performance requirements, literally a disappointment or a let down. And no one wants to do business with a failure. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 10

11. What is Testing Testing is a group of techniques to determine the correctness of the application under the predefined script but, testing cannot find all the defect of application. The main intent of testing is to detect failures of the application so that failures can be discovered and corrected. It does not demonstrate that a product functions properly under all conditions but only that it is not working in some specific conditions. Testing furnishes comparison that compares the behavior and state of software against mechanisms because the problem can be recognized by the mechanism. The mechanism may include past versions of the same specified product, comparable products, and interfaces of expected purpose, relevant standards, or other criteria but not limited up to these . Testing includes an examination of code and also the execution of code in various environments, conditions as well as all the examining aspects of the code. In the current scenario of software development, a testing team may be separate from the development team so that Information derived from testing can be used to correct the process of software development. The success of software depends upon acceptance of its targeted audience, easy graphical user interface, strong functionality load test, etc. Therefore , when an organization develops a software product, it can assess whether the software product will be beneficial to its purchasers and other audience. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 11

12. What are the benefits of Software Testing? Cost-Effective:  It is one of the important advantages of software testing. Testing any IT project on time helps you to save your money for the long term. In case if the bugs caught in the earlier stage of software testing, it costs less to fix. Security:  It is the most vulnerable and sensitive benefit of software testing. People are looking for trusted products. It helps in removing risks and problems earlier. Product quality:  It is an essential requirement of any software product. Testing ensures a quality product is delivered to customers. Customer Satisfaction:  The main aim of any product is to give satisfaction to their customers. UI/UX Testing ensures the best user experience. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 12

13. Type of Software testing 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 13

14. Manual testing The process of checking the functionality of an application as per the customer needs without taking any help of automation tools is known as manual testing. While performing the manual testing on any application, we do not need any specific knowledge of any testing tool, rather than have a proper understanding of the product so we can easily prepare the test document. Manual testing can be further divided into three types of testing, which are as follows: White box testing Black box testing Gray box testing STM(Unit 1) - Dr. B.Rajalingam 14 10 October 2023

15. Automation testing Automation testing is a process of converting any manual test cases into the test scripts with the help of automation tools, or any programming language is known as automation testing. With the help of automation testing, we can enhance the speed of our test execution because here, we do not require any human efforts. We need to write a test script and execute those scripts. STM(Unit 1) - Dr. B.Rajalingam 15 10 October 2023

16. Types of Manual Testing White Box Testing Black Box Testing Gray Box Testing STM(Unit 1) - Dr. B.Rajalingam 16 10 October 2023

17. White-box testing The white box testing is done by Developer, where they check every line of a code before giving it to the Test Engineer. Since the code is visible for the Developer during the testing, that's why it is also known as White box testing. STM(Unit 1) - Dr. B.Rajalingam 17 10 October 2023

18. Black box testing The black box testing is done by the Test Engineer, where they can check the functionality of an application or the software according to the customer /client's needs. In this, the code is not visible while performing the testing; that's why it is known as black-box testing. STM(Unit 1) - Dr. B.Rajalingam 18 10 October 2023

19. Gray Box testing Gray box testing is a combination of white box and Black box testing. It can be performed by a person who knew both coding and testing. And if the single person performs white box, as well as black-box testing for the application, is known as Gray box testing. STM(Unit 1) - Dr. B.Rajalingam 19 10 October 2023

20. Advantages of Manual Testing It does not require programming knowledge while using the Black box method. It is used to test dynamically changing GUI designs. Tester interacts with software as a real user so that they are able to discover usability and user interface issues. It ensures that the software is a hundred percent bug-free. It is cost-effective. Easy to learn for new testers. STM(Unit 1) - Dr. B.Rajalingam 20 10 October 2023

21. Disadvantages of Manual Testing It requires a large number of human resources. It is very time-consuming. Tester develops test cases based on their skills and experience. There is no evidence that they have covered all functions or not. Test cases cannot be used again. Need to develop separate test cases for each new software. It does not provide testing on all aspects of testing. Since two teams work together, sometimes it is difficult to understand each other's motives, it can mislead the process. STM(Unit 1) - Dr. B.Rajalingam 21 10 October 2023

22. Advantages of Automation Testing Automation testing takes less time than manual testing. A tester can test the response of the software if the execution of the same operation is repeated several times. Automation Testing provides re-usability of test cases on testing of different versions of the same software. Automation testing is reliable as it eliminates hidden errors by executing test cases again in the same way. Automation Testing is comprehensive as test cases cover each and every feature of the application. It does not require many human resources, instead of writing test cases and testing them manually, they need an automation testing engineer to run them. The cost of automation testing is less than manual testing because it requires a few human resources. STM(Unit 1) - Dr. B.Rajalingam 22 10 October 2023

23. Disadvantages of Automation Testing Automation Testing requires high-level skilled testers. It requires high-quality testing tools. When it encounters an unsuccessful test case, the analysis of the whole event is complicated. Test maintenance is expensive because high fee license testing equipment is necessary. Debugging is mandatory if a less effective error has not been solved, it can lead to fatal results. STM(Unit 1) - Dr. B.Rajalingam 23 10 October 2023

24. How to perform Manual Testing First, tester observes all documents related to software, to select testing areas. Tester analyses requirement documents to cover all requirements stated by the customer. Tester develops the test cases according to the requirement document. All test cases are executed manually by using Black box testing and white box testing. If bugs occurred then the testing team informs the development team. The Development team fixes bugs and handed software to the testing team for a retest. STM(Unit 1) - Dr. B.Rajalingam 24 10 October 2023

25. 25 A Model for Testing Environment Environment Model Tests Program Model Program Bug Model Nature & Psychology Outcome The World The Model World Expected Unexpected STM(Unit 1) - Dr. B.Rajalingam 10 October 2023

26. Model for Testing Model of testing process includes three models: A model of the environment, a model of the program and a model of the expected bugs. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 26

27. Environment Model A Program's environment is the hardware and software required to make it run. For online systems, the environment may include communication lines, other systems, terminals and operators. The environment also includes all programs that interact with and are used to create the program under test - such as OS, linkage editor, loader, compiler, utility routines. Because the hardware and firmware are stable, it is not smart to blame the environment for bugs. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 27

28. Program Most programs are too complicated to understand in detail. The concept of the program is to be simplified in order to test it. If simple model of the program does not explain the unexpected behaviour, we may have to modify that model to include more facts and details. And if that fails, we may have to modify the program. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 28

29. Bugs Bugs are more insidious (deceiving but harmful) than ever we expect them to be. An unexpected test result may lead us to change our notion of what a bug is and our model of bugs. Some optimistic notions that many programmers or testers have about bugs are usually unable to test effectively and unable to justify the dirty tests most programs need. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 29

30. Optimistic Notions about Bugs Benign Bug Hypothesis:  The belief that bugs are nice, tame and logical. (Benign: Not Dangerous) Bug Locality Hypothesis:  The belief that a bug discovered with in a component effects only that component's behaviour. Control Bug Dominance:  The belief that errors in the control structures (if, switch etc) of programs dominate the bugs. Code / Data Separation:  The belief that bugs respect the separation of code and data. Lingua Salvator Est:  The belief that the language syntax and semantics (e.g. Structured Coding, Strong typing, etc) eliminates most bugs. Corrections Abide:  The mistaken belief that a corrected bug remains corrected. Silver Bullets:  The mistaken belief that X (Language, Design method, representation, environment) grants immunity from bugs. Sadism Suffices:  The common belief (especially by independent tester) that a sadistic streak, low cunning, and intuition are sufficient to eliminate most bugs. Tough bugs need methodology and techniques. Angelic Testers:  The belief that testers are better at test design than programmers are at code design. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 30

31. Tests Tests are formal procedures, Inputs must be prepared, Outcomes should predicted, tests should be documented, commands need to be executed, and results are to be observed.  All these errors are subjected to error We do three distinct kinds of testing on a typical software system. They are: A   Unit  is the smallest testable piece of software that can be compiled, assembled, linked, loaded etc. A unit is usually the work of one programmer and consists of several hundred or fewer lines of code.  Unit Testing  is the testing we do to show that the unit does not satisfy its functional specification or that its implementation structure does not match the intended design structure. A  Component  is an integrated aggregate of one or more units.  Component Testing  is the testing we do to show that the component does not satisfy its functional specification or that its implementation structure does not match the intended design structure. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 31

32. Tests Integration Testing:   Integration  is the process by which components are aggregated to create larger components.  Integration Testing  is testing done to show that even though the components were individually satisfactory (after passing component testing), checks the combination of components are incorrect or inconsistent. System Testing:   A  System  is a big component.  System Testing  is aimed at revealing bugs that cannot be attributed to components. It includes testing for performance, security, accountability, configuration sensitivity, startup and recovery. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 32

33. Functional Testing Every program operates on a finite number of inputs. A complete functional test would consists of subjecting the program to all possible input streams. For each input the routine either accepts the stream and produces a correct outcome, accepts the stream and produces an incorrect outcome, or rejects the stream and tells us that it did so. For example, a 10 character input string has 280 possible input streams and corresponding outcomes, so complete functional testing in this sense is IMPRACTICAL. But even theoretically, we can't execute a purely functional test this way because we don't know the length of the string to which the system is responding. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 33

34. Structural Testing The design should have enough tests to ensure that every path through the routine is exercised at least once. Right off that's is impossible because some loops might never terminate. The number of paths through a small routine can be awesome because each loop multiplies the path count by the number of times through the loop. A small routine can have millions or billions of paths, so total  Path Testing  is usually IMPRACTICAL. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 34

35. Formal Proofs of Correctness Formal proofs of correctness rely on a combination of functional and structural concepts. Requirements are stated in a formal language (e.g. Mathematics) and each program statement is examined and used in a step of an inductive proof that the routine will produce the correct outcome for all possible input sequences. The IMPRACTICAL thing here is that such proofs are very expensive and have been applied only to numerical routines or to formal proofs for crucial software such as system’s security kernel or portions of compilers. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 35

36. Importance of Bugs The importance of bugs depends on frequency, correction cost, installation cost, and consequences. Frequency:  How often does that kind of bug occur? Pay more attention to the more frequent bug types. Correction Cost:  What does it cost to correct the bug after it is found? The cost is the sum of 2 factors: (1) the cost of discovery (2) the cost of correction. These costs go up dramatically later in the development cycle when the bug is discovered. Correction cost also depends on system size. Installation Cost:  Installation cost depends on the number of installations: small for a single user program but more for distributed systems. Fixing one bug and distributing the fix could exceed the entire system's development cost. Consequences:  What are the consequences of the bug? Bug consequences can range from mild to catastrophic. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 36

37. Importance of Bugs A reasonable metric for bug importance is Importance= ($) = Frequence * (Correction cost + Installation cost + Consequential cost) 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 37

38. Mild :  The symptoms of the bug offend us aesthetically (gently); a misspelled output or a misaligned printout. Moderate:  Outputs are misleading or redundant. The bug impacts the system's performance. Annoying:  The system's behaviour because of the bug is dehumanizing.  E.g.  Names are truncated or arbitarily modified. Disturbing:  It refuses to handle legitimate (authorized / legal) transactions. The ATM wont give you money. My credit card is declared invalid. Serious:  It loses track of its transactions. Not just the transaction itself but the fact that the transaction occurred. Accountability is lost. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 38 Consequences of Bugs

39. Very Serious:  The bug causes the system to do the wrong transactions. Instead of losing your pay check, the system credits it to another account or converts deposits to withdrawals. Extreme:  The problems aren't limited to a few users or to few transaction types. They are frequent and arbitrary instead of sporadic infrequent) or for unusual cases. Intolerable:  Long term unrecoverable corruption of the database occurs and the corruption is not easily discovered. Serious consideration is given to shutting the system down. Catastrophic:  The decision to shut down is taken out of our hands because the system fails. Infectious:  What can be worse than a failed system? One that corrupt other systems even though it does not fall in itself ; that erodes the social physical environment; that melts nuclear reactors and starts war. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 39 Consequences of Bugs

40. Taxonomy of Bugs There is no universally correct way categorize bugs. The taxonomy is not rigid. A given bug can be put into one or another category depending on its history and the programmer's state of mind. The major categories are: (1) Requirements, Features and Functionality Bugs (2) Structural Bugs (3) Data Bugs (4) Coding Bugs (5) Interface, Integration and System Bugs (6) Test and Test Design Bugs. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 40

41. What is testing? Testing is the practice of making objective judgments regarding the extent to which the system (device) meets, exceeds or fails to meet stated objectives 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 41

42. What the purpose of testing? There are two fundamental purposes of testing: verifying procurement specifications and managing risk. First, testing is about verifying that what was specified is what was delivered: it verifies that the product (system) meets the functional, performance, design, and implementation requirements identified in the procurement specifications. Second, testing is about managing risk for both the acquiring agency and the system’s vendor/developer/integrator. The testing program is used to identify when the work has been “completed” so that the contract can be closed, the vendor paid, and the system shifted by the agency into the warranty and maintenance phase of the project. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 42

43. Why is testing important? A good testing program is a tool for both the agency and the integrator/supplier; it typically identifies the end of the “development” phase of the project, establishes the criteria for project acceptance, and establishes the start of the warranty period. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 43

44. What methods are used to conduct testing? Inspection - Inspection is the verification by physical and visual examinations of the item, reviewing descriptive documentation, and comparing the appropriate characteristics with all the referenced standards to determine compliance with the requirements. Certificate of Compliance - A Certificate of Compliance is a means of verifying compliance for items that are standard products. Signed certificates from vendors state that the purchased items meet procurement specifications, standards, and other requirements as defined in the purchase order. Records of tests performed to verify specifications are retained by the vendor as evidence that the requirements were met and are made available by the vendor for purchaser review . Analysis - Analysis is the verification by evaluation or simulation using mathematical representations, charts, graphs, circuit diagrams, calculation, or data reduction. This includes analysis of algorithms independent of computer implementation, analytical conclusions drawn from test data, and extension of test-produced data to untested conditions. Demonstration - Demonstration is the functional verification that a specification requirement is met by observing the qualitative results of an operation or exercise performed under specific condition. This includes content and accuracy of displays, comparison of system outputs with independently derived test cases, and system recovery from induced failure conditions. Test (Formal) - Formal testing is the verification that a specification requirement has been met by measuring, recording, or evaluating qualitative and quantitative data obtained during controlled exercises under all appropriate conditions using real and/or simulated stimulus. This includes verification of system performance, system functionality, and correct data distribution. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 44

45. Productivity and Quality in software: In production of consumer goods and other products, every manufacturing stage is subjected to quality control and testing from component to final stage. If flaws are discovered at any stage, the product is either discarded or cycled back for rework and correction. Productivity is measured by the sum of the costs of the material, the rework, and the discarded components, and the cost of quality assurance and testing. There is a trade off between quality assurance costs and manufacturing costs: If sufficient time is not spent in quality assurance, the reject rate will be high and so will be the net cost. If inspection is good and all errors are caught as they occur, inspection costs will dominate, and again the net cost will suffer. Testing and Quality assurance costs for 'manufactured' items can be as low as 2% in consumer products or as high as 80% in products such as space-ships, nuclear reactors, and aircrafts, where failures threaten life. Where as the manufacturing cost of a software is trivial. The biggest part of software cost is the cost of bugs: the cost of detecting them, the cost of correcting them, the cost of designing tests that discover them, and the cost of running those tests. For software, quality and productivity are indistinguishable because the cost of a software copy is trivial. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 45

46. Phases in a testing Phase 0: (Until 1956: Debugging Oriented)  There is no difference between testing and debugging. Phase 0 thinking was the norm in early days of software development till testing emerged as a discipline. Phase 1: (1957-1978: Demonstration Oriented)  The purpose of testing here is to show that software works. Highlighted during the late 1970s. This failed because the probability of showing that software works 'decreases' as testing increases.  i.e.  The more you test, the more likely you'ill find a bug . Phase 2: (1979-1982: Destruction Oriented)  The purpose of testing is to show that software doesn’t work. This also failed because the software will never get released as you will find one bug or the other. Also, a bug corrected may also lead to another bug. Phase 3: (1983-1987: Evaluation Oriented)  The purpose of testing is not to prove anything but to reduce the perceived risk of not working to an acceptable value (Statistical Quality Control). Notion is that testing does improve the product to the extent that testing catches bugs and to the extent that those bugs are fixed. The product is released when the confidence on that product is high enough. (Note: This is applied to large software products with millions of code and years of use.) Phase 4: (1988-2000: Prevention Oriented)  Testability is the factor considered here. One reason is to reduce the labour of testing. Other reason is to check the testable and non-testable code. Testable code has fewer bugs than the code that's hard to test. Identifying the testing techniques to test the code is the main key here 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 46

47. Cont … Test Design:  We know that the software code must be designed and tested, but many appear to be unaware that tests themselves must be designed and tested. Tests should be properly designed and tested before applying it to the actual code. Testing isn't everything:  There are approaches other than testing to create better software. Methods other than testing include: Inspection Methods:  Methods like walkthroughs, desk checking, formal inspections and code reading appear to be as effective as testing but the bugs caught do not completely overlap . Design Style:  While designing the software itself, adopting stylistic objectives such as testability, openness and clarity can do much to prevent bugs. Static Analysis Methods:  Includes formal analysis of source code during compilation. In earlier days, it is a routine job of the programmer to do that. Now, the compilers have taken over that job. Languages:  The source language can help reduce certain kinds of bugs. Programmers find new bugs while using new languages. Development Methodologies and Development Environment:  The development process and the environment in which that methodology is embedded can prevent many kinds of bugs. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 47

48. Dichotomies Testing Versus Debugging:  Many people consider both as same. Purpose of testing is to show that a program has bugs. The purpose of testing is to find the error or misconception that led to the program's failure and to design and implement the program changes that correct the error. Debugging usually follows testing, but they differ as to goals, methods and most important psychology. The below table shows few important differences between testing and debugging. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 48

49. Testing Debugging Testing starts with known conditions, uses predefined procedures and has predictable outcomes. Debugging starts from possibly unknown intial conditions and the end can not be predicted except statistically. Testing can and should be planned, designed and scheduled. Procedure and duration of debugging cannot be so constrained. Testing is a demonstration of error or apparent correctness. Debugging is a deductive process. Testing proves a programmer's failure. Debugging is the programmer's vindication (Justification). Testing, as executes, should strive to be predictable, dull, constrained, rigid and inhuman. Debugging demands intutive leaps, experimentation and freedom. Much testing can be done without design knowledge. Debugging is impossible without detailed design knowledge. Testing can often be done by an outsider. Debugging must be done by an insider. Much of test execution and design can be automated. Automated debugging is still a dream. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 49

50. Function Versus Structure Tests can be designed from a functional or a structural point of view. In  functional testing , the program or system is treated as a black box. It is subjected to inputs, and its outputs are verified for conformance to specified behaviour. Functional testing takes the user point of view- bother about functionality and features and not the program's implementation.  Structural testing  does look at the implementation details. Things such as programming style, control method, source language, database design, and coding details dominate structural testing. Both Structural and functional tests are useful, both have limitations, and both target different kinds of bugs. Functional test can detect all bugs but would take infinite time to do so. Structural tests are inherently finite but cannot detect all errors even if completely executed. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 50

51. Designer Versus Tester   Test designer is the person who designs the tests where as the tester is the one actually tests the code. During functional testing, the designer and tester are probably different persons. During unit testing, the tester and the programmer merge into one person. Tests designed and executed by the software designers are by nature biased towards structural consideration and therefore suffer the limitations of structural testing. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 51

52. Modularity Versus Efficiency   A module is a discrete, well-defined, small component of a system. Smaller the modules, difficult to integrate; larger the modules, difficult to understand. Both tests and systems can be modular. Testing can and should likewise be organised into modular components. Small, independent test cases can be designed to test independent modules. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 52

53. Small Versus Large Programming in large means constructing programs that consists of many components written by many different programmers. Programming in the small is what we do for ourselves in the privacy of our own offices. Qualitative and Quantitative changes occur with size and so must testing methods and quality criteria . The different roles / users in a system include: Builder: Who designs the system and is accountable to the buyer. Buyer:  Who pays for the system in the hope of profits from providing services. User:  Ultimate beneficiary or victim of the system. The user's interests are also guarded by. Tester:  Who is dedicated to the builder's destruction. Operator:  Who has to live with the builders' mistakes, the buyers' murky (unclear) specifications, testers' oversights and the users' complaints. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 53

54. Path Testing 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 54

55. What is Path Testing in Software Testing ? Path testing is a structural testing method that involves using the source code of a program in order to find every possible executable path and NOT based on the specifications. It can be applied at different levels of granularity. It helps to determine all faults lying within a piece of code. This method is designed to execute all or selected path through a computer program. Any software program includes, multiple entry and exit points. Testing each of these points is a challenging as well as time-consuming. In order to reduce the redundant tests and to achieve maximum test coverage, basis path testing is used. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 55

56. Basis of Path Testing Basis Path Testing in software engineering is a White Box Testing method in which test cases are defined based on flows or logical paths that can be taken through the program. The objective of basis path testing is to define the number of independent paths, so the number of test cases needed can be defined explicitly to maximize test coverage. In software engineering, Basis path testing involves execution of all possible blocks in a program and achieves maximum path coverage with the least number of test cases. It is a hybrid method of branch testing and path testing methods . Path Testing Assumptions: The Specifications are Accurate. The Data is defined and accessed properly. There are no defects that exist in the system other than those that affect control flow. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 56

57. Path Testing Techniques Control Flow Graph (CFG) -  The Program is converted into Flow graphs by representing the code into nodes, regions and edges. Decision to Decision path (D-D) -  The CFG can be broken into various Decision to Decision paths and then collapsed into individual nodes. Independent (basis) paths -  Independent path is a path through a DD-path graph which cannot be reproduced from other paths by other methods. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 57

58. Path Testing E xample 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 58 Here we will take a simple example, to get a better idea what is basis path testing include. In the above example, we can see there are few conditional statements that is executed depending on what condition it suffice. Here there are 3 paths or condition that need to be tested to get the output, Path 1 : 1,2,3,5,6, 7 Path 2 : 1,2,4,5,6, 7 Path 3 : 1, 6, 7

59. Steps for Basis Path testing Draw a control graph (to determine different program paths) Calculate  Cyclomatic complexity  (metrics to determine the number of independent paths ). This will be the maximum number of independent paths in the graph. Identify independent paths in the control flow graph. Design test cases based on the independent paths identified so that the test cases execute all independent paths . 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 59

60. Advantages of Basic Path Testing Basis path testing reduces the number of redundant tests. All program statements are executed and tested at least once. It guarantees complete branch coverage. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 60

61. 61 Control Flow Graph A simplified, abstract, and graphical representation of a program’s control structure using process blocks, decisions and junctions. L1 Do Process A Process Block Decisions Junctions Case Statement IF A = B ? NO: ELSE DO YES: THEN DO 1 2 1 CASE-OF 2 N CASE 1 CASE 2 CASE N Control Flow Graph Elements 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

62. 62 Control Flow Graph Elements: Process Block: A sequence of program statements uninterrupted by decisions or junctions with a single entry and single exit. Junction : A point in the program where control flow can merge (into a node of the graph) Examples: target of GOTO, Jump, Continue Decisions : A program point at which the control flow can diverge (based on evaluation of a condition) . Examples: IF stmt. Conditional branch and Jump instruction. Case Statements: A Multi-way branch or decision. Examples: In assembly language: jump addresses table, Multiple GOTOs, Case/Switch For test design, Case statement and decision are similar. L1 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

63. Path Testing Concepts Path is a sequence of statements starting at an entry, junction or decision and ending at another, or possibly the same junction or decision or an exit point. Link is a single process ( block ) in between two nodes. Node is a junction or decision. Segment is a sequence of links. A path consists of many segments. Path segment is a succession of consecutive links that belongs to the same path. (3,4,5) Length of a path is measured by # of links in the path or # of nodes traversed. Name of a path is the set of the names of the nodes along the path. (1,2,3 4,5, 6 ) ( 1,2,3,4, 5,6,7, 5,6,7, 5,6) Path-Testing Path is an “entry to exit” path through a processing block . L2 10 October 2023 63 STM(Unit 1) - Dr. B.Rajalingam

64. 64 Path Testing Concepts.. Entry / Exit for a routines, process blocks and nodes . Single entry and single exit routines are preferable. Called well-formed routines. Formal basis for testing exists. Tools could generate test cases. L2 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

65. 65 Path Testing Concepts.. Multi-entry / Multi-exit routines: (ill-formed) A Weak approach : Hence, convert it to single-entry / single-exit routine. Integration issues : Large # of inter-process interfaces. Creates problem in Integration. More # test cases and also a formal treatment is more difficult. Theoretical and tools based issues A good formal basis does not exist. Tools may fail to generate important test cases. L2 Multi- entry routine Multi- Exit Routine Valid only for x Valid for x or y Valid only for Y Called by x, y Valid for caller A, B Valid for caller A Valid for caller B, C 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

66. 66 Path Testing Concepts contd.. Convert a multi-entry / exit routine to a single entry / exit routine : Use an entry parameter and a case statement at the entry => single-entry Merge all exits to Single-exit point after setting one exit parameter to a value . L2 Begin N Begin Begin 1 1 2 N Begin 2 Case Exit N Exit 1 Exit 2 1 2 N SET E = 1 SET E = 1 SET E = 1 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

67. 67 Path Testing Concepts Path Testing Criteria : Path Testing (P  ): Execute all possible control flow paths thru the program; but typically restricted to entry-exit paths. Implies 100% path coverage. Impossible to achieve. Statement Testing ( P 1 ) : Execute all statements in the program at least once under the some test. 100% statement coverage => 100% node coverage. Denoted by C1. C1 is a minimum testing requirement in the IEEE unit test standard: ANSI 87B. Branch Testing (P 2 ) : Execute enough tests to assure that every branch alternative has been exercised at least once under some test. Denoted by C2. Objective : 100% branch coverage and 100% Link coverage. For well structured software , branch testing & coverage include statement coverage L2 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

68. 68 Effectiveness of Path Testing Path testing (with mainly P1 & P2) catches ~65% of Unit Test Bugs ie ., ~35% of all bugs. More effective for unstructured than structured software. Limitations Path testing may not do expected coverage if bugs occur. Path testing may not reveal totally wrong or missing functions. Unit-level path testing may not catch interface errors among routines. Data base and data flow errors may not be caught. Unit-level path testing cannot reveal bugs in a routine due to another. Not all initialization errors are caught by path testing. Specification errors cannot be caught. L2 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

69. 69 Effectiveness of Path Testing Creating flow graph, selecting paths for coverage, finding input data values to force these paths, setting up loop cases & combinations. Careful , systematic, test design will catch as many bugs as the act of testing. Test design process at all levels at least as effective at catching bugs as is running the test designed by that process . L2 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

70. 70 Path, Predicates and Predicate Expressions Path A sequence of process links (& nodes) Predicate The logical function evaluated at a decision : True or False . ( Binary , boolean ) Compound Predicate Two or more predicates combined with AND, OR etc. Path Predicate Every path corresponds to a succession of True/False values for the predicates traversed on that path. A predicate associated with a path. “ X > 0 is True “ AND “W is either negative or equal to 122” is True Multi-valued Logic / Multi-way branching L3 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

71. Path Sensitizing Review of Achievable and Unachievable Paths. The act of finding a set of solutions to the path predicate expression is called path sensitization. If you can find a solution, the path is achievable. If you cannot find a solution, then the path is unachievable. If you cannot find a solution to any of the sets of inequalities, the path is not achievable. Pragmatic Observations Heuristic Procedures for Sensitizing Paths Identify Classify Start Path The predicates are correlated The predicates are incorrectly classified If coverage hasn’t been achieved using independent uncorrelated. If coverage hasn’t been achieved extend the cases to those that involve defendent Predicates. Last' use correlated' defendant predicates. Each path will yield a set of inequalities' which must be simultaneously satisfied to force the path. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 71

72. We want to select and test enough paths to achieve a satisfactory notion of test completeness, such as C 1  + C 2. Extract the programs control flow graph and select a set of tentative covering paths. In general, individual predicates are compound or may become compound as a result of interpretation Trace the path through by multiplying the individual compound predicates to achieve a Boolean expression such as       (A + BC) (D + E) (FGH) (IJ) (K) (I) (L) Multiply the expression to achieve a sum of products form:        ADFGHIJKL + AEFGHIKL + BCDFGHIJKL + BCEFCHIJKL Solve any one of the inequality sets for the chosen path, and you have found a set of input values for the path. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 72 • If coverage hasn%t been achieved using inde"endentuncorrelated& • If coverage hasn%t been achieved'e(tend the cases tothose that involve de"endent "redicates& • )ast' use correlated' de"endent "redicates& • *ach "ath will yield a set of ine+ualities' which must besimultaneously satis#ed to force the "ath&                                                                                                                                                                                                          

73. Heuristic Procedures for Sensitizing Paths This is a workable approach, instead of selecting the paths without considering how to sensitize, attempt to choose a covering path set that is easy to sensitize and pick hard to sensitize paths only as you must to achieve coverage. Identify all variables that affect the decision. Classify the predicates as dependent or independent. Start the path selection with uncorrelated, independent predicates. If coverage has not been achieved using independent uncorrelated predicates, extend the path set using correlated predicates. If coverage has not been achieved, extend the cases to those that involve dependent predicates Finally, use correlated and dependent predicates. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 73

74. Path Instrumentation Path instrumentation confirms that the outcome was achieved by the intended path. The displayed image is an example of a routine for the assumed input value x=4 If it yields the same output y=5 for all the case statements then it said to be co-incidental correctness. Otherwise it detects a bug 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 74

75. Co-incidental Correctness:   The co-incidental correctness stands for achieving the desired outcome for wrong reason The above image is an example of a routine, for the (unfortunately) chosen input value (X=16), yields the same outcome (y=2), no matter which case we select Therefore, the tests chosen this way will not tell us whether we have achieved coverage For example, the five cases could be totally jumbled and still the outcome would be the same 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 75

76. Types of Path Instrumentation 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 76

77. Interpretive Trace Program An Interpretive Trace Program is the one that executes every statement in order, and records the intermediate values of all calculations, the statement labels traversed etc. If we run the tested routine under a trace, then we have all the information needed to confirm the outcome and furthermore, to confirm that it was achieved by the intended path. The trouble with traces is that they give us far more information than we need. In fact, the typical trace program provides so much information that confirming the path from its massive output dump is more work than simulating the computer by hand to confirm the path. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 77

78. Traversal Marker or Link Marker : A simple and effective form of instrumentation is called a Traversal Marker or Link Marker. Name every link by a lower case letter. Instrument the links so that the links name is recorded when the link is executed. The succession of letters produced in going form the routines entry to its exit should, if there are no bugs, exactly correspond to the path name. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 78

79. Why single link markers aren't Enough :  Unfortunately, a single link marker may not do the trick because links can be chewed by open bugs. We intended to traverse the ' i ' km path. but because of a GOTO in the middle of the 'm' link, we go to process B. If conditional correctness is against us, the outcomes will be the same and we won't know about the bug. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 79

80. Two Link Marker Method The solution to the problem of single link marker method is to implement two markers per link, one at the beginning of each link and one at the end. The two link markers now specify the path name and confirm both the beginning and end of the link. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 80

81. Link Counter A less disruptive (and less informative) instrumentation method is based on outcomes. Instead of a unique link name to be pushed into a string when the link traversed, we simply increment a link counter, we now confirm that the path length is expected. The same problem that led us to double link markers also leads us to double link counters. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 81

82. Applications of Path Testing 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 82

83. Integration, Coverage and Paths in called Components Path testing methods are mainly used in unit testing, especially for new software The new component is first tested as an independent unit with all called components and co-requisite components replaced by stubs. A simulator of low-level components that is more reliable than the actual component Path testing clarifies the integration issues C 1  coverage at the system level ranges from 50% to 85% We gave no statistics for C 2  coverage in system testing because it is impossible to monitor C2 coverage without disrupting the system's operation 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 83

84. New Code New code should always be subjected to enough path testing to achieve C 2. Stubs are used where it is clear that the bug potential for the stub is significantly lower than that of the called components. Old, trusted components will not be replaced by stubs. Some consideration is given to paths within called components. Typically, we will try to use the shortest entry/exit path that will do the task. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 84

85. Maintenance There is a great difference between maintenance testing and new code testing. Maintenance testing is a completely different situation. It involves modifications which are accommodated in the system, as required. Path testing is used firstly on the modified component. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 85

86. Rehosting : Path testing with C 1 +C 2 coverage is a powerful tool for rehosting old software. We get a very powerful, effective, rehosting process when C 1 +C 2  coverage is used in conjunction with automatic or semiautomatic structural test generators. Software is rehosted because it is no longer cost effective to support the environment in which it runs. The objective of rehosting is to change the operating environment and not the rehosted software Rehosting from one COBOL environment to another is easy by comparison. Rehosted software can be modified to improve efficiency and/or to implement new functionality, which had been difficult in the old environments. The test suites(collection) and all outcomes of the old environment become the specification for the rehosted software. 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam 86

87. 87 For reference – just to see that we have covered these: Q. Define Path Testing. Explain three path testing criteria. Q . Illustrate with an example, how statement and branch coverage can be achieved during path selection. Write all steps involved in it. Q . Write the effectiveness and limitations of path testing. Q . Define Path Sensitization. Explain heuristic procedure for sensitizing paths with the help of an example. Q . How can a program control structure be represented graphically? Explain with the help of required diagrams. Q . How is a flowchart differed from a control flow chart? Q . Explain about Multi entry and Multi exit routines & fundamental path selection criteria Q . Explain about path instrumentation. How are link counters useful in Path Instrumentation method? Q . Write about implementation of path testing. What are the various applications of path testing? Q . Categorize different kinds of loops and explain . L7 10 October 2023 STM(Unit 1) - Dr. B.Rajalingam

88. Thank you STM(Unit 1) - Dr. B.Rajalingam 88 10 October 2023