dynamic testing for polytechnic students

Dynamic testing for polytechnic

1. UNIT 4 DYNAMIC TESTING AND TEST CASE DESIGN TECHNIQUES

2. 0. INTRODUCTION: during testing, one has to formulate testing specifications and design, test case design, test case design , record these test cases as documents, execute the test cases, report the results of test case execution and prepare defect list. Dynamic testing: The software is compiled and executed in order to find out defects if any Dynamic testing procedure is to be carried out meticulously Steps associated with dynamic testing are: Review work products Identify test objectives Identify test specification and carry out test design Design test cases Document test cases Execute test cases Generate incident report Log the defects

3. Review work products: when a software product is taken up for testing, one has to go through the work product details that have been prepared during the various stages of software development. Major work products that contribute the development of a software product are: the SRS document, the Design document, the Source code, the Draft user manual If any… Work products help in the development of test specifications The entity or object based on which you develop a test specification is termed as test basis. Accordingly, the test basis can be functional specification, a design strategy or a some part of source code that can be identified with work products. Therefore, quick overview of various work products pertaining to the software under test has to be made by the testers.

4. 3. Identify test objectives : once after going through the software’s work products thoroughly, it is equally important to identify test objective. First and foremost objective of testing is to verify whether the given software meets and satisfy all the functional requirements. The next step is to study the details of various quality requirements of the software product and list out as to what are all the quality parameters that needs to be tested. The quality attributes that are to be tested are: Conformance to a standard safety Portability reliability Performance External hardware/software interfaces. Usability Code coverage Security

5. Test specifications and test design: Test specification describes the overall requirements of testing the software product. “what are all to be tested?” These test specifications are obtained from the test basis and risk factors to be accommodated. Te test specifications can also be formulated based on answering to the following questions: Do non-functional tests have been conducted? What types? Do additional tests necessary to conduct even after black box and white box testing? Does structural testing required? Does conformance testing required? Any special requirements to be met? Does operational testing needed? If so how long to conduct beta testing? At how many places? Does acceptance testing needed? If so ……what type?

6. c) one has to prioritize the necessary tests that must be conducted d) test design procedure involve the following activities: Identify testing environment Identify testing instruments Identify the necessary automated software testing tools Formulate test case specifications and test data Generate test scripts Identify the requirements of the software, simulators, stubs to be written Workout test cases required for regression testing in view of pesticide paradox symptoms of test cases

7. Design test cases: the generation of the test case is the most crucial stage in testing and of course designing a test case is a creative task too For each and every test specification the test cases will have to be designed Test case design procedure involve the following activities: 1. identify test inputs 2. identify test data 3. identify entry criteria 4. identify exit criteria 5. formulate test procedure 6. write test scripts, if any 7. write drivers, stubs if any 8. identify the expected test results which include error messages if any.

8. The following 2 criteria are used to design test cases: Reliability: a set of test cases can be considered as reliable if they detect all errors and bugs. Validity: a set of test cases can be considered as valid if at least one test case reveals the errors and bugs Guidelines to be followed while designing test cases: Test case must be simple (one input and one output) Test case must reveal errors Test case must resemble the actual usage at software Use checklists for designing test cases Design separate test case for every quality parameter Make use of both valid and invalid inputs for test case design Test cases must not be redundant

9. Test case design techniques can be broadly classified into: Black box test case design techniques White box test case design techniques Experience based test case design techniques 5.1 black box test case design technique: is conducted to test and evaluate the functionality of the software. Test case design techniques concentrate on exactly how the end user make use of application to carry out activities with that software. But the test cases have to be designed and executed for both positive and negative testing The various methods used in black box test case design technique are: Specifications based testing Equivalence partitioning Boundary value analysis Decision tables Cause effect analysis State transition analysis/diagrams Use case scenarios

10. Specifications based testing : the name itself suggest that the test cases will have to be designed based on test specifications. For each of the specifications the input test data must be selected and the output must be cross checked. Ex: a program to compute the roots of the quadratic equation ax²+bx+c=0 The test specifications can be listed out as : a. read the values for a,b and c b. check if the roots are real or imaginary c. for real roots compute and print the roots d. for imaginary roots only print the message “ roots are imaginary” and program need not calculate roots. Test case 1: test input data: a=2 , b=4 , c=6 Test case 2: test input data: a=1 , b=8 , c=9 Test case 3: test input data: a=3 ,b=-2 , c=1 and so on…

11. Equivalence partitioning : here the test inputs are classified into valid and invalid inputs. For instance: to find a square root of a number. Test inputs can be chosen from two equivalence classes. a. set of positive integers leading to valid inputs b. set of negative integers leading to invalid inputs For instance: to check whether the 3 numbers form a triangle or not: Test inputs can be chosen from two equivalence classes. a. set of numbers that form a triangle (valid inputs) b. set of numbers that do not form a triangle (invalid inputs) A group of tests forms an equivalence class if: All the tests in a group are testing the same set of data If one of the tests of the group detects a bug or error then remaining tests of that group will also detect the bug. If one of the tests of that group did not detect a bug or error then remaining tests of that group will not detect the bug.

12. Executing a test case with multiple sets of data is termed as data-driven testing. The equivalence partitioning technique is used in testing of: Validating data inputs GUI inputs Data driven testing

13. Boundary value analysis: one has to identify valid or acceptable boundary values as upper limit and lower limit test data inputs for which the software under test is likely to fail. The programmer or developer can commit mistakes at extreme limits called boundary values. For instance: if salary field of employee is declared to be an int data type then it takes 2 bytes positive integer with a maximum integer value as 65535 One has to test by giving valid data with in boundary limit 65535 and invalid data input 65536 It is observed that whether data will be accepted or whether an appropriate error message will be displayed

14. 4. Decision tables: a. When a software under test consist a good number of conditional executions, then decision tables are useful means to verify the proper executions of all such conditions In such cases the test cases must be selected carefully such that – all the conditions must undergo testing at least once For instance: the chosen test case must verify both true-condition branching as well as false-condition branching to take place. d. For which decision tables must be constructed.

15. Cause effect analysis: in this technique, each input called cause there will be an identified output called action. For which a course-effect graph can be drawn. This graph can be converted into cause-effect table and this table can be used to generate necessary test cases Consider an ex: bank transactions I. withdrawing money from savings bank account using a cheque ii. The possible causes (inputs) and actions (outputs) of verifying the validity of cheque (authorization) and issuing money may include the following:

16. Causes(inputs) s/b account number is correct Customer signature on the cheque matches with that of bank’s record in the database Sufficient bank balance exist in that of s/b account …….. Actions(outputs) Issue requested amount to customer Inform police that it’s a fraud case since signature is forgery Convey the customer that enough bank balance does not exist in his s/b account ……

17. Therefore test cases will have to be framed for different combinations of inputs each with true or false status. And testing will have to carried out for the verification and confirmation of correct actions to be taken or not. Test case 1 a/c no is correct, signature matches, sufficient money exist in that account Test case 2 a/c no is correct, signature matches, sufficient money does not exist in that account Test case 3 a/c no is correct, signature does not match, sufficient money exist in that account Test case 4 a/c no is correct, signature does not match, sufficient money does not exist in that account Test case 5 a/c no is incorrect, signature matches, sufficient money exist in that account

18. State transition diagrams/analysis: are used to represent the different states of a system. a. when an input called “stimulus” is given to the system under test, it changes from one state to another state b. therefore, state transition diagrams help while formulating various test cases c. while testing, one has to ensure that each and every state is tested. Use case scenarios: suggest various actions that an end user of the software can take in sequence while operating that software a. test cases can be designed in such a way that all these user actions/responses are executed one after another and the expected results are obtained in one shot. b. whenever a customer submits his cheque in a bank counter for cash withdrawal, the bank clerk issues necessary commands to his computer to verify records from the bank database.

19. c. thus, validity of a cheque is confirmed by the software and cash will be issued to the customer accordingly. 5.2 white box test case design technique: emphasizes on “how the software under test is implemented…?” In particular to design the test cases this technique considers: The code at unit/component level The interconnections of components/units at module level The interconnection modules at sub systems at system level The interconnection of sub systems at system level. White box testing is difficult to conduct , it will explore all kinds of errors/defects of implementation level.

20. Some mechanisms of white box test case design techniques include: Code coverage Internal boundary value testing Structural testing at module level Structural testing at system level Code coverage: involve structural testing of: a. statement coverage b. condition coverage c. path coverage So the test cases must be designed to test the various constructs or control structures used in program code.

21. Internal boundary value testing: boundary values such as upper limits, lower limits must be mentioned and cross checked whether the software satisfy boundary limits or not. Example: for (k=0 ; k<=100 ; ++k) { ++count up ; ratio [k] = load – factor / count up; } Here the loop starts with k=0 and ends up with k< or =100. The loop will be executed from 0 to 100 means – 101 times since the condition k<=100 must be satisfied. 3. Structural testing at module level: modules are formed by integrating units or components and testing will be carried out on such functional modules.

22. Lets consider a bank account class: RD for recurring deposits SB for saving bank deposits CD for current deposits To test SBI account module: various account details such as RD,SB, CD (a/c no) along with customer details, bank deposits, etc will have to be given and cross check the connectivity of methods/ functions belonging to units/components such as RD ACCOUNT, SB ACCOUNT, CD ACCOUNT. SBI ACCOUNT RD ACCOUNT SB ACCOUNT CD ACCOUNT

23. 4. Structural testing at system level: the different modules are interconnected to form sub-systems and in turn into one complete fully functional Test cases will have to be designed here to check module interfaces and communication links across sub-systems for testing smooth functionality. 5.3 Experience-based test case design technique: indicates that growing experience in testing could reveal where exactly programmers and requirement analyst could commit mistakes. a. test cases will have to be designed in such a way that: b. they are 2 methods that can be used in experience-based test case 1. error guessing 2. exploratory testing

24. Error guessing: with growing years of testing experience that brings about the testing professional can guess errors or mistakes being committed by programmers at various stages of: Writing the code Design and implementation of functionality The relevant test cases can be formulated based on this expert guessing Error guessing is simple and short cut method based on which test cases can bring out errors/bugs in a very short interval of time. 2. Exploratory testing: testing carried out without the use of pre defined test cases in termed Exploratory testing. Test cases will be designed and executed in real-time. New test cases must be planned instantly afresh and trial out the execution to detect an error If such test cases succeeded to explore a defect then the corresponding test case must be documented.

25. iv. Scripted testing: testing carried out with the use of pre-defined test cases is termed scripted testing. 5.4 case study #1: test case for IVR system. IVR (interactive voice response) system can be used to know the arrival and departure timings of the trains User has to dial an IVR number and mention train number IVR system consist of both hardware and software and an associated software that responds to user requirements. IVR has a two line capacity and hence it should respond to two user simultaneously without any confusion or break. The below table has a list of different kinds of tests to be conducted on IVR system along with sample test cases:

26. Type of test to be conducted Functional testing(black box) Structural testing(white box) Gorilla testing Performance testing 5. Stress testing Description of test cases Test IVR system for a. Valid train numbers b. Invalid train numbers Draw the menu structure of IVR system and test all possible paths Dial the IVR phone number and key-in train number and now check whether random or wrong inputs are accepted with error message? Or system hanged? Test IVR system when two users access or enquire simultaneously: land lines, cell phones, one cell phone and one land line Test IVR system with 3 users accessing or dialing simultaneously: here 2 users must get IVR response and 3 rd user must listen to an ‘engage’ tone.

27. Reliability testing 7. Security testing Experience based testing Conformance testing test IVR system round the clock 24*7. switch off the IVR system for some time and switch it on and test the system for proper functionality Test the IVR system whether is is prone to security violation such as: “denial-of-service act”. A. It happens when 2 users dial IVR system and keep it engaged (ON) without keying train numbers. B. So the 3 rd user may not get a chance to avail this facility, but the software should quickly recover and get disconnected automatically Test IVR system should accept train no keying simultaneously as and when playing voice messages as its first response. play arrival and departure timings.IVR system should not respond to dealings from an old telephone set. Test IVR system whether it meets the standards put forth by telecommunication authorities

28. 5.5 case study #2: test cases for finger print recognition system. Example: A “thumb scanner” at card punching/swapping system to record reporting time for the job at various organizations. At the entry point, the finger print of the thumb or palm of the employee will be scanned and compared with the already recorded finger print database at computer’s hard disk In general, finger print recognition of the thumb is used to check the genuinity or authentication of the employees concerned to that work or to that organization. The test cases that can be used to test finger print recognition system are: Test the system for both valid and invalid entries: For unauthorized persons For authorized persons test the system performance by verifying as how many time it takes: to allow authorized persons to reject unauthorized persons

29. Test the system accuracy by verifying as how many in percentage: it allowed un authorized persons it disallowed authorized persons Test the system tolerance by verifying whether it identifies the authorized person: especially when ink is applied to fingers or when fingers are snubbed with wax or polish Test the system security by verifying password screens to access finger print database Test the “system overriding” feature as to what is the method to cancel or suspend the decision of the system in case if it has rejected an authorized person.

30. Document test cases: the test case design document must contain every details of all the test cases designed so far …to test a particular software product. The test case design document has to be kept strictly under configuration management and control When documenting the results – all these details must be updated with actual results i.e.; whether test passed-ok or test failed plus other adjustment/confusion if any are all to be recorded while documenting each test case must contain the following details: Author name Test case id Test case specification Test case design methods/techniques Functionality to be tested Test setup, test basis, test procedure Test pre-condition, test post-condition Test pass, fail criteria Expected test results Revision history Test case environment

31. Execute test cases: a test procedure is to be formulated in order to execute test cases in case of manual testing The actual test results will be compared with expected results and the test passed or failed will be declared based on this comparisons and pass-fail criteria Automated testing tools are used then test case execution is simple and straight forward The testing tool generates a test script automatically and this script can be executed for executing test case. This approach is useful and proven to be efficient in case of regression testing wherein the same test case is repeatedly executed Testing tools also provide useful information such as how long it takes to execute the test case and when this test was executed Blocked test case: a test case that cannot executed because of non-compliance of pre-conditions is termed as blocked test case.

32. 8. Generate incident report/Anomaly report: its generated to record the irregularities or deviations noticed during the execution of a test case or during test checking. Suppose, after executing a test case If the actual output does not match the expected output then it is an incident that needs to be investigated further for possible confusions/irregularities Such incident report is also required to be produced while carrying out the desk checking of different work products. The anomaly report must contain the following items of information: the incident anomaly noticed date. Name of the person/client/organization who discovered anomaly Name of the development phase in which anomaly was noticed Name of work product/test case that revealed anomaly Anomaly descriptions, severity of anomaly Expected result, actual result, present status. Status change history Priority assignment to investigate the anomaly and/or fix the problem Anomaly rectification date and closure.

33. Log the defects: the defects encountered during the execution of test cases must be recorded in a defect log. The defects log sheets will be maintained for every software development project using either MS-Excel or defect tracking software tool Person who detect defects will have to enter the defect log sheet. Further the test manager will verify the defect log to assign severity level and priority for rectification of the defects Now the debugging job called “defect removal work” is entrusted to developers and the status of defect will change as and when developer starts debugging. 10. Test documentation standard: