PRESENTATION ON ORTHOGONAL ARRAY TESTING SUBMITTED TO: DR. JAGTAR SINGH SUBMITTED BY: ATUL RANJAN KAMINI SINGH UTTAM KUMAR VIPIN Kr. SINGH
Contents: Introduction. Orthogonal Array. Terminology of OATS. Why OATS..?? Conventional testing issues. Example Application Advantage & disadvantage Degree of freedom
Introduction: Orthogonal Arrays (often referred to Taguchi Methods) are often employed in industrial experiments to study the effect of several control factors. Popularized by G. Taguchi. Other Taguchi contributions include: Model of the Engineering Design Process Robust Design Principle Efforts to push quality upstream into the engineering design process.
Orthogonal Array Theoretically- An orthogonal array is a type of experiment where the columns for the independent variables are “orthogonal” to one another . Analytically- An N x k array A with entries from some set S with v levels, strength t within the range 0 ≤ t ≤ k and index λ where every N x t sub array of A contains each t-tuple based on S exactly λ times as a row.
Why Orthogonal Array Testing (OATS)..?? Systematic, statistical way to test pair-wise interactions. Interactions and integration points are a major source of defects. Most defects arise from simple pair-wise interactions. “When the background is blue and the font is Arial and the layout has menus on the right and the images are large and it’s a Thursday then the tables don’t line up properly.” Exhaustive testing is impossible. Execute a well-defined, concise set of tests that are likely to uncover most (not all) bugs. Orthogonal approach guarantees the pair-wise coverage of all variables. 5
Taguchi OATS Approach
For OA, Must be identify : 1. Number of factors(K ) to be studied. 2. Levels(V ) for each factor 3. The specified Strength (t) 4. The special difficulties that would be encountered in running the experiment
Terminology for working with OA’s OA’s are commonly represented as : OA(Runs, Factors, Levels, Strength) OA λ (Runs(N); Factors(k), Levels(v), Strength(t)) is an N × k array on v symbols such that every N × t sub-array contains all tuples of size t from v symbols exactly λ times. Runs (N) – Number of rows in the array, which translates into the number of Test Cases that will be generated. Factors (k) – Number of columns in the array, which translates into the maximum number of variables that can be handled by the array. 8
Continued.. Levels (v) – Maximum number of values that can be taken on by any single factor. Strength (t) – The number of columns it takes to see all the possibilities equal number of times. No of runs= λ v t t is the strength, v is the number of levels λ -1 for software testing and is often omitted
Conventional Testing Issues: Conventional Test Cases: Variables:3 Input: 3 Possible cases: 27 =3 3 Variables: 3 Input: 5 Possible Cases: 243 = 3 5 … Variables: 5 Input: 5 Possible Cases: 3125 = 5 5 10 Orthogonal Test Cases Variables:3 Input: 3 Possible cases: 9 Variables: 3 Input: 5 Possible Cases: 11 … Variables: 5 Input: 5 Possible Cases: 21
OATS - Example A B C 1 1 1 3 2 1 2 2 3 1 3 1 4 2 1 2 5 2 2 1 6 2 3 3 7 3 1 1 8 3 2 3 9 3 3 2 TABLE1 Sample Array using OA Total cases = 9 which cover all pair-wise combinations of the 3 variables. Applying OATS 3 Parameters – A,B,C 3 Values – 1,2,3 All possible cases involving 3 parameters: 3*3*3 = 27 cases 11
Conventional Test Cases Example: If we have three variables (A,B,C), each can have 3 values say (Red, Green, and Blue). The possible combinations in conventional test cases would be 27 i.e. 3 3 12
By OATS: Example: If we have three variables (A,B,C), each can have 3 values say (Red, Green, and Blue). The possible combinations in OATS test cases would be 9. 13
Example 14 A Web Page has three distinct sections (Top, Middle, Bottom) that can be individually shown or hidden from user No.of Factors=3 ( Top,middle,Bottom ) No.of Levels =2 (Hidden or shown) Array Type = OA(4,3,2,2 ) If we go for exhaustive testing we need : 2 x 2 x 2 = 8 Test Cases OA(Runs , Factors, Levels, Strength)
contd.. 15 Fixed Level Array: L 4 (2 3 ) F1 F2 F3 Run1 Run 2 1 1 Run 3 1 1 Run 4 1 1 L 4 2 3- OA with 4 Runs 3 factors with 2 levels Top Middle Bottom Test 1 Hidden Hidden Hidden Test 2 Hidden Visible Visible Test 3 Visible Hidden Visible Test 4 Visible Visible Hidden The Four Test Scenarios (4 Vs. 8) 1 - Display the home page and hide all sections. 2 - Display the home page and show all except Top section. 3 - Display the home page and show all except Middle section. 4 - Display the home page and show all except Bottom section.
APPLICATIONS OF OA They are essential in statistics and they are used in computer science and cryptography. Orthogonal array are used in automobile design. They are immensely important in all areas of human investigation: for example in medicine , agriculture and manufacturing . orthogonal arrays are related to combinatorics , finite fields, geometry and error-correcting codes.
OATS advantage to select a test set: Guarantees testing the pair-wise combinations of all the selected variables. Creates an efficient and concise test set with many fewer test cases than testing all combinations of all variables. Creates a test set that has an even distribution of all pair-wise combinations. Exercises some of the complex combinations of all the variables. Is simpler to generate and less error prone than test sets created by hand. 17
Disadvantages of OATS: Can only be applied at the initial stage of the product/process design system. Arrays can be difficult to construct. With so many possible combinations of components or settings, it is easy to miss one. It covers 100% (9 of 9) of the pair-wise combinations, 33% (9 of 27) of the three-way combinations and 11% (9 of 81) of the four-way combinations.
Degrees of Freedom The number of degrees of freedom is very important value because it determines the minimum number of treatment conditions. It is equal to: Number of factors(no. of levels-1)+no. of interactions( no.of levels-1)( no.of levels-1)+one of the average.
EXAMPLE: Given four two level factors A,B,C,D and two suspected interactions, BC and CD,determine the degree of freedom,df . What is the answer if the factors are the three levels 1: df =4(2-1)+2(2-1)(2-1)+1=7 (for two level) 2:df=4(3-1)+2(3-1)(3-1)+1=17 (for three levels)