Validity of test

It will be helpful to undergraduate and post graduate students of Education and Psychology

1. Validity: Meaning, M ethods and Factors A ffecting V alidity of Test Dr. Sarat Kumar Rout Assist. Prof. Department of Education Ravenshaw University, Cuttack Email:[email protected]

2. What is Validity? Validity is the extent to which a test measures what it claims to measure . It is vital for a test to be valid in order for the results to be accurately applied and interpreted. http :// psychology.about.com/od/researchmethods/f/validity.htm The extent to which a test adequately represents the subject matter content or behavior to be measured commonly used in evaluating achievement or proficiency tests. http:// www.education.com/definition/content-validity

3. Content Validity If an assessment encompasses wide range or large proportion of syllabus or content from a book or area that will fall in the domain of content validity. When a test has content validity , the items on the test represent the entire range of possible items the test should cover . Individual test questions may be drawn from a large pool of items that cover a broad range of topics. http://psychology.about.com/od/researchmethods/f/validity.htm Content Validity is called appropriateness of the test ; that is appropriateness of the sample and the learning level . Content Validity is the most important type of validity which can be achieved through a careful examination of the test content . It provides the most useful subjective information about the appropriateness of the test .

4. Contd. Content validity • Example – Social Science Exam ; – Assume it contains all aspects of the Social Science (History, Political Science, Economics, Geography etc.,) If a person scores high on this test, we can ‘ infer’ that he knows much about the subject (i.e., our inferences about the person will right across various situations ). In contrast, if the exam did not contain anything about Political Science, the inferences we make about a high scorer may be wrong most of the time and vice versa.

5. Contd. Generally, a measures/scores that includes a more representative sample of the target behaviour will have more content validity and hence lead to more accurate inferences. Reliability places an upper limit on validity (the maximum validity is the square root of reliability coefficient.) the higher the reliability the higher the maximum possible validity . Example r tt =.79, validity index will be maximum=.88 One exception is that between internal consistency and validity (better to sacrifice IC to content validity) – The ultimate aim of test is inferential which depends more on content validity than internal consistency.

6. Formula for distribution marks Formula for allocation of marks to different units of a subject: = Total No. Pages in Unit/Total No. Pages of the books X 100 Formula for allocation of marks to subjective/objective type questions = (% weightage to subjective/objective type questions)⁄100 X Total No. of Marks Per Unit

7. Steps for content validation Step-I: Formulation of objectives Step-II: Specification of content, types of questions Step-III: Preparation of the test items Step-IV: Expert validation/Statistical verification

8. Marks allocation-Example Using content validity sheet a teacher can save the time and energy while developing exam papers. S. No. Chapter/Unit No. of Pages Allocated Marks Subjective (60 %) Objective (40 %) % Total Marks (%) 01 Rise of Nationalism in India 23 23/125X100=18 (18 % or 18 marks out of 100) 60/100X 18 =11 40/100 X 18 =7 18

9. 2. Criterion Validity Criterion-related validity investigates the extent of correspondence/agreement between the scores obtained from the newly-developed test and the scores obtained on an independent outside criterion available at present or available in the future . Two types Concurrent validity Predictive validity

10. Contd. Concurrent validity In order to find out concurrent validity both the newly-developed test and independent outside criterion measure will be administered to the same group at the same time. The extent of agreement/correspondence will be calculated by applying product moment coefficient correlation or discrimination. Example: scores obtained on a newly constructed achievement test on English Language of Class-X may be correlated with scores obtained on an already standardized achievement test on English Language. The resulting coefficient of correlation will be an indicator of concurrent validity.

11. Contd. Steps for concurrent validation Step-I: The newly constructed test will be administered to a defined group of individuals/learner. Step-II: The independent outside criterion measure/test will be administered to the same group at the same time. Step-III: The two sets of scores are correlated. Step-IV: The resulting coefficient indicates the concurrent validity. Concurrent validation is most suitable to tests meant for diagnosis of the present status rather than for prediction of future outcomes.

12. Contd. 2. Predictive validity In order to find out predictive validity of a newly constructed test, the test constructer will administered it on a defined group of individuals/learners and then a time gap of months or year will be allowed and subsequently, independent outside criterion measures/scores will be obtained by administering a test to the same group. The extent of agreement/correspondence between two sets of measures/scores will be calculated by applying product moment coefficient correlation or discrimination. Example: Predictive validity of entrance test for college admission to assess if a person will graduate or not.

13. Contd. Steps for concurrent validation Step-I: The newly constructed test will be administered to a defined group of individuals/learner. Step-II: The independent outside criterion measure/scores will be obtained from the same group after a gap (month/year). Step-III: The two sets of scores are correlated Step-IV: The resulting coefficient indicates the predictive validity. Predictive validity is needed for tests meant for long range forecast of academic achievement , vocational success.

14. Contd. A comparative study of predictive validity and the concurrent validity has revealed that for the same test predictive validity is usually lower than the concurrent validity . The degree of agreement between the test and criterion decrease over time. Example: The relationship between Academic aptitude test and the present achievement of learners. Test constructer face problem in selection of appropriate and adequate criterion which is more acute in predictive validity than the concurrent validity of a test.

15. Construct Validity Construct validation is a more complex and difficult process than the content and criterion validation. Hence the investigator decides to compute construct validity only when he is fully satisfied that neither any valid and reliable criterion is available to him. Other names of the construct validity factorial validity or trait validity. “The extent which the test may be said to measure a theoretical construct ”. What is a construct ? A construct is non observable trait ( anxiety, pain, intelligence ).

16. Contd. Steps for validation of the construct Step-I: Specifying the possible different measures of the construct. Step-II: Determining the extent of correlation between all or some of those measures of the construct. Step-III: Determining whether or not al or some measures act as if they were measuring the construct.

17. Steps for validation of the construct Step-I: Specifying the possible different measures of the construct . Example: Intelligence as a ‘Construct’ Possible different measures of Intelligence are Quick decision taking ability Ability to operate with symbols Ability to learn Capacity to modify behavior Verbal fluency Ability to comprehend verbal material . Original and critical thinking etc.

18. Contd. Step-II: Determining the extent of correlation between all or some of those measures of the construct. Empirical investigation to find out correlation to find out the extent to which various measures are correlated with each other. Some measures may correlate with each other and form cluster. As 1, 4 and 5 may be a cluster and 2, 6 and 7 like wise. All measures of a construct may highly correlate with each other. All measure may correlate with each other poorly.

19. Contd. Step-III : Determining whether or not all or some measures act as if they were measuring the construct . Now to determine whether or not such measures behave with reference to other variables of interest in an expected manner. For example, highly correlating measures from among the above seven supposed measures for intelligence should show at least some moderated correlation with teachers ratings, grades in class and examination marks.

20. Contd. Content and criterion can be established in one or two studies, but there is no single experiment that can prove a construct. Construct validation is an ongoing process, learning more about the construct, making new predictions and then testing them. Each supportive study strengthens the construct but one well designed negative study can question the entire construct.

21. Factors affecting Validity 1. Length of the Test: (Homogeneous Vs Heterogeneous) Formula for estimating validity by lengthening the test as r c ( nx ) =(n) ( r cx )/√ n+n (n-1) r tt r c ( nx ) = correlation between criterion and the test length ‘n’ times. n =number of times test has been lengthened. r cx = correlation between criterion and the test in original length. r tt =reliability coefficient of the test. 2. Range of ability 3. Ambiguous directions 4. Socio-cultural differences 5. Addition of inappropriate items

22. Relationship of validity to reliability Validity is the correlation of the test with some outside independent criteria and reliability of the test is the self-correlation of the test A test which is not correlating with itself is not expected to correlate with out side independent criteria. Thus validity depends on reliability.-For homogeneous test.

23. Contd. In heterogeneous test validity may be higher than the reliability. Thus reliability is necessary but not sufficient condition for validity-Example Achievement test in Social science having different sections of question from History, pol. Science, geography and economics . Theoretically, maximum validity coefficient between two variables is equal to the square root of the product of their reliabilities . A test constructor should not always aim at high reliability and high validity from the same test.

24. Thank you one and all