Challenges of cultural adaptation and translation of tests..pptx

Challenges of cultural adaptation

1. Challenges of cultural adaptation and translations of test

2. Cultural adaptation and translation of tests Psychometric tests are a common denominator across different fields in psychology Globalisation , however, is continuing to change assessment dynamics, making the test usage increasingly complicated. Researchers and practitioners from many fields across the globe are venturing into test development, or transferring existing tests developed for a particular language. In such a global economy, it is essential to have common grounds according to which individuals from different cultural backgrounds can be compared ( Daouk , Rust & McDowall, 2005), and guidelines that regulate the cross-cultural application of tests have become increasingly important (e.g., the International Test Commission (ITC), the European Federation of Psychologists’ Associations (EFPA)).

3. Test Translation vs. Test Adaptation Test translation  is a linguistically-focused process. It is used to create an instrument in one language that is linguistically equivalent to a testing instrument in another language. The skills required are linguistic in nature and performed by a translator, often with the assistance of a subject matter expert. Test translations are performed by a single translator or a team of translators who work with one or more subject matter experts. Thorough translation methods help reduce bias and enhance equivalence of multilingual versions of a test. Test translation always concerns two languages, but that may or may not be the case with test adaptation. Test adaptation, on the other hand, is a culturally-focused process. It is used to create an instrument that is culturally, linguistically and/or psychometrically equivalent to an original testing instrument. In this case, a broader range of skills are required, often requiring multiple professionals such as specialized translators, cultural experts, subject matter experts and psychometricians. Combining test translation and test adaptation in this way can help to ensure valid and fair assessments for all.

4. Types of Adaptation Construct Adaptation Testing instruments can be used to measure different types of knowledge, experiences and abilities. However, test takers must have a common understanding of what it is that will be measured. Construct adaptation ensures that a test’s results will be meaningful. Language Adaptation Languages are constructed and used differently. A common word in one language may not even exist in another language. Or a common grammatical structure in one language may not be common or appropriate in another language, or may even reveal too much information to the test takers. Language adaptation requires creative problem solving to ensure that the right meaning and intention are understandable and correct in all versions of a testing instrument. Translating an existing instrument to measure a phenomenon in people who speak another language.

5. Cultural Adaptation Different  cultures have different values and practices. What may be normal or usual in one culture may not be in another. Cultural adaptation levels the playing field for target culture test takers by making culturally-informed changes to ensure that they are tested on what an item is actually testing for and are not inadvertently derailed by cultural differences. Adapting an existing instrument to measure a phenomenon in a different culture. Familiarity Adaptation Different groups have different experiences and they may be more or less familiar with different ideas, practices and representations. Familiarity adaptation ensures that test takers are not unfairly burdened by being asked to recognize or do something that the original test assumes test takers already know or can do.

6. The guidelines identified characteristics of an appropriate translator: · An excellent knowledge of the target language · Experience in both languages and cultures · Experience with the target populations · Skills in test development. Verifying the translations Multiple-forward translation , Back-translation, Translation review by bilingual judges. Cultural and linguistic adaptation measures can enable inclusion of a broader study population and enhanced generalizability of results. However, if a measure has not been adapted or translated appropriately, then the population may not understand what is being asked of them. Therefore, the data collected from the measure may not accurately reflect the underlying construct(s) of interest( Gjersing et al. 2010). Test translation and test adaptation are economical ways of developing a new assessment. Combining test translation and test adaptation in this way can help to ensure valid and fair assessments for al l.

7. Stages of Translation Process and the Adaptation of Instruments

8. Instrument Translation into the New Language When adapting an instrument, the first step is its translation from the source language into the target language. It is a elaborate process and requires tremendous care to ensure that the final version is suitable for the new context and is also consistent with the original version. Previous literature highlights the need to avoid the literal translation of items (Hambleton, 1994, 2005) as it often results in incoherent statements or rather limited target language fluency.

9. Synthesis of the Translated Version The process of summarizing both version begins at this stage. The researches compares the different translations and assess their semantic, idiomatic, conceptual, linguistic and contextual difference with the aim of creating a single version. Inappropriate choices are identified and resolved though discussions among the judges and researchers responsible for adapting the instrument.

10. Evaluation of the Synthesized Version by Experts After the synthesis of translated versions, it is evaluated by experts in the area of psychological evaluation or those with specific knowledge of what the instrument assess. The experts assess other important aspects such as the structure, layout, instrument instructions, and the scope and adequacy of expression contained in the items. Translation, synthesis and evaluation are the first steps in the instrument’s adaptation into a new culture, of the translated version.

11. Back-Translation It is also suggested as an added quality control check ( Sireci et al., 2006). Back-translation refers to the translation of the synthesized and revised versions of the instrument into the source language. Its objective is to evaluate the extent to which the translated version reflects the item content of the original version.

12. Pilot Study One must perform a pilot study before claiming that a new instrument is ready for application. Pilot study refers to a previous application of the instrument in a small sample that reflects the sample/target population characteristics (Gudmundsson, 2009). During this process, the appropriateness of items regarding their meaning and difficulty, and instructions for conducting the test should be assessed. After considering the modifications suggested in the first pilot study, a second pilot study is recommended to assess whether the instrument is ready to be used.

13. Importance/Advantages of Cultural Adaptation of Tests

14. There is a growing interest in cross-cultural research and evaluation. The result of this interest is a boon in the call for multi-lingual versions of assessment instruments. Examples of psychological assessment available in various languages include, Wechsler Intelligence Scale for Children, the Children’s Hope Scale, the Sixteen personality Factor (16 PF) Questionnaire, Miller’s Analogy Test (MAT), the Wechsler Adult Intelligence Scale (WAIS) etc.

15. The process of adapting an existing instrument, rather than developing a new one, has considerable advantages. Comparison- by adapting an instrument, the researchers is able to compare data from different samples and backgrounds, which enables greater fairness in the evaluation as the same instrument assesses the construct. Generalization- use of adapted instruments enables a greater ability to generalize and also aids one to investigate differences within an increasingly diverse population (Hambleton, 2005; Vivas , 1999).

16. Challenges of Cultural adaptation and translations of test

17. C oncept of equivalence: The goal of test adaptation and translation is to create a test that Is equivalent to the original one. Equivalence is as important as reliability and construct validity. When a test is considered equivalent with another, two important implications arise. First, test scores derived from the two equivalent measures can be directly compared (at the level of equivalence they reflect. Second, relevant knowledge and evidence generated based on the first measure is also valid for the second measure. Lack of equivalence can be the result of bias in a number of components of the test. three sources of biases: Item bias Metho d bias construct bias,

18. Item bias Item bias refers specifically to threats that affect specific items of the test. The most common cause is poor item translation. The sources of item biases are: Linguistic bias : This type of bias can result from mistranslation, inappropriate use of wording, or even mistakes in the translation of idioms. In an example by Hambleton (1996), the question ‘Where is a bird with webbed feet most likely to live?’ proved to function differently with a Swedish sample than with most other European samples. The term ‘webbed feet’ was translated to ‘swimming feet’ in Swed- ish, which rendered the item easier in the Swedish version than the English one. Therefore members of one group (Swedish), with the same overall ability on the test as their counter- parts in the other group (English), were more likely to answer this question correctly

19. Psychological bias : This type of bias relates to situations in which the psychological impact of the item is not the same in the two or more target cultures. For example, emotions could have different intensity across cultures, which make respondents in different cultures exhibit stronger or weaker psychological reactions towards an item   Conceptual bias, also referred to as cultural bias, is the relevance of the item content to the target culture. That is, the concepts covered by a certain item need to be meaningful in the target culture. F. M. Cheung (2004) provides an example from the MMPI-2 that illustrates this source of bias. The item ‘I used to like to play hopscotch and jump rope’, even if well translated, will present a problem in cultures where hopscotch and jump rope are not common as children’s games.  

20. Method bias Method bias is related not to the conceptual development of the questionnaire but to the instrument itself, the data collection process, or the characteristics of the sample. I nstrument bias Instrument bias is associated with characteristics that can affect candidates' scores but that relate to the measurement tool rather than the characteristic being measured. sources: Familiarity with the response format : Psychometrics tests differ in the response format they employ (such as the Likert scale, multiple-choice pictorial items, or open-ended questions). However, certain cultures may be more familiar with one type of response format than others. In some countries, for example, questionnaires are only filled out for governmental or legal purposes (Fife-Schaw, 2006).

21. Response style: there is evidence that supports the existence of cultural differences in the style of responding to psychometric tests. For example, Extreme Response Style (ERS) is the tendency to endorse extreme answer options rather than middle ones. For example, questionnaires with a Likert scale from 1 to 5, participants who belong to groups high on ERS are likely to choose 1 and 5 and avoid the middle ones. Low ERS participants are likely to cluster around the middle (2, 3 and 4), whereas no ERS participants’ scores are likely to be spread on all the scales. Administration bias Administration bias is associated with environmental or communicational differences in administering the tests that affect the scores of the comparison groups differently.  Various sources of administration bias  are  Test instructions : Tests typically include a set of standardised instructions to be read during administration, as well as advice on best practices in administration situations. Any instructions made by the test administrator that do not follow these recommendations can potentially bias test takers’ responses either negatively or positively and result in error.

22. Differences in language: The test is given in a language that the participants do not speak very well, or the test administrator does not speak the language of the participants very well. Differences in the observance of target culture: The test administrator does not possess enough knowledge of the target culture Differences in technological administration conditions : The technology used in the target culture for administration is different than in the source culture or unfamiliar for the test takers Differential expertise of test administrators. The professionals administering the test in the source and target cultures have different levels of expertise in test administration. Sample bias Sample bias is a potentially dangerous type of bias, because it does not necessarily affect the equivalence between multilingual versions, but may affect the validity of the inferences that test users make from them.

23. Sources: Sample convenience and snowballing:  There are two systematic methods of sampling that maximise the likelihood of a sample resembling the general population: random sampling and matching many researchers tend to rely on samples of convenience and snowballing techniques as the most convenient sources of data which doesn’t represent the population Digital divide Collecting data electronically could also produce another example of sample bias if it results in favouring a subgroup of the general population that has access to computers. Self-selection bias : Pelham (1999) argues that ‘people who choose to answer surveys are systematically dif- ferent from people who choose not to do so’

24. Construct bias Such bias occurs when constructs being measured are not equivalent between given culturesNonequivalence may appear in construct-related issues if the construct that was initially targeted by the original version of the test does not exist in the same way in the target culture of the adaptation process. For example, the construct may not exist at all in the target culture, i.e., there is a complete lack of overlap of the construct between source and target culture. Differential construct manifestation : construct bias could result from the fact that the construct, although it exists in both cultures, is defined and exhibited differently in each. Differences in the symptoms of depression between Eastern and Western cultures are a good example of this concept. Construct under-representation : This source of construct bias is characterised by insufficient sampling of the behaviours that explain the construct (Messick, 1995). Generally, for constructs to be under-rep- resented, either the original test is too short to make valid deductions from or the items are too badly written to tap into the construct it is claiming to measure