Language assessment principles and class

3rd edition by h.douglas brown.

1. Language Testing & Assessment
Program: TEFL, BA
Session 5
Preliminaries (Continued)
Instructor: F. Behzadpoor
Azarbaijan Shahid Madani University
Spring, 1401

2. Backwash effect(also washbackeffect)
Theeffectofthenatureofatestonteachingandlearning
Characterizedintermsof impact,andincludesthepotentialimpactontesttakersandtheir
characteristicsonteachingandlearningactivities,andoneducationalsystemandsociety
Inlarge-scaleassessment,washbackgenerallyreferstotheeffectsthetestshaveoninstructioninterms
ofhowstudentsprepareforthetest.“Cram”coursesand“teachingtothetest”areexamplesofsuch
washback
Anotherformofwashbackthatoccursmoreinclassroomassessmentistheinformationthat
washes
back
tostudentsintheformofusefuldiagnosesofstrengthsandweaknesses.Students’incorrect
responsescanbecomewindowsofinsightintofurtherwork.Theircorrectresponsesneedtobepraised,
especiallywhentheyrepresentaccomplishmentsinastudent’sinterlanguage.

3. Backwash effect(also washbackeffect)
Washbackpromotesanumberofbasicvariablesinlanguageacquisition: intrinsic
motivation,autonomy,self-confidence,languageego,interlanguage,andstrategic
investment
Washbackcanvaryalongtwodimensions:intermsof kind(positiveornegative)andin
termsofdegree(fromstrongtoweak)
Thewashbackeffectisobviouslymuchstrongerin highstakessituationsthaninlow
stakessituations.
Teachersmightreasonablywanttodeterminethetypeofwashbackthatflowsfroma
giventest

4. Backwash effect(also washbackeffect)
Sevenothervariablesthatmayaffectthe kindanddegreeofwashback:prestige,
accuracy,transparency,utility,monopoly,anxiety,andpracticality
AldersonandHamp-Lyons(1996)arguedthatthe amountandtypeofwashbackdepend
on:
Thestatusofthetest(andthelevelofthestakesinvolved)
Thedegreetowhichthetestiscountertocurrentteachingpractices
Whatteachersandtextbookwritersthinkareappropriatetestpreparationmethods
Andhowmuchteachersandtextbookwritersarewillingandabletoinnovate

5. Backwash effect(also washbackeffect)
Typesofbackwash

Negative(orharmful)Washback
Iftestandtestingtechniquesareatvariancewiththeobjectivesofthecourse
Negativewashbackissaidtooccurwhentestitemsarebasedonanoutdatedview
oflanguagewhichbearslittlerelationshiptotheteachingcurriculum

6. Backwash effect(also washbackeffect)
Positive (or beneficial) washback:
Is said to result when a testing procedure encourages “good” teaching practice
Example:The consequences of many reading comprehension tests is a possible development of the reading skills
Note:When the testing procedure is too much in advance of the teaching method, it is unlikely that positive washback
will occur because teachers and learners may not understand what the test items require of them.
Informal performance assessment is by nature more likely to have built-in washbackeffects because the teacher is
usually providing interactive feedback
Formal tests can also have positive washback, but they provide if the students receive a simple letter grade or a single
overall numerical score

7. Backwash effect(also washbackeffect)
How to Achieve Beneficial Washback?
Test abilities whose development you want to encourage
Sample widely and unpredictably
Use direct testing
Make testing criterion-referenced
Base achievement tests on objectives
Ensure test is known and understood by students and teachers
Where necessary provide assistance to teachers

8. Backwash effect(also washbackeffect)
Other definitions and names:
Theconnectionsbetweentestingandlearning
Theinfluenceoftestingonteachingandlearning
Theutilizationofexternallanguageteststoaffectanddriveforeignlanguagelearning...thisphenomenonistheresultof
thestrongauthorityofexternaltestingandthemajorimpactithasonthelivesoftesttakers
Washbackincludestheeffectsofanassessmentonteachingandlearningpriortotheassessmentitself,thatis,onpreparation
fortheassessment
Thereareinstancesoflanguageteachingexpertscallingtheconcept testimpact,measurement-driveninstruction,
curriculumalignment,testfeedback,andeven(injest)bogwash

9. Backwash effect(also washbackeffect)
Impactvs.Washback:
Whileimpactreferstoanyoftheeffectsthatatestmayhaveonindividuals,policies,
andpractices,withintheclassroom,theschool,theeducationalsystem,orsocietyasawhole,washbackrefers
totheeffectsoftestsonteachingandlearning.
Washbackisadimensionof
consequentialvalidity.
Washbackeffectandtestingbiasareamongtheissuesthatariseintalkingabouttestvalidity
,
standards
,
politics
,etc.
Thephenomenon(washback)isreferredtoas ‘backwash’ingeneraleducationcircles,butithascometobe
knownas‘washback’inBritishappliedlinguistics

10. Authenticity
Degree of correspondence between test tasks, and activities of Target Language Use (TLU) or
Authenticity is the extent to which the tasks required on a given test are similarto normal real-life language
use ►
Situational Authenticity
→ The higher the correspondence, the more authentic the test
Note: Essentially, when we make a claim for authenticity in a test task, you are saying that this task is likely to be done in the
“real world”
Authenticityisalsodefinedasafunctionofinteractionbetweenthe testtakerandthetesttask►
InteractionalAuthenticity

11. Authenticity
Note:
Lateron,BachmanandPalmer(1996)separatedthenotionofauthenticityfrominteractiveness,defining
authenticityasthedegreeofcorrespondenceofthecharacteristicsofagivenlanguagetasktothefeaturesofa
TLUtask–thesameasthesituationalauthenticity.Theyalsoreplacedinteractivenessforinteractionalauthenticity.
In a test, authenticity may be present in the following ways:
The language in the test is as natural as possible
Items are contextualized rather than isolated
Topics are meaningful (relevant, interesting) for the learner
Some thematic organization to items is provided, such as through a story line or episode
Tasks represent, or closely approximate, real-world tasks

12. Test-wisenes
Atesttaker’scapacitytoutilizethecharacteristicsandformatsofthetestand/orthetest
takingsituationtoguessthecorrectanswerandhencereceiveahighscore
Itincludesavarietyofgeneralstrategiessuchasconsciouspacingofone’stime,reading
questionsbeforethepassageuponwhichtheyarebased,andrulingoutasmany
alternativesaspossibleinmultiple-choiceitemsandthenguessingamongtheones
remaining
Test-wisestudentsexperiencemetacognitivesuccesswhiletest-naïvestudents
experiencecognitivefailure.Ingeneralsuchstrategiesshouldfacilitatethetest-taking
processbyreducingtheeffectonperformanceofunfamiliaritywithfeaturesofagiven
testmethod

13. Other Miscellaneous Terms
Coaching Effect
Theeffectontestscoresofteachingtothetest
Theterm“teachingtothetest”impliesdoingsomethinginteachingthatmaynotbecompatiblewithteachers’own
valuesandgoals,orwiththevaluesandgoalsoftheinstructionalprogram
Coachingcanbedefinedasshortterminstructionin testwisenessandinansweringquestionssimilartothose
appearingonthetargetexamination
Test Compromise
Stealingor buyingthe content of and/or the answers to a test prior to the administration of the test. In this case, the test
would be completely invalidated. → Test security

14. Other Miscellaneous Terms
TestBias
Anyaspectofatestwhichyields differentialpredictionsforgroupsofpersonsdistinguishable
fromeachotherbyafactorwhich,inprinciple,shouldbeirrelevanttothetest
Standardizedtestsinvolveanumberoftypesoftestbias
Thatbiascomesinmanyforms: language,culture,race,gender,cognitivecharacteristicsand
learningstyles
Anitemthatisbiasedagainstonegroupofpeople istestingsomethinginadditiontowhatitwas
originallydesignedtotest,andsuchanitem cannotprovideclearandeasilyinterpretable
information

15. Other Miscellaneous Terms
Practice Effect (rehearsal effect)
Acertainamountofscoreimprovementresultingfromtheexperiencegainedbytheindividualsbetweenthe
firstandthesecondadministrationsofatest
Obviously,weexpecttesttakerswhoarerepeatingatest(whetherwiththesameoraparallelform)toscore
somewhathigherthantheydidthefirsttime,eveniftheirknowledgeofthesubjectbeingtestedhasnotitself
increased
CeilingEffect(alsoLackofHeadroom)
Variationintheopportunityforgainatdifferentscorelevels
Effectofmostcandidatesscoringnearthetopofthescaleonaparticulartest,withtheresultthatthetest doesnot
discriminateadequatelyamongsthigherabilitylearners

16. Other Miscellaneous Terms
Theterm
ceiling
isusedforthemaximumscoreonatestoralimitontheperformanceonsometasks
Thiseffectoccursparticularlywith
subjectiveratings
whereratersscoremostcandidateshighly
Solution:
Itcanbecounteractedthroughtheuseofmoredetaileddescriptors,byratertraining,orby
limitingthepercentageofhighscorersallowed
Itmayalsooccurthroughatestbeingtooeasyforthetestpopulationorwhereaninterviewerprovides
sucheasytasksthatacandidateisnotgiventheopportunitytodemonstratehisorherfullability
Theconverseiscalleda
basementeffect
ora
flooreffect
.

17. Other Miscellaneous Terms
Test Score Pollution
Thepracticeofteachingtothetest,inordertoraisetestscores,isgenerallycalled‘testscorepollution’
Pollutionherereferstoanincreaseinthetestscorewhichisnotconnectedtotheconstructbeingmeasured,
thatisitproduces
construct-irrelevant
testscorevariance
→Thisisimprovingtestscoreswithoutactuallymasteringtheskillsorconstructsbeingassessed
The‘LakeWobegon’Effect
TheLakeWobegonstoryisaboutteachingtothetestandtest-scorepollution

18. Other Miscellaneous Terms
Critical Language Testing (CLT)
Main Figure:
ElanaShohamy
RootedinprinciplesofCriticalPedagogy(CP)promotedbyPauloFreire
CPwassetinmotionasakindofoppositiontothe “BankingConceptofEducation”
CPisNOTatheory,butawayof‘‘doing’’learningandteaching;itisteachingwithanattitude
Whatcriticalpedagoguesareafteristhetransformationofsocietythrougheducation,includinglanguage
teaching
CPdealswithquestionsofsocialjusticeandsocialchangethrougheducation

19. Other Miscellaneous Terms
Criticalpedagoguesarguethateducationalsystemsarereflectionsofthesocietalsystemswithin
whichtheyoperate
Inallsocialsystemswehavediscriminationandmarginalizationintermsofrace,socialclass,or
gender,andthesamebiasesarereproducedineducationalsystems
Thesamepeoplewhohavethe powertomakedecisionsinsocietyatlargearetheoneswhoalso
havethepowertodesignandimplementeducationalsystems,andconsequently,theirideasand
valuesgetacceptedandpromotedwhilethevaluesandideasof OTHERSarenotgivenvoice
Educationisapoliticalactivityinwhichtherightsofcertainclassesaresystematicallydenied.

20. Other Miscellaneous Terms
Critical Language Testing (CLT)
AssumesthattheactoftestingisNOTneutral.Rather,itisbothaproductandanagentofcultural,
social,political,educationalandideologicalagendasthatshapethelivesofindividualparticipants,
teachersandlearners(Acaseinpointbeing
large-scalestandardizedtesting
)
Referstotheexaminationoftheusesandconsequencesoftestsineducationandsociety,tomonitor
theirpower,minimizetheirdetrimentalforce,revealthemisuses,andempowerthetesttakers
Testsrepresentasocialtechnologydeeplyembeddedineducation,governmentandbusiness;assuch
theyprovidethemechanismforenforcingpowerandcontrol.Testsaremostpowerfulastheyareoften
thesingleindicatorsfordeterminingthefutureofindividuals

21. Other Miscellaneous Terms
Some Tenets of CLT:
Test-takersarepoliticalsubjectsinapoliticalcontext
Itchallengespsychometrictraditionsandconsidersinterpretive,individualizedproceduresforpredictingsuccessandevaluating
ability
Testsaredeeplyembeddedincultureandideology
Testdesignershavearesponsibilitytooffermultiplemodesofperformance toaccountforvaryingstylesandabilitiesamongtest-
takers
Asksquestionsaboutwhatsortofagendasaredeliveredthroughtestsandwhoseagendastheyare
Itperceiveslanguageasbeingcaughtupinanarrayofquestionsconcerning educationalandsocialsystems;thenotionof
“justa
test”
isanimpossibilitybecauseitisimpossibletoseparatelanguagetestingfromthemanycontextsinwhichitoperates

22. Other Miscellaneous Terms
The problem of the test-oriented culture lies in the agendas of those who design and those who utilize the
tests:
Tests are used in some countries to
deny citizenship
Tests are by nature
culture-biased
and therefore may disenfranchise members of a
non-mainstream value system
Test givers are always in a position of power over test-takers and therefore can
impose social and political ideologies
on test-takers through standards of acceptable and unacceptable items
Tests promote the notion that answers to real-world problems have unambiguous right and wrong answers with no shades of
gray
A corollary to the latter is that tests presume to reflect an
appropriate core of common knowledge
and
acceptable
behavior;
therefore the test-taker must buy into such a system of beliefs in order to make the cut

23. Other Miscellaneous Terms
Fairness
Thedegreetowhichatesttreatseverystudentthesame,orthedegreetowhichitisimpartial
Teacherswouldgenerallyliketoensurethattheirpersonalfeelingsdonotinterferewithfairassessmentofthe
studentsorbiastheassignmentofscores
Theaiminmaximizingobjectivityistogiveeachstudentanequalchancetodowell.Equitabletreatmentinterms
oftestingconditions,accesstopracticematerials,performancefeedback,retestopportunities,andotherfeaturesof
testadministration,includingprovidingreasonableaccommodationfortesttakerswithdisabilitieswhenappropriate,
areimportantaspectsoffairnessunderthisperspective
Thistendencytoseekobjectivityhasledtotheproliferationof‘objective’testswhichminimizethepossibilityof
varyingtreatmentfordifferentstudents