How to develop completion and short answer tests?

Tasks and questions prompt learners to demonstrate their
knowledge, understanding and skills.
What learners say and do is then observed and interpreted, and
judgments are made about how learning can be improved.
These assessment processes are an essential part of everyday
classroom practice and involve both teachers and learners in
reflection, dialogue and decision-making.

1. Completion, Completion,
Short-Answer, and Short-Answer, and
True-False ItemsTrue-False Items
Educational Assessment of StudentsEducational Assessment of Students
Chapter 7Chapter 7
Anthony J. NitkoAnthony J. Nitko
(Brenda Stewart)(Brenda Stewart)

2. Three Fundamental Principles for Three Fundamental Principles for
Crafting AssessmentsCrafting Assessments

Focus each assessment task entirely on important Focus each assessment task entirely on important
learning targetslearning targets

Craft each assessment task to elicit from students only Craft each assessment task to elicit from students only
the knowledge and performance that are relevant to the the knowledge and performance that are relevant to the
learning targets you are assessinglearning targets you are assessing

Craft each assessment task to neither prevent nor Craft each assessment task to neither prevent nor
inhibit a student’s ability to demonstrate attainment of inhibit a student’s ability to demonstrate attainment of
the leaning targets you are assessingthe leaning targets you are assessing

3. Short-Answer ItemsShort-Answer Items

Require a word, short phrase, number, or Require a word, short phrase, number, or
symbol response.symbol response.

Three types of short-answer items:Three types of short-answer items:

Question – What is the capital of Texas?Question – What is the capital of Texas?

Completion – The capital of Texas is _________.Completion – The capital of Texas is _________.

Association – On the blank next to the state name, Association – On the blank next to the state name,
write the abbreviation for that state.write the abbreviation for that state.

TexasTexas TXTX

CaliforniaCaliforniaCACA

4. Usefulness of Short-Answer ItemsUsefulness of Short-Answer Items

Abilities AssessedAbilities Assessed

Lower-order thinking skillsLower-order thinking skills

RecallRecall

Comprehension of informationComprehension of information

Higher-level abilitiesHigher-level abilities

Make simple interpretations of data and applications of Make simple interpretations of data and applications of
rulesrules

Ability to solve numerical problems in science and Ability to solve numerical problems in science and
mathematicsmathematics

Ability to manipulate mathematical symbols and balance Ability to manipulate mathematical symbols and balance
mathematical and chemical equationsmathematical and chemical equations

5. Usefulness of Short-Answer ItemsUsefulness of Short-Answer Items

StrengthsStrengths

Relatively easy to constructRelatively easy to construct

Can be scored objectivelyCan be scored objectively

Lowers the probability of getting the answer correct Lowers the probability of getting the answer correct
by random guessingby random guessing

ShortcomingsShortcomings

Not free of subjectivityNot free of subjectivity

Cannot anticipate all possible responsesCannot anticipate all possible responses

Tends to lower the reliability of the obtained scoreTends to lower the reliability of the obtained score

6. Crafting Short-Answer ItemsCrafting Short-Answer Items

Importance of what is assessedImportance of what is assessed

How does it match the test blueprintHow does it match the test blueprint

Question format is the preferred format for a Question format is the preferred format for a
short-answer item. Focus the item on the short-answer item. Focus the item on the
specific knowledge soughtspecific knowledge sought

Word the items specifically and clearlyWord the items specifically and clearly

Put the blank near the end of the sentencePut the blank near the end of the sentence

If possible do not copy statements verbatimIf possible do not copy statements verbatim

7. Crafting Short-Answer ItemsCrafting Short-Answer Items

A completion item should omit important words A completion item should omit important words
and not trivial wordsand not trivial words

Limit blanks to one or two Limit blanks to one or two

Keep all blanks the same lengthKeep all blanks the same length

Specify the precision you expect in your answerSpecify the precision you expect in your answer

Avoid irrelevant cluesAvoid irrelevant clues

8. True-False ItemsTrue-False Items

Consists of a statement or proposition the Consists of a statement or proposition the
student may judge as true or falsestudent may judge as true or false

Six varieties:Six varieties:

True-FalseTrue-False

Yes-NoYes-No

Right-WrongRight-Wrong

CorrectionCorrection

Multiple True-FalseMultiple True-False

Yes-No with explanationYes-No with explanation

9. Usefulness of True-False ItemsUsefulness of True-False Items

AdvantagesAdvantages

Certain aspects of the subject matter readily lend themselves Certain aspects of the subject matter readily lend themselves
to verbal propositions that can be judged true or falseto verbal propositions that can be judged true or false

Relatively easy to writeRelatively easy to write

Scored easily and objectivelyScored easily and objectively

Can cover a wide range of content within a relatively short Can cover a wide range of content within a relatively short
periodperiod

CriticismsCriticisms

Poorly constructed assess only specific, frequently trivial factsPoorly constructed assess only specific, frequently trivial facts

Ambiguously wordedAmbiguously worded

Answered by random guessingAnswered by random guessing

Encourage students to study and accept only oversimplified Encourage students to study and accept only oversimplified
statements of truth and factual detailsstatements of truth and factual details

10. Usefulness of True-False ItemsUsefulness of True-False Items

Generalizations in a subject areaGeneralizations in a subject area

Comparisons among conceptsComparisons among concepts

Casual or conditional propositionsCasual or conditional propositions

Relationships between two events, concepts, Relationships between two events, concepts,
facts, or principlesfacts, or principles

Explanations for why events or phenomena Explanations for why events or phenomena
occurredoccurred

Instances or examples of a concept or principleInstances or examples of a concept or principle

11. Usefulness of True-False ItemsUsefulness of True-False Items

Evidential statementsEvidential statements

Predictions about phenomena or eventsPredictions about phenomena or events

Steps in a procedure of processSteps in a procedure of process

Computations (or other kinds of results Computations (or other kinds of results
obtained from applying a procedure)obtained from applying a procedure)

Evaluations of events of phenomenaEvaluations of events of phenomena

12. Validity of True-False Item FormatValidity of True-False Item Format

Essence of educational achievement is the Essence of educational achievement is the
command of useful verbal knowledgecommand of useful verbal knowledge

All verbal knowledge can be expressed in All verbal knowledge can be expressed in
propositionspropositions

Proposition is any sentence that can be said to Proposition is any sentence that can be said to
be true or falsebe true or false

Particular area of knowledge is indicated by Particular area of knowledge is indicated by
success in judging truth or falsity of propositionssuccess in judging truth or falsity of propositions

13. Guessing on True-False Items:Guessing on True-False Items:
An Ill Wind?An Ill Wind?

Guessing – getting 50% correctGuessing – getting 50% correct

Blind (complete random guessing) quite unlike Blind (complete random guessing) quite unlike
informed guessing (based on partial knowledge)informed guessing (based on partial knowledge)

Well-motivated guess blindly only on a small Well-motivated guess blindly only on a small
percentage of questions on a testpercentage of questions on a test

Difficult to obtain a good score on a test by blind Difficult to obtain a good score on a test by blind
guessing onlyguessing only

A given true-false test has a high reliability A given true-false test has a high reliability
coefficient – evidence blind guessing does not coefficient – evidence blind guessing does not
seriously affect the scoresseriously affect the scores

14. Guessing on True-False Items:Guessing on True-False Items:
An Ill Wind?An Ill Wind?

Random guessing – sometimes called blind Random guessing – sometimes called blind
guessingguessing

Partial knowledge – make an informed guessPartial knowledge – make an informed guess

Higher than 50-50 chance of successHigher than 50-50 chance of success

Measures of partial knowledgeMeasures of partial knowledge

Full knowledge – 100% chance of answering Full knowledge – 100% chance of answering
correctlycorrectly

Percentages of correct answers go down with Percentages of correct answers go down with
the number of blind guesses madethe number of blind guesses made

15. Suggestions for Getting Started Suggestions for Getting Started
ProperlyProperly

Create pairs of items, one true and one falseCreate pairs of items, one true and one false

Rephrase evaluative judgments into a Rephrase evaluative judgments into a
comparative statementcomparative statement

Write false statements that reflect the actual Write false statements that reflect the actual
misconceptions held by studentsmisconceptions held by students

Convert a multiple-choice item into two or more Convert a multiple-choice item into two or more
true-false itemstrue-false items

16. Suggestions for ImprovingSuggestions for Improving
True-False ItemsTrue-False Items

Importance of what is assessedImportance of what is assessed

How does it match the test blueprintHow does it match the test blueprint

Assess important ideas, rather than trivia, Assess important ideas, rather than trivia,
general knowledge, or common sensegeneral knowledge, or common sense

Make sure it is either definitely true or definitely Make sure it is either definitely true or definitely
falsefalse

Use short statements when possibleUse short statements when possible

Use exact languageUse exact language

Use positive statements and avoid double Use positive statements and avoid double negativesnegatives

17. Suggestions for ImprovingSuggestions for Improving
True-False ItemsTrue-False Items

Avoid copying sentence verbatimAvoid copying sentence verbatim

True and false statements should have approximately True and false statements should have approximately
the same number of wordsthe same number of words

Don’t present in a repetitive or easily learned patternDon’t present in a repetitive or easily learned pattern

Do not use verbal clues that give away the answerDo not use verbal clues that give away the answer

Attribute the opinion in a statement to an appropriate Attribute the opinion in a statement to an appropriate
sourcesource

Focus on one ideaFocus on one idea

18. Crafting Multiple True-False ItemsCrafting Multiple True-False Items

Looks like a multiple-choice item – followed by Looks like a multiple-choice item – followed by
several alternativesseveral alternatives

No single correct or best answerNo single correct or best answer

Response to each alternative is true or falseResponse to each alternative is true or false

Each alternative is scored correct or incorrectEach alternative is scored correct or incorrect

19. Crafting Multiple True-False ItemsCrafting Multiple True-False Items

AdvantagesAdvantages

Two or three multiple true-false responses in the Two or three multiple true-false responses in the
time it takes to make one multiple-choice responsetime it takes to make one multiple-choice response

Multiple true-false tests created from multiple-choice Multiple true-false tests created from multiple-choice
items have higher reliability than original multiple-items have higher reliability than original multiple-
choicechoice

Can assess same abilities as straight multiple-choice Can assess same abilities as straight multiple-choice
items that are crafted to assess parallel contentitems that are crafted to assess parallel content

20. Crafting Multiple True-False ItemsCrafting Multiple True-False Items

AdvantagesAdvantages

Students believe multiple true-false items do a better Students believe multiple true-false items do a better
job of assessing knowledge than multiple-choice job of assessing knowledge than multiple-choice
itemsitems

Students perceive them to be harderStudents perceive them to be harder

May be easier to write – not limited to one correct May be easier to write – not limited to one correct
answeranswer

LimitationsLimitations

Shares some of same limitations as multiple-choiceShares some of same limitations as multiple-choice

21. Works CitedWorks Cited

Nitko, A. J., (2004). Educational Assessment of Nitko, A. J., (2004). Educational Assessment of
Students, 4Students, 4
thth
ed. Upper Saddle River, NJ: ed. Upper Saddle River, NJ:
Pearson Education, Inc.Pearson Education, Inc.