Sampling and instrumentation

1. SamplingSampling
A sample is a small number of A sample is a small number of
individuals representing a larger individuals representing a larger
group.group.

2. Samples and PopulationsSamples and Populations
A A samplesample in a research study is a in a research study is a
relatively small number of individuals relatively small number of individuals
about whom information is obtained. The about whom information is obtained. The
larger group to whom the information is larger group to whom the information is
then generalized is the then generalized is the populationpopulation..

3. Why use samples?Why use samples?
Although the best data comes from Although the best data comes from
studying an entire population, samples are studying an entire population, samples are
used because they are smaller and less used because they are smaller and less
unwieldy. It can be too time consuming unwieldy. It can be too time consuming
and expensive to study an entire and expensive to study an entire
population.population.

4. Defining the populationDefining the population
Whether a researcher is drawing a sample Whether a researcher is drawing a sample
or is studying an entire population, the or is studying an entire population, the
population needs to be defined. This population needs to be defined. This
helps focus the research.helps focus the research.

5. Target vs. accessible populationsTarget vs. accessible populations
The The targettarget population is the population a population is the population a
researcher would like to generalize to. researcher would like to generalize to.
Often this isn’t possible, so the Often this isn’t possible, so the accessibleaccessible
population is used. For example, a population is used. For example, a
researcher might want to target all male researcher might want to target all male
elementary teachers in the United States, elementary teachers in the United States,
but actually collects data from the male but actually collects data from the male
elementary teachers in Hawaii.elementary teachers in Hawaii.

6. Random vs. nonrandom samplingRandom vs. nonrandom sampling
Random sampling is completely based on Random sampling is completely based on
chance. For example, one might identify chance. For example, one might identify
all members of a population, (n=250) write all members of a population, (n=250) write
their names on separate pieces of paper, their names on separate pieces of paper,
and then draw 25 names out of a hat to and then draw 25 names out of a hat to
determine who is actually to be included in determine who is actually to be included in
the study.the study.

7. Nonrandom samplingNonrandom sampling
In a nonrandom sample, members are In a nonrandom sample, members are
selected on the basis of a particular set of selected on the basis of a particular set of
characteristics, rather than a random characteristics, rather than a random
chance of being included.chance of being included.

8. Simple random sampleSimple random sample
In a simple random sample, each and In a simple random sample, each and
every member of a population has an every member of a population has an
equal and independent chance of being equal and independent chance of being
selected. selected.

9. Table of random numbersTable of random numbers
A table of random numbers is used to A table of random numbers is used to
identify the people to be included in a identify the people to be included in a
sample. These are usually found in sample. These are usually found in
statistics books, or can be generated by statistics books, or can be generated by
some calculators and computers.some calculators and computers.

10. Stratified random sampleStratified random sample
In stratified random sampling, subgroups In stratified random sampling, subgroups
within a target population are identified to within a target population are identified to
be included in proportion to the numbers be included in proportion to the numbers
in which they exist in the population. For in which they exist in the population. For
example, a researcher studying example, a researcher studying
aggressive behavior in dog breeds found aggressive behavior in dog breeds found
in Hawaii would want to include a sample in Hawaii would want to include a sample
of registered breeds in the proportion they of registered breeds in the proportion they
are found in the state.are found in the state.

11. Cluster samplingCluster sampling
In situations where simple random In situations where simple random
sampling isn’t possible, as is often the sampling isn’t possible, as is often the
case in schools, groups or case in schools, groups or clustersclusters are are
identified for inclusion in research. For identified for inclusion in research. For
example, a researcher might choose to example, a researcher might choose to
study all of the students in some specific study all of the students in some specific
classes.classes.

12. Two stage random samplingTwo stage random sampling
This technique combines random This technique combines random
sampling with cluster sampling. It allows a sampling with cluster sampling. It allows a
bigger group to be targeted for bigger group to be targeted for
generalization.generalization.

13. Systematic sampling with a random Systematic sampling with a random
startstart
In this procedure, a random number is In this procedure, a random number is
generated to identify the first member generated to identify the first member
selected for a sample, and then every selected for a sample, and then every nnth th
member of the population is selected for member of the population is selected for
inclusion. For example, the first member inclusion. For example, the first member
selected in a population of 500 might be # selected in a population of 500 might be #
412, and then every 7412, and then every 7
thth
person is chosen: person is chosen:
419, 426, 433, 440, 447, and so on. 419, 426, 433, 440, 447, and so on.
When you pass 500, you loop back to the When you pass 500, you loop back to the
beginning.beginning.

14. Sampling ratioSampling ratio
This is the proportion of individuals This is the proportion of individuals
selected for a study. For example, you selected for a study. For example, you
might select to study ten percent of the might select to study ten percent of the
population. The ratio is defined as the population. The ratio is defined as the
sample size divided by the population sample size divided by the population
size.size.

15. Convenience sampleConvenience sample
When it isn’t possible to draw a random or When it isn’t possible to draw a random or
systematic nonrandom sample, a systematic nonrandom sample, a
researcher might choose to study the researcher might choose to study the
individuals who are available. This is individuals who are available. This is
known as a convenience sample.known as a convenience sample.

16. Purposive samplingPurposive sampling
A purposive sample is one identified on A purposive sample is one identified on
the basis of specific characteristics the basis of specific characteristics
identified by the researcher. For example, identified by the researcher. For example,
if a researcher wanted to study all of the if a researcher wanted to study all of the
foreign-born teachers in a school district, foreign-born teachers in a school district,
he or she would try to identify all of those he or she would try to identify all of those
individuals and include only them. individuals and include only them.

17. External validityExternal validity
Since the entire point of sampling is to Since the entire point of sampling is to
generalize the results to a larger generalize the results to a larger
population, researchers need to be sure population, researchers need to be sure
their work actually does represent the their work actually does represent the
population. The extent to which population. The extent to which
information can be generalized to a larger information can be generalized to a larger
population is known as external validity.population is known as external validity.

18. Representative samplesRepresentative samples
A representative sample provides the A representative sample provides the
most accurate portrayal of the population most accurate portrayal of the population
being studied.being studied.

19. Replication studiesReplication studies
A replication study follows the format of a A replication study follows the format of a
previous study, but uses a new group of previous study, but uses a new group of
subjects or a new set of conditions or subjects or a new set of conditions or
both.both.

20. Ecological generalizabilityEcological generalizability
This term refers to the degree to which a This term refers to the degree to which a
study can be generalized to a different set study can be generalized to a different set
of conditions. For example, researchers of conditions. For example, researchers
studying rural schools might have difficulty studying rural schools might have difficulty
generalizing their results to urban schools.generalizing their results to urban schools.

21. DataData
Data is a plural word that refers to the Data is a plural word that refers to the
kinds of information researchers collect. kinds of information researchers collect.
Data should be followed by a plural verb, Data should be followed by a plural verb,
such as “Data are” or “Data were”.such as “Data are” or “Data were”.

22. InstrumentationInstrumentation
The process of preparing to collect data is The process of preparing to collect data is
called instrumentation. It involves the called instrumentation. It involves the
selection of the method by which data will selection of the method by which data will
be collected, as well as the procedures be collected, as well as the procedures
and conditions for collecting them.and conditions for collecting them.

23. ValidityValidity
This term refers to the defensibility of the This term refers to the defensibility of the
inferences a researcher can make from a inferences a researcher can make from a
study using an instrument. study using an instrument.

24. ReliabilityReliability
Reliability refers to consistency of results. Reliability refers to consistency of results.
If a study is repeated, will it yield similar If a study is repeated, will it yield similar
findings? A good example of reliability findings? A good example of reliability
might be having three different people might be having three different people
grading students’ essays. Will all three of grading students’ essays. Will all three of
them agree on what constitutes an A, B, them agree on what constitutes an A, B,
C, etc? Or will their scoring vary widely? C, etc? Or will their scoring vary widely?
If there is a large variety, the grades would If there is a large variety, the grades would
not be reliable.not be reliable.

25. ObjectivityObjectivity
This characteristic refers to the absence of This characteristic refers to the absence of
subjective bias on the part of the subjective bias on the part of the
researcher. For example, political analyst researcher. For example, political analyst
with a particular ideological bent might with a particular ideological bent might
conduct a poll differently from one who conduct a poll differently from one who
has no affiliation. has no affiliation.

26. Different types of instrumentsDifferent types of instruments
Researcher instruments are used by the Researcher instruments are used by the
researcher to collect data; a tally sheet or rubric researcher to collect data; a tally sheet or rubric
are examples.are examples.
Subject instruments are completed by the Subject instruments are completed by the
subject. A survey questionnaire is an example.subject. A survey questionnaire is an example.
Informant instruments are completed by Informant instruments are completed by
knowledgeable participants providing knowledgeable participants providing
information in addition to that collected by information in addition to that collected by
researchers and given by subjects.researchers and given by subjects.

27. Selecting instrumentsSelecting instruments
Instruments may be selected in one of two Instruments may be selected in one of two
ways. Either a researcher locates one ways. Either a researcher locates one
that has been developed by another that has been developed by another
person, or he/she designs a new one. person, or he/she designs a new one.
The advantage of selecting existing ones The advantage of selecting existing ones
is that they have often been field tested for is that they have often been field tested for
reliability and validity. reliability and validity.

28. Collecting dataCollecting data
Data may be collected in a variety of Data may be collected in a variety of
ways. Respondents might give written ways. Respondents might give written
responses, or they might perform a task. responses, or they might perform a task.
Doing a miscue analysis on a student is Doing a miscue analysis on a student is
an example of a performance analysis. an example of a performance analysis.

29. Rating scalesRating scales
The difference between observation and rating The difference between observation and rating
is that when a researcher rates a subject, he or is that when a researcher rates a subject, he or
she is making a judgment of some type. On the she is making a judgment of some type. On the
other hand, when a researcher makes an other hand, when a researcher makes an
observation, he or she is merely recording observation, he or she is merely recording
behavior and not judging it. For example, a behavior and not judging it. For example, a
rating might be that a girl made 3 baskets in 20 rating might be that a girl made 3 baskets in 20
attempts, thus scored 2 on a scale of poor to attempts, thus scored 2 on a scale of poor to
good on free throws, while an observation would good on free throws, while an observation would
just note the number of baskets/attempts.just note the number of baskets/attempts.

30. Researcher instrumentsResearcher instruments
Interview schedulesInterview schedules
Tally sheetsTally sheets
Performance checklistsPerformance checklists
Anecdotal recordsAnecdotal records
Time-and-motion logsTime-and-motion logs

31. Subject instrumentsSubject instruments
QuestionnairesQuestionnaires
Self-checklistsSelf-checklists
Attitude scalesAttitude scales
Personality inventoriesPersonality inventories
Achievement testsAchievement tests
Aptitude testsAptitude tests
Performance testsPerformance tests
Projective devicesProjective devices
Sociometric devicesSociometric devices

32. ScoresScores
Raw scores are the initial scores obtained Raw scores are the initial scores obtained
on a test. The number right out of a total on a test. The number right out of a total
number of questions is an example.number of questions is an example.
Derived scores have been scaled to show Derived scores have been scaled to show
their relative position with respect to other their relative position with respect to other
raw scores. raw scores.

33. Derived scoresDerived scores
Percentile ranksPercentile ranks
Age/grade equivalenceAge/grade equivalence
Standard scoresStandard scores

34. Norm-referenced vs. Criterion Norm-referenced vs. Criterion
referencedreferenced
 A norm-referenced test is developed to A norm-referenced test is developed to
provide scores that replicate a normal provide scores that replicate a normal
curve among the population tested. Thus, curve among the population tested. Thus,
among a population taking the test, half of among a population taking the test, half of
the people should score above average the people should score above average
and half below.and half below.
A criterion referenced test is based on a A criterion referenced test is based on a
goal and an identified percentage is goal and an identified percentage is
targeted to reach that goal.targeted to reach that goal.

35. Measurement scalesMeasurement scales
A nominal scale, the simplest scale, identifies A nominal scale, the simplest scale, identifies
groups by a number, e.g. “1” for male and “2” for groups by a number, e.g. “1” for male and “2” for
female.female.
An ordinal scale provides an rating from most to An ordinal scale provides an rating from most to
least. A Likert scale is an ordinal scale.least. A Likert scale is an ordinal scale.
An interval scale is an ordinal scale that has the An interval scale is an ordinal scale that has the
addition of equal distances between the points. addition of equal distances between the points.
IQ is measured using an interval scale.IQ is measured using an interval scale.
A ratio scale is an interval with a true zero and is A ratio scale is an interval with a true zero and is
rarely used in educational measurement.rarely used in educational measurement.