PILOT STUDY.pdf

pilot study vs pretesting

1. PILOT STUDY AND PRETESTING
BY
DR EBOT WALTER OJONG
SENIOR LECTURER/CHEMICAL PATHOLOGIST

2. INTRODUCTION
‘Noamountofintellectualexercisecansubstitute
fortestinganinstrumentdesignedto
communicatewithordinarypeople’(Backstrom
andHursch,1963)
Surveysmustbethoroughlypretestedbefore
theyarelaunched(Hullandetal.,2018)

3. INTRODUCTION
Toensurethatthemeaningofthequestionitems
arecleartotherespondents,andthatthe
respondentswillclearlyunderstandthe
questionsthewaytheyaredesignedand
intended,itisveyimportantthatasurvey
questionnairemustbepretested(Sekaran,
2003)

4. MEANING OF PILOT STUDY
•Aminiversionorsmallscaleversionoftheresearchstudyaimedat
tryingortestingtheresearchprocessandprotocolsinordertohelpin
assessingthefeasibilityofthemain/proposedresearchproject
•Couldbeaminiversionoftheentireprocessofthemainresearch
projectorofcertainaspectse.g.datacollection,analysistoolsetc.
•Examplespilotingofobservations,interviews,questionnairesorother
tools
•Inexperimentalstudiesitcouldbepilotingoftheexperiments/tests

5. OBJECTIVES OF PILOT STUDIES
•Thegeneralgoalofapilotstudyistosavesometime,effort
andmoney,whichcanbelostifamajorresearchstudyfails
becauseofunforeseenattributes
•Testadequacyofresearchinstruments
•Assessfeasibilityofafull-scalestudy
•Assesswhethertheresearchprotocolisrealisticand
workable
•Reveallogisticissues
•Collectpreliminarydata

6. OBJECTIVES OF PILOT STUDIES
•Ensurewhetherthesamplingframeandtechniqueare
effective
•Determinesamplesize(Thepilotstudysampleis
inappropriatetobeusedasthemainstudysample)
•Convincefundingbodiesthatthemajorstudyisfeasibleand
worthfunding

7. OBJECTIVES OF PILOT STUDIES
•‘Toseeifthebeastwillfly’(TeijingenandHundley,2002)
•‘Reassessmentwithouttears’(Blaxter,HughesandTight,
1996:121)
•‘Donottaketherisk.Pilottestfirst’(VanTeijilingenand
Hundley,2001:2)

8. IMPORTANCE OF PILOT STUDIES
•Helpsinestablishingthefeasibilityoftheresearchproject
•Feasibilityofdatacollectionandanalysistools/processes
•Authenticationandimprovementofresearchprocessesand
tools.Validityandreliabilityofresearchtoolsandprocesses
whichaddrigourtotheresearch
•Precautionsagainstpossiblelargescaleprojectfailures

9. IMPORTANCE OF PILOT STUDIES
•Data/resultsgeneratedfrompilotstudycouldbemade
partofthemainresearch.
•Establishingtheneed/importanceoffull/largescale
studies.
•Usefulforjuniorresearchersandstudentresearchers.
•Helpsinobtainingfundingforlargescalestudies.

10. HOW ARE PILOT STUDIES CARRIED OUT?
QUANTITATIVESTUDIES
•Surveysshouldbecarriedoutonsmallsamplesduringthe
pilotprocesswhicharesubsetsofthelarger
sample/populationforthemainstudy.
•Generallytheseminiversionsofsurveyswillestablish
validity,reliabilityandfeasibilityofdatacollection,analysis
andaccessprocesses
•Highlyimportant/essential

11. HOW ARE PILOT STUDIES CARRIED OUT?
QUALITATIVESTUDIES
•Pilotstudiesaregenerallyusefulforforimprovementof
datacollectiontoolsandprocesses(interviews,
observationprotocols)intermsoflanguage,contentand
dataobtained.
•Thishelpsinrefiningandauthenticatingthedataobtained
inthemainstudy
•Important/useful

12. HOW ARE PILOT STUDIES CARRIED OUT?
•MIXEDMETHODSSTUDIES
•Asaddressedabove
•Important/useful

13. PILOT STUDY OR PRETESTING
•Althoughpre-testingisoftenunderstoodas
apilotstudy,theybothservedistinctive
purposes.
•Theyaresimilarbutdifferentfromeach
other

14. PILOT STUDY OR PRETESTING
•Apilotstudyisamini-versionofafull-scalestudyora
trialrundoneinpreparationofthecompletestudyto
beundertakeninordertoensurethatafull-fledged
studywillbecarriedoutsuccessfully;afeasibility
study.
•Itisatry-outnotonlyofresearchtechniquesand
methodsbutalsoofquestionnairesandinterviews

15. SAMPLE SIZE FOR PILOT STUDY
•Asamplebetween25and100individuals(Cooperand
Schindler,2011)
•Arangefrom10to30individualsareenoughforapilot
test(Hill,1998)
•Asamplesizeshouldbe10percentofthesample
projectedforthemainstudy(Connelly,2008)

16. WHAT IS PRE-TESTING PROCESS?
•Objectivesofpre-testing
Toensurewhether
a)thewordingofthequestionsiscorrect
b)thesequenceofquestionsiscorrect
c)therespondentshaveclearlyunderstoodallthe
questions
d)Additionalquestionsareneededorsomequestions
shouldbeeliminated
e)theinstructionsareclearandadequate

17. WHAT IS PRE-TESTING PROCESS?
•Fundamentalissuesofpretestingaquestionnaire
a)Thespecificitemstobepretested
b)Themethodtobeusedtoconductthepretest
c)Theperson/swhoshouldconductthepretest
d)Theparticipantswhoshouldbethesubjectsofthe
pretest
e)Thesamplesizeneededforthepretest

18. THE SPECIFIC ITEM TO BE TESTED
Itemsaboutthequestionnaireitself
•Formatofthequestions
•Layout
•Length
•Numberoflinesincaserespondentsneedtowritea
response,andthesequencingofquestions
Itemsaboutspecificquestions
•Whethertherespondentinterpretedthequestion
differentlyorwhethertherespondentsdidnotunderstand
thestatementverywell

19. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive(personal)interviews
Debriefingmethod
Protocolmethod
Telephoneinterview
Mailself-reports

20. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive(personal)interviews
Debriefingmethod
Therespondentisaskedtoanswerthequestionnaire
completelywhiletheinterviewerobservesany
indicationgesturesofconfusionfromtherespondent
Aftercompletingthequestionnaire,theinterviewer
willasktherespondentforanypossibleproblems
whichhe/shemayhaveencounteredwhilefillingout
thequestionnaire

21. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive(personal)interviews
Protocolmethod
Therespondentisaskedtoexpresshisorher
thoughtsonceitoccurswhileansweringthe
questionnaire
Arecorderwillbehelpfultomakecarefulnotesabout
theverbalizationsoftherespondentsthoughts

22. THE PERSON TO CONDUCT THE PRETEST
Onlythebestinterviewersshouldconductpretestingas
theyhavethekeenabilitytoobserveanyuneasiness,
confusionandresistanceamongtherespondents
Theselectionofinterviewersshouldalsoinvolvedifferent
levelsofcompetence

23. THE PARTICIPANTS DURING THE PRETEST
Individualswhoaresimilartothetargetpopulationofthe
study
Individualswhoaretypicalorrepresentativesoftheactual
populationtobestudied
Individualsatasitedifferentfromtheactualstudysite

24. THE SAMPLE SIZE FOR THE PRETEST
Ifthequestionnaireislongandcomplex,itwould
needabiggersampleforpretesting
Ifthetargetpopulationofthestudyis
unsophisticated,abiggersampleforpretestingsizeis
alsonecessary

25. SUMMARY
PILOTSTUDY:Totryouttheresearchquestions,
methodology,researchinstrumentoranuaspectofthe
research
PRE-TESTING:Toensurethattheinstrumentisclearly
understoodbytherespondents