Data Quality Control for food and milk.

To know the steps necessary for ensuring quality assurance and control of data at various stages of a study

To understand the difference between pilot testing and pre-testing

To understand the importance of designing data collection instruments

To understand how data can be managed using an audit trail and the various techniques that can be used to inspect your dataset after it has been entered

1. Data Quality ControlData Quality Control
by
Naila Baig Ansari
Research Fellow
Dept of Community Health Sciences
The Aga Khan University
Karachi, Pakistan

2. Who am I?Who am I?
Education:
MSc (Epidemiology),
The Aga Khan University, 2001. Thesis:
Care and feeding practices and their
association with stunting among young
children residing in Karachi-s squatter
settlements
BBA (Management),
The College of William and Mary,
Williamsburg, VA, USA, 1989
Research interest: Nutritional and
behavioral epidemiology, methodological
issues in dietary assessment methods,
household food security and gender-
related issues, care and feeding
practices, management of data and
questionnaire designing

3. Learning ObjectivesLearning Objectives
To know the steps necessary for ensuring quality assurance
and control of data at various stages of a study
To understand the difference between pilot testing and pre-
testing
To understand the importance of designing data collection
instruments
To understand how data can be managed using an audit trail
and the various techniques that can be used to inspect your
dataset after it has been entered

4. Performance ObjectivesPerformance Objectives
Know the difference between quality assurance and quality
control and ways to ensure them
Know the objectives of a pilot test and a pre-test
Understand how data collection instruments should be
designed and coded
Be able to manage data using an audit trail
Be able to inspect datasets for errors and rectify them

5. Data Quality ControlData Quality Control
Quality Assurance
–Activities to ensure
quality of data before
data collection
Quality Control
–Monitoring and
maintaining the quality
of data during the
conduct of the study
• Data Management
– Handling and
processing of data
throughout the study

6. Steps in Quality AssuranceSteps in Quality Assurance
1.Specify the study hypothesis
2.Specify general design to test study hypothesis  Develop an
overall study protocol
3.Choose or prepare specific instruments
4.Develop procedures for data collection and processing  Develop
operation manuals
5.Train staff  Certify staff
6.User certified staff, pretest and pilot-study data collection and
processing instruments and procedures

7. Quality Assurance: Standardization of Quality Assurance: Standardization of
proceduresprocedures
Why is standardization important?
–In order to achieve highest possible level of uniformity
and standardization of data collection procedures in the
entire study population
Preparation of written manual of operations
–Detailed descriptions of exactly how the procedures
specific to each data collection instrument are to be
carried out (BP example)
–Q by Q’s (question by question) instructions for
interviews

8. Quality Assurance: Training of StaffQuality Assurance: Training of Staff
Aim to make each staff person
thoroughly familiar with procedures
under his/her responsibility
Training certification of the staff
member to perform a specific procedure

9. Quality Assurance: Pretesting and Pilot Quality Assurance: Pretesting and Pilot
testingtesting
Pretesting
–Involves assessing
specific procedures
on a sample in order
to detect major flaws
Pilot Testing
–Formal rehearsal of
study procedures
–Attempts to reproduce
the whole flow of
operations in a sample as
similar as possible to
study participants

10. Pretesting and Pilot testing resultsPretesting and Pilot testing results
Pretesting of questionnaire used to assess:
–flow of questions,
–presence of sensitive questions,
–appropriateness of categorization of variables,
–clarity of the q by q instructions to the interviewer
Pilot testing
–In addition to the above, flow of process

11. Quality Assurance: Data ManagementQuality Assurance: Data Management
Designing data collection
–Layout, questions to ask, sequence of questions,
phrasing of questions, response categories, skip
patterns
–Collect and record “raw”, not processed
information (eg. Age)
–Codebook: link between the questionnaire and the
data entered in the computer

12. Code book exampleCode book example
Variable QNo Meaning Codes Format
Q1Id Q1 Quest. No 1-750 C 3
Q2Sex Q2 Respondent’s sex1 male
2 female
N 1.0
Q3Child Q3 No of children99 no responseN 2.0
Q4Wt Q4 Weight in kg 999 not recordedN 3.1
Q5roof Q5 Roof type 1 RCC
2 Cement sheet
3 Tin sheet
4 Thatched
Other (specify)
N 2.0

13. Quality Assurance: Use of a Code bookQuality Assurance: Use of a Code book
Variable names
–Up to 8 characters a-z and 0-9, must start with a letter
–Combination of question number and description (eg.
q3age)
Meaning:
–short text description describing the meaning of the
variable
–SPSS software can incorporate this info as variable
labels and display it in the output

14. Quality Assurance: Use of a Code bookQuality Assurance: Use of a Code book
Codes
–Try and use numerical codes
Predecide codes for no response, missing values
–Question could not be asked or not applicable (eg.
pregnancy outcome)
–Question was asked but respondent did not reply (eg
salary)
–Respondent replied “don’t know”

15. Quality ControlQuality Control
Observation of procedures and performance of staff members
for identification of obvious protocol deviations
Strategies include:
–Over-the-shoulder observation of staff
–Taping all interviews and reviewing a random sample
–Ongoing field supervision
–field editing by interviewer as well as field supervisor
–Office editing which includes coding
–log book maintenance
–Statistical assessment of trends over time in the performance of
each observer/interviewer/technician

16. Data Management: Audit trailData Management: Audit trail
Researcher should be able to trace each piece of information
back to the original document:
–ID included in the original documents and in the dataset
–All corrections must be documented and explained
–All modifications to the dataset must be documented by command files
–Each analysis must be documented by a command file
Purpose of audit is to
–protect yourself against mistakes, errors, waste of time and loss of
information
–enable external audit (revision)

17. Data Management: Handling of DataData Management: Handling of Data
Entering data
– Use professional data entry program like
EpiData
Preparations
–complete codebook
–examine questionnaires for obvious
inconsistencies, skip patterns

18. Data Management: Handling of DataData Management: Handling of Data
Error prevention:
–Set up a data entry form resembling your
questionnaire
–Define valid values before entering data
–double data entry by two different operators

compare contents to get list of discrepancies (
EpiInfo)

correct errors in both files and run new comparison

19. First Inspection of data. Error FindingFirst Inspection of data. Error Finding
Add variable and value labels to your data using a syntax command
Searching for errors
–make printouts of codebook from the data, overview of variables, simple frequency
tables of appropriate variables
–compare codebook created with original codebook and see if label information is
correct
–Inspect the generated summary/frequency tables for illegal or improbable
minimum and maximum values of variables and inconsistencies (eg. 250 years
age, pregnant male; 23 yr woman with 19 yr son)
Calculate the error rate by
–randomly select 10% or at least 40 of your questionnaires and re-enter them into
new file

20. Correction of errors - DocumentationCorrection of errors - Documentation
If errors are discovered
–Make corrections in a command file (SPSS syntax
file), this will provide full documentation of
changes made to the dataset
If errors are discovered when comparing
files after double data entry
–you can make corrections directly in the data
entered, provided you end this step with a
comparison of the two files entered and corrected

21. Correction of errors - DocumentationCorrection of errors - Documentation
Split the process into distinct and well-
defined steps and that your documentation
from one step to another is consistent
Archive
–once you have a “clean” documented version of
your primary data, save one copy in a safe place
and do your work with another copy

22. AnalysisAnalysis
Make sure you use the right data set
–recommend to create command files for
analysis which start with the command reading
the dataset
Late discovery of errors and inconsistencies

23. Backing up vs ArchivingBacking up vs Archiving
Backing up
–everyday activity
–purpose to able you to restore your data and documents in case of
destruction or loss of data
–not only datasets, but also command files modifying your data,
written documents such as the protocol, log book and other
documenting information
Archiving
–takes place once or a few times during the life of the project
–purpose is to preserve your data and documents for a more distant
future, maybe to even allow other researchers access to the
information.