assessment of data quality in census.pptx

It is about assessment of data quality in census data collection.

1. 1 Assessment of data quality for censuses conducted during the Covid-19 pandemic –South Africa UN_EGM Meeting Session 4

2. Presentation outline Introduction 1 Methods of data collection 2 Changes in data collection method and their impact 3 Adapting the geography frame: 4 Conclusion 5 Change other census processes 6 Adapting the data collection tools: 7

3. Introduction (1) The 2020 round of Censuses adversely affected by the global COVID-19 pandemic Census plans, processes and activities have been disrupted and need for modification to mitigating risks posed by the pandemic Covid-19 pandemic created significant concerns for data quality attributed to the following: Changes in collection methods Disruption of planned processes and implementation of new untested processes Quality assurance procedures for Geography Frame Extending duration of enumeration

4. 4 CATI CAWI CAPI Computer Assisted Telephonic Interview  Computer Assisted Personal Interview  METHODS OF DATA COLLECTION Computer Assisted Web Interview  Administrative Data 

5. Methods of collection and associated risks to data quality FACE TO COLLECTION Risks linked to fieldwork Poor access to households Poor response rates BIG DATA TELEPHONE COLLECTION Non-contact Refusals Questionnaire length ONLINE COLLECTION DATA QUALITY Internet penetration Confidentiality and security concerns Cost of data Data Item non-response Non-response attributed to complex questionnaire Use of admin data- Some sources not fit for statistical use hence Require technical expertise in data modelling;

6. 6 Adapting the geography frame to new collection methods (1) Traditionally, conducting a successful Census relies on existence and availability of reliable national address frame. New Census data collection methods require inclusion of households info in the frame

7. 7 Census address frame - structures for habitation (Dwelling Units) and other structure types Changing collection methods dictate frame with households information; total household count per area of enumeration household contacts and preferred mode of collection Adapting Geography Frame to new collection methods (2)

8. 8 Inadequate geography frame - challenge to data quality; Multiple registrations of DUs & households Under/over estimation of DUs & households due to misconception of household concept) e.g. trial test results; Inability to clearly identify dwelling units and household frame shrinkage or growth. Insufficient frame update methodology and procedures for respondents- for self registration Inadequate quality assurance processes for self registrations before enumeration Adapting the geography frame to new collection methods (3)

9. South Africa: State of preparedness Upcoming Census 2021 plans Initial plan to conduct a digital Census Census procedures, methods and tools tested in various tests Methods of collection tested: CAPI, CATI & CAWI Changes introduced to deal with anticipated effects of Covid-19 pandemic. Working towards geo-statistical frame. Self registration of DUs and households by respondents introduced during census trial.

10. South Africa: State of preparedness Census trial test conducted in August 2020 - to assess the feasibility of collecting census data remotely. Specific objectives of Census trial relating to data quality: To introduce a suitable frame to facilitate self and assisted enumeration To test the adequacy of data collection instruments To test the data quality of both CAWI and CATI collection modes

11. Total No EAs 2134 EAs with at least one registration 672 (31.49%) EAs with zero registration 1462 (68.51%) Target sample (DUs) 426 466 Registered DUs 731 Special dwelling Institutions (SDI) registrations SDI registrations from in-sample SDIs completed registration from out of sample 8 Registration Duration: 4 weeks (10 August – 06 September 2020) Census Trial findings (Registrations)

12. Total household registrations 26 364 Completed household registrations 11 764 (44.62%) Respondent initiated registration 23 203 (88.01%) Assisted registration (through the call centre) 3 161 (11.99%) Incomplete household registrations ( no indication of household number and dwelling unit they were linked to and others no mode of collection selected ) 14 600 Household registration In-sample 1 063 Household registration out of-sample 10 701 Census Trial findings (Households)

13. 13 Households enumerated, mode of collection and questionnaire type

14. Adapting Census tools to new collection methods Data Collection Tool Considerations for review of collection tools Short vs Long questionnaire Reviewed questionnaire themes Inability to provide adequate frame for surveys Data Collection Tool Challenges Sample determination – Inclusion and exclusion Inability to adequately test adjusted instruments

15. Administrative records Surveys Alternative data sources Respondent burden: Questionnaire length Indicators Data quality: Modes of collection Training Quality of field staff Census Instrument/tool determination Questionnaire Themes - Data needs determined from stakeholders - Census time series? Development agenda? Rare events such as disability?

16. Process Issues and linked risks Training Virtual training on complex census methodologies, procedures and systems. Complex tools - limited understanding by field staff translating to poor data quality Self enumeration guidelines Inadequate self enumeration leads to poor quality data (CAWI) Testing Insufficient time for testing of changed processes and systems/ applications Inability to conduct Census evaluation Failure to conduct post enumeration survey Failure to conduct PES within stipulated timeframes Replacing PES with other methods of evaluation- Demographic methods for estimate on /adjustment Change in other census processes

17. Data quality from different collection methods require technical expertise in data integration, data editing and analysis. Collection methods Inadequate data integration & management

18. Census 2021 and data quality- Conclusion Constituting Dwelling Units and Households Census geography frame

19. 19 Thank You