Building Web Collection Capability for ONS Social Surveys Fiona Dawe and Laura Wilson
Outline • • Background Study methodology Results Way forward
Background • 2008 online pilot • 2010 Lound & Betts review of mixed modes Survey environment: • Falling response rates • Increasing costs of data collection • Respondent burden Labour Force Survey: • Shortest & most straightforward ONS social survey • Afford the largest cost savings • I
Issues to address • Household data – can this be collected? • Questionnaire length • Complex questions and routing • Will LFS specific questions work in this mode?
2010/11 Online pilot (a) Opinions Online Survey • Mode effect test • Comparison of online self completion vs interviewer coding (b) LFS Online Survey (wave 6) • Test of LFS questionnaire online • Comparison of LFS responses with online responses SIC and SOC (c) Cognitive testing • Real time usability testing • Investigation of response issues
2010/11 Online pilot : Response (a) November Opinions Online Survey (complete) • 8% (b) December Opinions Online Survey (complete) • 17% (c) LFS Online Survey (complete) § Total sample 25% - Non-email sample 13% - Email sample 40%
Questionnaire content - LFS Included: • Short household section • Basic socio-demographic information about each household member • Relationships to each other • Collected detailed information on one person only per household • Key Labour Market Division questions • ‘Unimode’ question design Questions on… • • • • Accommodation details Nationality Country of birth Ethnicity Religion Sexual identity Government training schemes Main job Home workers Employment pattern Second job Looking for work Benefit entitlement Employment 12 months ago Earnings
Questionnaire content - OPN Questions on… Included: • Short household section • Basic socio-demographic information about each household member • Relationships to each other • Interviewed one person in the household who was aged 16 or over • ‘Unimode’ question design LFS questions: - Country of birth - Nationality - Sexual identity - Health problems - Employment (including workplace and second job) - Benefits - SIC and SOC coding frames - Workplace coding frames - Nationality coding frames - Country of birth coding frames Sponsored OPN client Modules (with permission)
Questionnaire length LFS • Av. Completion - 18 minutes (estimate was 20) • Av. Partial - 9 minutes • Only 25 % of partial respondents spent more than 14 minutes on the survey. OPN Nov: • Av. Completion – 21 minutes (estimate was 20) • Av. Partial – 4 minutes Dec • Av. Completion – 29 minutes (estimate was 20; more modules) • Av. Partial – 7 minutes Abandoned the questionnaire quite soon, within only a few minutes.
Coding frames - OPN – face-to-face: – self-coded using laptop and interviewer coded also OPN – FTF self-complete frame: – 10% refused to complete frame – 53% reported SIC issues and 60% reported issues SOC issues – 18% SIC and 21% SOC differed at level 1 when compared
Coding frames - LFS – online: – free text description for industry (SIC) and occupation (SOC) – self-coded for workplace, nationality and country of birth LFS – coded at office: – Automated and SOC coders in Titchfield – compared these with wave 5 LFS interviewer codes – found discrepancies of 50% for SIC and 60% for SOC Conclusion: OPN and LFS were inconclusive in recommending whether to: – include frames for respondents to use themselves – or, to collect the free text and code in-house coding
Mode Effects Opinions Survey (OPN) and Opinions Online Survey (OPNO) • Online and face-to-face compared • 2 -tailed t-test used with 1% significance level Results: - Online sample comprised of: sig. more men sig. more people aged between 45 and 64 sig. less 16 to 24 year olds Due to significance, we calibrated the data to age and sex population totals Results: - Some significant differences observed on ‘non-sensitive’ questions - Average weekly income higher in online respondents
Way forward I • Identify software and hardware solution • Further testing of LFS suitability for online data collection • Learn more from the 2011 Census • Target: December 2013 to recommend extending the LFS mixed mode data collection to include online questionnaires
Thank you Contact: Fiona. dawe@ons. gov. uk Laura. wilson@ons. gov. uk
Скачать презентацию Building Web Collection Capability for ONS Social Surveys
4a06d04ab2e8cc513712d0fc81ae7f45.ppt