2011 Census 2007 Census Test emerging findings

2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007

Session Aim: • • • To share emerging findings from the 2007 Census Test Outline plan for publishing 2007 Census Test evaluation Present a brief update on other 2011 Census developments

Contents • • Test background Results: • • • Address register development Post-out/hand delivery Income question Key findings of 2007 Test Further evaluation and publication Did you know ….

2007 Census Test - Objectives Test objectives: • Assess the effect on response of: – Inclusion of an income question; and – the use of post-out to deliver questionnaires. • Assess the feasibility of major innovations in proposed 2011 Census operational procedures: – Outsourcing of field staff recruitment, pay and training; – development of an operational intelligence system to enable individual questionnaires to be tracked; and, – development of an address list and address checking procedures.

2007 Census Test - Sample sizes by LA and ETC

2007 Census Test – High-level Design • Address checking – Conducted in all Test areas during Sept and October – Split discretionary and full contact methods – Used controlled errors • Delivery – 50% Post-out, 50% hand delivery – 50% questionnaires income, 50% no income – For hand delivery 3 attempts at contact over 2 week period • Collection/Follow-up – – Central post-back 23 May – 22 June 3 attempts everywhere reminder letter to all outstanding addresses as at 31 May

2007 Census Test – Key constraints affecting evaluation of the Test: • Voluntary – Relied on public’s good will to complete a return • Publicity • Sample – skewed to harder to enumerate areas • Follow-up – Fixed number of follow-up attempts everywhere

Results

Household* return rates by LA * As at 15 July

Household* return rates by ETC * As at 15 July

Address register development Results

Key results • Address checking (AC) found about 12% new addresses – Time consuming to validate and update AR • 4. 4% (4, 400) of questionnaires undelivered – Of which 17% were added by address checkers – Reasons include vacant properties, non-existing addresses, duplicate addresses. • 1, 200 new addresses found during enumeration – Of which, over 50% found at follow-up. – Nearly 70% of new addresses were sub-premise addresses – suggest existed at time of AC.

Early conclusions • Suggests we need to plan to do a 100% address check in 2011 no matter what the delivery design. • Plan is for a rolling address check over a longer period of time (4 -6 months). – enables better quality address checking; and, – enables more time to update the final address register with Address check findings. • Consider a re-address check shortly before the Census in a small %’age of areas

Early conclusions cont … • Main enumeration and controlled errors didn't identify all missing addresses – need to improve methods and guidance • Issues around whether information with address suppliers can be shared; • Criteria for deciding which address products to use as a base under consideration.

Delivery Method Results

Household return rates* by delivery method by ETC * As at 15 July

Success rates* at follow-up by delivery method by ETC Post-out 1 38. 7% Hand delivery 39. 9% 2 29. 1% 29. 4% 3 24. 2% 26. 1% 4 19. 1% 18. 4% 5 14. 9% 16. 3% Overall 23. 9% 24. 3% * As at 15 July

Delivery method – estimated cost savings Estimated cost savings between 100% postout and 100% hand delivery Initial return* rate difference Estimated savings (%’age points) 5 £ 28 m - £ 35 m 6 £ 25 m – £ 33 m 10 £ 6 m - £ 21 m 15 -£ 18 m - £ 1 m * At the start of follow-up – 23 May

Address register coverage New addresses found during the 2007 Census Test by delivery method During delivery Delivery method Post-out No. %’age During follow-up No. %’age Total No. %’age 9 0. 02% 478 0. 94% 487 0. 95% Hand delivery 540 1. 06% 181 0. 36% 721 1. 42% Total 549 0. 54% 659 0. 65% 1, 208 1. 19%

Delivery method - conclusions • Post-out has an impact on return rates, minimal impact which can be addressed through additional follow-up. • A post-out methodology will provide significant savings (£ 25 -£ 33 m) to invest in targeted follow-up and community liaison. • Improvements identified for the address register and follow-up procedures suggest that the levels of AR undercoverage will be small and manageable. Decision: In E&W, post-out will be the primary means (at least 85%) of delivering questionnaires in 2011.

Income – Setting the scene Including income depends on: • results from the Test; and, • consultation on other topics and relative priority of income in relation to other demands. Further analysis required as follows: • Quality and accuracy of responses to income question • Item imputation rates • Public perception • Other data sources

Household return rates* by income/no income question by ETC * As at 15 July

2007 Census Test - Key findings Delivery method • In E&W, post-out will be the primary method (at least 85%) of delivering questionnaires in 2011. Address register development • Address checking will be required for 2011, currently planning 100% for E&W. Outsourcing recruitment, training and pay: • Worked well in the Test and are considering outsourcing for 2011;

Other key findings: Recruitment: • More difficult than expected in some areas, mostly delivery enumerators. • Good MI to identify and manage problems early. Training: • Some development issues with e-learning but overall well received/effective. Pay: • Much simpler system defined thereby avoiding some of the problems experienced in 2001. • Hourly pay worked well, provided good control and flexibility; travel expenses still considered cumbersome. Refinement for 2011 required.

Other key findings: LA Liaison: • Principles and benefits of LA Liaison proven. • Variable engagement across the LAs – some more engaged than others • Going forward we need to consider: – methods for achieving more consistency across LAs; – Making it simpler for ONS and LAs – Achieving Chief Exec buy-in. Follow-up: • Transfer of information held centrally to field staff worked well within the constraints of the Test. • A good start to follow-up is imperative – need to review start dates. • Organisation and management of field staff worked well, but more development required on doorstep interaction to convince respondent to respond.

2007 Census Test Evaluation – planned publications Publication Statistical Evaluation of the 2007 Census Test - Evaluation of the delivery method 2007 Census Test - Evaluation of the Income question Evaluation of 2007 Census Test – Summary Report

Did you know? Some other key Census developments: • Rehearsal - Spring 2009 • Route A – contract to be let in January 2008 • Route C – start procurement in January 2008 • Finalising questionnaire – Spring 2008 • White Paper – Autumn 2008

Questions ? ?

Household return rates* by delivery method by LA * As at 15 July

Household return rates* by income/no income question by LA * As at 15 July