IIPS 6 A 9 Interactive Quality assurance practices

IIPS 6 A. 9 Interactive Quality assurance practices A look at some methods for evaluating COOP data at the NOAA National Climatic Data Center Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC 28801 1. 828. 271. 4223 Karsten. Shein@noaa. gov NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

COOP Climate Data • ~ 8500 stations reporting • ~ 1, 000 observations per month • Manual observations and reporting – Most stations observe PRCP (SNOW) – Many also observe TMAX, TMIN, TOBS, SNWD – Few: DYSW, EVAP, WTEQ, WDMV, Soil T • Daily data / Monthly processing • Arrive at NCDC in electronic (daily) and paper (monthly) formats. 2 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Sources of bias in COOP data • Observations – Rounding, Instrument error, incorrect obs technique • Recording – Transposition, Wrong column, Wrong units, Wrong sign, illegibility, nonentries, wrong resolution, wrong date, wrong time. • Transcription (keying) – Most keying errors are due to bias already introduced by the recording step • Transmission (file corruption) • Validation / Quality Control – Failure to review forms prior to NCDC – Incorrect metadata – Inappropriate validation, flagging, estimation 3 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Not all COOP observations are created equally 4 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

5 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Error corrections or error creations? TMIN and TOBS in wrong columns. 10° F or 12° F ? TMAX of 19° F with TOBS of 21° F ? Keyers must “key what they see. ” 6 20 inches of snow or 2. 0? NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Automated COOP QC • Primary checks on Temperature and Precipitation elements – Around 10 million values per year • Internal consistency – Logical (e. g. , TMAX ≥ TMIN) – Spikes, flatliners, outliers, excessive range, change points – Date shifting • Spatial consistency • Values not automatically invalidated unless logically or meteorologically impossible. – Suspect data are reviewed by operators • Original values are NEVER changed or edited – unless they were keyed incorrectly 7 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

First stage Interactive QC “Quasi-Interactive” • Applied to values deemed suspect and where a logical operation will not provide resolution (e. g. , date shifting) • Value compared to climatological neighbors and to computed grids (GEA, Temp. VAL, Precip. VAL) – See extended abstract or http: //www. ncdc. noaa. gov/oa/hofn/coop-pubs-doc. html for references and details. • Decision is made to accept/reject valid status assigned by automated QC • Operator intervention is part of decision process 8 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Fully-Interactive QC • Health of the Network • Datzilla 9 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Health of the Network Web-based tool for viewing the results of NCDC QC processing. – Available once final QC has been completed – Output for: TMAX, TMIN, TOBS, PRCP, SNOW, SNWD – Accessible by anyone via the Internet – Track QC by station, state, WFO, NWS region, or RCC region. – Graphical and tabular reports to highlight quality issues. http: //www. ncdc. noaa. gov/oa/hofn/index. html 10 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Health of the Network • Data Completeness (number of obs) • Quality Assurance (invalid and missing) • Missing Data / Non-reporting stations • Data validity (% unflagged, non-missing) • Watch list (change points detected w/o corresponding metadata) 11 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

12 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Summary from Ho. N • All TMAX, TMIN, TOBS, PRCP, SNOW, SNWD data subjected to automated QC. • 96. 72% of checked 2006 COOP data declared valid (no further checks). • 3. 28% declared invalid (331, 777) – 14. 65% no estimate (48, 602) – 85. 36% estimated (283, 175) ◦ 31% of estimates supplied by Temp. Val (87, 820) Thanks to Helen Frederick, Ho. N administrator, for supplying the numbers. 13 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Datzilla • Fully manual, interactive quality assurance • Web-based tool to report and track errors in NOAA-held data, metadata or associated delivery systems. • Developed and maintained by Kevin Robins at the SRCC http: //datzilla. srcc. lsu. edu/datzilla/ 14 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

15 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Some possible reasons for a Datzilla ticket to NCDC • Data issue (usually) – That TMAX of 74 should be a 47! – Your #$&!*@ QC clobbered my data! • System issue – CDO inventory doesn’t match the data I got! • Metadata issue – This station’s COOP number is wrong! 16 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

When NCDC receives a Datzilla ticket … • • Datzilla gatekeeper Initial determinations Reassignment Investigation Course of action Resolution Closure 17 Your friendly Datzilla Gatekeeper NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Datzilla Summary • Began operation early 2005 • As of 1/15/08: – 865 Datzilla entries (564 NCDC) – 226 open (109 NCDC) Data quality ◦ Receive about 15 new tickets per month – 455 of the 564 resolved – 317 verified errors with archive fix ◦ Most are single values or metadata ◦ Few involve many values or larger station issues Data errors • Has resulted in improvement to the historical climate record. 18 NOAA National Climatic Data Center http: //www. ncdc. noaa. gov

Thank You Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC 28801 1. 828. 271. 4223 Karsten. Shein@noaa. gov NOAA National Climatic Data Center http: //www. ncdc. noaa. gov