Case Completeness and Data Accuracy in the National

Case Completeness and Data Accuracy in the National Program of Cancer Registries KK Thoburn, CDC/NPCR Contractor RR German, M Lewis, P Nichols, F Ahmed, CDC J Jackson-Thompson, University of Missouri-Columbia 2007 Annual NAACCR Conference Detroit, Michigan Tuesday, June 5, 2007

NPCR Cancer Surveillance System (NPCR-CSS) l Established in 2000 l In collaboration with SEER, collects cancer incidence data for the entire US population l NPCR-CSS data submissions are evaluated for case completeness (CC), percentage of death-certificate-only (DCO) cases, percentage of unknown/missing values for selected required data elements, and percentage of cases passing critical edits for data quality l Measures incidence rates by various characteristics l Data estimates CCR progress in meeting NPCR Program Standards

NPCR Technical Assistance and Audit Program (NPCR-TAA) l l l ORC Macro conducts NPCR-TAA Performs case completeness (CC) and data accuracy (DA) audits at the level of the hospital reporting to the CCR Uses sample of cases from selected facilities for reabstraction – – l Lung and bronchus, colon and rectum, prostate, and female breast These primary sites represent over 50% of cases reported to the NPCR-CSS Provides technical assistance to CCRs

NPCR Annual Program Evaluation Instrument (NPCR-APEI) l Web-based instrument completed annually by all NPCR-funded CCRs l Assessment of NPCR program objectives, registry operations, and data use

Present Study l l Usefulness of NPCR-CSS data depends on completeness of case reporting and accuracy of submitted information NPCR-CSS data are frequently used to measure incidence and survival – l Race, sex, age, subsite, and stage at diagnosis Present national-level study: – – Summary analysis of NPCR-TAA information by primary site Covariate analysis of NPCR-TAA results with information from the NPCR-APEI, and CCR achievement of USCS publication and NAACCR certification standards

Methods l NPCR-TAA audit results for 34 of 45 NPCR-supported CCRs audited at the time of analysis were included l 1998 through 2001 audit diagnosis years were included l Overall average case completeness and data accuracy rates, and site-specific data accuracy rates were calculated – l DA rates calculated by site for each of the 13 data audited data Average CC and site-specific DA rates were examined across the covariates – Small-sample Student’s t tests (α = 0. 05)

Methods CC rates (%) 100 - (# of missed cases / total # of cases identified) x 100 DA rates, overall and site-specific (%) = (# of data elements with no discrepancies / total # of data elements reabstracted) x 100 DA rates, audited data elements (%) = (# of reabstracted cases with no discrepancies on data element / total # of cases reabstracted) x 100

Methods: Sources of Variables NPCR-TAA: l l CC and DA rates Primary site Audited data elements Source of missed cases USCS publication: l Met publication standards? NAACCR Web site: l l Certified by NAACCR? Certification level NPCR-APEI: l l l l Type of current funding? Ratio of FTE positions to central registry caseload? Ratio of CTRs to central registry caseload? Supplementary reporting sources reporting cases? Case-finding audits at reporting facilities? Reabstracting audits at reporting facilities? Annual report issued?

Results: Case Completeness l 34 CCRs, diagnosis years 1998 to 2001 l 41, 521 lung and bronchus, colon and rectum, prostate, and female breast were identified l 1, 503 cases were missed l Overall case completeness rate of 96. 4%

Distribution of Missed Cases (N = 1, 503) Female breast 32. 7% (n = 491) Lung and bronchus 19. 9% (n = 299 ) Colon and rectum 18. 8% (n = 283) Prostate 28. 6% (n = 430)

Distribution of Missed Cases by Type of Case-finding Source Lung and bronchus Colon and rectum n = 324* n = 315* 29. 0% 43. 2% Medical records disease index Pathology report 0. 7% 1. 3% 0. 2% 40. 6% 55. 6% 62. 0% Female breast n = 544* 0. 6% 0. 2% 4. 6% 0. 3% 0. 6% 3. 7% 2. 2% 3. 1% Prostate n = 458* Radiation therapy log 45. 8% 53. 9% Cytology report 52. 0% Autopsy record *Counts include missed cases from more than one case-finding source. Other

Site-Specific Data Accuracy 99. 7 99. 8 100. 0 99. 9 99. 8 99. 9 99. 7 99. 5 Gender State of residence Date of birth Behavior Primary site Laterality Race Sequence number Diagnosis date Histology Subsite Grade SEER Summary Stage 81. 2 99. 5 99. 2 99. 4 99. 3 99. 8 98. 7 99. 9 98. 9 99. 4 97. 4 100. 0 99. 9 95. 4 99. 1 98. 4 97. 8 97. 5 98. 0 96. 9 97. 1 98. 2 97. 5 95. 5 98. 5 95. 3 96. 2 89. 1 88. 3 97. 6 89. 7 89. 1 93. 6 100. 0 Lung and bronchus (n = 2, 448) Colon and rectum (n = 2, 063) Prostate (n = 2, 289) Female breast (n = 3, 210) 89. 7 93. 9 89. 9 85. 9 83. 5 84. 3 92. 0 93. 5 0 10 20 30 40 50 60 70 80 90 100 Percentage of records containing no discrepancies

Covariate Analysis Ratio of FTEs and CTRs to CCR Caseload l l l Higher CC and DA rates were found for the more wellstaffed registries (not significant; P >. 05). Higher DA rates were found for registries staffed with a greater number of CTRs (significant for colon and rectum, prostate) Higher CC rate was found for registries staffed with a greater number of CTRs (not significant; P >. 05).

Covariate Analysis Supplementary Reporting Sources l l Higher CC rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (P <. 01). Higher site-specific DA rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (not significant; P >. 05).

Covariate Analysis Met USCS Publication Standards/Achieved NAACCR Certification l Higher CC and average site-specific DA rates were found for CCRs that achieved these 2 milestones (only colon and rectum significant; P =. 02, USCS; P =. 04, NAACCR certification) l NAACCR-certified CCRs generally had higher data element-specific DA rates than non-certified CCRs

Conclusions Summary Analysis of NPCR-TAA Data l Underscores importance and effectiveness of conducting CC and DA audits at reporting hospitals l Enables identification of general and site-specific casefinding and abstracting issues l Demonstrates overall high accuracy and completeness of NPCR-CSS incidence data on cancers of the lung and bronchus, colon and rectum, prostate, and female breast – – l Overall CC (96%) higher than NPCR-CSS 24 -month standard (95%) Overall DA rate (95%), and high DA rates found by demographic data elements and primary site lend confidence in incidence rates Provides guidance to users of the data

Conclusions Covariate Analysis l Shows that combining data from the NPCR-TAA and the NPCR-APEI provides additional valuable information that neither program can provide individually l Offers perspective on how CCR operations effect case completeness and data quality l Underscores importance of CCRs having well-trained staff; having supplemental reporting sources; and attaining compliance with national data standards

Conclusions l NPCR funding and technical assistance help CCRs in developing and enhancing effective registry operations—especially in areas such as staffing, training, and monitoring and in improving the completeness and quality of registry l Present study demonstrates: – – Positive outcome of enhanced registry operations Benefit of NPCR support for a high-quality, statewide, population-based CCR

Special Appreciation Arizona Arkansas Colorado District of Columbia Florida Georgia Idaho Illinois Kansas Maine Maryland Massachusetts Michigan Minnesota Missouri Montana Nebraska Nevada New Hampshire New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island Tennessee Texas Vermont Virginia Washington West Virginia Wisconsin