DATA QUALITY AND META DATA PRESENTED AT THE

DATA QUALITY AND META DATA PRESENTED AT THE TRAINING COURSE ON ‘INTERNATIONAL TRADE STATISTICS’ BRUNEI DARUSSALAM 20 -22 SEPTEMBER 2016

OUTLINE 2 1. IMTS 2010 RECOMMENDATIONS 2. DATA QUALITY 3. QUALITY REPORT 4. METADATA 5. DISSEMINATION

3 IMTS 2010 RECOMMENDATIONS

1. IMTS 2010 RECOMMENDATIONS 4 i. Systematic approach to data quality ii. Dimensions of data quality iii. Contents of quality reports iv. Quality indicators v. Cross-country data comparability

5 DATA QUALITY

2. DATA QUALITY 6 WHY Data Quality is Important Credibility To ensure reliable and accurate data for input into BOP and GDP And also for policy making

2. DATA QUALITY 7 Validation Errors Data is certainly incorrect May cause processing Problems Cannot be included in statistical analysis Must be removed/corrected Examples text characters in value field invalid commodity or country code

2. DATA QUALITY – cont. 8 Credibility Errors Data is valid but possibly incorrect Can be processed Examples: inflated value (currency) or quantity (*000) valid but wrong commodity code

2. DATA QUALITY – cont. 9 Validation Steps Electronic Entry Validity Check Valid Credibility Check Server JP Server SMK Credible Interim data of External Trade Statistics Invalid Auto-correction

2. DATA QUALITY – cont. 10 Quality Controls (a) Coverage check (b) Front-end check (c) Back-end check

2. DATA QUALITY – cont. 11 Electronic Hardcopy Statements from Free Zone Authority, Power Producer Company, Petroleum Company, MAS, Air Asia, etc) 11

2. DATA QUALITY – cont. 12 (b) Front-end Check High value transactions Cross-check Range-check Masterfile-check Computation-check

13 EXAMPLES OF VALIDATION CHECK

BXXXXW 14 COMPANY ABC JALAN ABCXYZ High Value Transactions COMPANY XYZ ADDRESS OVERSEAS AZ 9999 SYARIKAT XXX TAN AH BAH

15 BXXXXW COMPANY ABC JALAN ABCXYZ XXXX YYYYYYY NEGERI AA Country of Origin COMPANY XXXMMMM JALAN ABCXYZ XXXX YYYYYYY AZ 9999 COMPANY XXXMMMM TAN AH BAH

16 BXXXXW COMPANY ABC JALAN ABCXYZ XXXX YYYYYYY NEGERI AA Unit of Quantity COMPANY XXXMMMM JALAN ABCXYZ XXXX YYYYYYY NEGERI CINA AZ 9999 COMPANY XXXMMMM TAN AH BAH

17 BXXXXW COMPANY ABC JALAN ABCXYZ XXXX Country of destination COMPA JALAN ABCXYZ XXXX YYYYYYY NEGERI AA AZ 9999 COMPANY XX TAN AH BAH

18 BXXXXW Channel COMPANY ABC JALAN ABCXYZ XXXX YYYYYYY NEGERI AA COMPANY XXXMMMM JALAN ABCXYZ XXXX AZ 9999 COMPANY XXXM TAN AH BAH

19 Code BXXXXW COMPANY ABC JALAN ABCXYZ XXXX XXXY COMPANY XXX AZ 9999 COMPANY XXXMMMM TAN AH BAH

Quality Check (METSS) 20 BXXXXW COMPANY ABC JALAN ABCXYZ XXXXXX YYYYYYYXXX COMPANY XXXMMMMMMM AUP CHECK JALAN ABCXYZ XXXXXX YYYYYYYXXXXX AZ 9999 COMPANY XXXMMMMM TAN AH BAH

Quality Check (METSS) cont. 21 BXXXXW COMPANY ABC JALAN ABCXYZ XXXX YYYYYYY COMPANY X JALAN ABCXYZ XXXXMMMMM XXXX YYYYYY AZ 9999 Error in Tariff Code Change HS AHTN COMPANY XXXMMMMM TAN AH BAH ASEAN country

2. DATA QUALITY – cont. 22 Imports and Exports by Country Imports and Exports by Customs Station/ Channel Imports Exports Ratio Cross-check with Other Countries Checking of Selected Products (Aircraft, Rubber, Petroleum, Copper, etc)

23 QUALITY REPORT - MALAYSIA

3. QUALITY REPORT 24 Malaysia External Trade Statistics Quality Report was sent to ASEAN Secretariat in August 2013 – in line with the recommendation of IMTS 2010. Benchmark to UK’s Quality Report

3. QUALITY REPORT 25 Quality Dimensions 1. 2. 3. 4. 5. Timeliness Accuracy Accessibility & Clarity Comparability Coherence

3. QUALITY REPORT – cont. 26 Quality Indicators 1) Timeliness - Advanced Release Calendar (ARC) YEAR MONTHLY METS ANNUAL METS 2009 5 -6 weeks 10 months 2010 5 -6 weeks 10 months 2011 -current 5 -6 weeks 9 months

3. QUALITY REPORT – cont. 27 Quality Indicators – cont. 2) Accuracy 2. 1 Accuracy Characteristics 2. 2 Controls using primary information/ sources 2. 3 Controls using secondary information/source 2. 4 Confidentiality 2. 5 Actions to improve accuracy

3. QUALITY REPORT – cont. 28 Quality Indicators – cont. 2) Accuracy – cont. 2. 1 Accuracy Characteristics i) Statistical Value Exports – free on board (f. o. b. ) Imports – cost, freight and insurance(c. i. f. ) ii) Level of thresholds From January 2013, single transactions with a value ‘below RM 1, 000’ for Peninsular Malaysia are excluded except for major commodities and selected goods.

3. QUALITY REPORT – cont. 29 Quality Indicators – cont. 2) Accuracy – cont. 2. 1 Accuracy Characteristics – cont. iii) Exclusions Transit and goods for repair iv) Revisions - The monthly data are provisional when 1 st released and subject to revision in the following month. - For annual, it will be released 9 months after the reference year.

3. QUALITY REPORT – cont. 30 Quality Indicators – cont. 2) Accuracy – cont. 2. 2 Controls using primary information/sources i) General information ii) Information on controls and errors

3. QUALITY REPORT – cont. 31 Quality Indicators – cont. 2) Accuracy – cont. 2. 2 Controls using primary information/sources i) General information a) Data providers b) Number of transactions YEAR IMPORTS % CHANGE EXPORTS % CHANGE 2009 1, 340, 407 - 842, 529 - 2010 1, 447, 348 8. 0% 977, 095 16. 0% 2011 1, 479, 002 2. 2% 994, 848 1. 8% 2012 1, 555, 057 5. 0% 1, 118, 218 12. 0%

3. QUALITY REPORT – cont. 32 Quality Indicators – cont. 2) Accuracy – cont. 2. 2 Controls using primary information/sources i) General information – cont. c) Electronic declarations YEAR IMPORTS EXPORTS 2009 85. 0% 82. 0% 2010 95. 0% 93. 0% 2011 96. 0% 95. 0% 2012 97. 3% 96. 8%

3. QUALITY REPORT – cont. 33 Quality Indicators – cont. 2) Accuracy – cont. 2. 2 Controls using primary information/sources i) General information – cont. d) Information on controls and errors - Validity checks - Credibility checks - Methodology for credibility checks

3. QUALITY REPORT – cont. 34 Quality Indicators – cont. 2) Accuracy – cont. 2. 2 Controls using primary information/sources i) General information – cont. d) Information on controls and errors - Validity checks - Credibility checks - Methodology for credibility checks

3. QUALITY REPORT – cont. 35 Quality Indicators – cont. 2) Accuracy – cont. 2. 3 Controls using secondary information/source i) Statement on the imports/exports of oil and gas ii) Statement on the imports of tobacco iii) Report of total values and total number of transactions declared in Free Commercial Zone

3. QUALITY REPORT – cont. 36 Quality Indicators – cont. 2) Accuracy – cont. 2. 4 Confidentiality - Statistics Act 1965 (Revised – 1989) 2. 5 Actions to improve accuracy

3. QUALITY REPORT – cont. 37 Quality Indicators – cont. 3) Accessibility and clarity 3. 1 Accessibility - Publications - METS Online 3. 2 Clarity - Documentation available to users in all publications

3. QUALITY REPORT – cont. 38 Quality Indicators – cont. 4) Comparability and Issues 4. 1 Comparability over space - Methodological discrepancies in reporting - Triangular trade - Exchange rate - Reporting time lags - Classification of goods - Confidentiality 4. 2 Comparability over time

3. QUALITY REPORT – cont. 39 Quality Indicators – cont. 5) Coherence - Inclusion of ‘ Specific Movements’ data into METS - Mirror statistics analysis

4. METADATA 40 The term metadata defines all information used to describe other data. It is recommended that countries at least cover the following categories of metadata for their international merchandise trade statistics: (i) description of all underlying concepts and definitions, including the trade system used, and deviations from international standards, if any; (ii) legal framework, institutional arrangements and description of data sources; (iii) description of data collection and data processing procedures; (iv) description of estimation methods; (v) data dissemination policy including release and revision schedules; (vi) description of the all data fields/ variables (reference period, trade flow, commodity classification used, valuation, currency, quantity (net weight), weight unit used, supplementary quantity unit used, partner country (origin, last known destination, consignment) (this applies to all forms of data dissemination); (vii) explanations and footnotes concerning the data as required, i. e. informing about revisions, break in series and application of confidentiality; (viii) quality reporting

5. DISSEMINATION 41 The dissemination of data and metadata, therefore, should be treated as an integral part of the national programme of trade statistics and carried out with great care and attention to the needs of users, while at the same time ensuring adequate confidentiality of data providers. It is recommended that passive confidentiality be applied as much as possible unless the use of active confidentiality is already the established, desired and accepted practice. Reference period. It is recommended that countries make their data available on a calendar period basis, according to the Gregorian calendar and consistent with the present IMTS 2010 recommendations. Countries are encouraged to issue their first releases of data as follows: (a) Monthly totals of exports and imports within 45 days after the end of the month, at least by major trading partners and basic commodity breakdown; (b) Quarterly data within 60 days after the end of the quarter; (c) Annual data within 90 days after the end of the year Revision policy. Countries are encouraged to develop a revision policy that is well designed, carefully managed, transparent and well coordinated with other areas of statistics, and hence allows users to cope with revisions in a systematic manner. It is recommended that countries choose the dissemination format that best suits their user needs. It is further recommended that international merchandise trade statistics be accessed through the electronic dissemination databases maintained by the responsible agency.

42