UN Comtrade United Nations Commodity Trade Statistics Database Markie Muryawan Statistician Trade Statistics Branch United Nations Statistics Division Email: markie@un. org Tel: +1 -212 -963 -0692 ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 1
Importance of trade data (1) ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 2
Importance of trade data (2) China is now the biggest ICT Goods Exporter ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 3
What is UN Comtrade? • • International Merchandise Trade Statistics Database Maintained by UN Statistics Division / Trade Statistics Branch Inside UN Comtrade database: • – – – • Imports, Exports, Re-exports, Re-imports Commodity by Trading Partner Annual data (since 1962) Value in US dollars Net weight and secondary quantities Over 150 reporters per year (>95% of the World Trade) Received 2005 prestigious UN 21 Awards in the category of Knowledge Management ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 4
Key elements • Dimension – Classifications and Commodity Codes • Harmonized System (HS) • SITC • BEC – Reporter country – Partner country – Trade Flow – Year • Measure / indicator – Net weight – Supplementary quantity unit – value ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 5
Main trade data source • UN Comtrade is the largest database of annual commodity trade statistics • On average, more than 2 billion records downloaded every month • Newly published data synchronized with other international organizations such as World Bank, ECLAC, OECD, ITC, etc via web service Free Site License trial subscription (please contact comtrade@un. org) ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 6
Data collection & processing ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 7
Cooperation with OECD Joint UN/OECD Data Collection and Processing System (Copr. A) – Joint collection = Reduced response burden – Joint standards = Increased Quality – Joint processing = Full harmonization of data ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 8
Collection • Official annual request sent in February to national institution responsible for dissemination of trade statistics: – – – • Almost all data come from custom declarations, with an addition of postal services, oil ministry, electricity, sales/lease of aircraft/ship, etc About 80% data sent from National Statistical Offices and 20% sent from statistics unit in Custom Administration Most of reporters report data in more than 6 digit HS Most countries report in electronic format (via CD, FTP or email attachment) by June (for major countries), large chunk of data reported between March and June Since 2001, UN and OECD decided to collect data jointly: OECD collects OECD countries and shares them with UN. This cooperation aims to eliminate redundancy in data collection. • – • In the beginning 2006, OECD and UN released joint data processing system, named Copr. A. Reported data format must be refitted into standard format (see Key Element slide). – – This pre-processing task consumes a lot of time compare to processing itself (see next slide) SDMX format is intended to reduce pre-processing time and minimize error in transforming data format ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 9
Processing – UN/OECD Copr. A • Joint UN/OECD Processing System, Copr. A (2006) Main Processing Flows: • – – – – • Data format standardization Convert into US$ Codes Normalization Data Aggregation Quantity Estimation Data Checking Data Conversion Publish to UN Comtrade Processing flows controlled by system During processing, data is duplicated and stored – original, tariff line, complete and final data for publication ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 10
Challenges in processing • Original format is non-text or non-database – • How to handle confidential/higher level data Estimating and Checking quantities • – – – • • • Such as PDF or printed publication Unit Value rejected due to standard unit value Missing quantity units / net weight Outliers of unit value Different quantity units with WCO Non HS or high level HS (< 6 digit) reported Incomplete partners/commodities breakdown To make sure data is correct and make sense Lead to ALA BRASS Publishers’ Open Forum, January 2007 Delay in releasing data and/or lower data quality United Nations Statistics Division 11
Countries & Areas (1) • General rule – – Based on country definition of custom territories (legality) Copr. A processing system transforms detailed partner into UN Comtrade countries & areas • • For an example: US data includes US Virgin Islands and Puerto Rico, thus, any data referring US VI and PR as partners will be lumped together with US China Hong Kong SAR (344), China Macao SAR (446) and China (156) are two separate areas in UN Comtrade. They also declares trade to each others. Before 1981 USA was 841 (incl. PR) ALA BRASS Publishers’ Open Forum, January 2007 Since 1981 USA is 842 (incl. PR and USVI) United Nations Statistics Division 12
Countries & Areas (2) • Areas – – – • France (France, French Guiana, Guadeloupe, Martinique, Monaco, Réunion ) Italy (Holy See, Italy, San Marino) Norway (Norway, Svalbard and Jan Mayen Islands) USA (Puerto Rico, United States, US Virgin Isds) Switzerland (Liechtenstein, Switzerland) Serbia and Montenegro (Kosovo, Montenegro, Serbia) Any major splits or joints, new country code may be assigned – For an example: separate codes for Kosovo, Montenegro, Serbia ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 13
Data availability ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 14
General Data Availability • • • There are 5221 country-period datasets from 1962 -now, 1. 1 billion records, and 224 countries/areas as reporters (as of Jan 2007) Data availability is accessible via “Data Availability menu”, RSS and web services (http: //comtrade. un. org/kb/article. aspx? id=10089) Tariff Line data (detailed national level commodity codes) is not available for public use except in a project between international organizations and universities ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 15
Not yet available reporters • Every user must take into account “not yet available reporters” – – • • • ALA BRASS Publishers’ Open Forum, January 2007 Will have impact on calculation of country groups Not-yet-available reporters are shown in the front page 54% of countries represent 96% of world trade (based on mbs country tables) Normally, by June, almost all major countries have reported their data It is updated as soon as data comes in United Nations Statistics Division 16
HS 2007 & SITC Rev. 4 ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 17
HS 2007 & SITC Rev. 4 • • Addition of HS 2007 will increase UN Comtrade commodity classifications from 7 to 9 User must understand impact of break in time series HS-2007 will be the starting point of the classification conversion linking to earlier HS and SITC, and to CPC, ISIC, and BEC Correlation between SITC Rev. 4 and HS 2007 is available on UN Comtrade web sites ALA BRASS Publishers’ Open Forum, January 2007 • Time Series Continuation – – – HS-2007 appearing code 0105. 94 HS-2002 disappearing codes 0105. 92 and 0105. 93 0105. 92 = Chicken < 2 Kg 0105. 93 = Chicken > 2 Kg 0105. 94 = 0105. 92 + 0105. 93 Which code to choose for Time Series continuation in HS-2002 as of 2007 ? United Nations Statistics Division 18
UN Comtrade Portal ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 19
UN Comtrade Portal Page comtrade. un. org ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 20
UN Comtrade Sites • Database – • Contains the largest trade statistics database Publication – • Online version of International Trade Yearbook Web Services – • Facilitate automated large data transfer Knowledge base – • Guidelines, User Manuals, Reference Tables, Metadata, FAQ, Methodology, How-To, etc. Any knowledge related to trade statistics Community Forum – • Peer-to-peer support. Discuss anything from incorrect quantities to SDMX. Expert Forum – Restricted membership, by invitation only. To discuss various issues regarding IMTS and supporting tools. ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 21
Data Extraction • Do-it-your-self extraction – – – • • Comtrade Explorer (text search) Shortcut Query Express Selection Basic Selection Via data availability Via web services Selecting commodity classification is crucial for historical time series Download up to 20 million records per query via batch submission Special data extraction is not offered anymore ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 22
Express Selection Text sensitive search of commodities and countries EXAMPLE • Auto convert from text to code • More features than Basic Selection • Expert User Possibility to execute data requests off-line ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 23
Comtrade Explorer • • “Google like” search Data results in table and graph Modification in Basic or Express Selection For data mining and overviews EXAMPLE Data availability of search result shown in Tag Cloud ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 24
Summary of data request EXAMPLE View Map Germany = main exporter of cars over the last 4 years ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 25
Map shows World Exports of motor vehicles : Germany and Japan are leaders ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 26
Data Availability • • • ALA BRASS Publishers’ Open Forum, January 2007 Interactive In Graph Reporters Commodities Years By Date of Loading RSS United Nations Statistics Division 27
Data Availability by Reporter • Click to get all data • Drill down possibility • Modify with Express Selection EXAMPLE Commodity Classification Years available Iceland = Reporting Country ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 28
Data Availability by Year • • Overview per year and classification Click to get all data Drill down possibility Modify with Express Selection EXAMPLE All the Reporters available for Year = 2005 Classification = HS 2002 ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 29
Data Download For Subscribers, not for Guest User • Direct Download – Max. 150, 000 records to CSV output file – Instant extraction. Uncompressed file. • Batch Download – Max. 20, 000 records to CSV output file – Background extraction. Compressed file – Notification of completion via e-mail. • Output format can be customized ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 30
Personalization • • • For Subscribers, not for Guest User Favorites Saved Queries Front. Page Download Format Automated Data Extraction E-mail alerts ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 31
Favorites For Subscribers, not for Guest User • Country Groups (as reporter or partner) – Such as EU 25, ASEAN – During data extraction, it is considered as one country • Commodity Groups Commodities – Such as ICT products, Fuel, etc – During data extraction, it is considered as one commodity • Reporters • Partners • Years ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 32
Metadata & Reference • Commodity List – HS, SITC, Data Availability and Snapshot • Countries List – Data Availability, Snapshot, National Practices and ITC Country Profile • Glossary – Also refer to Knowledge Base which contains all detailed terminology used in UN Comtrade • Suggested Articles – Link to knowledge base with page sensitive articles ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 33
Summary ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 34
Pricing Policy for libraries http: //comtrade. un. org/db/help/Subscription. Prices. aspx • • Not for Profit Rate is 25% of Commercial Rate Additional discounts for libraries from certain countries Extra discount for UN Depository Libraries or library in the consortiums Best deal for library: Site Licence (including 5 individual users) Please contact subscriptions@un. org for further information ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 35
Future work • Methodology – - Supplement to the Compilers Manual • • – – • Part I: Questionnaire on National Compilation and Reporting Practices Part II: Special topics Singling out areas of increasing importance (Export Processing Zones/ Customs unions/ Affiliated trade/ Business Surveys) Database on National Compilation and Reporting Practices New trade statistics yearbook, additional online publication Writing IMTS, Rev. 3 (2010) UN Comtrade – – – Adding World and Region Totals On going interface improvement Improvement on web services (SDMX version 2. 0) Adding classification in different languages Better revision control ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 36
Summary • • Over 1 billion annual merchandise trade data, consists of detailed (6 digit) commodity with more than 209 countries & areas Coverage of reporting countries not available anywhere else Data extraction and Data mining Personalization of your requirements From 1962 until now, perfect for studying historical series Site License: institute-wide access and unlimited download UN Comtrade portal provides complete sets of tools: data extraction, comprehensive help and support and collaboration tools. Single point of contact: comtrade@un. org ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 37
Thank you for your attention Please visit UN Comtrade at booth 1337 ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 38
UN Comtrade Sites (@ booth 1337) • Database – • Contains the largest trade statistics database Publication – • Online version of International Trade Yearbook Web Services – • Facilitate automated large data transfer Knowledge base – • Guidelines, User Manuals, Reference Tables, Metadata, FAQ, Methodology, How-To, etc. Any knowledge related to trade statistics Community Forum – • Peer-to-peer support. Discuss anything from incorrect quantities to SDMX. Expert Forum – Restricted membership, by invitation only. To discuss various issues regarding IMTS and supporting tools. ALA BRASS Publishers’ Open Forum, January 2007 United Nations Statistics Division 39
Скачать презентацию UN Comtrade United Nations Commodity Trade Statistics Database
c579eb6eb22513e052604a4700071742.ppt