Machine Translation activities at WIPO Bruno Pouliquen Christophe

Machine Translation activities at WIPO Bruno Pouliquen, Christophe Mazenc Patentscope workshop June 2013

Agenda 1. History of machine translation activities at WIPO 2. Cross Lingual Search 3. Integration of third party MT engines 4. Development of in-house MT engines tuned for specific tasks 5. Strategy

1. History of MT activites At WIPO

MT at WIPO: history Why is WIPO interested in Machine Translation? The IB of the PCT is responsible for translating titles, abstracts, drawing legends, search reports, written opinions and IPRPs for the published PCT applications. (This represents xx millions of words translated per year) WIPO is disseminating multi lingual Patent Information through it’s portal PATENTSCOPE. Multi lingual functions are required to enable the largest number of users worldwide to search and browse patent applications in many different languages

MT at WIPO: an overview Mid 2007: International RFP to implement “cross lingual Search” functions in PATENTSCOPE End of 2008: project failure due to the supplier’s inability to deliver a quality product 2009: First Statistical Machine Translation experiments performed in-house. Development of a first engine to translate titles from English to French 2009 -2010: development of the PATENTSCOPE CLIR system in 5 languages (EN, FR, DE, ES, JA)

MT at WIPO: an overview Summer 2010: Integration of Google Translate in PATENTSCOPE to translate result lists, descriptions and claims March 2011: development and deployment of WIPO’s first own MT system tuned for patents’ titles and abstracts (TAPTA) April 2011: extension of CLIR to cover the Chinese, Korean, Russian and Portuguese languages August 2011: release of PCT corpus: COPPA

MT at WIPO: an overview November 2011: Integration of KIPO’s machine translation system in PATENTSCOPE (for the KO EN language pairs) (until December 2012) December 2011: Integration of Microsoft Translate into PATENTSCOPE January 2012: extension of CLIR to cover the Dutch, Italian, and Swedish languages Avril 2012: PATENTSCOPE CLIR functionality integrated into Minesoft’s PATBASE through a web service

MT at WIPO: an overview June 2012: provide MT transfer knowledge to UN and ITU October 2012: UN, ITU, Wipo Marks in production November 2012: Extension of Tapta to cover Japanese and German February 2013: Evaluation results: Tapta better than Microsoft and Google (title+abtract in all language pairs, similar results in UN) June 2013: Outsourcing contract using TAPTA for the EN=>FR language pair

2. CLIR （Cross Lingual Information Retrieval)

WIPO’s Cross-lingual search: principle ► Free tool available at http: //patentscope. wipo. int/search/clir. jsp? interface Language=en ► Enter a search query in either EN, DE, ES, FR, JP, RU, ZH, PT, IT, DU, SE and it will be expanded into the other languages (keywords translation) ► Automatic or supervised mode ► balance between precision and recall set by the user ► Disambiguation by technical domains and by selection of appropriate synonyms ► Built from bilingual dictionaries extracted statistically from Patent corpuses without supervision

Interface : Cross-lingual (CLIR)- Automatic

CLIR: automatically enriched query (EN_TI: ("hearing aids" OR "hearing prosthetic"~21 OR "auditory aids"~21 OR "auditory prosthetic"~21) OR EN_AB: ("hearing aids" OR "hearing prosthetic"~21 OR "auditory aids"~21 OR "auditory prosthetic"~21)) OR (DE_TI: ("Hörgeräte" OR "Hörhilfegeräten") OR DE_AB: ("Hörgeräte" OR "Hörhilfegeräten")) OR (ES_TI: ("audífonos") OR ES_AB: ("audífonos")) OR (FR_TI: ("audioprothèses" OR "appareils de correction auditive" OR "production d'appareils auditifs") OR FR_AB: ("audioprothèses" OR "appareils de correction auditive" OR "production d'appareils auditifs")) OR (JA_TI: ("穴形補聴器") OR JA_AB: ("穴形補聴器")) OR (KO_TI: ("보청") OR KO_AB: ("보청")) OR (PT_TI: ("audiofone" OR "auxìlio de audição") OR PT_AB: ("audiofone" OR "auxìlio de audição")) OR (RU_TI: ("слуха протезно"~22 OR "прослушивания протезно"~22 OR "слуха спидом"~22 OR "слуха наведения"~22 OR "прослушивания спидом"~22 OR "прослушивания наведения"~22 OR "слухоулучшающих протезно"~22 OR "слуховой протезно"~22 OR "слухоулучшающих спидом"~22) OR RU_AB: ("слуха протезно"~22 OR "прослушивания протезно"~22 OR "слуха спидом"~22 OR "слуха наведения"~22 OR "прослушивания спидом"~22 OR "прослушивания наведения"~22 OR "слухоулучшающих протезно"~22 OR "слуховой протезно"~22 OR "слухоулучшающих спидом"~22)) OR (ZH_TI: ("助听器") OR ZH_AB: ("助听器"))

Why use PATENTSCOPE CLIR? A) Search full text collections simultaneously in many foreign languages without knowing them (not English centric) B) Improve significantly the number of relevant results without increasing significantly the number of irrelevant results 3356 results in English titles or abstracts for hearing AND aids 3825 results obtained with CLIR searching in titles or abstracts in all languages C) Have confidence in your searches: No black box: users have access to the CLIR generated boolean queries (albeit complex) and have the full control on them D) Have a responsive system even for complex queries the query in the previous slide executes in less than 1/2 sec in PATENTSCOPE

What next? Improve terminology coverage of already supported languages Add other languages (Arabic)? Condition to add a language: Having more than 200’ 000 (ideally 500’ 000) titles and if possible abstracts in the language available with associated high quality translations in English

3. Integrated third-party MT engines

Integrated 3 rd party MT: principles ► Use free MT services available on the internet (so far Google Translate and Microsoft translate) ► Translates from the source language(s) to the language set by the user in the graphical interface ► Translates results lists and description and claims only when requested by the user ► 65 languages supported using Google Translate! ► Quality of Google Translate improved for patent texts thanks to EPO sharing patent corpora with Google

Search Results – machine translate

Description – machine translate

4. Development of in-house MT engines tuned for specific tasks

In-house MT engines MT systems building expertise developed in-house since 2009 Corpora approach: started using PCT corpus of titles and abstracts Uses open source Statistical Machine Translation: Moses (WIPO is a committer with a specific branch) First system developed: Translation Assistant for Patent Titles and Abstracts (TAPTA: publicly available at https: //www 3. wipo. int/patentscope/translate) Same system (trained on different corpora) developed for the United Nations, for ITU and for translation of Madrid Trademarks goods and services

TAPTA Hovering the mouse on the left highlights corresponding segment on the right (and viceversa)

How well does it work? Tapta better than Google and Microsoft for abstracts English->French: Tapta BLEU 46. 9 15 abstracts* Google 45. 9 / Google-EPO 45. 8 / Microsoft 36. 7 German->English: Tapta BLEU 38. 3 11 title & abstracts* Google 37. 8 / Microsoft 26. 8 Human evaluation: adequacy/fluency (Tapta: 79%, Google 65%, Microsoft 67%) English->Japanese: Tapta BLEU 25. 4 1000 segments (title & abstract)* Google BLEU 22. 3 English->Chinese: Tapta BLEU 22 1000 segments (title & abstract)* Google BLEU 17. 5 (*) from recent patent applications (published in March 2013), compared to one reference

Also in United Nations BLEU scores Language pair Google Bing Tapta ar-en Aims at assisting UN translators when translating UN official documents from AR, ES, FR, RU, ZH into EN (both directions) 55. 25 n/a[1] 51. 17 en-ar 44. 10 33. 74 28. 94 en-es 61. 81 53. 39 46. 86 en-fr 51. 23 45. 58 42. 19 en-ru 50. 85 39. 67 38. 96 en-zh 43. 17 34. 16 32. 77 es-en 60. 32 52. 54 49. 18 fr-en 53. 36 46. 46 43. 39 ru-en 58. 56 47. 71 47. 09 zh-en 42. 31 36. 55 30. 60

Findings Customized MT engines built on narrow language domains outperform state of the art general purpose MT engines TAPTA automatic evaluations are better than Google Translate on patent titles and abstracts (BLEU scores) Size of corpora matters, as well as quality of sentencepairs alignments Building customized SMT engines is sustainable and does not require large human, IT and financial resources

Bibliography TAPTA: A user-driven translation system for patent documents based on domain-aware Statistical Machine Translation, B. Pouliquen, C. Mazenc, A. Ioro in proceedings of the European Association for Machine Translation conference, May 2011, Leuven Belgium COPPA, CLIR and TAPTA: three tools to assist in overcoming the Patent language barrier at WIPO, B. Pouliquen, C. Mazenc in proceedings of Machine Translation Summit 2011, September 2011 Xiamen China Statistical Machine Translation prototype using UN parallel documents, B. Pouliquen, C. Mazenc, C. Elizalde, J. Garcia-Verdugo in proceedings of the 16 th EAMT conference, 28 -30 May 2012, Trento, Italy (forthcoming ) Large-scale multiple language translation accelerator at the United Nations, B. Pouliquen, C. Elizalde, M, Junczys-Dowmunt, C. Mazenc, J. Garcia-Verdugo in proceedings of Machine Translation Summit 2013, Nice, France

5. Strategy

WIPO’s MT strategy Make best use of state-of-the-art technologies available in open source and promote further their development Adapt these technologies to the patent domain (using Patent corpora, Patent classification, …) for practical use cases Develop patent MT systems and put them at disposal of the largest number of users to bridge the language barrier (notably in patent searching) Cooperate with interested offices by sharing experience, corpora and software solutions Adopt a barrier free dissemination of patent corpora when possible to foster research in MT for patent texts Investigate Cloud technologies to be able to ramp up to industrial internet solutions TAPTA: Extend coverage (languages, claims, descriptions)

Questions?