ITU-T Workshop on Telecommunications relay services for persons

ITU-T Workshop on “Telecommunications relay services for persons with disabilities ” (Geneva, 25 November 2011) Telecommunications Relay Services in Speech-to-Speech translation system in accordance with Recommendations F. 745 and H. 625 Chiori Hori Ph. D. Spoken Language Communication Laboratory National Institute of Information and Communications Technology (NICT) Geneva, 25 November 2011 2018/3/18 MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 1

Telecommunications Relay Services in Speech-to-Speech translation in accordance with ITU-T Recommendations F. 745 and H. 625 Network-based S 2 ST systems Speech-to-Speech Translation (S 2 ST) technologies are an effective means to break through language barriers between people who do not speak the same language. Communicating between more languages can be actualized using S 2 ST technology by connecting distributed S 2 ST servers, (i. e. , ASR, MT, TTS) all over the world. MC client Automatic Speech Recognition (ASR) Japanese w a t a sh i 「私は学校に行く」 w a g a xtu k o o n i…. . Convert from phoneme to word Machine Translation (MT) 私は 学校に行く Convert from Japanese text to English text Speaker of Language A Speech Synthesis (TTS) I go to school Communication between users who speak different languages English “I go to school” Digitalization of speech signals Speaker of Language B Digitalization of speech signals Network Convert from text to waveform Large amount of training data for machine learning Japanese speech and text corpora 2018/3/18 Japanese-to -English parallel corpora MC server English speech corpora MC server MC server ASR server MT server TTS server Conversion from speech signal to text in Language A Conversion from text in A to text in B Conversion from text in B to speech signal Conversion from speech signal to text in Language B Conversion from text in B to text in A Conversion from text in A to speech signal MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 2

Network-based Speech Translation System in accordance with ITU-T Recommendations F. 745 and H. 625 Network-based S 2 ST application via multilateral translation on smartphone/tablet/PC/TV Remote Communication お父さん，お母さん お元気ですか？ On-site communication Papa, maman, comment vas-tu? English speaker’s device 飲み水は 13： 00から市役 所前で配給します． Water to drink will be provided in front of the city hall from 13： 00. 从下午一点开始，在市 政府门前供应饮用水。 Chinese speaker’s device 2018/3/18 Japanese speaker’s device MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 3

Network-based Speech Translation System in accordance with ITU-T Recommendations F. 745 and H. 625 Modality Conversion Markup Language (MCML) XML schema, ITU-T name space (http: //www. itu. int/xml-namespace/itu-t/H. 645/MCML. xsd) MCML includes information for communication between multiple persons who use different modalities. Ex. speech, text, image, video data input by users or output by MCML servers such as ASR, MT, TTS , Sign Recognition systems. http: //www. itu. int/rec/T-REC-F. 745 -201010 -I/en 2018/3/18 MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 4

2018/3/18 MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 5

U-STAR Consortium The Universal Speech Translation Advanced Research (U-STAR) Consortium has been established as an international research collaboration entity with the goal of developing a world wide network-based speech-to-speech translation system. The consortium objective is to create a basic infrastructure for spoken language communication to overcome the language barriers that exist around the world. Currently, there are participant members from 14 countries (15 institutes). Plan for Field experiment Period: One year from April of 2012 including during the 2012 London Olympics Application: Multiparty conversation via a network-based S 2 ST system on i. Phones and Android phones (Free) MCML servers: ASR, MT, TTS servers will be provided by U-STAR members Potential languages: Chinese, Dzongkha, English, Filipino, Hindi, Indonesian Japanese, Korean, Mongolian, Malay Nepali, Sinhala, Thai, Urdu, Vietnamese and some European languages 2018/3/18 MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 6

The U-STAR members Institute Language DITT Bhutan Dzongkha UPD Philippines Filipino CDAC India Hindi BPPT Indonesian NICT Japanese ETRI Korean MUST Mongolian NUM Mongolian I 2 R Singapore Malay LTK Nepali UCSC Sri Lanka Sinhala NECTEC Thailand Thai KICS-UET Pakistan Urdu IOIT Vietnamese CASIA 2018/3/18 Country China Chinese MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 7

Potential European Language French, German, Italian, Portuguese, Spanish, Turkish, British English 2018/3/18 MASTAR Project, Universal Communication Research Institute E-mail: info@mastar. jp 8