Introduction to Grid Computing Deployment of a Language

Introduction to Grid Computing Deployment of a Language Detector Grid Service University of Amsterdam, 02 -11 -2005 Felix Hageloh Roberto Valenti

Overview n n n Introduction Required Steps Our Service n n n n n Introduction The basic idea Use Case Interface Implementation Problems Encountered Future Work Conclusions Questions

Introduction n Our chosen task: Grid Services n Task Goals: n Build a grid service. n Aggregate the service with another to provide additional, higher-level services

Steps n Get access to the systems n n Security issues n n n n Obtain User Certificate Obtain Host Certificate Implement the service Create required files n n Authentication WSDL QNames WSDD JNDI Compile and create GAR file As Globus user: n n Deploy service Start container

But you all know this… So… we jump to our service.

Our Service

Our Service: Introduction n We were requested to implement a useful service which could be integrated on other services n We are AI students so… Let’s Merge AI and Grid Computing!!

Our Service: The Basic Idea: Language Detection Is a necessary first step in a multitude of applications n Useful Web Service Examples: n n Email filtering n Information retrieval n Spell checkers n Can also be component of an aggregated grid service

Our Service: The Basic Idea n What about creating a Language Detector on the Grid? n Training and Testing can be extremely time consuming running on a single machine n Data difficult to obtain -> can be shared on the Grid n Duplicate data for parallel computing

Our Service: Use Case Simple Interface: n Receives a piece of text n Returns a string indicating the language

Our Service: Adding States Grid services can have states (as opposed to web services) n Not necessary for our service but for the learning factor n Added “dummy” states to our service: n n Last Operation n Times Used

Our Service: Statefull Use Case

Our Service: Interface n Requests and Responses

" src="https://present5.com/presentation/0bcd0b59d0b9ecd3a500427ce168fddd/image-14.jpg" alt="Our Service: Interface n Port Types " /> Our Service: Interface n Port Types

Our Service Implementation

Language Detection: Basic Idea n n Essentially based on probabilities of character combinations Every language has typical character combinations that are very frequent in that language n n n “th” in english “ij” in dutch Easy for humans to detect a language even when we don’t know that specific language

Language Learning: Standard Process n Standard machine learning process

Language Learning: Markov Models n Basic Markov Model n kth order Markov Model

Language Detection: Classification n Transitional probabilities estimated as n Classification

Language Detection: Example n The training text for a language consists of the string test text n Learned model: ( ^^, t, 1. 0 ) ( ^t, e, 1. 0 ) ( te, s, 0. 5 ) ( es, t, 1. 0 ) ( st, , 1. 0 ) ( te, x, 0. 5 ) ( ex, t, 1. 0 ) ( xt, , 1. 0 ) n the probability of the string test would be: P(test|L) = P(t|^^)*P(e|^t)*P(s|te) *P(t|es)*P(_|st) = 1*1*0. 5*1*1=0. 5

Language Detection: Performance

Problems Encountered Necessary tools had to be installed (ANT) n Problems on our machine (GRAM) n Conflicts with other team n Buggy shell script to build gar file n n Sensitive to path lengths/ names

Future Work n Connect with other services n Make training and evaluation a grid service n Make it part of a multi lingual retrieval engine n Web interface (interactive)

Conclusions n Successfully managed to create and deploy our own web service n Broke loose from the tutorial web service structure n Merged Grid Computing with AI Got hands on experience with Grid applications and structure n A lot of possibilities to integrate and/or extend the implemented service n

Questions ?