Named Entity Recognition Sobha Lalitha Devi AU-KBC Research Centre Chennai
Named Entity(NE) Recognition • • What is NE and What is not an NE How to identify NE Tagset and Annotation Guidelines Methods Used in developing NER 3/16/2018 IIIT Summer School 2
Why do NER? • Key part of Information Extraction system • Robust handling of proper names essential for many applications such as Summarization, IR, Anaphora, . . • Pre-processing for different classification levels • Information filtering • Information linking 3/16/2018 IIIT Summer School 3
What is NER ? • NER involves identification of proper names in texts, and classification into a set of predefined categories of interest. • Three universally accepted categories: • Person, location and organisation • Other common tasks: recognition of date/time expressions, measures (percent, money, weight etc), email addresses etc. • Other domain-specific entities: names of Drugs, Genes, medical conditions, names of ships, bibliographic references etc. 3/16/2018 IIIT Summer School 4
NER Definition • Named entity recognition (NER) (also known as entity identification (EI) and entity extraction) is the task that locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc. John sold 5 companies in 2002. <ENAMEX TYPE="PERSON">John</ENAMEX> sold <NUMEX TYPE="QUANTITY">5</NUMEX> companies in <TIMEX TYPE="DATE">2002</TIMEX>. 3/16/2018 IIIT Summer School 5
What is not NER? • NER is not event recognition. • NER does not create templates, • NER does not perform co-reference or entity linking, – though these processes are often implemented alongside NER as part of a larger IE system. • NER is not just matching text strings with pre-defined lists of names. It recognises entities which are being used as entities in a given context. • NER is not an easy task! 3/16/2018 IIIT Summer School 6
Named Entity and Philosophy of Language • Proper Names are defined by – Descriptivist's theory of Names • Frege, Russell, Ludwig , Wittgenstein and John Searle – Causal theory of Reference • Saul Kripke 3/16/2018 IIIT Summer School 7
Descriptivist's theory of Names Proper names either are synonymous with descriptions, or have their reference determined by virtue of the name's being associated with a description or cluster of descriptions that an object uniquely satisfies. Causal theory of Reference Proper names refer to an object by virtue of a causal connection with the object as mediated through communities of speakers. That is , proper names, in contrast to descriptions, are rigid designators. Rigid designators : A proper name refers to the named object in every possible world in which the object exists. Descriptions designate : a proper name as different objects in different possible worlds. 3/16/2018 IIIT Summer School 8
Proper Names and Definite Descriptions • A meaning of a Sentences involving Proper names could be substituted by a contextually appropriate description for a name. eg: Otto von Bismarck can be known or described as the first Chancellor of the German Empire Kripke argues that definite descriptions cannot be rigid designators. Because definite descriptions cannot be same/similar in all possible worlds More on Kripke’s Proper name in Naming and Necessity 1980 3/16/2018 IIIT Summer School 9
What is Named Entity • Named Entities are – A Noun Phrase – Rigid Designators : It designates/denotes the same thing in all possible worlds in which the same thing exists and does not designate anything else in those possible worlds in which that same thing does not exist 3/16/2018 IIIT Summer School 10
EXAMPLES for Named Entity and not a Named entity • Hotel & Taj Hotel • Flower & Rose Flower • Beach & Kovalam Beach • Airport & Indira Gandhi International airport • The School & Good Shepherd School • Prime Minister & Mr. Manmohan Singh 3/16/2018 IIIT Summer School 11
Some problems in indentifying NE • Variation of NEs. – Manmohan Singh, Manmohan, Dr. Manmohan Singh • Ambiguity of NE types: – 1945 (date vs. time) – Washington (location vs. person) – May (person vs. month) – Tata (person vs. organization) 3/16/2018 IIIT Summer School 12
Ambiguity Examples • Person vs Location – Sir C. P Ramaswamy was the Divan of Travancore (Per) – Sir C. P Ramaswamy Road is in Chennai (Loc) • Person vs Organization – Anil Ambani opened Reliance Fresh (Per) – Reliance Fresh is under Anil Amabani Group Ltd (Org) 3/16/2018 IIIT Summer School 13
More complex problems in NER Issues of style, structure, domain, genre etc. – Punctuation, spelling, spacing, formatting, …. all have an impact Dept. of Computing and Information Science Manchester Metropolitan University Manchester United Kingdom > Tell me more about Leonardo > Da Vinci 3/16/2018 IIIT Summer School 14
Problems in NE Task Definition • Category definitions are intuitively quite clear, but there are many grey areas. • Many of these grey area are caused by metonymy. Person vs. Artefact Organisation vs. Location Company vs. Artefact Location vs. Organisation 3/16/2018 IIIT Summer School 15
Tagset for Named Entity • ACE tagset is Hierarchical – ACE-Automatic Content Extraction • The tagset – CLIA-is Hierarchical -Similar to ACE – Developed for two domains • Tourism and Health 3/16/2018 IIIT Summer School 16
TAGSET • ENAMEX – Person • Individual – Family name – Title • Group – Organization • Government • Public/private company • Religious • Non-government – Political Party – Para military – Charitable – Association • GPE (Geo-political Social Entity) • Media – Location • Place – District – City – State – Nation – Continent • Address • Water-bodies • Landscapes • Celestial Bodies 3/16/2018 • • • IIIT Summer School • • • – Manmade » Religious Places » Roads/Highways » Museum » Theme parks/Parks/Gardens » Monuments Facilities – Hospitals Institutes Library – Hotel/Restaurants/Lodges – Plant/Factories – Police Station/Fire Services – Public Comfort Stations – Airports – Ports – Bus-Stations Locomotives Artifacts – Implements – Ammunition – Paintings – Sculptures – Cloths – Gems & Stones Entertainment – Dance – Music – Drama/Cinema – Sports – Events/Exhibitions/Conferences Cuisine’s Animals 17 Plants
Tagset Continued • NUMEX – Distance • Money – Quantity – Count • TIMEX – Time – Date – Day – Period 3/16/2018 Tagset Counts First Level Tags -3 Second Level -43 Third Level – 40 Total - 86 IIIT Summer School 18
How to Annotate • 1. ENAMEX – 1. 1 Person • 1. 1. 1 Individual • These refer to names of each individual person, also includes names of fictional characters found in stories/novels etc. Tag Structure: <ENAMEX TYPE= “PERSON” SUBTYPE_1= “INDIVIDUAL”> abc </ENAMEX> Examples: English: <ENAMEX TYPE= “PERSON” SUBTYPE_1= “INDIVIDUAL”>Abdul Kalam</ENAMEX> 3/16/2018 IIIT Summer School 19
Annotation continued 1. 1 Family name In general we find that a person name consists of a family name. Whenever an instance of individual name occurs with family name, then that part of the name, which refers to family name, must be tagged specifically with subtag “FAMILYNAME” as shown below. Tag Structure: <ENAMEX TYPE= “PERSON” SUBTYPE_1= “INDIVIDUAL” SUBTYPE_2= “FAMILYNAME”> abc </ENAMEX> Examples: English: <ENAMEX TYPE=”PERSON” SUBTYPE_1=”INDIVIDUAL”> Lalu Prasad<ENAMEX TYPE= “PERSON” SUBTYPE_1= “INDIVIDUAL” SUBTYPE_2= “FAMILYNAME”>Yadav</ENAMEX> 3/16/2018 IIIT Summer School 20
NE Types The Named entity hierarchy is divided into three major classes Entity Name, Time and Numerical expressions. ENAMEX NE TYPES NUMEX TIMEX 3/16/2018 IIIT Summer School 21
Entity Types 3/16/2018 IIIT Summer School 22
Entity Name Types q Persons are entities limited to humans. A person may be a single individual or a group. Individual refer to names of each individual person. Group refers to set of individual q Location entities are limited to geographical entities such as geographical areas like names of countries, cities, continents and landmasses, bodies of water, and geological formations. q Organization entities are limited to corporations, agencies, and other groups of people defined by an established organizational structure 3/16/2018 IIIT Summer School 23
![Examples for Entity Name Types q q En: [Sita]PERSON is working at [HCL]ORGANIZATION ,](https://present5.com/presentation/9d5c54cb78345f6d2133405fff530a8c/image-24.jpg "Examples for Entity Name Types q q En: [Sita]PERSON is working at [HCL]ORGANIZATION ,")
Examples for Entity Name Types q q En: [Sita]PERSON is working at [HCL]ORGANIZATION , which is in [Chennai] LOCATION Ta: [Seetha] PERSON [chennaiyilrukkira] LOCATION [HCLlil] ORGANIZATION En: Sita Chennai HCL velaiseikir. Al. Working Ml: [Seetha] PERSON [chennaiyillula] LOCATION [HCLlil] ORGANIZATION En: Sita Chennai HCL jolicheyyunnu. Working Hi: [Seetha] PERSON [HCL] ORGANIZATION main kaam kar raha hai, jo En: Sita HCL work is which [chennai] LOCATION main hain. Chennai in 3/16/2018 IIIT Summer School 24
Entity Name Types Facilities are limited to buildings and other permanent man-made structures and real estate improvements like hospitals, airport, colleges, libraries etc. En: [Appolo Hospital] FACILITY is in Chennai LOCATION Ta: [Appallo maruthuvaman. Ai]FACILITY [Chennaiyil]LOCATION irukkirathu Ml: [Appolo Asupathri]FACILITY [chennaiyil]LOCATION aa. N Hi: [Appolo aspathaal]FACILITY [chennai]LOCATION mein haim. 3/16/2018 IIIT Summer School 25
Entity Name Types A locomotive entity is a physical device primarily designed to move an object from one location to another, by carrying, pulling, or pushing the transported object. En: [Ananthapuri Express]LOCOMOTIVE departs from [Chennai] LOCATION at [7. 30 pm] Time. Hi: [Ananthapuri express] LOCOMOTIVE [Chennai] LOCATION se [r. Ath 7. 30] TIME ko ravana hoga Ml: [Ananthapuri eksprass] LOCOMOTIVE [chennaiyilninn] LOCATION [raathri 7. 30 ma. Nikk] TIME pu. Rappetum. Ta: [Ananthapuri viraivu rayil] LOCOMOTIVE [chennaiyilirunthu] LOCATION [i. Ravu 7. 30 ma. Nikku] TIME pu. Rappatukirathu 3/16/2018 IIIT Summer School 26
Entity Name Types Artifact entities are objects or things, produced or shaped by human craft, such as tools, weapons/ammunition, art paintings, clothes, ornaments, medicines En: [Vinayaga Statue] ARTIFACT is looking beautiful Ta: [Vinayakarin Silai] ARTIFACT p. Arpatharkku alak. Akairukkirathu Ml: [ganapathi vigraham]ARTIFACT ba. Mgiyaayi irikkunnu. Hi: [Vinayaka moorthi] ARTIFACT achi lagh rahi haim. 3/16/2018 IIIT Summer School 27
Entity Name Types Entertainment entities denote activities, which are diverting and hold human attention or interest, giving pleasure, happiness, amusement especially performance of some kind such as dance, music, sports, events. En: [Flower Exhibition] ENTERTAINMENT is held at [Hyderabad]LOCATION Ta: [Malar kank. Atchi] ENTERTAINMENT [hyderabaadil]LOCATION Nadaiperukirathu Ml: [pushpa pradarshanam] ENTERTAINMENT [hyderabaadil] LOCATION natakkunnu Hi: [phool pradarshnii] ENTERTAINMENT [hyderabad] LOCATION me. N Ayojith kiyaa j. Ath. A hai 3/16/2018 IIIT Summer School 28
Entity Name Types Materials refer to the names of food items, cuisines, chemicals and cosmetics En: [Honey]MATERIALS is good for face Ta: [Th. En]MATERIALS mukaththi. Rku nallathu Ml: [Madhu] MATERIALS mukaththinu nallath. AN Hi: [Shahad] MATERIALS chehare ke liye achcha hai. 3/16/2018 IIIT Summer School 29
Entity Name Types ORGANISMS: These are the names of different animal species including birds, reptiles, viruses, bacteria and names of herbs, medicinal plants, shrubs, trees, fruits, flowers etc. En: [Peacock] ORGANISM is the national bird of [India] LOCATION Ta: [Mayil] ORGANISM [Inthiy. Avin] LOCATION th. Eciyappa. Ravai Akum. Ml: [Mayil] ORGANISM [indyayute] LOCATION raashtrapakshi AN. Hi: [Mor] ORGANISM [bhaarath] LOCATION kaa raashtr. Iya pakshi hai. 3/16/2018 IIIT Summer School 30
Entity Name Types Disease: Names of disease, symptoms, diagonisis and treatment are comes under this type. En: Smoking Causes [Cancer] DISEASE Ta: Pukaippithithal. Al [pu. RRu. Noi] DISEASE varuki. Rathu Ml : pukavali [a. Rbhudham] DISEASE u. Nt. Akkunnu Hi: dhumrapan [kaansar] DISEASE ka kaara. N banaatha hai. 3/16/2018 IIIT Summer School 31
Numerical Expressions DISTANCE QUANTITY NUMEX MONEY COUNT 3/16/2018 IIIT Summer School 32
Numerical Expressions 3/16/2018 Distance refers to the distance measures such as kilometers, Centimeters, acres, feet etc. Example: 10 cm. , twenty feet, 15 hectares Money specifies the different currency value such as rupee, euro, Dinar, dollar etc. Example: Rs. 1000, 250 Euro, $160 Count denotes the number (or counts) of Items/ articles/things etc. Example: 5 subjects, 12 students, 20 books Quantity measurements like liters, tons, grams, volts etc. are comes under this category. Example: 20 litres, 22 kg, 50 g, 100 volts IIIT Summer School 33
Time Expressions TIMEX DAY 3/16/2018 DATE MONTH TIME PERIOD IIIT Summer School YEAR SPECIAL DAY 34
Temporal Expressions Temporal expressions are the entities refers to time, date, year, month and day Time: These refer to expressions of time, includes different forms of expressing time. This also includes Hours, minutes and seconds. Example 5’o clock in the morning 9. 30 a. m. Evening 6. 30 p. m. Date: This refers to expressions of Date such as 13/12/2001 etc in different forms. This also includes month, date and year Example August 15 1947 1956 September 11 3/16/2018 IIIT Summer School 35
Temporal Expressions Day: These are expressions, which convey days in a year. Also it can include days occurring weekly /fortnightly/ monthly /quarterly/ biennial etc. Example Sunday Tomorrow Today Yesterday Special Day: refers to special days in a year Example Gandhi Jayanthi Rama Navami 3/16/2018 IIIT Summer School 36
Temporal Expressions Period: refers to expressions, which express duration of time or time periods or time intervals. Example 17 th century 10 minutes 10 a. m. to 12 p. m. One year 3/16/2018 IIIT Summer School 37
Methodologies Methods: 1) Rule Based 2) Machine Learning Hidden Markov Model (HMM) Naïve Bayes Classifier Maximum Entropy Markov Model (MEMM) Conditional random Fields (CRF) 4) Hybrid Approach 3/16/2018 IIIT Summer School 38
Challenges of NER in Indian Languages Following are the major challenges encountering in Indian Languages. § Agglutination § Ambiguity §Between Proper and common nouns §Between named entities § Lack of Capitalization 3/16/2018 IIIT Summer School 39
Challenges of NER in Indian Languages Agglutination In Dravidian languages, words consist of a lexical root to which one or more affixes are attached. Example in Tamil: 1) Ta: Ramanaiththavira (otherthan Raman) 2) Ta: Cevvaiyandru (On Tuesday) 3) Ta: Inthiyavilllula (In India) 4) Ta: Kannanaippa. RRikkondu (hold onto Kannan) 3/16/2018 IIIT Summer School 40
Challenges of NER in Indian Languages Example in Malayalam: 1) Ml: hemayilu. Ntaayirunna (that which Hema have) 2) Ml: Chennaiyilethunna (reach in Chennai) 3) Ml: arabikatalina. Bimukhamaayi (towards the arabian sea) 4) Ml: kaa. Siyilekkozhukunna ( flowing towards kaa. Si) 3/16/2018 IIIT Summer School 41
Challenges of NER in Indian Languages Ambiguity Comparatively Indian languages suffer more due to the ambiguity that exists between common & proper nouns and between named entities itself. In some cases same word can refer to different named entity types. Those instances can recognized by contextual information. Examples: Hi: Akash - Person name and Sky Hi: Sooraj - Person name and Sun Hi: Chaanth – Moon and Silver Hi: Aam – Mango and Common Ml: Roopa – Person name and Rupee Ml: Madhu – Person name and Honey Ml: Mala – Person name and Garland 3/16/2018 IIIT Summer School 42
Challenges of NER in Indian Languages Ta: Malar - Person name and Flower Ta: Chevvai - Day and planet Ta: Shakthi – Person name and Power Ta: MAlai – Evening and Garland Ta & Ml: Velli – Silver, Planet, Day 3/16/2018 Thinkal - Day and Month IIIT Summer School 43
Challenges of NER in Indian Languages Spell Variation: Due to the different writing styles same entity is represented in various word forms. In Tamil, sanskirit letters such as “ja”, “sha”, “sri” “Ha” are replaced by “sa”, “ciri”, “ka” Example: Roja can be written as Rosa Srimathi - cirimathi Raja - rasa Shajah. An - sajak. An 3/16/2018 IIIT Summer School 44
Challenges of NER in Indian Languages Lack of Capitalization In English and some other European languages capitalization is considered as the important feature to identify proper noun. It plays a major role in NE identification. Unlike English capitalization concept is not found in Indian languages. 3/16/2018 IIIT Summer School 45
Nested Entities: Refers to the named entities which occurs within another named entities. Also called as embedded entities. Ta: [[Mathurai] LOCATION [Meen. Atchi Amman]PERSON Koyil]RELPLACE En: Mathurai Meenatchi Amman Temple Ml: [[Nittoor] PERSON Srinivasa rao] PERSON En : Nitoor Srinivasa rao Hi: [[Rajeev] PERSON MArg] ROAD En : 3/16/2018 Rajeev Road IIIT Summer School 46
Approaches in Named Entity Resolution • Dictionary Look-up • Rule based ( Using lexical, contextual and morphological information) • Maximum entropy theory based • Hidden Markov Model • Conditional Random Fields • Hybrid methods (Statistical+ Linguistics) 3/16/2018 IIIT Summer School 47
Dictionary (Gazetteers) Look-up Approach • Uses Dictionaries for identifying NERs ( Gazetteers) • Gazetteer contains NEs from all domains • Advantage – Very simple approach – Gives very high precision 3/16/2018 IIIT Summer School 48
Disadvantages of Dictionary Approach • Preparation of exhaustive dictionary is a tedious and expensive process. • The dictionary should cover the different spellings of the same place. 3/16/2018 IIIT Summer School 49
Rule Based Approach • Rule Based System – Needs more rules to tag all kinds of NE • Advantages: – Rich and expressive rules – Good results • Disadvantages: – Requires huge experience and grammatical knowledge – Experts to craft rules are expensive – Highly domain specific ( not portable to a new domain) 3/16/2018 IIIT Summer School 50
General difficulties “ Italy's business world was rocked by the announcement last Thursday that Mr. Verdi would leave his job as vice-president of Music Masters of Milan, Inc. to become operations director of Arthur Andersen". • Capitalization useless for first word • S not part of name "Italy" • Date is "last Thursday" not "Thursday" • Milan is location, not organization • Arthur Andersen is organization, not person 3/16/2018 IIIT Summer School 51
Rules success and failure Title Capitalized_Word Title Person_Name Correct: Mr. Jones Incorrect: Mrs. Field's Cookies (corporation) Month_name number_less_than_32 Date Correct: February 28 Incorrect: Long March 3 (a Chinese Rocket) From Date to Date Correct: from August 3 to August 9 Incorrect: I moved my trip from April to June (two separate dates) 3/16/2018 IIIT Summer School 52
Statistical based approach • Need to identify features • Feature selection has to be correct for all types of NE • Development of Tagged Corpus • The Corpus should contain all types of tags in appropriate number • Domain based corpus has to be generated. 3/16/2018 IIIT Summer School 53
Automated approaches Address drawbacks of hand-coded system Automated training • Human-annotated (with desired output standards) training data • Annotation requires less effort and expertise than hand-coding rules • Annotation accuracy • Two annotators for checking, third annotator to resolve disputes 3/16/2018 IIIT Summer School 54
Literature Survey 1) Named Entity Recognition was one of the tasks defined in Message Understanding Conference(MUC) 6. 2) A survey on Named Entity Recognition was done by David Nadeau (2007). 3) Techniques used include: - rule based technique by Krupka (1998) 1) - using maximum entropy by Borthwick (1998) 2) - using Hidden Markov Model by Bikel (1997) 3) - bootstrapping approach using concept based seeds (Niu et al. , 2003) 2) - hybrid approaches such as rule based tagging for certain entities such as date, time, percentage and maximum entropy based approach for entities like location and organization (Rohini et al. , 2000) 3)4) The Stanford NER software (Finkel et al. , 2005), uses linear chain CRFs in their 4) NER engine. Here they identify three classes of NERs viz. , Person, Organization 1) and Location. 3/16/2018 IIIT Summer School 55
References § Arulmozhi, P. and Sobha, L. (2006). HMM-based Part of Speech Tagger for Relatively Free § Word Order Language. Advances in Natural Language Processing, Research in Computing Science Journal, Mexico Volume 18, pp. 37 -48. § Bikel, D. M. Miller, S. Schwartz, R. Weischedel, R. (1997). Nymble: A high-performance learning name-finder. In Fifth Conference on Applied Natural Language Processing. pp. 194201. § Borthwick, A. Sterling, J. Agichtein, E. and Grishman, R. (1998). Description of the MENE named Entity System. In Seventh Machine Understanding Conference (MUC-7). § Chen, W. Zhang, Y. and Isahara, H. (2006). Chinese Named Entity Recognition with Conditional Random Fields. In Fifth SIGHAN Workshop on Chinese Language Processing, Sydney. pp. 118 -121. § Ekbal, A. Bandyopadhyay, S. (2009). A Conditional Random Field Approach for Named Entity Recognition in Bengali and Hindi. Linguistic Issues in Language Technology, 2(1). pp. 1 -44. 3/16/2018 IIIT Summer School 56
References § § § § Finkel, J. N. Grenager, T. and Manning, C. (2005). Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling. In 43 nd Annual Meeting of the Association for Computational Linguistics (ACL 2005). pp. 363 -370. Finkel, J. Dingare, S. Nguyen, H. Nissim, M. Sinclair, G. and Manning, C. (2004). Exploiting Context for Biomedical Entity Recognition: from Syntax to the Web. In Joint Workshop on Natural Language Processing in Biomedicine and its Applications, (NLPBA), Geneva, Switzerland. Gali, K. Surana, H. Vaidya, A. Shishtla, P. Sharma, D. M. (2008). Aggregating Machine Learning and Rule Based Heuristics for Named Entity Recognition. In Workshop on NER for South and South East Asian Languages, IJCNLP-08, Hyderabad, India. Kumar, K. N. Santosh, G. S. K. Varma, V. (2011). A Language-Independent Approach to Identify the Named Entities in under-resourced languages and Clustering Multilingual Documents. In International Conference on Multilingual and Multimodal Information Access Evaluation, University of Amsterdam, Netherlands. Lafferty, J. Mc. Callum, A. Pereira, F. (2001). Conditional Random Fields for segmenting and labeling sequence data. In ICML-01, pp. 282 -289. Loinaz, I. A. Uriarte, O. A. Ramos, N. E. Castro, M. I. F. D (2006). Lessons from the Development of Named Entity Recognizer for Basque. Natural Language Processing, 36. pp. 25 – 37. Mc. Callum, A. and Li, W. (2003). Early Results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons. In Seventh Conference on Natural Language Learning (Co. NLL). 3/16/2018 IIIT Summer School 57
References § § § § Nadeau, David and Sekine, S. (2007) A survey of named entity recognition and classification. Linguisticae Investigationes 30(1). pp. 3– 26. Niu, C. Li, W. Ding, J. Srihari, R. K. (2003). Bootstrapping for Named Entity Tagging using Concept -based Seeds. In HLT-NAACL’ 03, Companion Volume, Edmonton, AT. pp. 73 -75. Pandian, S. Lakshmana, Geetha, T. V. and Krishna. (2007). Named Entity Recognition in Tamil using Context-cues and the E-M algorithm. In the Proceedings of the 3 rd Indian International Conference on Artificial Intelligence, Pune, India. pp. 1951 -1958. Sasidhar, B. , Yohan, P. M. , Babu, V. A. , Govarhan, A. (2011). A Survey on Named Entity Recognition in Indian Languages with particular reference to Telugu. J. International Journal of Computer Science Issues, Volume. 8, pp. 1694 -0814. Sobha, L. , Vijay Sundar Ram. R. (2006). "Noun Phrase Chunker for Tamil", In Proceedings of Symposium on Modeling and Shallow Parsing of Indian Languages, Indian Institute of Technology, Mumbai, pp 194 -198. Srihari, R. K. Niu, C. Yu, L. (2000). A Hybrid Approach for Named Entity Recognition in Indian Languages. In 6 th Applied Natural Language Conference, pp. 247 -254 Gupta, S. and Bhattacharyya, P. (2010). Think globally, apply locally: using distributional characteristics for Hindi named entity identification. In 2010 Named Entities Workshop, Association for Computational Linguistics Stroudsburg, PA, USA Vijayakrishna, R. and Sobha, L. (2008). Domain focused Named Entity for Tamil using Conditional Random Fields. In IJNLP-08 workshop on NER for South and South East Asian Languages, Hyderabad, India. pp. 59 -66 3/16/2018 IIIT Summer School 58
Literature Survey Indian Languages: 5) Named Entity recognition for Hindi, Bengali, Oriya, Telugu and Urdu (some of the major Indian languages) were addressed as a shared task in the NERSSEAL workshop of IJCNLP. The tagset used here consisted of 12 tags. 6) Vijayakrishna & Sobha (2008) worked on Domain focused Tamil Named Entity Recognizer for Tourism domain using CRF. It handles nested tagging of named entities with a hierarchical tag set containing 106 tags. They considered root of words, POS, combined word and POS, Dictionary of named entities as features to build the system. 7) Pandian et al (2007) have built a Tamil NER system using contextual cues and E-M algorithm. 8) The NER system (Gali et al. , 2008) build for NERSSEAL-2008 shared task which combines the machine learning techniques with language specific heuristics. The system has been tested on five languages such as Telugu, Hindi, Bengali, Urdu and Oriya using CRF followed by post processing which involves some heuristics. 3/16/2018 IIIT Summer School 59
Thank you 3/16/2018 IIIT Summer School 60
Скачать презентацию Named Entity Recognition Sobha Lalitha Devi AU-KBC Research
9d5c54cb78345f6d2133405fff530a8c.ppt