A Panoramic Approach to Integrated Evaluation of Ontologies

A Panoramic Approach to Integrated Evaluation of Ontologies in the Semantic Web S. Dasgupta, D. Dinakarpandian, Y. Lee School of Computing and Engineering University of Missouri-Kansas City

Overview • Motivation • Approach - Pan-Onto-Eval 1. Triple Centricity 2. Theme Centricity 3. Structure Centricity 4. Domain Centricity • Experiments • Evaluation • Conclusion

Related Work • Ontology ranking by cross-references: Swoogle [3, 6], Onto. Select [7] and Onto. Khoj [4] • Structural richness – Tartir et al [8]: distribution and generic/specific super/sub concepts# [Alani et al. 16 -18]. Density measure [16], centrality measure [18]. • Relational richness – Tartir et al [8] - ratio of #non-IS-A to #rels. – Sabou et al [2] - no consideration of the roles of concepts of relationships. • Very limited work on Thematic richness - multiple hierarchies in a single ontology

Are they similar? udge not j u can yo their l by m al the ers". "cov Actually, they are similar NO! They live in the same house They have the same last name They have the same children ….

Ontology Evaluation • How to evaluate ontology? – Some ontologies are strong in terms of structure while their relationships are weak. • We need to evaluate ontologies considering different perspectives.

Onto. Snap Framework Onto. Snap Ontology Summarization Ontology Evaluation Ontology Integration Ontology Categorization Ontology Query & Reasoning

Summary - WINE Ontology • http: //www. w 3. org/2002/03 owlt/miscellaneous/consistent 001 • Total Number of Classes: 138 (Defined: 77, Imported: 61) • Total Number of Datatype Properties: 1 • Total Number of Object Properties: 16 (Defined: 13, Imported: 3) • Total Number of Annotation Properties: 2 • Total Number of Individuals: 206 (Defined: 161, Imported: 45

Summary - Wine 3 Ontology

Pan-Onto-Eval A comprehensive approach to evaluating an ontology by considering its structure, semantics, and domain 1. Triple Centricity: • • Information sources 2. Theme Centricity: Relation Classification 3. Structure Centricity: Relationship Inheritance 4. Domain Centricity

Triple Centricity capturing Information source is. Made. From Subject (Domain) Relation (Property) Object (Range)

Theme Centricity Classification of Relations in Wine Domain Relation Functional • has. Maker • drink • cause Attributive compositional Spatial Comparative Temporal Conceptual • has. Region • taste. Better • made. In. Year • is. Located. In • Expensive • adjacent. To Relations between domain and range concepts carry different semantic ‘senses’. for better understanding of thematic categories of the ontology

Structure Centricity beverage IS-A Distribution of non-IS-A relations Beer IS-A is. Made. From has. Color has. Sugar Wine has. Maker IS-A Cause

WIKIpedia Semantic implication of each hierarchy is different - contributes differently to the semantics of the ontology as a whole. Domain Centricity

Pan-Onto-Eval Ontology O 1 Hierarchies Panoramic Metrics H 1 IC IR D R H 2 R R IC D R IR H 3 R R IC IR D R Evaluation Score DMF 2 DMF 3 DMI 1 DMI Domain Importance DMF 1 DMF DMI 2 DMI 3 ρ R R

Information Content (IC) Triple: Domain-Property-Range Domain Concepts Range Concepts R 1 D 1 R 2 D 2 R 3 Which information sources are important How Range concepts are associated - with which Domain concepts - through which Relation types Information Sources

Information Content (IC) Domain Concept Range Concept IS-A Relation type 1 Relation type 2. . . Relation type 7 Information Entropy is used to measure the significance of information sources • the overall uncertainty of Range concept association

Inheritance Richness (IR) All Domain Concepts X For each X IR(X) = R(X)*S(X) Average of IRs Domain Concept Range Concept IS-A Non IS-A X N: Number of domain concepts in H R(DCi)): Number of relations associated with the domain concept DCi S(DCi) Number of children under the domain concept DCi

Dimensional Richness (DR) {DCi, RCj DCk, RCl. . . }. The dimensional coverage of relationships in a hierarchy. The richness of these relationships are measured by selected range concepts corresponding domain concepts

Relational Richness (RR) {Ri, Rj. . . }. {Rk, Rl. . . }. {Rm, Rn. . . }. {Ro, Rp. . . }. The dimensional coverage of relations in a hierarchy. The richness of these relations are measured by selected relations for categories in a hierarchy

Domain Importance (DMI) • The richness of the core domain(s) of hierarchy Hk compared to other hierarchies.

Ontology Evaluation Score • Combine the richness of hierarchies together into a single model that can effectively evaluate ontologies. K: the number of hierarchies in a given ontology

Experiments • We analyze three related university ontologies – http: //www. ksl. stanford. edu/projects/DAML/ksl-daml-desc. daml – http: //www. ksl. stanford. edu/projects/DAML/ksl-daml-instances. daml – http: //www. cs. umd. edu/projects/plus/DAML/onts/univ 1. 0. daml. • Preprocessing – convert the DAML files to OWL using a mindswap converting tool – assign a type to the relations in these ontologies – generate summaries. • The application is implemented in Java using the Protégé OWL 3. 3 beta API.

H 5: Document - attributive, functional and temporal H 7: Organization - conceptual and attributive H 6: Organism The evaluation score of the University-I (ρ) is 6. 109

The best hierarchy in O 2 is H 6 vs. O 1's is H 5 The evaluation score of the ontology (ρ) is 3. 909.

The evaluation score of the University-III (ρ) is 4. 567.

Comparison of the three ontologies

Conclusions • Pan-Onto-Eval – A comprehensive approach to evaluating an ontology considering various aspects - structure, semantics, and domain. – A formal treatment of the model • The experimental results demonstrate benefits of the proposed model. • Overall, the model has great potential on evaluation of distributed knowledge in the Semantic Web. • Limitations – Lack of rigorous evaluation by experts. – Preprocessing – summarization, relation type assignment – Verified for real applications.