Quality Interoperability Approaches, case studies and open issues Rome, 28 th May 2010 DL. org Quality Working Group DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality comprehensive models Fuhr et al. , 2001 Digital Libraries: A Generic Classification and Evaluation Scheme DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality comprehensive models Goncalves et al. , 2006 What is a good digital library? A quality model for digital libraries DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality comprehensive models Zhang, 2010 Holistic DL evaluation model DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality comprehensive models Candela et al. , The DELOS RM Quality concept map, 2008 Annotations by the DL. org Quality WG DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality is subjective Quality is dynamic Quality is vague Quality needs policies … DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality WG meeting results • Our motivating scenario: consider that representatives of two (or more) DLs have a round table to negotiate a service level agreement (SLA) defining their interoperability requirements and for this establish a quality threshold that each individual DL has to meet or exceed; “Quality” would provide transparent qualitative or quantitative parameters for defining the threshold • Our approach is practical: Quality Interoperability Survey, Quality scenarios • The Cookbook TOC • The interoperability scenarios 1&2 DL. org All WGs Meetings, Rome, 26 -28 May 2010
Quality Interoperability Survey • • Survey Pilot questionnaire + results Simplification and improvement Disambiguation (Glossary) Collection strategy Data analysis and interpretation Results expected by June 2010 Best practices DL. org All WGs Meetings, Rome, 26 -28 May 2010
The Cookbook Introduction • • • Definitions of quality Context: how the Quality WG faced the investigation challenges Consistent terminology DL. org All WGs Meetings, Rome, 26 -28 May 2010
The Cookbook Looking at Web solutions at a technical level FRONT END: Web Interfaces (usability, accessibility), eg. W 3 C BACK END: Web services ontologies (Qo. S), eg. FIPA, WS-Qo. S, MOQ, DAML-Qo. S DL. org All WGs Meetings, Rome, 26 -28 May 2010
The Cookbook Quality insurance procedures for specific DL systems Semantic and organisational levels • Document repositories (eg. DINI, DRIVER) • Research data archives (eg. Data Seal of Approval) • Preservation systems (eg. TRAC, DRAMBORA) ↓ Template for comparison within each class DL. org All WGs Meetings, Rome, 26 -28 May 2010
The Cookbook – Case studies Template Aspect DINI DRIVER Explicit quality policy for protocol and metadata implementation Yes Explicit policy for operations (personell, support etc. ) Yes No Personal quality check (questionaire, on-site review) Yes No Intellectual quality check (remote) Yes Automatic self validation No Yes Organized through sustainable Organisation DINI COAR Explicit branding when checked Yes No Translation in English, Spanish, Portuguese, Japanese No Yes Green and Gold Yes No Strictly full-text oriented Yes Certificate DL. org All WGs Meetings, Rome, 26 -28 May 2010 Guidelines
The Cookbook Best practices from the professional community The results of the Survey will be included as best practices from the professional community. We are aware that quality is subjective, that we are dealing also with two “primitive” interoperability challenges 1. researchers vs professionals 2. different disciplines involved But we want to know from DLs people! DL. org All WGs Meetings, Rome, 26 -28 May 2010
The Cookbook The Quality WG checklist Light-weight document – based on the Quality Core Model parameters - with practical recommendations from the DL. org Quality WG based on the QCM parameters and the Quality Interoperability Survey results DL. org All WGs Meetings, Rome, 26 -28 May 2010
Interoperability Scenarios Scenario no. 1 – Quality and Functionality issues § Metadata Evaluation § Formats quality § Geographic referencing (Content) and related pinpointing issues (Functionality) § Controlled vocabulary + related Functionalities issues (search, browsing, etc) § Multilingualism § Quality of the annotations (User profiling, User authentication) § Sources evaluation, i. e. Provenance DL. org All WGs Meetings, Rome, 26 -28 May 2010
Interoperability Scenarios Scenario no. 1 – Policy issues § Copyright issues § Reuse issues § Content policy issues Possible solutions: change the copyright policy, use creative commons license Suggestions: review the scenario and make it consistent (logical incoherencies on map reproduction, restored video, 3 D Model regeneration), less chaotic and more realistic DL. org All WGs Meetings, Rome, 26 -28 May 2010
Interoperability Scenarios Scenario no. 2 – Quality issues • Merged query results • Machine translations for supporting two languages • Articles quality (one has editorial reviews the other has not) • Mark-up of content • Metadata evaluation: Accuracy and Completeness • System performance • Technical correctness DL. org All WGs Meetings, Rome, 26 -28 May 2010
Interoperability Scenarios Scenario no. 2 - Approaches • Two localizations, one for each country, each supporting one language for content from both systems (very difficult!) • Functionalities of both source DLs (i. e. union of), including: access control, search, results views • Metadata quality: solved selecting targets, customizing search screen based on targets e. g. which common metadata fields they support • Search Qo. S: solution in the user interface design to indicate how far along the search is and cluster the results • Ownership of images: search results must contain author, photographer, agency and rights metadata so the user can see if they need to pay, who they pay, and what rights they get DL. org All WGs Meetings, Rome, 26 -28 May 2010
Some conclusive thoughts • Quality: dynamic, subjective, not completely definable • Provenance = the resource story = how to establish quality • DL systems and their implementation teams will always struggle with quality, but users will have the last word DL. org All WGs Meetings, Rome, 26 -28 May 2010
THANK YOU DL. org All WGs Meetings, Rome, 26 -28 May 2010
Скачать презентацию Quality Interoperability Approaches case studies and open issues
7aafad7976a1ffdc6df143be2e60b2e5.ppt