Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC Berkeley Di. GIR 1
Distributed Databases – Discipline-specific n n n n The Species Analyst (TSA) The Integrated Taxonomic Information System (ITIS) Fish. Net The Mammal Networked Information System (Ma. NIS) Herp. NET The Ornithological Information System (ORNIS) … Di. GIR 2
Distributed Databases – International n n n European Natural History Science Information Network (ENHSIN) Biological Collection Access for Europe (Bio. CASE) Australia Virtual Herbarium (AVH) Red Mundial de Información Sobre Biodiversidad, Comisión Nacional para el Conocimiento y Uso de la Biodiversidad (REMIB, CONABIO) … Di. GIR 3
Distributed Databases – Regional n n n n Mountain and Plains Spatio-Temporal Database. Informatics (Ma. PSTe. DI) Ocean Biogeographic Information System (OBIS) Pacific Basin Information Node, National Biological Information Infrastructure (PBIN, NBII) Species Link, Centro de Referência em Informação Ambiental (Species Link, CRIA) A Virtual Herbarium of the Chicago Region (v. Plants) Spatial Analysis of Local Vegetation Inventories Across Scales (SALVIAS) … Di. GIR 4
Distributed Databases – Intra-institutional n n n Berkeley Natural History Museums (BNHM) Association of Biological Collections, UC Davis … Di. GIR 5
Distributed Databases – “Nodes” n n n Life. Mapper National Biodiversity Information Infrastructure (NBII) Global Biodiversity Information Facility (GBIF) Di. GIR 6
GBIF Work Programmes n n n NODES ECAT – Electronic Catalogue of Names of Known Organisms DIGIT – Digitisation of Natural History Collections OCB – Outreach and Capacity Building DADI – Data Access and Database Interoperability Di. GIR 7
Taxonomic Database Working Group Standards development and maintenance n n n Access to Biological Collections Data (ABCD) Darwin Core Version 2 (Dw. C 2) Structure of Descriptive Data (SDD) Di. GIR Others… Di. GIR 8
Di. GIR Distributed Generic Information Retrieval John Wieczorek, Stan Blum, Dave Vieglais, P. J. Schwartz Di. GIR 9
Information Retrieval n n Distributed - a protocol for retrieving structured data from multiple, heterogeneous databases across the Internet. Generic - a protocol independent of the data retrieved and of the software to retrieve it. Di. GIR 10
Project Rationale n n n Avoid multiple incongruous development efforts Pool resources and create a support community of experts Solve scalability problems Di. GIR 11
Design Goals n n Use open protocols and standards, such as HTTP and XML Decouple the protocol, software and semantics Make new data provider installations as easy as possible Develop open source software with GNU General Public Licensing (It’s free). Di. GIR 12
Di. GIR Component Summary Di. GIR 13
Di. GIR Architecture n. Provider Di. GIR 14
Provider n n n Receives requests Retrieves data from database Sends results to requestor Supplies metadata to describe content, contacts, and capabilities Logs requests Di. GIR 15
Di. GIR Architecture n. Portal Engine Di. GIR 16
Portal Engine n n The entry point for an application Can query a registry to discover potential providers Can determine, based on provider metadata, whether a provider should be queried Can send requests to multiple providers Di. GIR 17
Portal Engine, continued n n Assembles responses from providers Returns packaged results to the requesting application Communicates via protocol compliant messaging only Logs activity Di. GIR 18
Registry n n Provides a “yellow pages” to advertise the existence and capabilities of a provider Provides a means to discover potential providers of interest May be public or private Need not be a part of the architecture Di. GIR 19
Di. GIR Architecture n. Provider n. Registry (register) Di. GIR 20
Di. GIR Architecture n. Portal Engine n. Registry (discover) Di. GIR 21
Di. GIR Protocol n Defines request and response message formats for communication between provider, portal engine, and applications n n Metadata requests Search requests Inventory requests Remains unfettered by the structure of the data it transfers Di. GIR 22
Di. GIR Architecture n. Application Di. GIR 23
Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine Di. GIR 24
Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (request) n. Provider Di. GIR 25
Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (response) n. Provider Di. GIR 26
Di. GIR Architecture n. Application n. Protocol (response) n. Portal Engine Di. GIR 27
Applications n n Must be able to assemble and send a request document to a portal Must be able to receive and interpret a response document from the portal Must do something incredibly useful and interesting with the data This is where the real fun is! Di. GIR 28
Hot topics – Interesting problems n n n n Persistent unique identifiers Web services (converters, translators, calculators, transformation services) Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection, …) Spatial query interfaces Data quality feedback mechanisms Automated georeferencing, event gazetteers Concept libraries, schema extensions, schema libraries, federated ontologies Di. GIR 29
Ma. NIS – The Mammal Networked Information System It’s more than just a pangolin… Di. GIR 30
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 31
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 32
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 33
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 34
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 35
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 36
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 37
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 38
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 39
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 40
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 41
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 42
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 43
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 44
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 45
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 46
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 47
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 48
Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 49
Ma. NIS Network Configuration … MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider … Ma. NIS Di. GIR Portal … … UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer … Di. GIR Provider Di. GIR UAM Oracle Database … LACM MS Access Database … CAS SQL Server Database Di. GIR Provider … 50
“Cal. Net” Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Cal. Net Di. GIR Portal Di. GIR 51
Event Gazetteer Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Cal. Net Di. GIR Portal Bio. Geomancer Web Service Di. GIR 52
NBII Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider NBII Di. GIR Portal Di. GIR 53
GBIF Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider NBII Di. GIR Portal GBIF Presentation Layers Di. GIR 54
Intra-Network Configuration (BNMH) Essig Working Database PHMA Working Database UCBG Working Database UCJEPS Working Database UCMP Working Databases (4) Online Database Online Database Di. GIR Provider BNHM Di. GIR Portal BNHM Presentation Layer Di. GIR 55
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 56
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 57
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 58
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 59
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 60
Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Di. GIR 61
Project Information n Di. GIR is a collaborative open source development project on Source. Forge (https: //sourceforge. net/projects/digir). Software and documentation are available on the Di. GIR web site (http: //digir. net). Ma. NIS is an international network collaboration among mammal specimen collections (http: //elib. cs. berkeley. edu/manis). Di. GIR 62
Hot topics – Interesting problems n n n n Persistent unique identifiers Web services (converters, translators, calculators, transformation services) Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection, …) Spatial query interfaces Data quality feedback mechanisms Automated georeferencing, event gazetteers Concept libraries, schema extensions, schema libraries, federated ontologies Di. GIR 63
Скачать презентацию Distributed Databases and Applications John Wieczorek Museum of
c923896a94e67141078547789c099a3e.ppt