Скачать презентацию Distributed Databases and Applications John Wieczorek Museum of Скачать презентацию Distributed Databases and Applications John Wieczorek Museum of

c923896a94e67141078547789c099a3e.ppt

  • Количество слайдов: 63

Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC Berkeley Di. GIR Distributed Databases and Applications John Wieczorek Museum of Vertebrate Zoology, UC Berkeley Di. GIR 1

Distributed Databases – Discipline-specific n n n n The Species Analyst (TSA) The Integrated Distributed Databases – Discipline-specific n n n n The Species Analyst (TSA) The Integrated Taxonomic Information System (ITIS) Fish. Net The Mammal Networked Information System (Ma. NIS) Herp. NET The Ornithological Information System (ORNIS) … Di. GIR 2

Distributed Databases – International n n n European Natural History Science Information Network (ENHSIN) Distributed Databases – International n n n European Natural History Science Information Network (ENHSIN) Biological Collection Access for Europe (Bio. CASE) Australia Virtual Herbarium (AVH) Red Mundial de Información Sobre Biodiversidad, Comisión Nacional para el Conocimiento y Uso de la Biodiversidad (REMIB, CONABIO) … Di. GIR 3

Distributed Databases – Regional n n n n Mountain and Plains Spatio-Temporal Database. Informatics Distributed Databases – Regional n n n n Mountain and Plains Spatio-Temporal Database. Informatics (Ma. PSTe. DI) Ocean Biogeographic Information System (OBIS) Pacific Basin Information Node, National Biological Information Infrastructure (PBIN, NBII) Species Link, Centro de Referência em Informação Ambiental (Species Link, CRIA) A Virtual Herbarium of the Chicago Region (v. Plants) Spatial Analysis of Local Vegetation Inventories Across Scales (SALVIAS) … Di. GIR 4

Distributed Databases – Intra-institutional n n n Berkeley Natural History Museums (BNHM) Association of Distributed Databases – Intra-institutional n n n Berkeley Natural History Museums (BNHM) Association of Biological Collections, UC Davis … Di. GIR 5

Distributed Databases – “Nodes” n n n Life. Mapper National Biodiversity Information Infrastructure (NBII) Distributed Databases – “Nodes” n n n Life. Mapper National Biodiversity Information Infrastructure (NBII) Global Biodiversity Information Facility (GBIF) Di. GIR 6

GBIF Work Programmes n n n NODES ECAT – Electronic Catalogue of Names of GBIF Work Programmes n n n NODES ECAT – Electronic Catalogue of Names of Known Organisms DIGIT – Digitisation of Natural History Collections OCB – Outreach and Capacity Building DADI – Data Access and Database Interoperability Di. GIR 7

Taxonomic Database Working Group Standards development and maintenance n n n Access to Biological Taxonomic Database Working Group Standards development and maintenance n n n Access to Biological Collections Data (ABCD) Darwin Core Version 2 (Dw. C 2) Structure of Descriptive Data (SDD) Di. GIR Others… Di. GIR 8

Di. GIR Distributed Generic Information Retrieval John Wieczorek, Stan Blum, Dave Vieglais, P. J. Di. GIR Distributed Generic Information Retrieval John Wieczorek, Stan Blum, Dave Vieglais, P. J. Schwartz Di. GIR 9

Information Retrieval n n Distributed - a protocol for retrieving structured data from multiple, Information Retrieval n n Distributed - a protocol for retrieving structured data from multiple, heterogeneous databases across the Internet. Generic - a protocol independent of the data retrieved and of the software to retrieve it. Di. GIR 10

Project Rationale n n n Avoid multiple incongruous development efforts Pool resources and create Project Rationale n n n Avoid multiple incongruous development efforts Pool resources and create a support community of experts Solve scalability problems Di. GIR 11

Design Goals n n Use open protocols and standards, such as HTTP and XML Design Goals n n Use open protocols and standards, such as HTTP and XML Decouple the protocol, software and semantics Make new data provider installations as easy as possible Develop open source software with GNU General Public Licensing (It’s free). Di. GIR 12

Di. GIR Component Summary Di. GIR 13 Di. GIR Component Summary Di. GIR 13

Di. GIR Architecture n. Provider Di. GIR 14 Di. GIR Architecture n. Provider Di. GIR 14

Provider n n n Receives requests Retrieves data from database Sends results to requestor Provider n n n Receives requests Retrieves data from database Sends results to requestor Supplies metadata to describe content, contacts, and capabilities Logs requests Di. GIR 15

Di. GIR Architecture n. Portal Engine Di. GIR 16 Di. GIR Architecture n. Portal Engine Di. GIR 16

Portal Engine n n The entry point for an application Can query a registry Portal Engine n n The entry point for an application Can query a registry to discover potential providers Can determine, based on provider metadata, whether a provider should be queried Can send requests to multiple providers Di. GIR 17

Portal Engine, continued n n Assembles responses from providers Returns packaged results to the Portal Engine, continued n n Assembles responses from providers Returns packaged results to the requesting application Communicates via protocol compliant messaging only Logs activity Di. GIR 18

Registry n n Provides a “yellow pages” to advertise the existence and capabilities of Registry n n Provides a “yellow pages” to advertise the existence and capabilities of a provider Provides a means to discover potential providers of interest May be public or private Need not be a part of the architecture Di. GIR 19

Di. GIR Architecture n. Provider n. Registry (register) Di. GIR 20 Di. GIR Architecture n. Provider n. Registry (register) Di. GIR 20

Di. GIR Architecture n. Portal Engine n. Registry (discover) Di. GIR 21 Di. GIR Architecture n. Portal Engine n. Registry (discover) Di. GIR 21

Di. GIR Protocol n Defines request and response message formats for communication between provider, Di. GIR Protocol n Defines request and response message formats for communication between provider, portal engine, and applications n n Metadata requests Search requests Inventory requests Remains unfettered by the structure of the data it transfers Di. GIR 22

Di. GIR Architecture n. Application Di. GIR 23 Di. GIR Architecture n. Application Di. GIR 23

Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine Di. GIR 24 Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine Di. GIR 24

Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (request) Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (request) n. Provider Di. GIR 25

Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (response) Di. GIR Architecture n. Application n. Protocol (request) n. Portal Engine n. Protocol (response) n. Provider Di. GIR 26

Di. GIR Architecture n. Application n. Protocol (response) n. Portal Engine Di. GIR 27 Di. GIR Architecture n. Application n. Protocol (response) n. Portal Engine Di. GIR 27

Applications n n Must be able to assemble and send a request document to Applications n n Must be able to assemble and send a request document to a portal Must be able to receive and interpret a response document from the portal Must do something incredibly useful and interesting with the data This is where the real fun is! Di. GIR 28

Hot topics – Interesting problems n n n n Persistent unique identifiers Web services Hot topics – Interesting problems n n n n Persistent unique identifiers Web services (converters, translators, calculators, transformation services) Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection, …) Spatial query interfaces Data quality feedback mechanisms Automated georeferencing, event gazetteers Concept libraries, schema extensions, schema libraries, federated ontologies Di. GIR 29

Ma. NIS – The Mammal Networked Information System It’s more than just a pangolin… Ma. NIS – The Mammal Networked Information System It’s more than just a pangolin… Di. GIR 30

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 31

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 32

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 33

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 34

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 35

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 36

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 37

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider Ma. NIS Di. GIR Portal UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer Di. GIR UAM Oracle Database Di. GIR Provider 38

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 39

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 40

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 41

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 42

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 43

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 44

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 45

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 46

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 47

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 48

Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Ma. NIS Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Ma. NIS Di. GIR Portal MVZ-Ma. NIS Presentation Layer Di. GIR 49

Ma. NIS Network Configuration … MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Ma. NIS Network Configuration … MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider … Ma. NIS Di. GIR Portal … … UMNH-Ma. NIS Presentation Layer MVZ-Ma. NIS Presentation Layer UWBM-Ma. NIS Presentation Layer … Di. GIR Provider Di. GIR UAM Oracle Database … LACM MS Access Database … CAS SQL Server Database Di. GIR Provider … 50

“Cal. Net” Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac “Cal. Net” Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Cal. Net Di. GIR Portal Di. GIR 51

Event Gazetteer Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Event Gazetteer Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider Cal. Net Di. GIR Portal Bio. Geomancer Web Service Di. GIR 52

NBII Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database NBII Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider NBII Di. GIR Portal Di. GIR 53

GBIF Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database GBIF Network Configuration CAS SQL Server Database MVZ Sybase Database LSUMZ 4 D-Mac Database Online MS Access Database Di. GIR Provider LACM MS Access Database Online MS Access Database Di. GIR Provider UAM Oracle Database Di. GIR Provider NBII Di. GIR Portal GBIF Presentation Layers Di. GIR 54

Intra-Network Configuration (BNMH) Essig Working Database PHMA Working Database UCBG Working Database UCJEPS Working Intra-Network Configuration (BNMH) Essig Working Database PHMA Working Database UCBG Working Database UCJEPS Working Database UCMP Working Databases (4) Online Database Online Database Di. GIR Provider BNHM Di. GIR Portal BNHM Presentation Layer Di. GIR 55

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 56

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 57

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 58

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 59

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Working Database Di. GIR Portal Di. GIR Working Database Online Database Di. GIR Provider Di. GIR Portal 60

Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Di. Other Network Configurations Working Database Online Database Di. GIR Provider Di. GIR Portal Di. GIR 61

Project Information n Di. GIR is a collaborative open source development project on Source. Project Information n Di. GIR is a collaborative open source development project on Source. Forge (https: //sourceforge. net/projects/digir). Software and documentation are available on the Di. GIR web site (http: //digir. net). Ma. NIS is an international network collaboration among mammal specimen collections (http: //elib. cs. berkeley. edu/manis). Di. GIR 62

Hot topics – Interesting problems n n n n Persistent unique identifiers Web services Hot topics – Interesting problems n n n n Persistent unique identifiers Web services (converters, translators, calculators, transformation services) Data validation (taxonomic and geographic thesauri, ecological niche modeling, expedition analysis, outlier detection, …) Spatial query interfaces Data quality feedback mechanisms Automated georeferencing, event gazetteers Concept libraries, schema extensions, schema libraries, federated ontologies Di. GIR 63