Do D Metadata WG Dr Glenda Hayes NCES

Do. D Metadata WG Dr. Glenda Hayes NCES Data Advisor Glenda. Hayes. ctr@disa. mil 8 Dec 2005

Do. D Net-Centric Data Strategy Goals Make the Department’s Data: Visible Accessible Understandable Trusted Is an information resource and associated POC discoverable by most users? Is it connected to the network(s), and are tools readily available to use it? Can it be intelligibly used? Are the semantics well documented? Is the source, accuracy and currency of the resource available to users? Interoperable Can it be easily combined or compared with other information or mediated? Responsive Is the resource answering user needs? Are robust, direct user feedback mechanisms in place to guide development? 2

Differentiating Types of Metadata l Structural Metadata – “Rules governing a chunk” - Name, description, data constraints, and relationships of tags used in information resources to delimit one chunk of data from another chunk – Artifacts where structural metadata is described: XML schemas, RDBMS structures, taxonomies – Register in Do. D Metadata Registry, use submission pkg l Resource Metadata – “Advertisement” - Terms to aid in the recall and retrieval of artifacts – Artifacts that we collect resource metadata on: PPT, DOC, GIF, JPG, MPG, RDBMS – Register in “Data Catalog”, use DDMS bibliographic information 3

Working Group Relationships GES Data Strategy WG Chair: NII IIPT COI Forum Ch: NII NCES Eng WIPT Do. D Metadata WG Chair: NE SOAF WG Metadata Registry FG Service Mgmt CES Metadata FG Security Service Do. D Core Taxonomy FG DDMS FG equals Focus Area A Identity Mgmt Metadata Services TBD FG Mediation Focus Area B M 2 M Messaging Defense Online Portal WG Pilot WG DOD Enterprise Collaboration WG Architecture WG Content Discovery & Delivery WG 4

Do. D Metadata Registry Engineering-level Processes Participates in DISA Engineering Staff Data Working Group Do. D Metadata WG ICMWG Reports to Hosts DHS MCOE MDR FG Develop & Publish Operates, Maintains Reports to Participate in CES FG Governs/Coordinates Acts as Registry CM Board Do. D Metadata Registry Ops Staff Controls/Displays on Web Guides Participates in Consults & Submits to/Downloads from XML Registry Evaluate, Use, Refine Namespace Managers && Managers WGs WGs Participates in DOD Developer 5

Metadata Registry FG 6

Metadata Registry Working Group MRWG Objectives: l Review and implement Do. D Metadata policy l Develop and promote best practices in metadata l Seek opportunities for convergence l Define Registry requirements, oversee development and operation l Determine what metrics to use, analyze and make recommendations to Do. D and other policies l Participate, Respect, and Influence industry, international and coalition metadata (e. g. , SQL, XML) standards Status: l 1999 -05 – COE XML Registry IOC l 2000 -10 – Namespace Mgrs Forum (NSMF) established (DISA COE CRCB) l 2005 -06 -08 – Do. D Metadata Registry v 5. 0 beta status 7

Do. D Metadata Registry l “One Stop” Publish & Subscribe for: – Build Time Structural Metadata Registration – Pulling Re-usable Data Components – Information Community Management – Current XML Gallery supporting 100, 854 items (11 Nov 2005) l Now Supporting: – Database Structures – Reference Sets – XML Structures – Taxonomies – Transformations – More Products soon (e. g. , ontologies, symbology) http: //metadata. dod. mil 8

Do. D Metadata Registry Users Discover and Pull metadata they are cleared for l Unclassified (Open) – NIPRNET w/Internet Access – http: //metadata. dod. mil l Unclassified (Sensitive) – Open Registry contents plus “password protected” components – NIPRNET with Internet Access – http: //metadata. dod. mil l Secret – Unclassified (Sensitive) content plus secret components – SIPRNET – http: //diides. ncr. disa. smil. mil l Top Secret (SCI) Managed by ICMWG Top Secret (SCI) Secret Unclassified (Sensitive) Unclassified (Open) Synchronized Components on Multiple Networks – Secret content plus SCI components – JWICS 9

Do. D Metadata Registry Supports Mediation XSLT l l Convert XML to XML (and non-XML) – e. g. , FIPS to ISO country codes, inches to centimeters Report status (degree of success/failure) 2004 -ATO. xsd (schema) Has. XSLT 2005 -ATO. xml (sample) notional stylesheet sample 2004 -2005 -ATO. xsl (stylesheet) Transforms. To. Xml. Schema 2005 -ATO. xsd (schema) Described. By schema Has. XSLT Transforms. To. Xml. Schema 2005 -2004 -ATO. xsl (stylesheet) Legend XML Gallery Organizes Components for Mediation Registered in Do. D Metadata Registry 10

Namespace Management Tasks Coordination & Configuration Management l Coordinate Stakeholders – Establish and Run Coordination Venue(s) – Resolve Issues within Namespace – Coordination across Namespaces (bilateral, multilateral and/or via Metadata Registry Focus Group) l Oversee Development of structural metadata products l Submit Information Resource Packages to Metadata Registry l Vet Information Resources and Publish Status l Perform Versioning l Participate in Metadata Registry WG Leverage Existing Metadata Management 11

DDMS FG 12

DDMS - Catalog Card Template for the GIG Do. D Discovery Metadata Specification (DDMS) Modern Warfare Lee Grant US 491. 2. W 66 1864 Coordination reflected in EO 13388 • Do. D Further Strengthening The • IC Sharing Of Terrorism • DHS Information To Protect • Justice Americans Registered in Data Catalog(s) – NCES Content Discovery 13

DDMS FG l DDMS 1. 0 – 2003 -12 – DDMS FG established – Configuration Manage (CM) DDMS Specification – Develop and CM DDMS XML Schema (for use in HF & NCES pilots) l DDMS 1. 1 – Simplification of DDMS 1. 0 – Phase 1 response to Executive Order 13356 – “Sharing of Terrorism Information” – included in EO 13388 – Import of ICISM l DDMS 1. 2 – Security Info on • Title, description, creator, publisher l DDMS 1. 3 – Geospatial references using GML – Import of GML • Postal Address Example • Bounding Box Example • Bounding Geometry Example 14

Patterns for Visibility and Accessibility (content) xyz. xsd Do. D Metadata Registry ddms. xsd . xml. html. jpg DDMS . doc DDMS . html Register Catalog Service Endpoint Catalog Introspection . gif DDMS Data Catalog Web Service DDMS Payload Description Service Registry Query Available Catalogs COI Service Discovery sws. wsdl Unanticipated Consumer Enterprise Search SWS Query Catalog Structural Metadata Discovery Content Discovery Enterprise 15

DDMS in NCES l Federated Search – Search Web Service (SWS) l DDMS-enabled Catalogs– implementing SWS – Enterprise Catalog – in pilot environment – Resource Metadata Harvesting Tools l Do. D Metadata Registry – Taxonomy Gallery – inline DDMS record 16

Taxonomy FG 17

Practical Utility for Taxonomies l Aid for organizing – Content – Services – Structural Metadata Country. sql l ISO. xsd FIPS. xsd Aid precision search via DDMS : …/My. COI. owl#Al. Qaida l Aid meaning resolution 18

Taxonomy references in DDMS 19

Taxonomies to Support Discovery Do. D Core taxonomy Organization Political. Organization Terrorist. Organization My. COI Ur. COI taxonomy Foreign. Terrorist. Organization same. As Al. Qaida same. As Producer View : Group Terrorist. Group al-Qaeda Consumer View : …/My. COI. owl#Al. Qaida …/Ur. COI. owl#Terrorist. Group 20

Do. D Core Taxonomy v 0. 75 c Product of Taxonomy Focus Group Encoded in OWL, then Registered in Do. D Metadata Registry 21

Beyond Schema Validation l eb. XML and the ICMWG expressing concern re: CM of enumerations l ICMWG support proposes enumerations not be included in the schema, but that an instance document identify its source/version for valid enumerations. l If some version of this approach is adopted, the run-time requirement will exist to query the enumeration source identified in the XML instance and any mediation that exists to retrieve the meaning for the valid value or to convert the code from one representation to the desired. These 2 XML chunks conform to the same schema, look different, but represent the same entity. 22

Taxonomy Gallery l Purpose – Provide visibility to manage and consume taxonomies • Enable COI oversight • Reuse • Harmonization Includes Do. D Core Taxonomy! – Support Discovery • Enable navigation within and among taxonomies – Even when producer and consumer do not share common taxonomy – Ensure availability • Persistent, immutable URL lowers risk of broken links l Technical Details – Implements W 3 C OWL (XML) as syntax for taxonomies – Each taxonomy is self-describing (registers via embedded DDMS) 23

Taxonomy Tie-ins l Core Taxonomy Development – C 2 IEDM – METOC – DTIC l Registered Taxonomies Inline DDMS record – Focused Logistics – DHS l Work-in-Progress: Reference Sets – Feature Codes – FIPS Country Codes – Administrative Political Subdivision – Language Codes 24

Tax. FG Status l 2004 -01 – Taxonomy Focus Group formed, Objectives: – Core Taxonomy – Syntax for Core and COI taxonomies – Implementation of taxonomies via DDMS l 2004 -05 – Taxonomy Gallery prototype l 2004 -10 – Taxonomy Gallery IOC l 2005 -02 – Finalized beta version of Core Taxonomy (v 0. 75 c) l 2005 -02 – White paper on stubbing l 2005 -03 – Core (v 0. 75 c) registered in Taxonomy Gallery 25

Core Enterprise Services (CES) Metadata FG 26

Core Enterprise Services (CES) Metadata FG Status l 2003 -01 – CES Metadata WG formed l 2003 -12 – DDMS XML Schema FG formed l 2004 -01 – Taxonomy FG formed l 2005 -02 – DDMS FG Status Briefed – DDMS 1. 2 – Core Taxonomy FG Status Briefed – “Stubbing Exercise” l 2005 -04 – DDMS FG Status Briefed – Change Requests – Core Taxonomy FG Status Briefed l New Initiative: “Design Guidelines for CES” – Naming conventions – Use of enumerations – Beyond Schema Validation 27

28

DDMS in SOAP Response - notional DDMS record 29

DDMS in Service Registration - notional DDMS record 30

Do. D Guidance l Do. D Dir 8320. 2 Data Sharing in a Net-Centric Department of Defense 2004 -12 -02 – 4. 2. Data assets shall be made visible by creating and associating metadata (“tagging”), including discovery metadata, for each asset. Discovery metadata shall conform to the Department of Defense Discovery Metadata Specification (reference (d)). http: //www. dtic. mil/whs/directives/corres/html/83202. htm l Supporting Data Asset Visibility Guidance 2003 -10 -24 – A key Do. D goal is to institutionalize the practice of advertising all data assets on the GIG by FY 2008. – By the FY 2006 program review, the Program Manager for any application, system and platform being developed, modified, or acquired will incorporate a requirement or capability description to create DDMS compliant discovery metadata for their data assets, and identify a date or release for implementation. The Program Manager will also incorporate a requirement or capability description to provide that metadata to a catalog and/or to generate an Enterprise search query response compliant with the defined discovery interface specification. http: //www. defenselink. mil/nii/org/cio/doc/Data_Visibility_Component_Guidance_10 -24 -03 sign. URL. pdf 31