Quick Guide to CAM Blueprints Using blueprints to

Quick Guide to CAM Blueprints Using blueprints to develop XML Author: exchange templates David RR Webber and schema Chair OASIS CAM TC January, 2010 http: //www. oasis-open. org/committees/cam (CAM – Content Assembly Mechanism Specification) 1

Introduction • Configuring formal information exchanges using XML transactions today requires a set of exchange schema that describe those interactions • An exchange schema provides the structural layout for the XML transaction along with rules detailing the content expected • Exchange schema development requires knowledge of XML best practices, industry specifications, data modelling techniques and guidelines for exchanging XML transactions • How does the CAM blueprint approach enable a shift away from a purely manual process today to one that can be highly automated, consistent and repeatable? 2

Solution Approach • Top Down development • Allows developers to quickly outline the information exchange • Provides XML dictionaries of canonical reusable components • CAM tool then expands the details and fill-in the blanks • Review in visual editor to confirm or extend results • Iterative approach until complete • Automated exchange package generation • Schemas, XML, documentation, mapping crosswalk • Test generation of example XML with rules validation • Principles and Rules evaluator • Provides best practices for interoperability • Checks for specific schema techniques and details • Dictionary driven reuse • Provides industry components / Enterprise Data Model • Ensures consistency of definition and use • Roll your own dictionaries 3

• Blueprints and Dictionaries Agenda • Exchange Assembly Process Steps Overview • Leveraging re-use – canonical XML dictionaries • Introducing Exchange Assembly Blueprints • Modelling information structures in XML • Using namespaces and prefixes • Constructing your Blueprint templates • Running the Expander • Finishing your Exchange Template • Generating Exchange Artifacts • NDR evaluation, Exchange schema, mapping crosswalk, XML instances, realistic data use, business rules documentation • Summary ecification Development Related Materials 4

Exchange Assembly Process Overview 1 Project Inception 2 Business Modelling Charter Work Requirements Group Search tools Exchange Content Modelling CAM Enabled Model Exchange Blueprints Dictionary Import XSD Web based Desktop Dictionary Excel 3 4 5 6 Expand, Review, Refine Expand Evaluate Results Review / Tailor Exchange Structures Generate Package Artifacts and Test them Exchange Package ecification Development Related Materials 5 Dictionary Generation / Selection NDR Evalu - ation Naming Import EDM / NDR / spell check Generate Dictionar y Complete d Exchange Structures dictionaries Completed Exchange artifacts XML XSD examples schemas Rules Report Subset Crosswalk Schema Mapping Spreadsheet Rendering Stylesheet Test Examples and Rules Schema Generation Validation Engine Test Cases XSD Schema Writer Cross Reference Mapping

Leveraging Re-use : Canonical XML Dictionaries • What is a canonical dictionary • Styles of dictionary • How do I obtain a dictionary? • How do I use it with CAM? ecification Development Related Materials 6

Canonical XML dictionary • A collection of distinct components that represent discreet business information for an application domain • Includes singleton components and combinations of related components together as sub-assemblies • Information is represented in a simple neutral conceptual data format that captures the critical concepts about the data e. g. name, description, content type, contextual usage pattern, hierarchy • Wikipedia definition: http: //en. wikipedia. org/wiki/Canonical#Computer_science ecification Development Related Materials 7

XSD Schema v CAM Dictionary • There are generally two “styles” of XSD schema • Distinct information exchange schema structures • Collections of schema model components, possibly abstract, and possibly recursive • CAM dictionaries built from distinct exchange structures generally mimic those components directly • Parent / Child sets are visible in dictionary as ABIE/BBIE pairings • “children” column in dictionary contains direct descendents • “context” column reflects component relationships • CAM dictionaries built from models with abstract and recursive parts may have fragments of components only • Assembly of exchange structures using references to piece parts, rather than complete structure assemblies • Reuse is then around these extension part collections • Context may need to be overridden to meet desired use pattern ecification Development Related Materials 8

Available XML Dictionaries • LEXS 3. 1. 4 dictionary • OASIS EDXL dictionary • OASIS EML dictionary Packaged with CAM editor XML XML see dictionary folder of install + spreadsheet + blueprint samples • NIEM 2. 1 dictionaries • • • CBRN dictionary Emergency dictionary Family dictionary Immigration dictionary Infrastructure dictionary Intelligence dictionary Justice dictionary Maritime dictionary Screening dictionary Trade dictionary NIEM core dictionary Immigration blueprint ecification Development Related Materials 9 Note: Those marked in bold are model style dictionaries with recursive components. Available from download site direct link: http: //sourceforge. net/projects/camprocessor/files + includes spreadsheets and sample blueprint XML XML XML

Drag and Drop dictionary into Spreadsheet Item names Parent item indicator List of Child items Child item indicator Item Properties Unique ID ecification Development Related Materials 10 Drag and Drop XML dictionary format into Microsoft Excel

Creating your own Dictionaries • The CAM toolkit includes dictionary generation tools • can harvest components from existing exchange schema • will import model components from ERwin modelling tool • can ingest and use subset schema or component schema • A separate companion Quick Guide describes the dictionary creation processes and tools • Look for existing dictionary packs from Sourceforge files location of CAM download http: //www. sourceforge. net/projects/camprocessor/files ecification Development Related Materials 11

Using Dictionaries • Determine your business information exchange components at conceptual level • Search and locate candidate components from appropriate domain dictionary collections • Catalogue the parts to be used • Dictionary components can be referenced individually or as collections by an assembly blueprint that puts them all together to create a complete information exchange • Components can be selected from multiple dictionaries • Note any new extension pieces as needed • A control file configures which physical dictionary files to reference and associates one or more particular namespace prefixes with each dictionary ecification Development Related Materials 12

Dictionary Control File Associates physical dictionary files to namespace prefix details Control files should be manually edited in a XML editor tool to make configuration changes as needed (e. g. arrowed text areas). ecification Development Related Materials 13

Introducing Exchange Assembly Blueprints • What is a blueprint template • Example blueprints • Blueprint tools overview • How do I create a blueprint? • How do I use it with CAM? ecification Development Related Materials 14

Introducing Blueprints • A blueprint is the outline of your information exchange and the top level sets of components needed • Allows rapid iterative exchange structure design • Start with a basic idea of what is needed, extend and refine as you clarify groups of components and their relationships • Can import components from one or more domain dictionary collections • Allows you to focus on the conceptual and hierarchical information model components and their purpose • The low level specific information details will be filled in for you by the expander tool from the matching dictionary definitions • Review results and add local extensions / exclusions as needed • Blueprints themselves also have high re-use value • Can be sub-assemblies and patterns not just exchange models ecification Development Related Materials 15

Example Blueprints OASIS EDXL HAVE message LEXS messaging blueprint Business functional components Reusable messaging envelope constructs Individual component Top level sets of business information components these examples available from CAM editor install package ~ CAMeditoreclipseworkspaceCAMEditordictionaryblueprints ecification Development Related Materials 16 Message handling, delivery and control Payload goes here

Blueprint Tools Overview Industry Domain Excel dictionary Web tool 1 Blueprint Designer Component Definitions 2 Search Tools Insert Dictionary Parent Components ecification Development Related Materials 17 3 Component Definitions Expander Tool 4 5 Completed Exchange Template

How do I create a blueprint? • Use CAM Eclipse visual editor • create basic blueprint from scratch or from XML instance • Clone and edit an existing blueprint • use a XML editor to manipulate raw XML of blueprint (e. g. search and replace namespace details) • modify blueprint using CAM Eclipse visual editor • Beware of namespaces! • You need to pre-declare namespaces in the blueprint – that may appear later in the finished expanded exchange • Use CAM Eclipse visual editor root node menu to add namespace definitions • Tip: use your XML editor to cut and paste namespaces into raw blueprint XML file directly ecification Development Related Materials 18

Blueprint Structure Overview ecification Development Related Materials 19 Short Name for Exchange Root node name and optional namespace XML container elements to facilitate use patterns Dictionary component references Dictionary Type annotation reference (note red paper clip icon) Namespace declarations

Adding Components from a Dictionary • Understanding the 4 basic types of component referencing available • Direct component (BBIE) • Direct collection (ABIE) • Indirect type (extension) • Indirect sub-component (contextual re-use) • Control of the XML structure hierarchy • Grouping collections of components • Extending an existing component • Building new extension from existing components • Namespaces – when to use them ecification Development Related Materials 20

Example XML Techniques ecification Development Related Materials 21 (Immigration Alien Alert example blueprint) Direct collections referencing (ABIE) Grouping collection of components Direct component referencing (BBIE) Building new component from existing ones Indirect type extension via annotation Type (red paper clip icon denotes annotation) Namespaces – root namespace optional (aaa) Dictionary component namespaces are required

Making New Blueprint from scratch 2 1 Open new template ecification Development Related Materials 22 3 Complete structure by adding elements and dictionary component s Assign root element

Running Blueprint Expander • Blueprint check list • Using Expander tool dialogue • Reviewing results • Hints and Tips ecification Development Related Materials 23

Using Your Blueprints • What you need • • CAM Eclipse visual editor Blueprint template Dictionary or dictionaries of components Dictionary control file • Contains linkage of namespace prefix to each dictionary • Contains physical filename path of each dictionary • Getting Started • Install CAM editor tool from Sourceforge. net • Open example blueprint from EDXL folder ~ CAMeditoreclipseworkspaceCAMEditordictionaryblueprintsEDXL • Try blueprint expander with control file EDXL-dictionary-control-file. xml ecification Development Related Materials 24

Running Blueprint Expander This example is available in the NIEM dictionaries download pack from Sourceforge Select Dictionary Control File Pick output result filename (default is -expanded. cam) Note: entries are “sticky” so are remembered for next time ecification Development Related Materials 25 Determine rule generation selections from dictionary definitions

Blueprint Expander Example COMPLETED EXCHANGE TEMPLATE 2 Expander Tool 1 Dictionary Lookups Exchange components outline Structure Details Expanded 3 EXCHANGE BLUEPRINT Blueprint follows model ecification Development Related Materials 26 Console Log detail

Reviewing Results ecification Development Related Materials 27 2 1 3 Exchange Structure items Rules for each item Domain Component s Namespace s

Hints and Tips Blueprint Remedy Node not expanded, or removed! Override default dictionary structure context definition using an annotation Type reference to the node instead – see Alien alert example blueprint – aaa: Officer. Details Added parent and child combination. All children added instead of just one. Expander inserts all children of parent. Either delete unwanted children from expanded result (see next section), or create custom parent node, and pick children components directly – see Alien alert blueprint example. Node not expanded Make sure namespace prefix and spelling exactly matches dictionary and dictionary control file prefixes. Undefined namespace prefix error Expander added dictionary components from another when opening expanded template namespace. Declare that namespace prefix back in blueprint and re-run expander again in CAM editor Expanded result missing all desired sub-components Create your own blueprint extension parent and combine parts to create desired result – see Alien alert example – aaa: Further. Details Expanded result has too many sub-components In expanded result mark unneeded parts with “exclude. Tree” rules, then run File / Export / Compress to strip out; or use extension technique above to pick only few child elements instead of whole parent ecification Development Related Materials 28

Completing the Exchange Template • • 29 Adding nodes Deleting nodes Editing Rules Adding annotations

Exchange Template Editor ecification Development Related Materials 30 1 2 Rules for each item 3 Can add new custom domain Items May need to 4 add a namespace for new items

Deleting Exchange Components Select item to be removed Right click for popup menu 1 2 3 4 From Rules exclude item or tree After marking all items to be excluded, save template, then run Files / Export / Compress template to physically remove them. ecification Development Related Materials 31

Editing Rules 1 Select item to add new rule Right click for popup 2 menu Use rule wizard to complete ecification Development Related Materials 32 3

Adding annotations 1 Select item to add new annotation Right click for popup 2 menu Use annotation wizard 3 to complete ecification Development Related Materials 33

Generating Exchange Artifacts • • 34 Dictionary crosswalk mapping Exchange Schema XML instances Business rules documentation

Exchange Generation Steps EXCHANGE TEMPLATE Structure Toolkit for exchange artifacts generation Rules 1 1. Run template Evaluation Report 2 2. Compare to industry dictionary - create mapping spreadsheet - create crosswalk xml Documentation Suite of menu options and tools in desktop CAM toolkit editor (each run custom xslt scripts on the exchange CAM template xml) CAM Template ecification Development Related Materials 35 3 Generate business rules report 4 3. Generate exchange XSD schema 5 4. Generate XML test instance(s)

Using NDR Evaluation Report • Provides scoring, alerts, warnings and potential issues including: • • • Naming and content model conventions Naming and restriction consistency checks Interoperability enablers/inhibitors checks Rules integrity and duplicates Statistics on exchange size Spell checking on component names Access from Tools menu, Evaluate Template option ecification Development Related Materials 36

Evaluation Report (NDR) example ecification Development Related Materials 37 Part of an example report for LEXS get. Data. Item. Request template displayed in toolkit HTML viewer

Compare to industry dictionary • • References industry dictionary of names and properties Matches on physical names Reports mapping details Compatible with Microsoft Excel Report can be used to do spell checking Generates crosswalk xml file When NIEM-dictionary selected: • generates NIEM SSGT wantlist. xml file • provides % re-use statistics for NIEM components ( note NIEM-dictionary. xml is located in install directory ~ CAMeditoreclipseworkspaceCAMEditordictionary folder ) ecification Development Related Materials 38

Example cross-reference spreadsheet ecification Development Related Materials 39 Formatted view in Microsoft Excel of import of crossreference report details (from generated XML file)

Generate Documentation and Schema • Documentation: • Create HTML report of exchange schema details and associated content and business rules • Report layout and content designed to be reviewed by business analysts • Schema: • Generate XSD schema for exchange • Customizable exchange folder layout management by namespace for extension, subset and exchange schema components • Writes XSD schema in syntax that is clear, simple and compatible with deployment tooling environments ecification Development Related Materials 40

Business Rules Documentation ecification Development Related Materials 41 Part of the example rules for LEXS get. Data. Item. Request template displayed in toolkit HTML viewer

Export Exchange to XSD Schema ecification Development Related Materials 42 Export Template to Exchange XSD: Completed Exchange Structure Set Exchange Output Option Complete set of exchange schemas generated

Exchange Schema Generated ecification Development Related Materials 43 Each namespace file is import for those specific type definitions Set of XSD files with filename and namespace suffix Reviewing XSD results in a schema editor tool

XML Testing Examples Generation • Support for software development testing process • Designed to allow creation of concrete realistic • • examples not just random value based Hinting system allows insertion of actual test system values into XML examples Can create both valid and invalid examples to support unit testing of application software Exclude capability allows generator to create examples that contain only a portion of the entire exchange Control over random seed value used allows regeneration of identical test cases ecification Development Related Materials 44

XML example generation wizard ecification Development Related Materials 45 View of CAM toolkit with LEXS get. Data. Item. Request and dialogue for XML test example generator tool

Running validation rules tests • Built-in CAM validation engine allows testing of XML • • instances against actual exchange rules (CAMV). Critical to ensure that the exchange validates actual live production example scenarios correctly Allows deployed solution to match exchange schema details Errors can be reviewed interactively in exchange visual interface Post-processing of validation results allows unit regression tests to be created with reporting of errors, warnings and information level notes ecification Development Related Materials 46

Run Exchange Template ecification Development Related Materials 47 Pick XML test case to validate Run validation Review results in visual editor

Example Exchange Packaging Details Package Artefact Description CAM Generates Files NDR Evaluation Report Evaluation of exchange template compared to set of Naming and Design Rules (NDR) and interoperability factors / principles Crosswalk XML Itemized list of each dictionary component element and attribute included in the exchange. Exchange Schema Base document schema that defines the full XML structure for the exchange and is generally named after the exchange itself. Extension Schema Specification for extended components—separate local name-spaces of components not contained in dictionary Sample XML Instance Example instance(s) Documentation Master Documentation The Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, the overview, as well as details surrounding the exchange, business drivers and requirements Business rules in plain or structured English, generated by CAM development tool. Mapping to Dictionary Mapping of domain components, tagged with constraints (i. e. , cardinality, etc. ) to dictionary components as a spreadsheet. Extended components Components created because they were not in dictionary—may be part of mapping spreadsheet and include structure and definitions of new components. ecification Development Related Materials 48

Summary Dictionary driven exchanges Blueprint enabled reuse Automated exchange package generation Alignment to NDR Principles and Rules Testing and validation support ecification Development Related Materials 49

• Top Down development Review • Reference dictionary components • Create exchange blueprint • Run Expander tool • Refine desired structure in visual editor • NDR Principles and Rules • Best practices for interoperability and schema techniques • Dictionary driven reuse • Select Enterprise Data Model and industry components • Ensures consistency of definition and use • Automated exchange package generation • Schemas, XML, documentation, mapping crosswalk • Test generated example XML with rules validation ecification Development Related Materials 50

Links and Resources • CAM Toolkit download • http: //sourceforge. net/projects/camprocessor • NIEM dictionaries and sample blueprint • http: //sourceforge. net/projects/camprocessor/files • NIEM tools and IEPD tools tips and hints • http: //wiki. oasis-open. org/cam/NIEM_Tools_and_CAM ecification Development Related Materials 51