Using XML to Encode Questionnaires Existing Standards

Using XML to Encode Questionnaires Existing Standards & Technical Implementation “The Why and How of QEDML” Philip Cookson Director of Research Philology Pty. Ltd. Email: philip@philology. com. au Jason Sobell Chief Technologist Philology Pty. Ltd. Email: jason@philology. com. au Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 1

About the Authors … Philip. Cookson is an internationally experienced Marketing Researcher, with expertise in the area of Internet-based market research and survey automation technology. Philip is the Director of Research and co-founder of Philology Pty. Ltd. Philip’s previous industry experience include: Director of Server & Development Tools Business Research at Microsoft Corporation, where he was responsible for Microsoft’s global research for its Server and Development Tools products; and Senior Director for Global Market Research & Analysis with Apple Computer Inc. , where he was responsible for conducting global market analysis and customer research relating to Apple's new products. Philip is a full member of the Market Research Society of Australia, and has achieved QPMR accreditation. Philip holds a Masters Degree in Mathematical Physics, and a Post-graduate diploma in Computer Science. Jason Sobell is a professionally qualified software engineer and database designer who has extensive international experience as an Information Technology consultant. Jason has a Masters Degree in Information Technology from RMIT University, and lectures in commercial database applications at RMIT University and the University of Melbourne. Jason is co-developer of the QEDML standard for encoding survey questionnaire designs; and is the Chief Technologist at Philology Pty. Ltd. where he is responsible for the technical development of the QEDML product family. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 2

Objectives of the presentation… • Explain the overall architectural design issues and technical constraints/challenges in creating an XML standard (QEDML) for encoding questionnaire designs and respondent data sets. • Discuss the practical (operational) issues associated with developing a software for questionnaire design and deployment … using the QEDML software suite as a case study. • Where relevant, contrast and compare the design approaches for other XML–based standards for questionnaire design and deployment. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 3

Target Audiences … • Architects, Developers, Implementers, and “Power Users” of questionnaire design and deployment software systems. • Those responsible for developing or deciding on standards for the storage, manipulation, and/or retrieval of questionnaires and/or respondent data sets. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 4

Agenda… for the next 45 minutes … • Introduction: Why Standards? Why XML? Why QEDML? – – – Why a standard is needed? The benefits and limitations of using XML to encode questionnaire designs The “Why” of QEDML – QEDML “raison d’être” • QEDML (XML) Architectural Design Principles – – The “How” of QEDML – QEDML Design philosophy/guidelines/decisions/compromises Developing a framework for “comparing” XML implementations • Practical (Technical) Implementation Issues … and “Solutions” – – – Questionnaire Design vs. Questionnaire Storage/Deployment/Data Collection Extensibility: New Question Types, Dealing with “compound” questions Managing Multi-lingual questionnaires • Summary: Key Takeaways & Next Steps – XML Design Approach; XML implementation issues, Future Directions for QEDML • Question & Answer – Question/Discuss/Debate the key conclusions and implications! Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 5

Why Standards? : Survey Automation Systems Design Integrated Software Suite • All components of the system are built by a single vendor and are explicitly designed to work together. Integration of Point Solutions • Components from different vendors are used as point solutions and are linked together via a set of standardized interfaces. • PRO – Ability to choose “Best of Breed” – Seamless exchange and integration • PRO of data between components • CON – Constrained to the capabilities provided by a single vendor. – Often impractical to enforce a single solution across a wide variety of market research survey settings. solutions. – Greater flexibility in vendor choice. • CON – Requires someone to take on a “systems integrator” role – Lack of common standards for data interchange (or sometimes too many incompatible “standards” for data/design interchange) Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 6

XML: What is it, and why is it important? • SGML – Structured General Markup Language • XML – EXtensible Markup Language A subset of SGML constituting a particular text markup language for the storage and exchange of structured information, retaining both the content and context of data. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 7

Sample XML document

XML as an Information Interchange Standard • Human readable – Encoded using ASCII or Unicode (multiple-language support) • Future-safe – Self-documenting – (As long as we have the means to read the storage media) • Extensible – New tags and language elements may be introduced while maintaining backwards compatibility • XML documents can be validated ( DTD or XML Schema) – Ideal as an interchange format between organisations • Flexible rendering tools – Tools such as XSLT renderers allow multiple output formats to be automatically generated Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 9

XML as a standard for encoding questionnaires XML is an ideal means for the storage and interchange for questionnaire data because questionnaires… • Consist of a tree structure (natural hierarchy) • Implement looping and conditional sections • Contain meta-data or attributes relating to questions and/or sections of a questionnaire • Require rendering to a variety of deployment platforms and output document formats • Must be retained for future access if subsequent respondent data is to be of long lasting value Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 10

What is QEDML? Questionnaire Exchange, and Deployment Markup Language QEDML is an XML-based standard for encoding questionnaire designs so that they can be deployed in a variety of output formats and contexts, ranging from paper based documents, to online tests/surveys, to structured output data types suitable for integration into Survey Automation and e-Learning systems. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 11

QEDML Architectural Overview Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 12

Example of a simplified QEDML snippet … Q 1 Which one of the following is not a right hemisphere function? A … … creating a poem completing a jigsaw puzzle feeling sensations on the left side admiring artwork … Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 13

QEDML Design “Philosophy” • The key to representing, storing, and correctly interpreting respondent data sets is through a comprehensive description of the questionnaire design. – Resultant data sets can be stored as a simple table referenced via a unique Survey/Questionnaire ID link if there is a complete representation of the questionnaire design. – XML implementations that focus on the resultant data sets are missing key aspects of the design that are essential to correct interpretation of the results (e. g. why responses are skipped/missing). Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 14

QEDML Design “Philosophy” … • A complete questionnaire design must explicitly factor in the operational characteristics and constraints of the deployment (data collection) systems that may be used to field the questionnaire. – Must be able to accommodate the requirements of multi-modal deployment (Print, Web, CATI, CAPI) – Self-Administered vs. Interviewer Administered; Question sequencing (logical flow) – Some questionnaire design “entities” only exist during deployment (e. g. “piping”, “list filtering”) Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 15

QEDML Design “Philosophy” … • Questionnaire design is not restricted to the domain of market research surveys, and so a standard for encoding questionnaires should be flexible enough to adapt to the full spectrum of questionnaire design applications – Market Research Survey applications – Educational and/or Assessment Testing applications – Forms based (intelligent) data collection Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 16

QEDML/XML Standard Architectural Design Requirements

QEDML/XML Standard Design Requirement … #1 • A standards based representation of a questionnaire design should be able to accommodate multiple language variants within the context of a single document. – Each “translatable” element needs to be represented as a distinct entity – The standard should support all Unicode languages – The translation status of each element should be preserved throughout the editing phase – “Language variants” should accommodate different deployment requirements (e. g. Web vs. CATI) Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 18

QEDML/XML Standard Design Requirement … #2 • The encoded questionnaire design should be able to be automatically translated into any other functionally complete XML representation of the questionnaire using a simple XSLT transform. – All complete XML representations of a questionnaire design should be functionally equivalent. – Each element of the design must be “atomic” in terms of its XML representation (i. e. you should not have to parse the content within a pair of XML tags to extract a value). Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 19

QEDML/XML Standard Design Requirement … #3 • The encoded questionnaire design and associated resultant data sets, should be able to be automatically translated into a wide variety of output document and script formats compatible with those commonly used for questionnaire design and deployment in market research, and educational testing environments. – Transformed using XSLT transforms, via report templates, or standard programming interfaces – For example: MS Word, PDF, Quancept Script … SPSS . sav, MS Excel, CSV Export etc. – Needs to be able to represent a “superset” of question data types, logical constructs, etc. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 20

QEDML/XML Standard Design Requirement … #4 • The standard should be flexible enough to incorporate the full spectrum of existing and emerging questionnaire data types and constructs. – Base data types: Number (Integer, Real); Text (Char, Memo); Binary (Boolean, BLOB) – Multimedia data types: Audio (In/Out), Video (In/Out), Ink, Image etc. – Standard question types: SP, MP, Ratings Grid, 1 D and 2 D Grids; Compound Questions – Question constructs: Groups, Loops, Sections, Instruction, etc. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 21

QEDML/XML Standard Design Requirement … #5 • The standard should support vendor, application, and customer specific extensions to the standard through a generic “metadata” mechanism. – Each component of the design should be able to have an associated metadata tag – The metadata should be able to include structured (hierarchical) XML data Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 22

QEDML/XML Standard Design Requirement … #6 • The standard should be capable of accurately representing the wide variety of hierarchical (treelike), grouped, and looped (iterated) data structures that are found in real world market research and educational testing applications. – Sections, sub-sections of questions; Instructional (i. e. non-data) information fields – Logical groupings of questions; Iterative data structures (looped list) Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 23

QEDML/XML Standard Design Requirement … #7 • The standard should be capable of accurately representing the logical sequencing (program logic) of the questionnaire. – Issue of how to accommodate the differences in the proprietary programming/scripting environments used in real-world survey deployment systems. Ideally, the code elements should be represented “atomically” in XML (QEDML Script) Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 24

QEDML/XML Standard Design Requirement … #8 • The questionnaire design should be stored as a standard XML document (native file format), and the resultant data set should be defined as a standard XML document … though not necessarily use XML as the native data storage format (for efficiency/practicality of use). – Pure XML representation; Compatibility with commonly used formats for information storage Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 25

QEDML/XML Standard Design Requirement … #9 • The standard should be capable of accurately representing the resulting data set in a complete and efficient manner that is compatible with commonly used methods for storing and reporting on respondent data sets. – Efficient = compact storage, standard retrieval for analytical use Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 26

A framework for “comparing” XML Implementations A Aspect XML Implementation Multilingual Support Pure XML document format Atomic representation Data types and construct support Single document architecture Preserve Translation Status Multi-modal Support Application Support Industry Support ………………… QEDML Rating Vendor X Rating (0 -10) TOTAL (Importance Weighted) C (0 -10) Feature B Importance Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 = SUM (A *B) = SUM (A *C) 27

XML/QEDML Practical Implementation Issues Where the “Rubber” (of XML theory & benefits), meets the “Road” (of developing real-world questionnaire design and deployment software systems).

Technical Implementation: XML File Size Issues Problem: XML is a verbose form of representation, and as a result questionnaire designs encoded in XML are often considerably larger than questionnaire designs encoded using a proprietary/binary format. For example, an “empty” QEDML project is 7 Kbytes, a standard (single language) QEDML survey questionnaire is 100 - 200 Kb, and a complex (multi-lingual, adaptive) QEDML survey questionnaire may be as large as 1 - 2 MB. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 29

Technical Implementation: XML File Size Issues … Solution: – For single-user, local-disk based questionnaire design applications, the XML overhead is of little practical consequence. However, for the implementation of networked, multi-user, questionnaire repositories that support the ad-hoc searching and retrieval of questions/questionnaires from a central repository … the overhead of shunting multi-megabyte survey files across the network can pose a potential performance bottleneck. – Fortunately, XML documents are amenable to “on-the-fly” compression and decompression of the document, which can dramatically reduce the overall file size. For example a 1 MB QEDML survey questionnaire document can be compressed to 50 Kb using the standard WIN. ZIP utility. – For networked questionnaire repository systems it is often worthwhile to store the XML file in both raw and compressed formats to reduce the processing need for file compression prior to every transmission of the questionnaire document. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 30

Technical Implementation: Multi-lingual Surveys … Problem: How do you maintain multiple language variants of a questionnaire within a single document where the text associated with questions in the survey may be different “states” of being translated … while preserving the “translation” state during the questionnaire editing process. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 31

Technical Implementation: Multi-lingual Surveys … Solution: – Need to represent and separately tag each translatable element of each questionnaire in each of the languages implemented in the questionnaire, along with the “Translation Status” of the element. – The Questionnaire Editor application must explicitly reset the “Translation Status” of all language variants when ever the corresponding default (base) language text is changed. Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 32

Technical Implementation: Multi-lingual Surveys … • “Languages” may be “deployment” specific language or spoken/written languages Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 33

Technical Implementation: Multi-lingual Surveys … Default/Invariant Language fr de Joint RSS/ASC meeting on Systems Architecture for Statistical Applications: January 2006 34