Model-Driven Engineering of User Interfaces Promises Successes Failures

Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http: //www. isys. ucl. ac. be/bchi Place des Doyens, 1 – B-1348 Louvain-la-Neuve (Belgium) 1 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Talk’s outline • What is model-based UI design? – What is a UI model? – What do we need to get a quality UI model? – How many models do we need to characterize a UI? • From model-based UI design to model-driven UI engineering – Three dimensions • What are the models? Which language? • What is the methodological approach to manipulate them? • What is the tool support? – Promises – Successes vs failures – Challenges • What can we do today together? 2 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is the today’s situation regarding user interfaces? • Technological aspects of user interfaces progress significantly faster than – Software engineering aspects • It takes time to develop a user interface with a new device, a new interaction technique • It takes more time to develop a toolkit • It takes even more time to rely on a model-driven approach – Usability engineering aspects • New user interfaces are shipped with usability problems because – Little or no experience – No past, no empirical evidence • Empirical experiments require a lot of resources if done carefully 3 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Dragicevic et al. , 2004]

What is model-based UI design? • Classical approch to developing UIs with a UI builder (IDE) – Describe it, prototype it, code it with trial and errors, repeat 4 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • UI builders offer only partial solution Desired Interface l l l Table with dynamic data Gantt chart Direct manipulation of widgets UI Builder Solution l l l 5 Window Menu bar Palette (icons) Scrollbars Other widgets: drop-down list, etc. Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • UI builders cannot produce their own UI (no bootstrapping) 6 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • Classical approch to developing UIs – Advantages • UI is graphical by nature • A UI can be – Rapidly prototyped – Easily modified – Visually demonstrated – Shortcomings • No structured method for layout and programming • Selection of widget can be inappropriate • Layout of widget can be tedious, repetitive • Problem of the spaghetti of callbacks • Any UI modification can lead to unstructured design 7 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • Model-based UI design consists of – Describing a UI in a UI model – Relying on this UI model to drive the UI development lifecycle – While pursuing the following goals: • To provide comprehensive UI development environments • To deliver robust development lifecycle support • To improve portability • To integrate usability with UI development – A structure in 3 dimensions Development methodology Software tools support Step-wise method Models Usi. XML Language 8 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 involves described in

What is model-based UI design? • What is a model? – Webster’s definitions: • « One who is employed to display clothes or other merchandise » • « A set of plans for a building » (good analogy) • « A system of postulates, data, and inferences presented as mathematical description of an entity or state of affairs » – Our definition • A structured set of plans for developing a UI • A system of postulates, data, and inferences presented as a UI description (or specification) that drives the UI development lifecycle 9 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – A simple example for understanding purposes: a system login – Finding the right model abstractions is NOT • Replicating code in another way • Capturing all physical properties 10 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – Finding the right model abstractions IS • Turning important aspects into UI design options – E. g. Label. Alignement instead of absolute coordinates • Deriving final properties from these options – E. g. , compute coordinates from Label. Alignement = {left, right, aligned, centred, justified} Label. Alignement = left Button. Placement = Total. Equil 11 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – Finding the right model abstractions IS • Describing UI widgets independently from technology • Why? Identification label Input text Push button 12 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – Because different evolutions Language Platform User Environment time 13 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – Because different evolutions 14 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – Because same widget appears in many different technologies Check Box (Windows) Xm. Toggle. Button (OSF-Motif) Box. Array (Garnet) – Because many different ways to reach the same goal (e. g. , selector) 15 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – In order to derive UIs systematically Grafi. XML 16 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – In order to derive UIs systematically Grafi. XML 17 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What is a model? – In order to derive UIs systematically A click on the tree highlights the corresponding Usi. XML Edit the Usi. XML Show attributes Grafi. XML 18 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What is model-based UI design? • What do we need to get a quality UI model? – Abstractions are always limited, but extensible CUI Model Plateau effect: • Do not introduce too many, otherwise too complex (<> simple) • Do not introduce too few, CIO otherwise too simplistic (<> simple) – C 1: Need to ensure quality properties of a model • Completeness graphical. CIO • Consistency • Correction • Expressiveness graphical. Container graphical. Individual. Component • … 19 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What do we have so far? • With this first set of abstractions, we go from Final UI (FUI) to Concrete UI (CUI) • Consequently, a CUI is independent from – Any language: (X)HTML, Java, Visual Basic, C++ – Any computing platform: Windows, Linux, Mac. OS Concrete user Interface S Final user Interface S 20 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

And now what? • UIs are not only graphical, but could be vocal, tactile, 3 D, haptic, an multimodal • So, there is a need for more abstraction Abstract Container Abstract Individual Component with Output Abs. Ind. Comp. with Input Abs. Ind. Comp. with Control 21 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What do we have so far? • With this second set of abstractions, we go from CUI to Abstract UI (AUI) • Consequently, a AUI is independent from – Any interaction modality Abstract user Interface S Concrete user Interface S Final user Interface S 22 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

And now what? • UIs, even if they are abstract, are structured according to the target system as opposed to be user-centered • So, there is a need for an abstraction that is computing independent – Task and domain models (UML Class Diagram) System Login Enter information ||| Enter Username 23 []>> Check U+P ||| Enter Password Check Verify existence correspondence Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What do we have so far? • With this third set of abstractions, we go from AUI to Task and Domain (T+D) • Consequently, a T+D is independent from any implementation Task and Domain S Abstract user Interface S Concrete user Interface S Final user Interface S 24 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Do we need more models? • Users want to determine their path on each platform 25 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Forrester Research, 2003]

Demo Flashi. XML 26 Do we need more models? Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Do we need more models? • Multiplicity of contexts of use Location Role Device Experience Sporting Working Travel booking site Powerful interface for complex operations Travelling Booking notification Everywhere connectivity for simple data exchange Family 27 Multimedia Travel programme TV is multi-media family device #1 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Demo Flex. Clock Do we need more models? • Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest 28 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Grolaux et al. , 2002]

So what we have now is S=Source context of use User S Platform S Environment S Task and Domain S Abstract user Interface S Concrete user Interface S Final user Interface S 29 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 In Grafi. XML [Calvary et al. , 2003]

Multi-platform for Emergency • Three platforms – Pocket PC – Desktop PC – Wall Screens 30 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Multi-platform for Emergency • Model and method – Design the reference screen first – Refine the others screens later • By applying graceful degradation • By applying transformation techniques 31 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Graceful degradation 32 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Florins & Vanderdonckt, 2004]

Transformation rules 33 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Transformation rules Add all >> Add > < << Remove all 34 > < Remove Item 1 >> << Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 1 Item 2 > Item 3 Item 4 Item 5 < Item 6 Item 7 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 Group box Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8

Transformation rules 35 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

What do we have now • Cameleon Reference Framework S=Source context of use User S Platform S Environment S Task and Domain S T=Target context of use User T Platform T Environment T http: //www. plasticity. org Task and Domain T Usi. XML supported model Abstract user Interface S Abstract user Interface T Concrete user Interface S Concrete user Interface T Usi. XML unsupported model Reification Abstraction Final user Interface S 36 Final user Interface T Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 Reflexion Translation [Calvary et al. , 2003]

Other challenges raised • C 2: Need to cover Root – Semantics: metamodel as a UML Class Diagram – Syntax: Usi. XML language – Stylistics: view results insert search criteria edit submit aux show item details submit select item show list items insert criteria show • C 3: Minimum amount of models • C 4: Maximum amount of models select view details view items view results 37 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 show

Towards model-driven engineering • Possible definition – MDA is an OMG initiative that proposes to define a set of non-proprietary standards that will specify interoperable technologies with which to realize model-driven development with automated transformations. Not all of these technologies will directly concern the transformation involved in MDA does not necessarily rely on the UML, but, as a specialized kind of MDD (Model Driven Development), MDA necessarily involves the use of model(s) in development, which entails that at least one modeling language must be used. Any modeling language used in MDA must be described in terms of the MOF language to enable the metadata to be understood in a standard manner, which is a precondition for any activity to perform automated transformation. 38 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Towards model-driven engineering • C 5: Keep decision as SDLC is evolving – Support for annotation-based design 39 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Example of preference-based design • Goal = to input the temperature of a human body – – – 40 Solution Solution n° 1: n° 2: n° 3: n° 4: n° 5: edit box list box drop down list field with scroll bar thermometer Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

How to argue for preference? • Use the QOC notation Option 1 Option 2 Criteria 2 Option i Criteria j Option n Question? Criteria 1 Criteria m = positively affects = negatively affects 41 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Mc. Lean & Belotti, 1998]

Preference example (A 1) avoids respects suggests Solution 1 (RE 1) radio buttons (A 2) contradicts Standard compatibility (e. g. Windows) Cognitive respect (A 3) Minimal actions (A 4) Dialog representativity Problem Solution 2 suggests (RE 2) (A 5) contradicts Prompting drop down list = for = against (A 6) Clarity (A 1) = drop down list requires less space than radio buttons (A 2) = some possible values become obsolete when they are infrequently used (A 3) = drop down list shows better then current value than the possible values (A 4) = radio buttons are faster and easier to manipulate than drop down list (A 5) = radio buttons are recommended in Microsoft Windows style guide (A 6) = radio buttons do not allow users to change the values (A 4) is contradicted by (A 2) and (A 3): the widget should be more used for output than input. (A 6) is contradicted by (A 3): it is better to present all possible values than only one at a time (A 5) is an autority argument, and can fall in front of (A 1)+(A 2)+(A 3) 42 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Vanderdonckt, 1997]

Theory of argumentation Design problem Is solved by Accepted solution Is described by Design option parameter (parameter, value) corresponds to Set of admissible solutions Solution 1 can be rejected because of Counter-argument (rebuttal, counter-claim) Solution 2 Solution n Can be accepted thanks to Is justified by Argument (ground, typically data) Is justified by Is reinforced by Design claim Warrant Is qualified by satisfied Qualifier unsatisfied Is related to 43 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Toulmin, 1975] [Perelman, 1978]

Demo Formi. XML Towards model-driven engineering • C 6: Need to support (de)composition 44 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Towards Model-Driven Engineering • • C 7: Support for multi-path development Different types of engineering – – • Forward engineering Reverse engineering Lateral engineering Diagonal engineering (with or without shortcuts) Different approaches – Top-down – Bottom-up – Middle-out • Different development paths – Example: Round-trip engineering = composition of • Reification CUI -> FUI • Reflexion FUI -> FUI • Abstraction FUI -> CUI • Reflexion CUI -> CUI 45 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Revisitation 46 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Bouillon, 2006]

Demo Sketchi. XML Towards Model-Driven Engineering • C 8: Support for multi-fidelity 47 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Towards model-driven engineering • Other challenges (C 13 -> C 16) – High ceiling – Low threshold – Wide walls Resource win for applications supported by MDA-compliant tools Ceiling Interface builders and integrated environments MDA CASE tools Second generation First generation 100% 50% Capabilities Resource win for applications supported by first-generation Threshold Resources (time, experience, …) 48 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Towards model-driven engineering Third generation Second generation First generation 50% 100% – High ceiling – Low threshold – Wide walls Capabilities Threshold s pe I ty U 49 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 Ceiling Integrated Development Environments • Other challenges (C 13 -> C 16) Resources (time, experience, …) Walls

Towards model-driven engineering • C 17: Effort incrementality and C 19: rendering engines – Usi. XML interpreter for • Graphical user interfaces: XHTML, XUL, Java, Flash, Tcl-Tk • Vocal user interfaces: Voice. XML • Multimodal user interfaces: X+V • Haptic user interfaces: Hapti. XML – Support for • Distributed user interfaces • Mixed-reality user interfaces 50 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Developed by Donatien Grolaux • Problem: how to design a UI that takes care of multiple displays? • Solution: Migration = DEMIPLAT • Distri. XML = software architecture for migrating UIs from one platform to another at run-time Painting Pencil Painting tool Palette 51 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Grolaux & Vanderdonckt, 2005]

Demonstration 52 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 [Grolaux & Vanderdonckt, 2005]

Demonstration using two displays from two different computers 53 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Example using a Pocket PC 54 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

UI Migration: DEMIPLAT • Detach 55 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

UI Migration: DEMIPLAT • Detach - Migrate 56 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

UI Migration: DEMIPLAT • Detach - Migrate - Plastify 57 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

UI Migration: DEMIPLAT • Detach - Migrate - Plastify - Attach 58 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

This is not a floating toolbar Process 59 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

This is migration ! Computer A Process 60 Computer B Process Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Minority report Towards model-driven engineering • C 18: More dynamic aspects – Mixed reality 61 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Conclusion • What can be really supported? 16000 Application logic User interface 14000 12000 Generated code Word 10000 8000 6000 4000 Models 2000 62 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008 Interactions Update Actions Presentation UI states Network ops. Query KB Parser KB 0 [Skezely, 1996]

Conclusion • Successes: – Work for very-well defined domains – Really fosters collaboration – Supports flexibility, maintainability (Belgium) • Failures – Does not work for very complex, dynamic UIs – Takes a lot of resources for • Models (some) • Method (more) • Tool (even more) Support for mnemonics and shortcuts 63 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

How to participate? • Join the Usi. XML Consortium today! – Anybody can be a member: observer, participant, developer – Send an e-mail to bchi@uclouvain. be and/or « Register » on the web site: for this purpose, provide • Your firstname, last name, affiliation, desired level of membership, and motivations – Think of your own research in terms of models, e. g. , • Instead of evaluating a Graphical UI for a certain platform, conduct its evaluation on the corresponding CUI model: it is more general, you will gain more audience • Instead of developing a separate tool, make it Usi. XML-compliant: you will gain more audience, the results are more largely applicable • Output – Usi. XML session at conferences – Usi. XML CD-ROM, etc. 64 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

How to participate? • Develop web services for Usi. XML • Open positions in Usi. XML project (from April 2009) – Ph. D students – Internships for MSc students 65 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008

Thank you very much for your attention http: //www. usixml. org User Interface e. Xtensible Markup Language http: //www. similar. cc European network on Multimodal UIs For more information and downloading, http: //www. isys. ucl. ac. be/bchi 66 Keynote address – ROCHI’ 2008, Iasi, 17 -18 September 2008