DECISIONARIUM Aiding Decisions Negotiating and Collecting Opinions on

DECISIONARIUM Aiding Decisions, Negotiating and Collecting Opinions on the Web www. decisionarium. hut. fi Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology www. raimo. hut. fi JMCDA, Vol. 12 , No. 2 -3, 2003, pp. 101 -110. S ystems Analysis Laboratory Helsinki University of Technology v. 3. 2006

DECISIONARI UM global space for decision s upport group decision making group collaboration GDSS, NSS multicriteria decision analysis decision making Joint Gains multi-party negotiation support with the method of improving directions Opinions. Online RICH Decisions CSCW DSS platform for global participation, voting, surveys, and group decisions internet computer support preference programmi ng, PAIRS Smart-Swaps Web. HIPRE value tree and AHP based decision support WINPR E rank inclusion in criteria hierarchies Windows software for decision analysis with PRIME imprecise ratio statements Decisions web-sites www. decisionarium. hut. fi www. dm. hut. fi www. hipre. hut. fi www. jointgains. hut. fi www. opinions. hut. fi www. smartswaps. hut. fi www. rich. hut. fi PRIME Decisions and WINPRE downloadable at www. sal. hut. fi/Downloadables selected publications elimination of criteria and alternatives by even swaps S ystems Systems Analysis Laboratory Updated 25. 10. 2004 Helsinki University of Technology J. Mustajoki, R. P. Hämäläinen and A. Salo: Decision support by interval SMART/SWING – Incorporating imprecision in the SMART and SWING methods, Decision Sciences, 2005. H. Ehtamo, R. P. Hämäläinen and V. Koskinen: An e-learning module on negotiation analysis, Proc. of HICSS-37, 2004. J. Mustajoki and R. P. Hämäläinen, Making the even swaps method even easier, Manuscript, 2004. R. P. Hämäläinen, Decisionarium - Aiding decisions, negotiating and collecting opinions on the Web, J. Multi-Crit. Dec. Anal. , 2003. H. Ehtamo, E. Kettunen and R. P. Hämäläinen: Searching for joint gains in multi-party negotiations, Eur. J. Oper. Res. , 2001. 2 J. Gustafsson, A. Salo and T. Gustafsson: PRIME Decisions - An interactive tool for value tree analysis, Lecture Notes in Economics and Mathematical Systems, 2001.

Mission of Decisionarium Provide resources for decision and negotiation support and advance the real and correct use of MCDA History: HIPRE 3+ in 1992 MAVT/AHP for DOS systems Today: e-learning modules provide help to learn the methods and global access to the software also for non OR/MS people S ystems Analysis Laboratory Helsinki University of Technology 3

Opinions-Online (www. opinions. hut. fi) • Platform for global participation, voting, surveys, and group decisions Web-HIPRE (www. hipre. hut. fi) • Value tree based decision analysis and support WINPRE and PRIME Decisions (for Windows) • Interval AHP, interval SMART/SWING and PRIME methods RICH Decisions (www. rich. hut. fi) • Preference programming in MAVT Smart-Swaps (www. smart-swaps. hut. fi) • Multicriteria decision support with the even swaps method Joint Gains (www. jointgains. hut. fi) • Negotiation support with the method of improving directions S ystems Analysis Laboratory Helsinki University of Technology 4

New Methodological Features • Possibility to compare different weighting and rating methods • AHP/MAVT and different scales • Preference programming in MAVT and in the Even Swaps procedure • Jointly improving direction method for negotiations S ystems Analysis Laboratory Helsinki University of Technology 5

e. Learning Decision Making www. dm. hut. fi SAL e. Learning sites: Multiple Criteria Decision Analysis www. mcda. hut. fi Decision Making Under Uncertainty Negotiation Analysis www. negotiation. hut. fi S ystems Analysis Laboratory Helsinki University of Technology 6

Opinions-Online Platform for Global Participation, Voting, Surveys and Group Decisions www. opinions. hut. fi www. opinions-online. com Design: Raimo P. Hämäläinen Programming: Reijo Kalenius S ystems Analysis Laboratory Helsinki University of Technology Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi 7

Surveys on the web • • • Fast, easy and cheap Hyperlinks to background information Easy access to results Results can be analyzed on-line Access control: registration, e-mail list, domain, password S ystems Analysis Laboratory Helsinki University of Technology 8

Creating a new session • Browser-based generation of new sessions • Fast and simple • Templates available S ystems Analysis Laboratory Helsinki University of Technology 9

Possible questions • Survey section Multiple/single choice • Best/worst • Ranking • Rating • Approval voting • Written comments S ystems Analysis Laboratory Helsinki University of Technology 10

Viewing the results • In real-time • By selected fields • Questionwise public or restricted access • Barometer • Direct links to results S ystems Analysis Laboratory Helsinki University of Technology 11

Approval voting • The user is asked to pick the alternatives that he/she can approve • Often better than a simple “choose best” question when trying to reach a consensus S ystems Analysis Laboratory Helsinki University of Technology 12

Advanced voting rules www. opinion. vote. hut. fi • Condorcet criteria – Copeland’s methods, Dodgson’s method, Maximin method • Borda count – Nanson’s method, University method • Black’s method • Plurality voting – Coombs’ method, Hare system, Bishop method S ystems Analysis Laboratory Helsinki University of Technology 13

Examples of use • Teledemocracy – interactive citizens’ participation • Group decision making • Brainstorming • Course evaluation in universities and schools • Marketing research • Organisational surveys and barometers S ystems Analysis Laboratory Helsinki University of Technology 14

Global Multicriteria Decision Support by Web-HIPRE A Java-applet for Value Tree and AHP Analysis www. hipre. hut. fi Raimo P. Hämäläinen Jyri Mustajoki S ystems Analysis Laboratory Helsinki University of Technology Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi 15

Web-HIPRE links can refer to any web-pages S ystems Analysis Laboratory Helsinki University of Technology

Direct Weighting Note: Weights in this example are her personal opinions S ystems Analysis Laboratory Helsinki University of Technology

SWING, SMART and SMARTER Methods • SMARTER uses rankings only S ystems Analysis Laboratory Helsinki University of Technology

Pairwise Comparison - AHP • Continuous scale 1 -9 • Numerical, verbal or graphical approach S ystems Analysis Laboratory Helsinki University of Technology

Value Function • Ratings of alternatives shown • Any shape of the value function allowed S ystems Analysis Laboratory Helsinki University of Technology

Composite Priorities • Bar graphs or numerical values • Bars divided by the contribution of each criterion S ystems Analysis Laboratory Helsinki University of Technology

Group Decision Support • Group model is the weighted sum of individual decision makers’ composite priorities for the alternatives S ystems Analysis Laboratory Helsinki University of Technology

Defining Group Members • Individual value trees can be different • Composite priorities of each group member - obtained from their individual models - shown in the definition phase S ystems Analysis Laboratory Helsinki University of Technology

Aggregate Group Priorities • Contribution of each group member indicated by segments S ystems Analysis Laboratory Helsinki University of Technology

Sensitivity analysis • Changes in the relative importance of decision makers can be analyzed S ystems Analysis Laboratory Helsinki University of Technology

Future challenges Web makes MCDA tools available to everybody Should everybody use them? It is the responsibility of the multicriteria decision analysis community to: • Learn and teach the use different weighting methods • Focus on the praxis and avoidance of behavioural biases • Develop and identify “best practice” procedures S ystems Analysis Laboratory Helsinki University of Technology 26

Sources of biases and problems S ystems Analysis Laboratory Helsinki University of Technology 27

Visits to Web-HIPRE S ystems Analysis Laboratory Helsinki University of Technology 28

Visitors’ top-level domains S ystems Analysis Laboratory Helsinki University of Technology 29

Visitors’ first-level domains S ystems Analysis Laboratory Helsinki University of Technology 30

Visits through sites linking to Web-HIPRE S ystems Analysis Laboratory Helsinki University of Technology 31

Literature Mustajoki, J. and Hämäläinen, R. P. : Web-HIPRE: Global decision support by value tree and AHP analysis, INFOR, Vol. 38, No. 3, 2000, pp. 208 -220. Hämäläinen, R. P. : Reversing the perspective on the applications of decision analysis, Decision Analysis, Vol. 1, No. 1, pp. 26 -31. Mustajoki, J. , Hämäläinen, R. P. and Marttunen, M. : Participatory multicriteria decision support with Web-HIPRE: A case of lake regulation policy. Environmental Modelling & Software, Vol. 19, No. 6, 2004, pp. 537 -547. Pöyhönen, M. and Hämäläinen, R. P. : There is hope in attribute weighting, INFOR, Vol. 38, No. 3, 2000, pp. 272 -282. Pöyhönen, M. and Hämäläinen, R. P. : On the Convergence of Multiattribute Weighting Methods, European Journal of Operational Research, Vol. 129, No. 3, 2001, pp. 569 -585. Pöyhönen, M. , Vrolijk, H. C. J. and Hämäläinen, R. P. : Behavioral and Procedural Consequences of Structural Variation in Value Trees, European Journal of Operational Research, Vol. 134, No. 1, 2001, pp. 218 -227. S ystems Analysis Laboratory Helsinki University of Technology 32

New Theory: Preference programming Analysis with incomplete preference statements (intervals): ”. . . attribute is at least 2 times as but no more than 3 times as important as. . . ” Windows software • WINPRE – Workbench for Interactive Preference Programming Interval AHP, interval SMART/SWING and PAIRS • PRIME-Preference Ratios in Multiattribute Evaluation Method Incomplete preference statements Web software • RICH Decisions – Rank Inclusion in Criteria Hierarchies S ystems Analysis Laboratory Helsinki University of Technology 33

Preference Programming – The PAIRS method • Imprecise statements with intervals on – Attribute weight ratios (e. g. 1/2 w 1 / w 2 3) Feasible region for the weights – Alternatives’ ratings (e. g. 0. 6 v 1(x 1) 0. 8) Intervals for the overall values – Lower bound for the overall value of x: – Upper bound correspondingly S ystems Analysis Laboratory Helsinki University of Technology 34

Interval statements define a feasible region S for the weights S ystems Analysis Laboratory Helsinki University of Technology 35

Uses of interval models New generalized AHP and SMART/SWING methods DM can also reply with intervals instead of exact point estimates – a new way to accommodate uncertainty Interval sensitivity analysis Variations allowed in several model parameters simultaneously - worst case analysis Group decision making All members´ opinions embedded in intervals = a joint common group model S ystems Analysis Laboratory Helsinki University of Technology 36

Interval SMART/SWING • A as reference - A given 10 points • Point intervals given to the other attributes: – 5 -20 points to attribute B – 10 -30 points to attribute C • Weight ratio between B and C not explicitly given by the DM S ystems Analysis Laboratory Helsinki University of Technology 37

WINPRE Software S ystems Analysis Laboratory Helsinki University of Technology 38

PRIME Decisions Software S ystems Analysis Laboratory Helsinki University of Technology 39

Literature – Methodology Salo, A. and Hämäläinen, R. P. : Preference assessment by imprecise ratio statements, Operations Research, Vol. 40, No. 6, 1992, pp. 1053 -1061. Salo, A. and Hämäläinen, R. P. : Preference programming through approximate ratio comparisons, European Journal of Operational Research, Vol. 82, No. 3, 1995, pp. 458 -475. Salo, A. and Hämäläinen, R. P. : Preference ratios in multiattribute evaluation (PRIME) – Elicitation and decision procedures under incomplete information, IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans, Vol. 31, No. 6, 2001, pp. 533 -545. Salo, A. and Hämäläinen, R. P. : Preference Programming. (Manuscript) Downloadable at http: //www. sal. hut. fi/Publications/pdf-files/msal 03 b. pdf Mustajoki, J. , Hämäläinen, R. P. and Salo, A. : Decision Support by Interval SMART/SWING - Incorporating Imprecision in the SMART and SWING Methods, Decision Sciences, Vol. 36, No. 2, 2005, pp. 317 -339. S ystems Analysis Laboratory Helsinki University of Technology 40

Literature – Tools and applications Gustafsson, J. , Salo, A. and Gustafsson, T. : PRIME Decisions - An Interactive Tool for Value Tree Analysis, Lecture Notes in Economics and Mathematical Systems, M. Köksalan and S. Zionts (eds. ), 507, 2001, pp. 165 -176. Hämäläinen, R. P. , Salo, A. and Pöysti, K. : Observations about consensus seeking in a multiple criteria environment, Proc. of the Twenty-Fifth Hawaii International Conference on Systems Sciences, Hawaii, Vol. IV, January 1992, pp. 190 -198. Hämäläinen, R. P. and Pöyhönen, M. : On-line group decision support by preference programming in traffic planning, Group Decision and Negotiation, Vol. 5, 1996, pp. 485 -500. Liesiö, J. , Mild, P. and Salo, A. : Preference Programming for Robust Portfolio Modeling and Project Selection, European Journal of Operational Research (to appear) Mustajoki, J. , Hämäläinen, R. P. and Lindstedt, M. R. K. : Using intervals for Global Sensitivity and Worst Case Analyses in Multiattribute Value Trees, European Journal of Operational Research. (to appear) S ystems Analysis Laboratory Helsinki University of Technology 41

RICH Decisions www. rich. hut. fi Design: Ahti Salo and Antti Punkka Programming: Juuso Liesiö Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi S ystems Analysis Laboratory Helsinki University of Technology 42

The RICH Method Based on: Incomplete ordinal information about the relative importance of attributes • ”environmental aspects belongs to the three most important attributes” or • ”either cost or environmental aspects is the most important attribute” S ystems Analysis Laboratory Helsinki University of Technology 43

Score Elicitation • Upper and lower bounds for the scores • Type or use the scroll bar S ystems Analysis Laboratory Helsinki University of Technology 44

Weight Elicitation The user specifies sets of attributes and corresponding sets of rankings. Here attributes distance to harbour and distance to office are the two most important ones. The table displays the possible rankings. S ystems Analysis Laboratory Helsinki University of Technology 45

Dominance Structure and Decision Rules S ystems Analysis Laboratory Helsinki University of Technology 46

Literature Salo, A. and Punkka, A. : Rank Inclusion in Criteria Hierarchies, European Journal of Operational Research, Vol. 163, No. 2, 2005, pp. 338 -356. Salo, A. and Hämäläinen, R. P. : Preference ratios in multiattribute evaluation (PRIME) – Elicitation and decision procedures under incomplete information, IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans, Vol. 31, No. 6, 2001, pp. 533 -545. Salo A. and Hämäläinen, R. P. : Preference Programming. (manuscript) Ojanen, O. , Makkonen, S. and Salo, A. : A Multi-Criteria Framework for the Selection of Risk Analysis Methods at Energy Utilities. International Journal of Risk Assessment and Management, Vol. 5, No. 1, 2005, pp. 16 -35. Punkka, A. and Salo, A. : RICHER: Preference Programming with Incomplete Ordinal Information. (submitted manuscript) Salo, A. and Liesiö, J. : A Case Study in Participatory Priority-Setting for a Scandinavian Research Program, International Journal of Information Technology & Decision Making. (to appear) S ystems Analysis Laboratory Helsinki University of Technology 47

Smart-Swaps Smart Choices with the Even Swaps Method www. smart-swaps. hut. fi Design: Raimo P. Hämäläinen and Jyri Mustajoki Programming: Pauli Alanaatu Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi S ystems Analysis Laboratory Helsinki University of Technology 48

Smart Choices • An iterative process to support multicriteria decision making • Uses the even swaps method to make trade-offs (Harvard Business School Press, Boston, MA, 1999) S ystems Analysis Laboratory Helsinki University of Technology 49

Even Swaps • Carry out even swaps that make Alternatives dominated (attribute-wise) • There is another alternative, which is equal or better than this in every attribute, and better at least in one attribute Attributes irrelevant • Each alternative has the same value on this attribute These can be eliminated • Process continues until one alternative, i. e. the best one, remains S ystems Analysis Laboratory Helsinki University of Technology 50

Supporting Even Swaps with Preference Programming • Even Swaps process carried out as usual • The DM’s preferences simultaneously modeled with Preference Programming – Intervals allow us to deal with incomplete information – Trade-off information given in the even swaps can be used to update the model Suggestions for the Even Swaps process S ystems Analysis Laboratory Helsinki University of Technology 51

Decision support Even Swaps Problem initialization Eliminate dominated alternatives Preference Programming Initial statements about the attributes Practical dominance candidates Updating of Eliminate irrelevant attributes No the model More than one remaining alternative Yes Make an even swap Even swap suggestions Trade-off information The most preferred alternative is found S ystems Analysis Laboratory Helsinki University of Technology 52

Smart-Swaps • Identification of practical dominances • Suggestions for the next even swap to be made • Additional support Information about what can be achieved with each swap Notification of dominances Rankings indicated by colours Process history allows backtracking S ystems Analysis Laboratory Helsinki University of Technology 53

Example • Office selection problem (Hammond et al. 1999) 25 78 Practically An even swap dominated Commute time removed by as irrelevant Montana (Slightly better in Monthly Cost, but equal or worse in all other attributes) Dominated by Lombard S ystems Analysis Laboratory Helsinki University of Technology 54

Problem definition S ystems Analysis Laboratory Helsinki University of Technology 55

Entering trade-offs S ystems Analysis Laboratory Helsinki University of Technology 56

Process history S ystems Analysis Laboratory Helsinki University of Technology 57

Literature Hammond, J. S. , Keeney, R. L. , Raiffa, H. , 1998. Even swaps: A rational method for making trade-offs, Harvard Business Review, 76(2), 137 -149. Hammond, J. S. , Keeney, R. L. , Raiffa, H. , 1999. Smart choices. A practical guide to making better decisions, Harvard Business School Press, Boston. Mustajoki, J. Hämäläinen, R. P. , 2005. A Preference Programming Approach to Make the Even Swaps Method Even Easier, Decision Analysis, 2(2), 110 -123. Salo, A. , Hämäläinen, R. P. , 1992. Preference assessment by imprecise ratio statements, Operations Research, 40(6), 1053 -1061. Applications of Even Swaps: Gregory, R. , Wellman, K. , 2001. Bringing stakeholder values into environmental policy choices: a community-based estuary case study, Ecological Economics, 39, 37 -52. Kajanus, M. , Ahola, J. , Kurttila, M. , Pesonen, M. , 2001. Application of even swaps for strategy selection in a rural enterprise, Management Decision, 39(5), 394 -402. S ystems Analysis Laboratory Helsinki University of Technology 58

Joint-Gains Negotiation Support in the Internet www. jointgains. hut. fi Eero Kettunen, Raimo P. Hämäläinen and Harri Ehtamo Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi S ystems Analysis Laboratory Helsinki University of Technology 59

Method of Improving Directions • Interactive method for reaching efficient alternatives Efficient frontier Utility of DM 2 Ehtamo, Kettunen, and Hämäläinen (2002) . . Utility of DM 1 • Search of joint gains from a given initial alternative • In the mediation process participants are given simple comparison tasks: “Which one of these two alternatives do you prefer, alternative A or B? ” S ystems Analysis Laboratory Helsinki University of Technology 60

Mediation Process Tasks in Preference Identification • Initial alternative considered as “current alternative” • Task 1 for identifying participants’ series of most preferred directions pairwise comparisons • Joint Gains calculates a jointly improving direction • Task 2 for identifying participants’ most preferred alternatives in the jointly improving direction series of pairwise comparisons S ystems Analysis Laboratory Helsinki University of Technology 61

Joint Gains Negotiation • User can create his own case • 2 to N participants (negotiating parties, DM’s) • 2 to M continuous decision variables • Linear inequality constraints • Participants distributed in the web S ystems Analysis Laboratory Helsinki University of Technology 62

DM’s Utility Functions • DM’s reply holistically • No explicit assessment of utility functions • Joint Gains only calls for local preference information • Post-settlement setting in the neighbourhood of the current alternative • Joint Gains allows learning and change of preferences during the process S ystems Analysis Laboratory Helsinki University of Technology 63

Case example: Business delivery (days) • Two participants 30 buyer and seller • Three decision variables unit price ($): 10. . 50 amount (lb): 1. . 1000 1 delivery (days): 1. . 30 1 amount (lb) • Delivery constraint (figure): 999*delivery - 29*amount ³ 970 • Initial agreement: 30 $, 100 lb, 25 days 1000 S ystems Analysis Laboratory Helsinki University of Technology 64

Creating a case: Criteria to provide optional decision aiding S ystems Analysis Laboratory Helsinki University of Technology

Sessions • Participants take part in sessions within the case • Sessions produce efficient alternatives Joint Gains - Business • Case administrator can start new Session 1 efficient sessions on-line and define new point Session 2 efficient initial starting points point Session 3 efficient • Sessions can be parallel. point. . • Each session has an independent Session n efficient mediation process point S ystems Analysis Laboratory Helsinki University of Technology 66

New comparison task is given after all participants have completed the first one Not started Preference identification task 1 Preference identification task 2 JOINT GAIN? Stopped S ystems Analysis Laboratory Helsinki University of Technology

Session view - joint gains after two steps S ystems Analysis Laboratory Helsinki University of Technology

Literature Ehtamo, H. , M. Verkama, and R. P. Hämäläinen (1999). How to select Fair Improving Directions in a negotiation Model over Continuous Issues, IEEE Trans. On Syst. , Man, and Cybern. – Part C, Vol. 29, No. 1, pp. 26 -33. Ehtamo, H. , E. Kettunen, and R. P. Hämäläinen (2001). Searching for Joint Gains in Multi-Party Negotiations, European Journal of Operational Research, Vol. 130, No. 1, pp. 54 -69. Hämäläinen, H. , E. Kettunen, M. Marttunen, and H. Ehtamo (2001). Evaluating a Framework for Multi-Stakeholder Decision Support in Water Resources Management, Group Decision and Negotiation, Vol. 10, No. 4, pp. 331 -353. Ehtamo, H. , R. P. Hämäläinen, and V. Koskinen (2004). An E-learning Module on Negotiation Analysis, Proc. of the Hawaii International Conference on System Sciences, IEEE Computer Society Press, Hawaii, January 5 -8. S ystems Analysis Laboratory Helsinki University of Technology 69

e. Learning Decision Making www. mcda. hut. fi e. Learning sites on: Multiple Criteria Decision Analysis Decision Making Under Uncertainty Negotiation Analysis Prof. Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology http: //www. sal. hut. fi S ystems Analysis Laboratory Helsinki University of Technology 70

e. Learning sites Material: • Theory sections, interactive computer assignments • Animations and video clips, online quizzes, theory assignments Decisionarium software: • Web-HIPRE, PRIME Decisions, Opinions-Online. vote, and Joint Gains, video clips help the use e. Learning modules: • 4 - 6 hours study time • Instructors can create their own modules using the material and software • Academic non-profit use is free S ystems Analysis Laboratory Helsinki University of Technology

S ystems Analysis Laboratory Helsinki University of Technology 72

Learning paths and modules Learning path: guided route through the learning material Learning module: represents 2 -4 h of traditional lectures and exercises Learning Paths Theory Cases Quizzes Videos Assignments Evaluation Introduction to Value Tree Analysis Module 2 Module 3 S ystems Analysis Laboratory Helsinki University of Technology

Learning modules Learning Theory Paths Quizz Videos. Assignments Evaluation es Introduction to Value Tree Analysis Module 2 Cases Module 3 • motivation, detailed instructions, 2 to 4 hour sessions Theory • HTML pages Case • slide shows • video clips Web software • Web-HIPRE • video clips S ystems Analysis Laboratory Helsinki University of Technology Assignments • online quizzes • software tasks • report templates Evaluation • Opinions Online

Cases Family selecting a car Job selection case • basics of value tree analysis Theory Assignments • how to use Web-HIPRE Intro Evaluation Car selection case Problem structuring • imprecise preference statements, interval value trees • basics of Prime Decisions software Preference elicitation Family selecting a car Theoretical foundations • group decision-making with Web-HIPRE • weighted arithmetic mean method S ystems Analysis Laboratory Helsinki University of Technology 75

Video clips • Recorded software use with voice explanations (1 -4 min) • Screen capturing with Camtasia • AVI format for video players – e. g. Windows Media Player, Real. Player • GIF format for common browsers - no sound S ystems Analysis Laboratory Helsinki University of Technology Learning Theory Paths Cases Quizzes Videos Assignments Videos Working with Web-HIPRE Structuring a value tree Entering consequences of. . . Assessing the form of value. . . Direct rating SMART SWING AHP Viewing the results Sensitivity analysis Group decision making PRIME method 76

Learning Theory Paths Cases Quizzes Videos Assignments testing the knowledge on the subject, learning by doing, individual and group reports Software use • value tree analysis and group decisions with Web-HIPRE Report templates • detailed instructions in a word document • to be returned in printed format S ystems Analysis Laboratory Helsinki University of Technology 77

Academic Test Use is Free ! Opinions-Online (www. opinions. hut. fi) Commercial site and pricing: www. opinions-online. com Web-HIPRE (www. hipre. hut. fi) WINPRE and PRIME Decisions (Windows) RICH Decisions (www. rich. hut. fi) Joint Gains (www. jointgains. hut. fi) Smart-Swaps (www. smart-swaps. hut. fi) Please, let us know your experiences. S ystems Analysis Laboratory Helsinki University of Technology 78

Contributions of colleagues and students at SAL • HIPRE 3 +: Hannu Lauri • Web-HIPRE: Jyri Mustajoki, Ville Likitalo, Sami Nousiainen • Joint Gains: Eero Kettunen, Harri Jäälinoja, Tero Karttunen, Sampo Vuorinen • Opinions-Online: Reijo Kalenius, Ville Koskinen Janne Pöllönen • Smart-Swaps: Pauli Alanaatu, Ville Karttunen, Arttu Arstila, Juuso Nissinen • WINPRE: Jyri Helenius • PRIME Decisions: Janne Gustafsson, Tommi Gustafsson • RICH Decisions: Juuso Liesiö, Antti Punkka • e-learning MCDA: Ville Koskinen, Jaakko Dietrich, Markus Porthin Thank you! S ystems Analysis Laboratory Helsinki University of Technology 79

Public participation project sites • PÄIJÄNNE - Lake Regulation (www. paijanne. hut. fi) • PRIMEREG / Kallavesi - Lake Regulation (www. kallavesi. hut. fi, www. opinion. hut. fi/servlet/tulokset? foldername=syke) • STUK / Milk Conference - Radiation Emergency (www. riihi. hut. fi/stuk) S ystems Analysis Laboratory Helsinki University of Technology 80

SAL e. Learning sites • www. dm. hut. fi Decision making resources at Systems Analysis Laboratory • www. mcda. hut. fi e. Learning in Multiple Criteria Decision Analysis • www. negotiation. hut. fi e. Learning in Negotiation Analysis • www. decisionarium. hut. fi Decision support tools and resources at Systems Analysis Laboratory • www. or-world. com OR-World project site S ystems Analysis Laboratory Helsinki University of Technology