9dc50b3735718b6f6ac7d177b937adc7.ppt

- Количество слайдов: 51

A Core Course on Modeling Week 7 -A working model – and then? Contents • The Need for Interpretation • Approach • Criteria for Modeling • Genericity • Scalability • Specialization • Population • Convincingness • Distinctiveness • Surprise • Impact • Criteria for modeling and purposes • Summary • References to lecture notes + book • References to quiz-questions and homework assignments (lecture notes)

A Core Course on Modeling Week 7 -A working model – and then? The Need for Interpretation 2 A model outcome in itself cannot solve a problem. Interpretation is necessary final step. How to achieve a goodquality interpretation? Model makers in 19 th century London predicted 3 meter rising of horse manure levels by 1950 Model makers in 21 st century predict 0. 5 meter rising of sea levels by 2050

A Core Course on Modeling Week 7 -A working model – and then? Approach 3 We want to say that one model (including the interpretation of the outcomes) is better than another. ‘Better’ means: with respect to fulfilling the model’s purpose. Therefore we need criteria to compare. The first step to quality assessment is the ability to compare things that are utterly different. Comparison invariably needs criteria.

A Core Course on Modeling Week 7 -A working model – and then? Approach So: we will try to formulate criteria for comparing problem As yet, we have no means to solutions. compare ‘quality’ of interpretations Problem solution = model outcome + interpretation can be assessed (compared) with the tools of Chapter 6 (sensitivity analysis, uncertainty analysis) 4

A Core Course on Modeling Week 7 -A working model – and then? Criteria for Modeling 5 When formulating criteria for comparing problem solutions, the main problem is the immense variety of problems asking for modeling. Our criteria need to apply to all of them. The large variety of possible modeling processes is a complication for finding a single generic set of criteria.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for Modeling 6 The standard approach to finding a complete set of ‘things’, is to construct a complete taxonomy. A complete taxonomy requires enumerable properties: properties where all possible values can be enumerated. The quarto-game is based on 4 enumerable properties: color(black, white), size(tall, short), shape(round, square), interior(hollow, solid). Rows of 4 must be formed of pieces which all have a common value for one property

A Core Course on Modeling Week 7 -A working model – and then? Criteria for Modeling 7 Property 1: Does the criterion regard the begin (definition stage of the modeling process) or the end (conclusion stage)? • Example of criterion related to definition: what is the scale of the problem? (scalability) • Example of criterion related to conclusion: how much depends on it? (impact) The input-side of a problem relates to stage 1 (problem definition); the output-side relates to stage 5 (conclusion: presentation and interpretation). This photo seems to suggest that input and output are relative notions …

A Core Course on Modeling the model: Week 7 -A working model – the modeled system: part of present painting and then? the world, Criteria for Property 2: including the Modeling model and some stakeholder 2: stakeholders the modeler stakeholder 1: the customer 8 Does the criterion regard the modeled system or the (stakeholders + context) ? • Example of criterion related to modeled system: what is the scale of the problem? (scalability) • Example of criterion related to stakeholders: how much knowledge do the stakeholders need? (specialization) In this famous Velasquez painting, the stakeholders and the modeled system are deliberately confused. In modeling practice, the distinction between subject (the modeled system) and object (the stakeholder(s)) are clearly distinct. Hopefully.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for Modeling 9 Property 3: Does the criterion regard qualitative or quantitative aspects ? • Example of criterion related to quantitative aspect: what is the scale of the problem? (scalability) • Example of criterion related to qualitative aspect: how different can modeled systems be? (genericity) The distinction quantitative – qualitative is sometimes associated to the closed – open distinction. ‘Closed’ means that all possible outcomes can be foreseen or enumerated; ‘open’ means that this is not possible.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for Modeling 10 Combinations define/ conclude modeled system / stakeholders qualitative / quantitative criterion define modeled system qualitative genericity define modeled system quantitative scalability define stakeholders qualitative define stakeholders quantitative conclude modeled system qualitative conclude modeled system quantitative conclude stakeholders qualitative conclude stakeholders quantitative specialization population convin. The 3 distinctions together form a imaginary cube, cingness each corner corresponds to one of the criteria. distinctiveness surprise impact

A Core Course on Modeling Week 7 -A working model – and then? Genericity: to which extent is the approach capable to handle various types of modeled systems / purposes? Examples: Physics: involve more effects, principles and interactions (flow, continuum mechanics) Mathematics: abstraction Computer Science: generic tools (graphs, databases, compilers, XML, . . . ) 11

A Core Course on Modeling Week 7 -A working model – and then? Genericity 12 Genericity: example Given an object of some shape and density ; we want to know its mass. 1. If it is a cube with side p: m= a 3 2. if it is a rectangular block with height h: m= h*Areatop 3. if it is a truncated pyramid: m= h*(Areatop + (Areatop*Areabottom) + How heavy is the Borobodur? (E. g. , in the context of Areabottom)/3 (works for all these shapes!) risks for landslides etc. : the Borobodur is one of the Above models are increasingly generic; each can deal with the simpler cases as well. heaviest temples in the world, restauration in the 1980 -ies required partial reinforcement of the hill on which it stands to prevent landslides). Only formula (‘model’) 3 is sufficiently generic

A Core Course on Modeling Week 7 -A working model – and then? Scalability 13 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional Many problems contain some characteristic dimension (e. g. , n=number of boxes). If n grows, O(np) – SE polynomial there will come a point where the model fails when O(2 n), O(n!), … – SE worse than polynomial trying to solving the problem. The larger n can be, the better the scalability of the solution.

A Core Course on Modeling Week 7 -A working model – and then? Scalability 14 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n The size of Google’s data repository, according to http: //www. worldwide O(n log n) – SE slightly worse than proportional www. worldwidewebsize. com , varied O(np) – SE polynomial between 15 and 55 billion webpages in less O(2 n), O(n!), … – SE worse than polynomial than 4 months. Still, the response time stays near constant due to sophisticated indexing O(n) – SE proportional to n

A Core Course on Modeling Week 7 -A working model – and then? Scalability 15 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2 n), O(n!), … – SE worse than polynomial To tell if there is a needle in the haystack is easy, provided the hay is sorted: check the halfway element. If this is larger than the needle, discard the upper half, otherwise discard the lower half – and checking the halfway element in the remaining part until either found or not found.

A Core Course on Modeling Week 7 -A working model – and then? Scalability 16 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2 n), O(n!), … – SE worse than polynomial For many tasks, the effort of doing n things is proportional to n. If one chicken lays three eggs per week, 5 chicken lay 15 eggs per week. This does not hold for everything, though … climbing n flights of stairs may take more effort than n the effort of one. But computers don’t get tired, and many computer tasks scale similarly to chicken laying eggs.

A Core Course on Modeling Week 7 -A working model – and then? Scalability 17 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2 n), O(n!), … – SE worse than polynomial Sorting an array of unsorted things can be done in O(n log n). Military go faster, but they work in parallel.

A Core Course on Modeling Week 7 -A working model – and then? Scalability 18 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2 n), O(n!), … – SE worse than polynomial Straightfowardly sweeping a matrix takes O(n 3), but faster methods exist that bring this down to something like O(n 2. 5). The difference is noticable only for n sufficiently large.

A Core Course on Modeling Week 7 -A working model – and then? Scalability 19 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) O(1) Performance: n O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2 n), O(n!), … – SE worse than polynomial The travelling salesman problem asks for the shortest route connecting n places. Exact solution is believed impossible in O(np) but good approximations sometimes take only O(np)

A Core Course on Modeling Week 7 -A working model – and then? Specialization 20 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? How much control should the modeler give the problem owner over • validity of assumptions, • regime of application, • use of results? A model, intended for specialists has to face the challenge of withstanding expert scrutiny. On the other hand, a model intended for laymen has to be robust against unintended misuse.

A Core Course on Modeling Week 7 -A working model – and then? Specialization 21 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? 1. Invest in presentation • graph style (line plot, scatterplot, polar plot, barchart, pie chart, 3 D surface, …) • labels, captions, scales (zero, lin/log, units, major / minor scale lines …) • how to represent uncertainty? • which plots to combine in a single graph? The appropriate visualization style is the answer to the question: what message should this graph convey?

A Core Course on Modeling Week 7 -A working model – and then? Specialization 22 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? 2. When in doubt, be conservative A modeler resembles a blindfolded torreador: without full knowledge, consequences of the modeled system being mischievous may be quite harmful. For good reasons, the jackass adagium reads: ‘don’t try this at home!’

A Core Course on Modeling Week 7 -A working model – and then? Specialization 23 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? 3. Be warned for biased users • give a full and detailed account of as much as possible assumptions and estimates • insist on the report being indivisible • refrain from easy-to-misapprehend results • Get a second opinion Users having personal interests in model outcomes may not always take a balanced stand.

A Core Course on Modeling Week 7 -A working model – and then? Specialization 24 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? 4. Take care for self-fulfilling and self-denying prophecies • wicked problems: the problem solution post-hoc modifies the problem statement Both legendary prophet Michael Nostradamus (glass bowl? ) and present-day opinion investigators (black ballot box? ) relate to the working principle of self-fulfilling and self-denying prophecies.

A Core Course on Modeling Week 7 -A working model – and then? Population 25 Population: what size of intended audience does the model address? • Large size: low level of specialization • Large size: no option for bi-directional communication • Large size: consider interactive model (example: ‘stemwijzer’) In the 1999, Wachowski-brothers’ movie ‘the Matrix’, a simulation’s intended audience size is stretched so as to encompass the entire world’s population – with the extra that nearly everybody is ignorant of the fact.

A Core Course on Modeling Week 7 -A working model – and then? Convincingness 26 Convincingness: a model is more convincing if it contains fewer and/or less implausible assumptions. • Are assumptions logically deducible from other, less problematic assumptions? • If not, are there any first-principle ‘laws’ to back up assumptions (example: if this component behaves as a lever, we can use the physical laws of torque in a lever)? • If not, can we construct a simplified model system (example: tidal motions as a lumped system) with outcomes that can be empirically verified? • If not, can we get support from comparison to empirical observations on an existing system (example: behavior of visitors in shopping malls to predict buying behavior)? • If not, is there at least any argument for consistency of the assumptions? (example: price elasticity, ‘fun’ in the taxi model) In the early days of photography, a photo (as prototypical empirical evidence) counted as proof. As this no longer holds, we need to rethink what constitutes ‘plausible evidence’.

A Core Course on Modeling Week 7 -A working model – and then? Convincingness 27 Convincingness: a model is more convincing if it contains fewer implausible assumptions. • Convincingness in the absence of ‘laws’: • conceive a plausible formal model system to which laws do apply (e. g. , replace the world + oceans by a homogenous ball covered by a sheet of water). The measure of correspondence between these two is left out of the discussion; • convincingness relies on the agreement with empirical data. Tides are caused by subtle interplay between gravity, water (viscosity vs. inertia), and the complex geometry of the earth’s sea floors. A (glass box) model to predict tidal motions contains none of these, but a hypothetical ‘sheet of water on a perfect sphere’ instead.

A Core Course on Modeling Week 7 -A working model – and then? Convincingness 28 Convincingness: a model is more convincing if it contains fewer implausible assumptions. • Convincingness in the absence of a formal model system and/or no empirical validation: • look for an empirical model system • physical: water tank, wind tunnel • economical, social: comparative populations, historical survey data • biological: the guinea pig! Scale models are physical replica of the modeled system, usually at a smaller scale. Applications include windtunnel (flow, drag) and hydromechanical investigations, but also scale models as used in architecture and urban studies. These are examples of empirical model systems. BTW: J. Swift’s story seems to relate to a scale model; it is actually a social model.

A Core Course on Modeling Week 7 -A working model – and then? Distinctiveness 29 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. • Most purposes ask for some form of distinctiveness. Examples: • prediction 1: something is correctly predicted to happen at time T 1 rather than T 2; how close can T 1 and T 2 be? • explanation: Q 1 causes P 1 and not P 2; Q 2 causes P 2 and not P 1. How close can Q 1 vs. Q 2, or P 1 vs. P 2 be so that the explanation holds? • optimization: input P 1 yields output Q 1, and input P 2 yields output Q 2, where Q 1 correctly is found to Many modeling purposes boil down to the ability of distinguishing between two alternatives that are as similar as possible. This relates to accuracy and precision, but also to non-numerical purposes.

A Core Course on Modeling Week 7 -A working model – and then? Distinctiveness 30 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. This hinges on the notion of ‘distance’, explained in week 6. The quantitative measure for distinctiveness is usually a similarity measure: the more similar A and A’ whereas the model can successfully distinguish them, the better its distinctiveness.

A Core Course on Modeling Week 7 -A working model – and then? Distinctiveness 31 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. Distinctiveness relates to two common types of errors: • false positive (conclude X where there is no X) • false negative (don’t conclude X where there is X) Successful spreading of false money is an example of false negative: the counterfeit goes unnoticed. Coin operated vending machines have to safeguard against both false positives and false negatives; the consequences of the two types of errors being very different. This is true for most models where distinctiveness plays a role.

A Core Course on Modeling Week 7 -A working model – and then? Surprise 32 The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas. There are Open and closed spaces of outcomes. Examples of closed outcomes: • A model computing the probability of X can only produce a number between 0 and 1. • A model verifying Y can only produce ´true´ or ´false´. A widespread opinion is, that computations cannot ‘output more than you put in’. This underestimates computers’ scope of computation, or overestimates humans’ scope of anticipation.

A Core Course on Modeling Week 7 -A working model – and then? Surprise 33 The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas. Examples of open outcomes: • result of using ontologies • outcome of evolutionary algorithms • outcome of PCA, abstraction, … Common feature: output space has size O(2 n) whereas input space has size O(n) Evolutionary programming, applied to shape design. Given a comprehensible genotype representation, the space of phenotypes is virtually infinity. Xai Xu et. al. , Special Issue of SIGGRAPH, Vol. 31, No. 4, pp. 57: 1 -57: 10, 2012

A Core Course on Modeling Week 7 -A working model – and then? Impact 34 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders (either beneficially or adversely). Two perspectives: • prestige and profit: the more impact the better • risk and responsability: the less impact the better ‘Impact’ is a double-edged sword: some modelers attempt to increase the impact of their models whereas other modellers prefer to play safe and seek modeling with less impact.

A Core Course on Modeling Week 7 -A working model – and then? Impact 35 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. 0 | | 1: magnitude of impact Intended impact can • be capitalized: • If of model positive r 1=revenues in absence 0 1: outcome; impact (e. g. , models for optimization); To assign a quantitative value to a model outcome r 2=revenues with model outcome present; we must assume some market mechanism where • if -1 0 negative impact (e. g. , risk we identify revenues and costs of ownership, both prediction of model c 1=cost of ownership in absence or analysis) in the absence of the model outcomes and in their outcome; presence. c 2=cost of ownership with model outcome present; =((r 2 -r 1)-(c 2 -c 1))/(|r 2 -r 1|+|c 2 -c 1|)

A Core Course on Modeling Week 7 -A working model – and then? Impact The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. Adverted impact can be quantified or capitalized: C=the estimated chance (per time interval) of an incident (=an erroneous model outcome, e. g. a type-I or type-II error) V=the estimated value loss per incident Both C and V can serve to quantify the negative impact of a model. Their product CV has dimension of money / time and can be added to c 2 in the formula for . 36

A Core Course on Modeling Week 7 -A working model – and then? Impact The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. The impact of a model should balance with its reliability. 37

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes Not each of the eight criteria is equally relevant for all purposes. Given the purpose of the model, the modeler should identify the most relevant criteria to seek improvement when necessary. For instance: … (see next sheets) 38

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 39 Not each of the eight criteria is equally relevant for all purposes. prediction (1, 2): convincingness, distinctiveness, impact An almanac is a model to serve the purpose of predictions (both 1 and 2). Its impact is rather large (given the long historical record of almanac-use), but the distinctiveness is small, and its convincingness decreases over time.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 40 Not each of the eight criteria is equally relevant for all purposes. compression: scalability, population, distinctiveness Printing is a form of compression (transferring written text into printed text) with huge population and distinctiveness; by present day’s standards, the scalability is not impressive

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 41 Not each of the eight criteria is equally relevant for all purposes. inspiration: surprise All the world’s religious traditions acknowledge holy scriptures of some sort as primary source of inspiration. Their potential for surprise is sometimes rather limited.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 42 Not each of the eight criteria is equally relevant for all purposes. unification: genericity, convincingness, surprise In alchemy, the pre-scientific forerunner of both chemistry and psychology, unification of conflicting properties was one of the leading purposes. In modern times, scores on all three criteria are low – when seen through modern eyes.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 43 Not each of the eight criteria is equally relevant for all purposes. abstraction: distinctiveness, convincingness, surprise The term ‘abstraction’ in modern art no longer defines a relation between a modeled system and a model; rather, the absence of any modeled system is the defining feature for true abstract art. The potential for surprise, however, sometimes still applies.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 44 Not each of the eight criteria is equally relevant for all purposes. verification: scalability, convincingness, impact Biting a golden coin is the proverbial way to verify its authenticity. Scalability and convincingness are probably low, but if the coin contains chocolate, or if the biter has metal inlays, the impact may be quite considerable.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 45 Not each of the eight criteria is equally relevant for all purposes. exploration: genericity, surprise Few models for exploration will have a larger potential of surprise than the Mars exploration robot.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 46 Not each of the eight criteria is equally relevant for all purposes. decision: convincingness, distinctiveness, impact The Delphi oracle was a decision-making model that derived its convincingness from its deliberately low distinctiveness – thereby being known for its major impact.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 47 Not each of the eight criteria is equally relevant for all purposes. optimization: genericity, scalability, convincingness, impact Of all generic optimization models for continuous systems, hill climbing is perhaps one of the most generic ones. It often suffers from scalability, though – even when implemented by modern, automated devices.

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes Not each of the eight criteria is equally relevant for all purposes. specification: genericity, distinctiveness Sometimes the distinction between a specification model (blueprint) and a realization is somewhat blurred. 48

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes Not each of the eight criteria is equally relevant for all purposes. realization: genericity, distinctiveness, impact Realization … 49

A Core Course on Modeling Week 7 -A working model – and then? Criteria for modeling and purposes 50 Not each of the eight criteria is equally relevant for all purposes. steering & control: distinctiveness The prototypical stick-and-carrot steering, with the stick tied to the donkey’s back is an example of open-loop control without a set-point.

A Core Course on Modeling Week 7 -A working model – and then? Summary 51 • Leading question: to what extent has the initial problem been solved? • Approach: criteria to assess the quality of the modeling process • Taxonomy: Input or output side? Modeled system or stakeholders? Qualitative or quantitative? • Resulting criteria: • Genericity: how many di®erent modeled systems can we handle? • Scalability: how large can the size of the problem be? • Specialization: how much should the intended audience know? • Population: how large can the intended audience be? • Convincingness: how plausible are the assumptions? • Distinctiveness: e. g. , how accurate, how certain, how decisive can the model outcome be? • Surprise: to what extent can the model outcome give new insight? • Impact: how big can the consequences of the model outcome be?