Скачать презентацию Visual Communication in the AQUA environment Technical aspects
3676fb8f47d2e3102f8841e05a8b71f2.ppt
- Количество слайдов: 65
Visual Communication in the AQUA environment Technical aspects and solutions Stefan Seipel Lars W. Pettersson Björn Andersson Uppsala University
Informationsteknologi We are situated in a co-located multiple viewing environment Institutionen för informationsteknologi | www. it. uu. se
![Informationsteknologi Co-located multiple viewing environments n Example: i. Room [Fox 2000] Based on conventional](https://present5.com/presentation/3676fb8f47d2e3102f8841e05a8b71f2/image-3.jpg "Informationsteknologi Co-located multiple viewing environments n Example: i. Room [Fox 2000] Based on conventional")
Informationsteknologi Co-located multiple viewing environments n Example: i. Room [Fox 2000] Based on conventional 2 D GUI http: //graphics. stanford. edu/projects/iwork/room/images/using-room-feb-00/ Institutionen för informationsteknologi | www. it. uu. se
![Informationsteknologi Co-located multiple viewing environments n The AQUARIUM [Sundin 2000] Beyond the 2 D](https://present5.com/presentation/3676fb8f47d2e3102f8841e05a8b71f2/image-4.jpg "Informationsteknologi Co-located multiple viewing environments n The AQUARIUM [Sundin 2000] Beyond the 2 D")
Informationsteknologi Co-located multiple viewing environments n The AQUARIUM [Sundin 2000] Beyond the 2 D graphical user interface Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Why is 3 D so interesting ? Example of what could be done with 3 D representations Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Do existing tools fix the job? n Net. VR ® ® Latency issues not a predominant issue ® n Full transparency and SG/DB replication Concurrency of media streams more important Cluster/Tiled Rendering ® Optimization ® Single data multiple processes ® Single data multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi AQUARIUM – A 3 D Interactive Environment n Specific requirements ® Tiled displays ® Multiple view ports ® Image superimposition ® Multiple views on a shared 3 D scene ® Individual views on private data ® Co-location of multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi AQUARIUM – A 3 D Interactive Environment n Specific requirements ® Tiled displays ® Multiple view ports ® Image superimposition ® Multiple views on a shared 3 D scene ® Individual views on private data ® Co-location of multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi AQUARIUM – A 3 D Interactive Environment n Specific requirements ® Tiled displays ® Multiple view ports ® Image superimposition ® Multiple views on a shared 3 D scene ® Individual views on private data ® Co-location of multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi AQUARIUM – A 3 D Interactive Environment n Specific requirements ® Tiled displays ® Multiple view ports ® Image superimposition ® Multiple views on a shared 3 D scene ® Individual views on private data ® Co-location of multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi AQUARIUM – A 3 D Interactive Environment n Specific requirements ® Tiled displays ® Multiple view ports ® Image superimposition ® Multiple views on a shared 3 D scene ® Individual views on private data ® Co-location of multiple displays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi How to design applications… …that support “ 20 eyes upon 15 visuals” ? ® Multiple processes ® Allow for flexible configuration ® Management of visual views ® Provide multiple interaction contexts ® Easy to use programming concepts Institutionen för informationsteknologi | www. it. uu. se
![Informationsteknologi Controlling by sharing data n Shared State Repository [Lindkvist 2001] • Client-Server architecture](https://present5.com/presentation/3676fb8f47d2e3102f8841e05a8b71f2/image-13.jpg "Informationsteknologi Controlling by sharing data n Shared State Repository [Lindkvist 2001] • Client-Server architecture")
Informationsteknologi Controlling by sharing data n Shared State Repository [Lindkvist 2001] • Client-Server architecture • Lean and easy to use API Client 1 Client 2 sr. API sr. Alloc sr. Free sr. Get sr. Update • Based on TCP, UDP or MC . . . Virtual Shared Memory • Small data subscription packages TCP/UDP • No concurrence control sr. API Client n Institutionen för informationsteknologi | www. it. uu. se sr. API Server
Data Pools Informationsteknologi n Pooling context relevant data projector pool Application VR-Tool sr. API Application Low Level 3 D (Open. GL) pipe pool Rendering HW sr. API VR-Tool Low Level 3 D (Open. GL) Rendering HW Display AQUARIUM application node pool Institutionen för informationsteknologi | www. it. uu. se
Data Pools Informationsteknologi n Pooling context relevant data projector pool how? Viewing frustum Head position Real world metrics pipe pool where? Buffer specific Pixel metrics Reference to a projector what? node pool Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi An Example Scenario Retroscope 1 Retroscope 2 Vsionarium 1 Vsionarium 2 View Manager Tracker 1 Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi What about performance? n Latency and frame incoherence will introduce visual artifacts! ® Little research done ® Edge discontinuity ® Hyper- or hypo-parallax ® Vertical parallax ® Peripheral viewing artifacts Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi General test set-up: pool ® 2 Rendering processes ® 1 Animation process ® Simple scene ® Register output Server 1 ® Analyze differences ® Client 1 Client 2 Render Observation: # frames out-of-sync. Capture& Analysis Institutionen för informationsteknologi | www. it. uu. se
stefan: Test A : Local host Informationsteknologi Host 1 200 frames AVI Sequence Client 1 800 x 300 fr~80 Hz fan Client 2 800 x 300 fr~80 Hz frame grabber 800 x 600 20 Hz server PIII/2. 2 GHz, Ge. Force 4 4600 1 animated node va = 90°/sec. const. Animation rate: 10 Hz, 20 Hz, 30 Hz, 40 Hz, 50 Hz Institutionen för informationsteknologi | www. it. uu. se Differential Analysis
stefan: Test B : Local area network Informationsteknologi Host 2 Host 1 200 frames AVI Sequence Client 1 800 x 300 fr~80 Hz fan LAN 10/100 MBit Client 2 800 x 300 fr~80 Hz frame grabber 800 x 600 20 Hz server PIII/500 MHz, E&S 1 animated node va = 90°/sec. const. Animation rate: 10 Hz, 20 Hz, 30 Hz, 40 Hz, 50 Hz Institutionen för informationsteknologi | www. it. uu. se PIII/2. 2 GHz, Ge. Force 4 4600 Differential Analysis
stefan: Results for Test A and Test B Animation rate Local host Frame number #differing pixels Animation rate Frame number Institutionen för informationsteknologi | www. it. uu. se LAN Informationsteknologi #differing pixels
stefan: Informationsteknologi Results for Test A and Test B Institutionen för informationsteknologi | www. it. uu. se
stefan: Test C and D : Traffic Informationsteknologi ® LAN configuration as in Test B ® Increasing the number of shared states Host 2 Host 1 200 frames AVI Sequence Client 1 800 x 300 fr~80 Hz fan LAN 10/100 MBit Client 2 800 x 300 frame grabber 800 x 600 fr~80 Hz 20 Hz server PIII/500 MHz, E&S PIII/2. 2 GHz, Ge. Force 4 4600 va = 90°/sec. const. Test C: (10 Hz; 1, 10, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200) Test D: (20 Hz; 1, 10, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200) Institutionen för informationsteknologi | www. it. uu. se Differential Analysis
stefan: Informationsteknologi Results for Test C and Test D Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Conclusion n Framework allows for building modular complex visualization environments n Flexible to use and combine with existing tools n No need for fancy protocols to maintain adequate frame-sync. Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The fusion 2 D and 3 D Interfaces n The AQUARIUM is a 3 D environment n How can legacy code be re-used? n Can 2 D applications be instantiated within a 3 D environment? Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Virtual Network Computing n System for sharing frame-buffers/applications n Developed by AT&T Laboratories Cambridge, 1999 n Open Source n Remote Frame Buffer Protocol well documented n Servers and clients readily available for Microsoft Windows, Unix, Linux och Mac. Os n Bold servers, thin clients Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi 2 D Virtual Network Computing Client Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi A 3 D VNC Client VNC Server 3 D VNC Client x • Decode RFB protocol • Administrate local texture buffers y x • Handle 3 D input and map to 2 D server coordinates Institutionen för informationsteknologi | www. it. uu. se y y’ x’
Informationsteknologi 3 D VNC Application n Example in the AQUARIUM Contextual browsing of auxiliary information on the WEB Institutionen för informationsteknologi | www. it. uu. se
3 D VNC Performance Assessment Informationsteknologi ® Benchmarking § Frame rate § Delay ® Common Interaction § Text editing § Mouse movement § Web browsing § Streaming videos Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Modular Application Development n The VASE development framework n n n Background The framework & component design Communication between components Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Background n n Until now – monolithic application Different but similar applications Many components reappear in several of the applications ® Makes modular development natural ® n Framework that handles this is needed (VASE) ® n Easy script (XML) The modular approach enables a parallel development process Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Middlewares used n VRT – Virtual Reality Toolkit ® For graphics handling ® Built on top of Open. GL ® Implements a scenegraph n STREEP ® For network communication ® Shared state repository ® Supports both TCP and UDP Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Design of the framework and the basic structure of the plugins n n n The framework defines the basic structure for the components (plugins) Implements a main module Each plugin is written as a DLL and implements a class, which structure is inherited from a base class Plugins are dynamically linked during execution Easy to change or create new plugins when new functionality is to be added Institutionen för informationsteknologi | www. it. uu. se Main module Plugin instance Plugin DLL Class implementation
The main module Informationsteknologi n The main module (vase. exe) handles: ® Parsing of a configuration file (*. vas) ® Loading and creation of plugin instances ® Distributes user interaction events ® Handles message passing ® Handles shared states ”myfile. vas”
Informationsteknologi Communication n ® ® n Main module Local communication Function calls to the main module Method calls Plugin 1 Plugin 2 Network communication ® ® ® Uses STREEP Through the main module Message passing Shared state repository § Receiving plugin concatinated to the message ® Shared states of who is interested in what Institutionen för informationsteknologi | www. it. uu. se Client 1 Client 2 Main module Plugin 1 § Subscription lists keeps track Plugin 2
Example of Aqua components (plugins) n Aqua. Controller Informationsteknologi ® ® n Aqua. Echelon ® ® ® n 3 D-buttons Sends messages to other plugins Draws an echelon graph Listens to parsed Stratmas data Implements an internal XML structure for units Aqua. Environment ® Handles the drawing of the § map § grid n Aqua. Parser ® ® ® n Aqua. Viewer ® ® n Inuput: Stratmas generated data file Output: Parsed binary file Distributes the parsed data to other plugins Handles projectors and pipes Tracking devices Aqua. Whiteboard ® ® Overlay for the map Stores strokes in RT 90 coordinates Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi How do we connect the AQUA environment to the rest of the world? n The STRATMAS-Link ® Modifications to STRATMAS ® Socket connectivity ® File connectivity ® XML encoding Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The STRATMAS Link - Modifications to STRATMAS TCP socket communication add in a thread n conf. dat (link to) n ID number for each Unit n Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The STRATMAS Link - Socket connectivity Connection on an hostname and port specified in conf. dat n XML data is sent in fixed sized blocks over the socket connection n Communication is one way n Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The STRATMAS Link - File connectivity n Generation of data. txt on the Mac running STRATMAS ® Xml encoded (see next slide) Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The STRATMAS Link - XML for Units int ID; long superior_ID; // -1 if no superior int rank; // 0 -6 - enum Rank. Type float latitude; float longitude; int purpose; // 0 -13 - enum Purpose. Type int vehicle_type; // 0 -6 - enum Veh. Type, -1 if no vehicle int unit_type; // 0 -8 - enum Unit. Type int condition; // 0 -100 int health; // 0 -2 - enum Health. Type long my_threat; // Larger value - larger threat float sens_range; // Sensor range in km, -1 if no sensors float veh_sens_range; // Vehicle sensor range in km, -1 if no vehicle float weapon_range; // Shoot radius in km, -1 if no vehicle long shots_fired; // Number of shots fired long fi_injured; // Number of enemies injured long fi_killed; // Number of enemies killed float vx; // vehicle velocity vector in degrees *lng* per // iteration, -1 if no vehicle float vy; // vehicle velocity vector in degrees *lat* per // iteration, -1 if no vehicle float speed_kph; // vehicle speed in kilometers per hour, -1 if no vehicle float goal_x; // goal position RT 90 x float goal_y; // goal position RT 90 y int flag; // Flag for red, green or blue unit. Values: 0, 2 or 1. Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The STRATMAS Link - XML for Grid int rows; // Number of rows in the grid. int cols; // Number of columns in the grid. int nc; // Number of cells holding relevant data. int nlayers; // Number of layers. char lname[MAX_GRID_LAYERS][64]; // Layer names. int *index; // Array of the cell-indices corresponding to the data array indices float *data; // The values for each layer are stored sequencially after eachother. for red, green or blue unit. Values: 0, 2 or 1. Institutionen för informationsteknologi | www. it. uu. se
Map View Informationsteknologi 15º 48’ 29. 8’’Ö n n n Topographic context Based on RT 90 coordinate reference Mapped upon internal grid 0/1600000 7700000/1600000 50 kmx 50 km Scale 1: 100. 000 0/0 700000/0 0 000 190 0 000 Institutionen för informationsteknologi | www. it. uu. se 150 120 28 x 64 cells á 25 x 25 km (1: 25000) 6100000
2 textures 1024 x 1024 texels 00 00 0 [m et er s (R T 9 0) ] 1024 x 1024 texels 16 Informationsteknologi Map View contd. 700000 [met ers (RT 90)] Map cell 25 x 25 km 32 x 32 texels 1 texel ~ 780 meters Modeling unit in the virtual environment is 1 cm (1: 1000000) -> Map (70 cmx 160 cm) fits visionarium Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Map View summary n Map granularity can be adapded At present topography is schematic n High resolution map data can be loaded dynamically per map cell (e. g. 1: 25000 maps per cell) depending on zoom level n Advantage of RT 90 reference frame internal representation of 3 D data always Euklidean orthogonal system eases texture mapping use lat/long conversion routines by metria Institutionen för informationsteknologi | www. it. uu. se
Cell Grid Layer Co-located with the RT 90 map n Mapps geographically related information upon map n Grid cell contain aggregated data for military units n Grid cell contain substrate information n Grid cell can visualize itself n Grid size and resolution can be chosen arbitrarily n Informationsteknologi n Current limits: Max cell size: 200 x 200 km 2 => 32 cells/RT 90 map Min cell size: 3. 125 x 3. 125 km 2 => 114688 cells/RT 90 map Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Cell Properties n Cell geometry (polygon) n Cell texture n Cell related data num_red : 0, 1, 2, 3, 4, 5 => number of red troups in 5 levels 0 = no red troups num_blue: 0, 1, 2, 3, 4, 5 => number of blue troups in 5 levels 0 = no red troups num_green: 0, 1, 2, 3, 4, 5 => number of green troups in 5 levels 0 = no green troups age: 0, 1, 2 => 0 = very up-to-date information; 1 = older; 2=very old sensor: 0, 1 => 0 = there is no sensor coverage; 1 = sensor coverage disease: 0, 1, 2 => 0 = no disease; 1 = disease 1; 2 = disease 2 strenght_red: 0, 1, 2, 3 => 0 = perfect strenght, 1, 2, 3 dicreasing moral/strenght_blue: 0, 1, 2, 3 => 0 = perfect strenght, 1, 2, 3 dicreasing moral/strenght_green: 0, 1, 2, 3 => 0 = perfect strenght, 1, 2, 3 dicreasing moral/strenght combat: => 0 = no fight; 1 = little fight; 2 = heavy fight 0, 1, 2 Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Mapping Data from external Sources (e. g. Stratmas) n Continuous data (unit data) . . . n Discrete data (raster data) Stratmas grid rows="79” cols="36" Cell. Grid: : Set. Stratmas. Grid. Data Cell. Grid: : Set. Stratmas. Unit. Data Institutionen för informationsteknologi | www. it. uu. se
Visualizing Cell Data Informationsteknologi n Visual pattern library n Patterns associated with properties (32 x 32 texels) n Patterns can be combined n Examples: Cell property Number of troups (ordinal) No sensor coverage Diseases Troup strength Combat Age Institutionen för informationsteknologi | www. it. uu. se Visual pattern(s)
Visualizing Cell Data contd. Informationsteknologi n Rules, based on actual cell data - Chosing the correct base-pattern - Applying modifiers - Texture combiner Examples of precompiled patterns (512 x 512 texels) Institutionen för informationsteknologi | www. it. uu. se
Visualizing Cell Data contd. Informationsteknologi n 3 forces (red, green, and blue) n 3 disease conditions Þ 3 x 8 megabye/texture = 75 MB texture memory Þ This is equivalent to 2304 unique patterns Þ One more attribute will double the memory requirements Þ This is quite unlikely there are only 1800 grid cells in the default resolution not all cells contain interesting data Cell pattern compilation at run-time for individual cells Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Cell Grid Visualization Example Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Visualization of continuously positioned data – Echelon graphs Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Aqua. Echelon in detail n Function Draws tactical information in 3 D ® Gives both a spatial and hierarchical understanding at a glance ® Interactable ® n XML ® Bottom-up vs. Top-down Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi The XML formats n Internal format ® Top-down ® Shows a static situation Example of the internal XML format
Informationsteknologi The XML formats – cont. n Stratmas generated ® Bottom-up ® Dynamic, can play whole scenarios Example of a Stratmas generated file
Informationsteknologi Symbol construction n Symbol construction ® Previously manually generated bitmaps . . . ® . . . Now 3 D-models of the basic shapes Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Construction of the symbols n n Make 3 D-models of the basic shapes Merge several of these into a complete unit Easy to alter the apperance of the symbols by changing the basic models 3 D instead of 2 D Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Overlays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Overlays Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Overlays n Storage data structure § A document. . . • Has one or several pages. . . • A page has none or several strokes n Rendering in a bitmap texture ® n Strokes are rendered in their correct RT 90 context File format ® Text/Ascii based format. Institutionen för informationsteknologi | www. it. uu. se
Informationsteknologi Interaction in with the AQUA environment n Demo. . . Institutionen för informationsteknologi | www. it. uu. se