Testbed for Mobile Augmented Battlefield Visualization September 2003

Testbed for Mobile Augmented Battlefield Visualization September, 2003 William Ribarsky and Nickolas Faust GVU Center and GIS Center Georgia Institute of Technology Virtual Worlds Lab

Matrix of Proposed Activities and Results Virtual Worlds Lab

Mobile Situational Visualization An extension of situation awareness that exploits and integrates interactive visualization, mobile computing, wireless networking, and multiple sensors: • Mobile users with GPS, orientation sensing, cameras, wireless • User carries own 3 D database • Servers that store and disseminate information from/to multiple clients (location, object/event, weather/NBC servers) • Location server to manage communications between users and areas of interest for both servers and users • Ability to see weather, chem/bio clouds, and positions of other users • Accurate overviews of terrain with accurately placed 3 D buildings • Ability to mark, annotate, and share positions, directions, speed, and uncertainties of moving vehicles or people GPS and • Ability to access and playback histories of movement • Placement of multiresolution models from MURI team members into orientation tracker environment New lightweight wearable system Results of realtime collection of GPS path at night (left); screen shot with annotated path in red (right). Virtual Worlds Lab

Mobile Situational Visualization Spread of dynamic Sarin gas cloud with positions of first responders Weather/Atmospheric Server Accurate Shared Locations Annotation Server Annotated views with updated user location and orientation Virtual Worlds Lab

Mobile Situational Visualization System Buttons Pen Tool Drawing Area collaborators Collaboration Example Mobile Team Shared observations of vehicle location, direction, speed Virtual Worlds Lab

Collaborative Environment • Everybody has a location in space and Location Server time in the Virtual World Traffic Server • Geographic server Annotation lookup approach Server – Users – Location Servers – Data Servers User Geo. Data Server User Weather Server User Virtual Worlds Lab

Mobile Situational Visualization Video Virtual Worlds Lab

What is Novel and Compelling About Mobile Situational Visualization? Mobile battlefield visualization was an original proposed (and accepted) task. That’s pretty compelling! But, beyond that Instant placement of environmental activity information within the geospatial environment combined with fast sharing and use are novel and compelling. -Fast, accurate, and specific annotation of activity information (both user-controlled and automated logging) -Immediate updates of databases with this information -Server structure for sharing this with collaborators or commanders in the area of interest -Use in computations and simulations (some launched automatically) Virtual Worlds Lab

Matrix of Proposed Activities and Results Virtual Worlds Lab

Integrated, Comprehensive Modeling High Mid Berkeley Hundreds of automatically modeled buildings USC Tens to Hundreds of semiautomatically modeled buildings Georgia Tech Thousands of semi-automatically of Hundreds to tens of thousands buildings and trees modeled buildings Low Geo-accuracy • To build comprehensive models, we need a range of modeling techniques. • We also should combine techniques for richer and more complete models. Low Mid High Model Detail Virtual Worlds Lab

Integrated, Comprehensive Modeling High Mid Integrated, comprehensive models with combined techniques Low Geo-accuracy • To build comprehensive models, we need a range of modeling techniques. • We also should combine techniques for richer and more complete models. Low Mid Detail High Virtual Worlds Lab

New Results on Modeling Large Collections • Generic models extruded from accurate footprints with accurate locations. (11, 000 automatically generated from insurance GIS databases). (individual 3 D buildings have brown roofs) -Complete models with roofs -Generic façade textures -Databases available for automatically generating whole city (hundreds of thousands) • Automatic generation of accurately located tree models (thousands) from high-res imagery. • Creation of hundreds of specific buildings using commercial or selfdeveloped (semi-automatic) software. 3 D CAD modeled objects on high resolution terrain Virtual Worlds Lab

Automatic Identification and Placement of Trees, Shrubs, and Foliage This can be used with Ulrich Neuman’s or Avideh Zakhor’s results to automatically identify, remove, and model foliage. Virtual Worlds Lab

Application to Tree Modeling Automated identification and modeling of trees Accurate placement of 3 D modeled trees Virtual Worlds Lab

Matrix of Proposed Activities and Results Virtual Worlds Lab

Organizing Large Collections of 3 D Models for Interactive Display • Merging of different types and formats • Automated replacement of structures for overlapping areas Common format and organization for different types Q Q Q Linked Global Quadtrees Q Q Q Q Virtual Worlds Lab

Paging, Culling, and Fast Rendering Block Q Q Q Out-of core Storage Q Linked global quadtree Block k Bloc l Q dcel ua Virtual Worlds Lab

Integrated, Interactive Visualization of Large Collections of Models Video Virtual Worlds Lab

Matrix of Proposed Activities and Results Virtual Worlds Lab

Handling Complicated Models View-Dependent LOD for large collections of complicated models Q Q Q Q Linked Global Quadtrees Q Q N Levels Q Bounding box Results of view-dependent simplification. The blue box is the viewing window; fully textured models with and without meshes displayed are shown on the left and right, respectively. (Top) Full resolution mesh and textures within the window. (Bottom) Significantly reduced resolution mesh and textures within the window without reduction in visual quality. Viewpoint Selected LOD Virtual Worlds Lab

Quadric Error Approach to Simplification • Initial development Garland Heckbert, SIGGRAPH, 1997 • Quadric approach yields “optimal” simplification by permitting generalized contractions between vertices and keeping track of the deviation from the original mesh v 1 v 2 contraction v 1 Use quadric matrix to find a vertex with error within ε; Δ is the surface at error value ε. Non-topological simplification v 2 general contraction Virtual Worlds Lab

Limitations on Basic Quadric Approach [ ] • No concept of view-dependence and continuous LOD • No structure for large collections of objects • Geometry error metric; no appearance-preserving metric (e. g. , for textures, shading, lighting). A combined metric is best. Application of appearance-preserving metric to a textured object (Cohen et al. , SIGGRAPH 98) full resolution w/o appearance with appearance metric Virtual Worlds Lab

View-Dependent Continuous LOD Tree Q Q Linked global quadtree Block Façade 1 … LOD Hierarchy Façade N … … … The vertex front is circled. Green nodes are active-interior, blue nodes are active-boundary, and orange nodes are inactive. Here, vertex V 7 is split and vertices V 10 and V 11 are merged. … Object 1 … … Object M … … … The pink, purple, and dark gray triangles are subfaces of V 7, V 5, and V 4, respectively. (a) Full mesh. (b) Tree on left. (c) Tree on right. Virtual Worlds Lab

View-Dependent Appearance-Preserving Simplification original surface M 0 current surface Mi-1 (A) (E) (C) possible surface Mi deviation vectors P A PC Va PO Vb Va Va Vb Vc Vc Collapse Distance Deviation P P B Vb C Vc Quadric Error Deviation Two-Way Incremental Texture Deviation (F) (D) (B) P Va Vb Vc Two-Way Incremental Distance Deviation Va C Vb Vc One-Way Incremental Texture Deviation V P V a O b V c Total Texture Deviation Virtual Worlds Lab

View-Dependent Appearance-Preserving Simplification Video Virtual Worlds Lab

Matrix of Proposed Activities and Results Virtual Worlds Lab

Implementing and Using the Testbed • Merging of tens of thousands (and more) of models from multiple sources. • Efficient organization and culling of massive collections of 3 D objects. • Integration of view-dependent methods for accurate and efficient display of complex models. • Deployment and use of mobile situational visualization capability. Virtual Worlds Lab

Technology Transfer • The VGIS visualization system with capabilities developed here (including mobile visualization) was a key part of the Georgia Tech Homeland Defense Workshop and will be part of the GT Homeland Defense Initiative with support at the State and National levels. • The system is being used as part of the Sarnoff Raptor system, which is deployed to the Army and other military entities. In addition our visualization system is being used as part of the Raptor system at Scott Air Force Base. • We are in discussion with the Department of the Interior on use of our mobile situational visualization capability to develop Anytime-Anywhere information system resource accessibility for countering asymmetric threats. Virtual Worlds Lab

Plans for Next Year • Full deployment of mobile situational visualization capability with sharing of the system and the results with team members. • Further development of automated model building from multisource data. This will be a collaborative effort with other team members. We will move towards a robust system with ability to merge and increment model sets and update models (adding improvements to make generic models more detailed and specific as data are available). • Development of fully scalable 3 D object organization and interactive visualization capability extending to hundreds of thousands of accurately located buildings and trees (or more). • Full integration of view-dependent capability for complex models. Virtual Worlds Lab