ARIVA PI Conference November 2005 ARIVA SAGE UNCLASSIFIED FOR

ARIVA PI Conference, November 2005 ARIVA SAGE UNCLASSIFIED//FOR OFFICIAL USE ONLY

Team Roles 4 Manage project/sustain vision 4 Intel and human factors expertise 4 Cognitive Science research 4 Build Viz. Arch, sim. Borgs 4 Build Slate 4 Cognitive Systems Engineering – Nutech ABEM 4 HII Research – Network data – Image management 4 Novel Gaming Interfaces – CMU/Dynamix ARGUS SAGE 2

(Not based on ppt source. ) The Slate System: Four New Developments Selmer Bringsjord Andrew Shilliday, Joshua Taylor Konstantine Arkoudas Sangeet Khemlani, Eric Pratt, Gabe Mulley, Bettina Schimanski Rensselaer AI & Reasoning (RAIR) Laboratory Department of Cognitive Science Department of Computer Science Rensselaer Polytechnic Institute (RPI) Troy NY 12180 US 11. 30. 05 SAGE

Original Concept

Current Release: v 2. 0

Some Innovative Features Reflects new theories of hypothesis generation E. g. , MMOI-based hypo gen E. g. , scenario generaion via MDF Automatic (v 1) report generation Includes a proved-to-be sound system, S, for assembling proofs, arguments, composite arguments, meta-arguments — and these arguments can be automatically assessed Facilitates not just deduction, but abduction, induction, and model-based reasoning Seamless integration with the world’s best automated and interactive machine reasoning systems and model finders (Vampire, Paradox, Athena, etc. ) IKL-compliant (empirically confirmed)

http: //www. cogsci. rpi. edu/research/rair SAGE

http: //www. cogsci. rpi. edu/research/rair/projects. php SAGE

SAGE

All of CS #4 Translated into Common Logic So, we’re making progress, and will have this “mega” round trip working @ Nov/Dec 05 PI meeting SAGE

The Visualization “Hookup” Challenge We are swimming in an expanding sea of systems for exploring and visualizing data in the IA domain. We need interoperability here as well! SAGE

SAGE

Challenge Reduced to Software Engineering: • SAGE The Vivid family, mathematically, up to the challenge of visual data integration and interoperability.

The Uncertainty Challenge can be met; we have the formalism; simple example SAGE

SAGE

79% 82% 77% SAGE

Argument against an argument SAGE

Novel Methods of Human-Computer Interaction Network Data Visualization Catherine Plaisant, Benjamin B. Bederson Bongshin Lee, Hyunmo Kang Human-Computer Interaction Lab University of Maryland SAGE

Two applications for Network Data Visualization ¢ Tree. Plus l Node-link graph visualization Tree-centered solution l Improve incremental navigation l ¢ Net. Lens l l l SAGE Interactive coordinated overviews Dynamic filtering Bipartite graphs

Challenge of Network Data Visualization SAGE

Tree. Plus Project Goals ¢ Develop a good tree-centered solution for iterative inspection of large networks + ty + li abi d ea +r + and in cooperation with the team of Wayne Gray Rensselaer ¢ ¢ SAGE Determine for which tasks and data sets different solutions works better Better understand how people use visualization tools to explore graphs

Tree. Plus ¢ ¢ Extraction of spanning tree “Plant a seed and watch it grow” l l l ¢ Interaction techniques l l l SAGE Dynamic root selection Incremental Exploration + Integrated Search Pan/Zoom + Animation Highlight and preview of adjacent nodes Hints of graph structure Animated transitions

Tree. Plus Written in C# with Piccolo. NET SAGE Bars give a preview of how fruitful it would be to go down a path When cursor hovers over a node, a preview of all nodes connected to that node appear on the right

Controlled experiment Tree. Plus Graph. Plus SAGE

Promising Results • 28 subjects • Compared Tree. Plus to Graph. Plus • Controlled interface “density” SAGE Study in collaboration with Wayne Gray - RPI

Two applications for Network Data Visualization ¢ Tree. Plus l Node-link graph visualization Tree-centered solution l Improve incremental navigation l ¢ Net. Lens l l SAGE Interactive coordinated overviews Dynamic filtering Interaction history Bipartite graphs

Net. Lens – Papers (ACM DL) & Authors Papers on the left Authors on the right Overviews provided for all attributes (here for number of papers per year) Filtered to show only papers related to visualization, and the people who wrote those papers are shown on the right side, aggregated by institution type. SAGE

Net. Lens –Email (Enron) & People Emails on the left People on the right Overviews provided for all attributes (here for emotional tone on emails side) Filtered to show only emails related to CA energy crisis; and the people who sent them are shown on the right side. SAGE

SAGE Complex Visualizations of Massive Data: The Impact of Design on Mind Rensselaer Cognitive Science ARDA ARIVA Workshop Orlando, Florida 30 November 2005

Intelligently Designed Interfaces for Intelligence Analysts SAGE • NIMD will bring a superhighway of information technology to the Intelligence Analyst • But, by themselves the NIMD technologies will not bridge the crucial human-computer-information gap • All of these technologies REQUIRE a user interface for the IA Rensselaer Cognitive Science

Interface = Visualization + Interaction Is this a good interface? Rensselaer Cognitive Science SAGE

Interface = Visualization + Interaction Is this? Rensselaer Cognitive Science SAGE

Interface = Visualization + Interaction How about this? Rensselaer Cognitive Science SAGE

Interface = Visualization + Interaction Or this? Rensselaer Cognitive Science SAGE

Understanding the Impact of Design on Mind SAGE • A good interface is one that optimizes performance on a task that you care about • Understanding what makes an interface good or bad requires understanding the constraints and affordances in the interactions between Mind, Design, and Tasks Design Mind Rensselaer Cognitive Science Performanc e Tasks

Our Goals: Forecasting Dynamic Changes in Cognitive Workload SAGE • Create a new generation of tools inspired by cognitive science theory that will enable us to predict the changing demands on human cognition, perception, and action imposed by an interface during task performance • Apply these tools to aid the evaluation and design of the next generation of visualization and interaction techniques for software intended for use by Intelligence Analysts Cognitive Metric Profile -- showing dynamic changes in task demands on human cognitive, perceptual, and action resources Rensselaer Cognitive Science

Our Approach • SAGE 1 st – Play well with others (VIA) – Develop tools and techniques so that our data collection and modeling systems can interact with tools built by others • 2 nd – Simulated Human Visualizers (sim. Borgs) – Develop generation of cognitive theory based models of human visualization and interaction • 3 rd – Cognitive Metrics Profiling – Toolkit for accessing and predicting dynamic changes in demands that an interface makes on human cognition, perception, and action as the human user performs a typical task • Human data – eye fixations, scanpaths, mouseclicks, performance time • sim. Borg data – predictions of changes in internal use of memory, attention, and perception Rensselaer Cognitive Science

Cog. Works VIA: SAGE Visualization-Interaction Architecture Current research focuses on three nested technologies Rensselaer Cognitive Science

SAGE VIA • VIA release 0. 0 used at UMd in August/September to evaluate an innovative design for visual display of abstract data • VIA release 0. 9 used to interact with a random application downloaded from the internet – required addition of two lines of code Rensselaer Cognitive Science

SAGE Demos • VIA in action -- demo of ease of instrumenting C# applications to run with VIA • Playback of logfile created during UMd session • Logfiles analyzed for UMd dissertation • Eye data analyzed for UMd dissertation • sim. Borg written that uses VIA to perform tasks on UMd software • Information scent implemented in sim. Borgs in real-time use Rensselaer Cognitive Science

Bridging the Human-Computer. Information Gap by. . . SAGE • Engineer interfaces for the next generation tools developed for the IA by creating a new generation of human performance models focused on cognitive issues in usability • To tailor the designed environment to best fit the ways in which the IA –thinks –perceives –and acts Rensselaer Cognitive Science

SAGE Questions? Rensselaer Cognitive Science