The Sprint Essistant Interactive Services Through a Natural

The Sprint Essistant: Interactive Services Through a Natural UI James Schumacher [email protected] com January 2002 Sprint Advanced Technology Laboratories Burlingame, California Advanced Technology Laboratories

The Sprint Essistant A federated system of loosely coupled services made cohesive through an innovative multi-modal interface. Leveraging such components as animated agents and Text to Speech engines, the Essistant enables the user to make a personalized connection to the services. Advanced Technology Laboratories

What is the Essistant? § § § Not the desktop Ubiquitous computing Smart House / Smart Space Pervasive Computing Speech & Transparent User Interfaces Advanced Technology Laboratories

LAN as Client § § Federated Services Backend Services feed the LAN as intermediary LAN Services function autonomously Advanced Technology Laboratories

The LAN & Connectivity § § § § “Broadband is five years away. ” Long live narrowband (DSL & Cable) Down rates average ~400 kb/s Leveraging “Always On” Fat Content Over Skinny Pipes Keep Data Flowing. Maximize the use of narrowband. Advanced Technology Laboratories

Fat Content over Skinny Pipes § Assumptions: • • • Typical broadband connectivity is 0. 3 -1. 5 Mbps. BW bottlenecked pipes not necessarily at the last mile. Multimedia content (mostly video) is emerging. High percentage of multimedia content is stored. Most of multimedia content is currently encoded/delivered at a constant bitrate. • Broadcast quality content > 1. 5 Mbps § Can this content be sent over a slow pipe? Advanced Technology Laboratories

Fat Content over Skinny Pipes § LAN utilization of fat content: • • Active Prefetched media Video-on-demand Purchase of multi-media (Music, DVDs, etc) Rich media web sites pre-cached. Advanced Technology Laboratories

Services of the Essistant § § § Fat Content download manager Various content agents (news, traffic, etc) Room control (lights, thermostat, security) Entertainment system (set-top box, VCR) Chatter-bots (e. g ALICE) Robots Advanced Technology Laboratories

Essistant User Interface § § § Speech Recognition Animated Characters Text-to-Speech response Face recognition / Motion Detectors Other sensors (e. g thermostat, microphone) Advanced Technology Laboratories

Essistant Sample Advanced Technology Laboratories

Enabling Technologies § § § § Connectivity & Bandwidth Processing Power Speech Recognition Text-To-Speech Real-Time Animation Face Recognition Robots Artificial Intelligence Advanced Technology Laboratories

Enabling Technology: Bandwidth § § § “Broadband is five years away” What to do with 100 mb? Not just faster web pages Entertainment, Cable TV, VOD. Telephony and Vo. IP Advanced Technology Laboratories

Enabling Tech: Mobile Bandwidth § Network is always with you § Mobile, G 3 connectivity § Connectivity to home LAN from anywhere Advanced Technology Laboratories

Generic Architecture Agent Preference database Intelligent Agent Network Provider User-Agent Net-Agent Databases AI User-Network Prefs Home User-Agent User-Network Mobile User Advanced Technology Laboratories

Enabling Tech: Artificial Intelligence § § § Speech Recognition Natural Language Processing Chatter-Bots (ALICE) Profiling, and auto preferences Semantic Web & Ontologies Inference Engines (e. g Cycorp) Advanced Technology Laboratories

Use of AI to predict user requirements § Wide availability of AI engines. § AI capabilities have increased, cost has decreased significantly. § Ideally suited to alter the form of interfaces to the network based on user requirements and user input. § Advanced form of user profiling and personalization. § AI predictions may alter the network usage and its patterns considerably. Advanced Technology Laboratories

Current Trends: Personalized Interactive Services § Goal: • Development of a distributed network and client-aware interactive system which integrates and delivers personalized content and services to users in a seamless manner through agent technology and network-based user information. § Adds value by: • • Personalizing content. Personalizing interaction. Seamlessly introducing new services. Allowing accessibility by different devices/networks (PCS, broadband, etc. ). • Providing for targeted advertisement. Advanced Technology Laboratories

Enabling Technology: Animation & 3 D § Three dimensional displays § Three dimensional cameras § Dynamic real-time animation Advanced Technology Laboratories

Live 3 D capture technologies § Z-Cam Advanced Technology Laboratories

3 D Display Technology § Stereoscopic monitors § Multi-planar monitors Advanced Technology Laboratories

Robotics and Physical Presence § Capabilities of robots have increased at a much lower cost § Virtual Presence • Virtual Telecommuter • Human/Robot remote collaboration (POP troubleshooting…) • Users can have a physical presence Advanced Technology Laboratories

Robotics and Physical Presence (cont’d) Advanced Technology Laboratories

MPEG-4 Applicability § Emerging international standard for delivery of multimedia traffic. § Provides: • Delivery of all known media types over any transport mechanism (IP, ATM, RTP, MPEG-2 Transport Stream, etc. ). • Scene coherency and synchronization. • Scalability due to bandwidth (BW) or client device limitations. • Interactivity with 3 D objects and object-based compression and delivery. • Encode-once/use-everywhere content. Advanced Technology Laboratories

High-Level Architecture MPEG 4/Anim Server Side Client Side Agent Client MPEG-4 Player MPEG-4 Server component w profiler Animation Engine (Server) w profiler Signalling path Voice/Face Recognition Data path Sprint Controller Profile Manager AI engine Interaction Engine Content Databases Backend AAA Billing Service Control Text-to-Speech Server Service 1 Service 2 Advanced Technology Laboratories

Current Status § Prototype of a network to user interface. § Platform for test of intent interpretation technologies. § Open user-end platform for quick addition of novel services. § Ongoing Integration with MPEG-4 infrastructure. Advanced Technology Laboratories

A Glimpse of a Possible Future… Vo. D Advanced Technology Laboratories

Economic Models § § § Flat-Rate Pricing currently in use Usage-Based Pricing? Service-Based Pricing? On-Demand Self-Enrollment for Services Models for Revenue Sharing in “wallgarden” networks with multiple service providers Advanced Technology Laboratories

Conclusions § Several technological advances enable the next wave of services based on: • • New UIs Network Intelligence Advanced personalization User anticipation § Advanced Services not very far out § Business Models not clear yet Advanced Technology Laboratories

That’s all folks! James Schumacher [email protected] com Advanced Technology Laboratories