Smart Frog Overview HP Labs Version 0 6

What is Smart. Frog? Smart. Frog: “Smart Framework for Object Groups” Describes services as collections of components • Activates services by realizing service descriptions • • Framework elements: language – engine – components – A framework to build from, not a packaged solution • Mature; already embedded in products • July 2003 2

Smart. Frog framework Language language transforms parser security model config. models live-ness process model templates transport protocol discovery scripting config servers Components July 2003 Engine transactional lifecycle naming workflows component model binding deployment reliability services 3

What problem does it solve? • How to configure & automatically activate complex, distributed apps flexible – repeatable – sequencing, failure-recovery, evolution, adaptation, … – • Ideal for the utility computing model automatic app activation on utility resources – enables rapid resource repurposing for different apps – July 2003 4

How does Smart. Frog address the problem? 1. The common problems • No standard method to capture service config. , multiple: definitions of each value – mechanisms for access – notations (XML, ini files, SQL, . . . ) – • No method of validation for the configuration as a whole variable lifecycles – no up-front consideration of complete system startup/shutdown – lack of facility to compose systems from sub-systems – • No separation of concerns functionality and configuration mixed together – rigid location and binding integrated into code – • Lack of auto-discovery, self-monitoring, and automation July 2003 5

How does Smart. Frog address the problem? 2. Philosophy • Services are collections of components – • software, data: disk files, hardware Components collaborate to achieve some goal for which they must be appropriately organised: required components must be defined – correctly initialised, with appropriate attributes – be able to locate each other as required – exchange information regarding their state – July 2003 6

How does Smart. Frog address the problem? 3. Approach Language allows rich system descriptions • Engine automates application deployment, activation, management and shutdown • Component model allows the engine to be extended • July 2003 7

Examples • Network monitoring application many monitoring components spread throughout the network – each component must be correctly configured – service as a whole must be correctly configured / customized – service components must come up in the correct sequence – • Three-tier web application database, application logic engine and web-server must each be installed, configured and started correctly – must be started on the correct host(s), in the correct sequence – July 2003 8

Smart. Frog elements Language – templates / descriptions • Engine – interprets descriptions to activate running services • Components – make up the running service – deployed, configured & activated by the engine • Language which service components? running where? how is each component initialised? how are components related? how are the component lifecycles sequenced? Engine activates running components July 2003 • • • managed, monitored through lifecycle 9

Language properties Attribute: value pairs in a structured containment hierarchy • Properties • – – – • declarative (a data description language, not a programming language) instance-inheritance (prototype-based); allow templates to be progressively redefined flexible linking between attributes and values value resolution can be delayed to deployment time composition: build larger descriptions out of smaller parts Note: not XML-based, but can use XML as an input or output format if needed July 2003 10

Language example wstemplate. s f web. Server. Template extends { sf. Process. Host “localhost”; port 80; use. DB; } dbtemplate. s f db. Template extends { user. Table extends { columns 4; rows 3; } data. Table extends { columns 2; rows 5; } July 2003 webservice. s f #include “wstemplate. sf” #include “dbtemplate. sf” sf. Config extends { common. Port 8080; ws 1 extends web. Server. Template { sf. Process. Host “webserver 1. hpl. hp. com”; port ATTRIB common. Port; } ws 2 extends web. Server. Template { sf. Process. Host “webserver 2. hpl. hp. com”; port ATTRIB common. Port; use. DB LAZY ATTRIB db; } db extends db. Template { user. Table: rows 6; } } 11

Language: composing descriptions software component descriptions E. g. Apache Web-server resource descriptions composed: solution templates application patterns e. g. servers, storage networking e. g. 3 -tier web service on Linux cluster front-end, Superdome back-end. e. g. reliability & recovery thresholding customer preferences e. g. SLAs July 2003 parameterized fully-specified solution description ‘farm’ patterns e. g. n-tiers including firewall rules 12

Engine • Fully distributed engine comprised of Smart. Frog daemons that run on each node – • deploy service descriptions on any node and the engine will create components in the right place Daemons interpret Smart. Frog descriptions load components in the right sequence, in the right place, with correct configuration parameters – orchestrate the whole system by stepping components through their complete lifecycles – Provides liveness and component-tree navigation mechanisms • Many services can be deployed simultaneously using the same engine • Service un-deployment allows clean service removal • July 2003 13

Security • • • Engine provides secure deployment based on PKI All nodes have certificates installed All component code and descriptions must be signed If node has valid certificate and code/descriptions are correctly signed, then deployment is allowed Single-level security model (all deployed services operate within the same trust domain for a given certificate set) July 2003 14

Component model Smart. Frog components all implement a simple lifecycle that allows the engine to control them • Allows systems of components to be started consistently • import com. hp. Smart. Frog. Prim. *; import java. rmi. *; public class My. Prim extends Prim. Impl implements Prim, … { /* any component specific declarations */ public My. Prim() throws Remote. Exception {} public void sf. Deploy() throws Exception { super. sf. Deploy(); /* any component specific initialization code */ } public void sf. Start() throws Exception { super. sf. Start(); /* any component specific start-up code */ } public void sf. Terminate. With(Termination. Recored tr) { /* any component specific termination code */ super. sf. Terminate. With(tr); } /* any component specific methods */ } July 2003 15

Components Smart. Frog has an extensible component set • Some basic components form the core of the system, e. g. : • “Prim”: the primitive component – everything inherits from it – “Compound”: implements a “shared-fate” lifecycle for groups of components – Sequence: implements a sequential lifecycle for components – • Other components provide system services: discovery mechanisms, reliability building blocks, host scripting, JMX support, … • Components are written as needed to support new July 2003 16

Encapsulating non-Smart. Frog components We don’t need to write our whole application/service in Smart. Frog Java components -- it’s easy to encapsulate non-Smart. Frog things, e. g. • Apache web-server: wrapper components to • install and configure software (and remove) – stop and start the service – We describe Apache and its required configuration in a Smart. Frog description (using an Apache SF template) • When deployed, the SF Apache component can install, configure, start, stop and remove Apache • Apache can be used as a component of larger services • July 2003 17

Implementation details • Implemented completely in Java (using JDK 1. 4) – works across all standard Java platforms Uses Java RMI as transport • Core system size • ~10 K non-commented lines of Java – ~750 lines of description files – binaries: 800 -1200 KBytes depending on services included – July 2003 18

Research questions • Rich research space spanning language, engine, and components – – – – – July 2003 reliability (basic mechanisms, generic recovery patterns) security combining reliability and security scalability language specification service debugging tools standardized interfaces to resource management subsystems standardized interfaces to node image deployment subsystems generic system/service visualization tools 19

Business opportunity • App developers better enable apps for Grid and utility computer models – optimize application for specific hardware configurations – improve development (rapid deployment) – • Solution providers improve app configuration, deployment, and management – more secure and reliable solutions – quicker time to deployment and adaptation – • Happier users more stable systems – customization to specific needs – more control over solutions – July 2003 20

Framework status Developing an open source (LGPL-licensed) release of the core framework (availability calendar ‘ 03) • Earlier preview / alpha releases planned • Deployed within HP/Agilent products • • Experience with: Apache – BEA Weblogic – Maya CGI rendering engine – Oracle – July 2003 21

Some related work • • • Grid standards for Grid/utility computing Imaging / provisioning systems: Altiris, Rembo, Novadigm (+ many others) MS Dynamic Systems Initiative, MS Provisioning System Adaptive systems: Think. Dynamics, Corosoft IBM e. Liza, autonomic computing Proprietary (product-specific) solutions in place today for installation / configuration / activation July 2003 22

Standardization goals • Improve Grid applications – specification, deployment, configuration & life-cycle mgmt Standardize SF language, app templates, and SF engine • Create multiple reference implementations of • engine – app templates (Application Server, JBOSS, Web. Logic) – Define SF integration with Grid • Form a GGF WG • July 2003 23

Smart. Frog benefits Increased operational reliability • Improved quality • Assured correctness and consistency • Increased security • Reduced cost • Improved customer experience • July 2003 24

Summary • Smart. Frog framework (language, templates, engine) supports: – specification, deployment, configuration, and life-cycle mgmt • • • Well suited for a utility computing model Addresses needs of complex distributed applications Business benefits for application developers, ISPs, & users • We are seeking partners & lighthouse users – – July 2003 jointly standardize key elements develop with reference implementations create templates for specific applications establish a wide technical community 25

HP logo July 2003 26