Putting It All Together Trends in Business Intelligence

Putting It All Together: Trends in Business Intelligence Claudia Imhoff, Ph. D Intelligent Solutions, Inc. CImhoff@Intel. Sols. com www. intelsols. com Blog: http: //www. b-eye-network. com/blogs/imhoff/ Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved

Claudia Imhoff President and Founder Intelligent Solutions, Inc. A thought leader, visionary, and practitioner in the rapidly growing fields of business intelligence and customer focused-strategy – Claudia Imhoff, Ph. D. , is an internationally recognized expert on analytical CRM, business intelligence, and the infrastructure to support these initiatives – the Corporate Information Factory (CIF). Dr. Imhoff has co-authored five highly-regarded and popular books on these subjects and writes monthly columns (totaling more than 100) for technical and business magazines. Email: cimhoff@intelsols. com Phone: 303 -444 -6650 Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 2

Putting It All Together § Going Beyond Traditional BI – Operational BI Takes the Stage § Data Warehouse Appliances and Analytic Databases – Making Life Simpler § BI Software as a Service – Feeling Saa. S-y? § Open Source BI – Free Software Anyone? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 3

The Three Levels of Business Intelligence Strategic BI timeframe ~ months Tactical BI timeframe ~ days or weeks Operational BI timeframe is intra-day Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 4

The Three Levels of Business Intelligence Strategic BI Tactical BI Operational BI Business focus Achieve long-term business goals Manage tactical initiatives to achieve strategic goals Monitor & optimize operational business processes Primary users Executives & business analysts Business analysts, & LOB managers, operational users & operational processes Timeframe Months to years Days to weeks to months Intra-day to daily Data Historical data Real-time, low-latency, & historical data Paradigm Shift Mode of operation User driven Data centric Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved Event driven Process centric 5

What is Operational BI* A set of services, applications and technologies for monitoring, reporting on, analyzing and managing the business performance of an organization’s daily business operations *From research study. “Embedded BI”, written by Colin White and Judy Davis, www. B-EYE-Research. com Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 6

Operational BI – Answers to Day-to. Day Business Questions ü picked ü packed q shipped q invoiced What is my customer’s order status? What can I offer based on customer’s life-time value? What is my current inventory level world wide? Is it sufficient to meet demands? Yield What is my production yield right now? Am I at par with acceptable standards? Can I afford to make this move at current margin rates? Operational BI § Helps front-line workers make immediate business decisions to squeeze out inefficiencies. New Data Needs § Information on demand § Real-time + historical data § Access to SAP, Siebel, Oracle and BI results Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 7

Real Time Decision-Making* § Operational BI optimizes time latency between when a business event occurs and when an appropriate action is taken § The goal – to “right-size” the decision-making cycle § Compressing time lag between knowing what is happening and taking action based on that knowledge § Real-time must consider potential trade-off between timeto-action and business value of actions * From “Right-Time Business Intelligence: Optimizing the Business Decision Cycle” By Judy Davis, www. B-EYE -Network. com Research paper Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 8

Impact on BI Environment § History of BI § Extract usable information from operational systems § Users, technologies, processes, procedures – all independent of operations § Now what? § Impact on BI environment is significant § Increase in number of users, volume of data, and faster performance § Operational BI – MUST be integrated into the operational environment § Requires understanding of operational systems, processes, procedures, workflows, personnel Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 9

Impact on BI Environment § Numbers of users increase significantly § Traditional BI rarely supported a few hundred, maybe a thousand or so users § Opening BI up to operational personnel means ramping up into tens of thousands of users § These users have very different interface requirements § Means BI implementers must rethink how BI is delivered to business users § Means tighter and faster connectivity of enterprise decision support environment to rest of the company. Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 10

Impact on BI Environment § Volumes of data increase substantially § Detailed intraday snapshots of data are loaded or tricklefed into data warehouses § Tens of terabytes to hundreds of terabytes are not unusual storage requirements for operational BI § Scalability now a mandatory requirement in any BI technology § Whether in processing and integration of data, storage of massive volumes, or retrieval of query responses Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 11

Impact on BI Environment § Faster performance § Query performance must mimic or emulate response times in operational systems § Sub-second to just a few seconds to return data from a query. § Ability to prioritize queries not only according to their importance but also their response requirements is mandatory success criterion § This last feature has stumped many BI implementers and BI vendors § Must have ability to handle mixed work load gracefully and simultaneously Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 12

Getting Started – Assess Reality § First step – perform honest assessment of existing data delivery capabilities – available technologies, maturity of the BI architecture, existing personnel, etc. § Combine these with solid understanding of business requirements for operational BI data § Important to understand which weaknesses discovered in assessment will be exaggerated as you speed up the enterprise Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 13

Operational BI Requirements § Continuous availability of operational data and BI results § Current information from operational systems § Integrated with BI data on demand § Minimal impact on operational systems performance § Presented in a proactive manner § Make decisions – act on information presented § Easy to understand use § Dynamic modeling § Ability to change business rules on the fly § Show different set of metrics depending on situation Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 14

Picking a Project § Look for workflow activities that have significant impact on costs or revenues § Bottlenecks today that can be made more efficient through use of operational BI § Don’t make big changes to operational processes § Just speed up or make more efficient processes you already have in place § You will have to retrain personnel and retool SOPs § Project managers may not realize operational BI application has ramifications beyond project’s immediate boundaries Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 15

Putting It All Together § Going Beyond Traditional BI – Operational BI Takes the Stage § Data Warehouse Appliances and Analytic Databases – Making Life Simpler § BI Software as a Service – Feeling Saa. S-y? § Open Source BI – Free Software Anyone? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 16

Data Warehouse Appliances § BI and data warehousing technologies continue to evolve and innovate § Produce more efficient & cost effective ways to deliver BI § Latest innovations are DW and BI appliances § Definition of an appliance* § § One purpose One package One installation One vendor * From the B–EYE-Research. com paper titled “Data Warehouse Appliances: Evolution or Revolution? ” by Colin White, and Richard Hackathorn Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 17

Data Warehouse Appliances § All-in-one box that provides a hardware server preconfigured with all software components § Designed for a specific purpose – supporting data warehouse processing § Offers ease of use, simplicity, and compatibility – tested, ordered and delivered as a single system § Simple to understand even though mechanism may be complex § Low cost in terms of TCO § High performance in achieving its purpose § Single point of service provided by single vendor Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 18

Data Warehouse Appliances § Cost effective solution § TCO of a data warehouse appliance is lower because cost of hardware and software is cheaper § Also because simplicity and ease of reduces installation, administration and support cots § Improved usability of a data warehouse appliance means projects can be developed and deployed faster § Includes popular BI capabilities § Interactive dashboards, analysis, reports, alerting, and data integration Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 19

Sweet Spot for Data Warehouse Appliances Size of Data Multiple Terabytes Data Warehouse Appliances Mega- to Gigabytes Focused Purpose Specialized Databases (e. g. , Teradata, IBM) Any database vendor Complexity of Workload Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved Mixed Purpose 20

Data Warehouse Appliances § Pros § Cons § Immediate visibility & interaction into business performance § Non-disruptive to existing infrastructure § Faster deployment § Low maintenance – black box Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved § Still some opposition to use of appliances by IT departments § Loss of “control” over moving parts § DW and BI appliance scalability § Customization to fit each company’s needs 21

Sample Data Warehouse Appliance Vendors § § § § Netezza Teradata DATAllegro (now Microsoft) Sun + Green Plum Sun + Vertica Sun + Par. Accel Sun + Kognitio IBM Info. Sphere Warehouse Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 22

Role of Appliances in BI Saa. S § Many data warehouse appliance and BI Saa. S vendors are forming partnerships § Gives Saa. S vendors scalability, reliability, performance § Gives appliance vendors applications, new markets, greater exposure § Gives customers more confidence that solution is on solid technological footing § § Performance Support for multi-tenancy Scalability Applications Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 23

Analytic Databases § Many are Massive Parallel Processing (MPP) § Can use commodity hardware § Many have column-based data organization § Limit I/O by putting similar data together – reduces reads to only columns needed for query § Single data type per column allows for significant compression § Data compression § Compression can be optimized for particular data types § CPU is not the bottleneck, only I/O is Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 24

Analytic Databases § Built-in intelligence § Allows decompression of only data that must be for query resolution and ignore all others § Is major factor in overall improved performance § Load times remain constant regardless of table size § Should also have query times that remain constant regardless of table size § Bottom line – technology must be seamlessly scalable Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 25

Analytic Databases § Many new vendors on the market (sample): § § § § Green Plum Vertica (Michael Stonebraker*) Par. Accel (Barry Zane**) Dataupia (Foster Hinshaw**) Info. Bright (Warsaw University) Aster Data (Stanford University) illuminate (Former Synerra Systems founders) One well-established vendor: Sybase IQ since 1993 § Most are column based, MPP, shared nothing architectures (not all though) * Ingres and Illustra founder Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved ** Netezza founders 26

Analytic Databases – Really Fast: TPC-H 1 TB Copyright © 2008, Intelligent Solutions, Inc.

Analytic Databases – Really Fast and Really Inexpensive Solution Pricing model Price/unit 1 TB solution Remarks Vertica Data Volume (raw) $ 100, 000/TB $ 200, 000, - Based on 5 nodes, $ 20, 000, - each Par. Accel Node $ 40, 000, (+$10, 000/TB) $ 310, 000, - Based on 5 nodes, $ 20, 000, - each Par. Accel Data Volume (raw) $ 1, 000, -/GB $ 1, 250, 000, - From TPC-H publication Info. Bright Data Volume (raw) $ 40, 000, -/TB $ 140, 000, - Based on 5 nodes, $ 20, 000, - each Dataupia Node $ 19, 500/2 TB $ 19, 500, - You can not buy a 1 TB Satori server Exa. Sol Data Volume (active) $ 675 - $1, 750 per GB $ 940, 000, - From TPC-H publication Graph compliments of Jos van Dongen, Tholis Consulting, NL. Numbers are estimated. Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 28

Analytic Databases § Pros § § Excellent performance Very cost-effective Low maintenance Partnering with hardware vendors (DW appliance) § Cons § Many are small companies § May not handle mixed work load well § New (unknown) technology for IT Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 29

Putting It All Together § Going Beyond Traditional BI – Operational BI Takes the Stage § Data Warehouse Appliances and Analytic Databases – Making Life Simpler § BI Software as a Service – Feeling Saa. S-y? § Open Source BI – Free Software Anyone? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 30

BI Delivery Models § There are two BI delivery models today § On-premises –

On-premises – Traditional Model § Internal IT is responsible for entire environment from first project § Find excess capacity on machines § Upgrade memory on existing machine for usage § Leverage installed end user access tools § Buy smaller platforms that can scale § Migrate to bigger box when necessary § Use smaller box for data mart(s) § Look into data warehouse appliances for very large, focused BI analytics Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 32

Software as a Service (Saa. S) § Characteristics* § Secure, flexible, and efficient business processes & workflows § Service level agreements § Value-added business services such as analytics & best practices § Extensive use of service-oriented architecture (SOA) to enable scaling, configurability, and integration § Subscription monitoring & usage-based billing * From www. sandhill. com, “Get Ready for Saa. S 2. 0” by Bill Mc. Nee, Saugatuck Technology, May 8. 2006 Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 33

Advantages for Saa. S Vendors § Vendors support only one platform and one version of the application § No need to support multiple operating systems, platforms, and older versions of the software § Decreases development costs significantly § Saa. S gives vendor great visibility into how their customers are actually using their software § See every move, every feature, every function used by customers § Gives vendor great intelligence on how to build a better product based on actual usage Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 34

Advantages for Saa. S Vendors § Saa. S model gives vendor a predictable cash flow § Subscription model is reliable for cash flow estimation § Improves start-up estimations and growth track § Vendors don’t get trapped in “feature bloat” § No need to keep adding feature after feature to get customers to buy new versions § Create only features that are needed based on actual customer usage Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 35

Disadvantages for Saa. S Vendors § Saa. S produces lower revenues at first than traditional vendor models § Must attain critical mass of subscribing customers § Vendor must have enough funding to tide them over § More time is needed to ramp up to mature status § Higher customer set up costs § Traditional vendor model – send customer a CD § Saa. S vendors must allocate space, set up customer support, etc. § Saa. S vendor becomes IT support for their customers (higher costs for customer service? ) Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 36

Disadvantages to Saa. S Vendors § Customers still need ability to integrate Saa. S application data with other enterprise data § Need mechanism to export data out of Saa. S environment § Who supplies integration of Saa. S data with customer’s other data? § If customer is not SOA-compliant yet, what does this mean to Saa. S model? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 37

Reasons for Adoption: Ease of Deployment § This is the Saa. S model’s greatest advantage § § No installation of hardware No installation of software No administration of new versions of either No need for IT expertise in the tool or application § Set up consists of getting a login and password set up for the business users Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 38

Reasons for Adoption: More Flexibility for Evolving Needs § Perhaps… § You can certainly change Saa. S vendors quite easily § If you are unhappy with one vendor, changing to another one is about as easy as getting a new login and password § You can influence the direction and R & D of the current Saa. S vendor § You can easily add or subtract users § You can easily add or subtract functionality § It may not be as easy to customize the Saa. S offering to your specific needs Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 39

Reasons for Adoption: Not Locked into Long Licenses § True § Great advantage in BI world where technology is moving very fast § Can switch from one Saa. S vendor to another § But watch for cancellation fees § And make sure you know what the subscription fee is based on § Reduction or addition of users may be cross price break threshold § Salesforce. com model is typical Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 40

Considerations for BI Saa. S § Saa. S – good at supporting particular types of users § Highly mobile work force § Field sales personnel § Product support specialists at customer sites § Telecommuters § Highly geographically disbursed workforce § International enterprises § Non-office workers (virtual offices) § Customer or partners worldwide § Must include support for various mobile devices § Phones, mobile PCs, handheld devices, PDAs, etc. Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 41

Considerations for BI Saa. S § Ensuring quality of delivered environment § § Correct mappings, verified data lineage, transformations Sufficient data quality processing Data represented in analytic engine correctly Appropriate presentation of information, e. g. , personalized dashboards § Scalability of environment § Data volumes – small beginnings to 100’s of terabytes? § From a few users to 1000’s Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 42

Considerations for BI Saa. S § Performance § From simple to complex queries § Response times – operational to strategic BI § Getting right data to right people at right time § Open Architecture § Compliance with best practices? § Non-proprietary infrastructures? § Integration with existing infrastructure? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 43

Considerations for BI Saa. S § What does Saa. S vendor bring to the table? § Best practices § Quick start BI components like a library of reports, analytic calculations, KPIs, etc. § Industry-specific knowledge § Horizontal business knowledge § Support for all employees in all levels of enterprise Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 44

BI Saa. S § Pros § Fixed cost – subscription model § Fixed time § Flexibility / customization § Single vendor responsible for entire environment § Quick ramp up § Cons § New paradigm – nervousness? § Can a company maintain its uniqueness? § Loss of “control” over data, quality, access § Vendor’s timeliness in response to changes § Vendor’s industry knowledge Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 45

Sample BI Saa. S Vendors § § Lucid. Era Xactly Eyeris Pivot. Link (was Sea. Tab) § Oco § On Demand IQ § § § § Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved Actuate Cognos SAP ERP SAS Business Objects Sales. Force. com Dimensional Insight 46

Putting It All Together § Going Beyond Traditional BI – Operational BI Takes the Stage § Data Warehouse Appliances and Analytic Databases – Making Life Simpler § BI Software as a Service – Feeling Saa. S-y? § Open Source BI – Free Software Anyone? Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 47

Open Source Vendors Face Questions § The myths and doubts: § § Is there support for open source BI? How many people are really using it? Will it scale? Is it considered enterprise class? Is it only for developers? § These are being overcome… § According to Aberdeen*, 25% of survey respondents will adopt open source BI in next 12 to 24 months § CEOs agree – open source is a worldwide growth story in 2008** § First nine months of 2007, open source deal flows doubled each quarter*** § Sun’s commitment to open source - $1 Billion for My. SQL * Source: “The TCO of Business Intelligence – Open Source Takes on Traditional BI”, www. aberdeen. com ** Source: www. opensolutionsalliance. org *** Source: www. the 451 group. com Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 48

Why Use Open Source? § Price! § Open Source software can be downloaded, installed and operated free of charge § Return on investment (ROI) of Open Source model is good § Open Source software is reliable and scalable § Just look at the Internet – its infrastructure relies heavily on Open Source software § Wall Street – 8 of top 10 banks use Open Source technologies Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 49

Why Use Open Source? § Open Source community grown to significant size § Millions of developers contributing everyday § Cost of development is externalized § Ability to adapt or customize § Many companies don’t want or need feature bloat § Easy integration and performance § New tools for building browser based reports and dashboards accessible to more people § Ad-hoc report designers with drag and drop capabilities § Enhanced wizards for custom data source implementation Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 50

Open Source Offerings § BI projects won’t be consumed by license fees § No huge up-front fees to justify before commencing a project § “Safe choice” § § Many successful deployments Professional services experts to work with you Professional, public training Support from the experts – project leaders and sponsor Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 51

Open Source BI Vendors § Proponents of open source BI point out low cost of entry, flexibility and variety of applications available § Opponents believe open source BI lacks functionality needed to succeed right now § Regardless, don’t be fooled by “numbers of downloads”… § Vendors* – Actuate, Jasper. Soft, Jpivot, Mondrian, Pentaho, Spado. BI * For a more complete list, go to http: //www. manageability. org/blog/stuff/open-source-java-business-intelligence Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 52

Open Source ETL § ETL alternative follows industry standards for ease of use, quick deployment, and fit into company’s needs § Users download open source ETL code and get started § Can collaborate with open source community to share integrations and extend tool’s functionality § Will probably need to buy support and services from company’s professional services and support § Sample Vendors*: Talend, Jitter. Bug, KETL, Pentaho, Octopus, Clover. ETL * For a more complete list, go to http: //www. manageability. org/blog/stuff/open-source-etl Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 53

Open Source § Pros § Cons § Cost effective § Easy to install and deploy § Large development community Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved § How do they make money? § Many are small companies § Some offerings not truly open source 54

Right Place at Right Time – Get Going! § Once you have your ducks in a row, you are ready to create the proper environment § Create an infrastructure that can withstand change – you’ll need it § Pick technologies that support that infrastructure and move you toward SOA compliance § Constantly monitor business community usage § Measure ROI and publish it § IT infrastructure should be to information as a power grid is to electricity § Information should flow as freely as electricity does Copyright © 2008, Intelligent Solutions, Inc. , All Rights Reserved 55