SOFSEM 2013 The Challenges in ICT

SOFSEM 2013 : = The Challenges in ICT: Debunking the Hype Keith G Jeffery Science and Technology Facilities Council Harwell Oxford Rutherford Appleton Laboratory, OX 11 0 QX UK e-mail: keith. jeffery@stfc. ac. uk ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 1

SOFSEM 2013 : = The Challenges in ICT: Debunking the Hype Or the nebulous concept of CLOUD Computing ©STFC/Keith G Jeffery Science and Technology Facilities Council Harwell Oxford Rutherford Appleton Laboratory, OX 11 0 QX UK e-mail: keith. jeffery@stfc. ac. uk The Challenges in ICT: Debunking the Hype 20130127 2

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 3

Rutherford Appleton Laboratory ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 4

Brief Biography • Degree and Ph. D in Geology • Led teams working on information systems • (R&D and services) • Director IT • 1100 servers, 360, 000 users, 8 Pb/year ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 5

Associations =: SOFSEM : = ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 6

EC CLOUDs Expert Group Report 1: Work 2010, Event January 2011 Published January 2011 http: //cordis. europa. eu/fp 7/ict/ssai/docs/cloud-report-final. pdf Report 2: Work 2011, Event May 2012 Published December 2012 http: //cordis. europa. eu/fp 7/ict/ssai/home_en. html ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 7

So? This background gives you some idea of ‘where I’m coming from’ – Advanced research problems requiring ICT solutions – Research practical (yet leading edge) • But depends on ‘blue sky’ – International working – consultancy, reviewing, expert – Strategic thinking for / using blue sky research to plan roadmaps for ICT R&D – Design authority for large industrial-scale projects ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 8

So? This background gives you some idea of ‘where I’m coming from’ – Advanced research problems requiring ICT solutions – Research practical (yet leading edge) • But depends on ‘blue sky’ – International working – consultancy, reviewing, expert – Strategic thinking for / using blue sky research to plan roadmaps for ICT R&D – Design authority for large industrial-scale projects And what I am going to talk about is the ICT of the future that we shall be using and/or developing And the research challenges we have to overcome to make it happen ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 9

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 10

Internet Users by Region ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 11

The ASIA Timebomb ASIA has largest population, largest number of users but relatively low penetration ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 12

Mobile Internet • Global mobile data traffic in 2011 (597 petabytes per month); • Global mobile data traffic grew 2. 3 -fold in 2011, more than doubling for the fourth year in a row; • The number of mobile-connected devices exceeded the world's population in 2012. • There will be over 10 billion mobile-connected devices in 2016; • The average mobile network connection speed (189 kbps in 2011) will exceed 2. 9 megabits per second (Mbps) in 2016. ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 13

Mobile Traffic Growth ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 14

Mobile Traffic Sources ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 15

Non-user devices • The vast majority of computers – 98% - do not have traditional keyboard, mouse, screen – They are in cars, planes, washing machines, mobile phones • The most-used operating system is NOT Windows (or Unix / Linux) – Symbian in mobile phones i. OS Android – or specialised operating systems (e. g. Contiki) in embedded systems ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 16

Social Context • The number of computers will vastly outnumber humans on the planet very soon; • Everything will be computerised; – Sensor networks • Home, healthcare, environment, industrial processes, transport systems…. – Control systems • Industrial, transport, home (central heating)… Just think what a neutron bomb in the atmosphere could do ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 17

So… • This is the ‘internet of things’ or ‘future internet’ • We need to : – Manage the huge numbers, sizes – Integrate the different kinds of systems – Into one environment leading to human decision-making • whether managing a business, shopping, media choice, social interaction • But there is a problem…in last 20 years – Data storage density increased ~10**18 – Processor power increased ~10**15 – BUT broadband capacity increased ~10**4 • This has implications for Information Systems Engineering! • In fact the requirement and limitations challenge the very basis of traditional computer science / ICT ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 18

Issues Elastic scalability costs, green Trust & security & privacy confidence Manageability else many administrators Accessability different modes of use Useability natural – fits with user model of the world Representativity of the real world ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 19

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 20

Era 1 User request, programmer, punched cards, low-level program, mainframe ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 21

Era 2 User interacts with in-house-written software in high level language on mainframe or mini. Network proprietary. ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 22

Era 3 User interacts with off-the-shelf software on PC which interacts with in-house -written or purchased software on mainframe. Client-server to 3 -tier. Network is (becoming) internet ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 23

Era 4 User interacts with pre-written software on mobile device which interacts with pre-written software on mainframe. Network is internet Wi. Fi, 3 G, 4 G… and using WWW ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 24

CLUSTERs & GRIDs Having virtualised the way the user interconnects to the application on the mainframe (Client-Server or 3 -tier) the next logical step is to virtualise the mainframe CLUSTER • Racked mainframe in-house • Homogeneity • Dynamically reassigned resources GRID • Distributed racked mainframes • Heterogeneity • Dynamically reassigned resources • Mobile Code Which leads us towards CLOUD Computing ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 25

Software Just to mention there has been a parallel evolution in: • Programming languages • Machine code, assembler, autocode, 3 G (imperative), functional, 4 G (declarative), scripting, mobile • Data modelling and management • Lists, hierarchies, networks (E-R), graphs (EER, ORM) • Software and systems design • Including human factors, adopting new modalities (mouse, gesture, speech, brain-connected) • Systems development methods (CASE to IDE) • HIPO, Jackson, SSADM, PRINCE (waterfall to spiral) • Object-orientation (aspect orientation) ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 26

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 27

Definition cloud (klaʊd) an elastic execution environment of resources involving multiple stakeholders and providing a metered service at multiple granularities for a specified level of quality (of service). From report EC Cloud Computing Expert Group January 2011 ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 28

Cynicism 2010 we recognized that all our processes were far too complex so we put them in the cloud ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 29

CLOUDs in use • Social networking • E. g. Facebook • Email systems • E. g. Gmail • Office systems • E. g. Google Docs • Shared storage • E. g. Google Drive ©STFC/Keith G Jeffery But also 1. Software / system development away from production systems 2. Experimental techniques (like a sandbox) The Challenges in ICT: Debunking the Hype 20130127 30

Gartner Hypecycle ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 31

Gartner Hypecycle Opportunity gap ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 32

Cloud Computing • Very old idea • Use of cloud to depict a computing service or network The premise: Most compute centres utilise only 10% of capacity but need 100% for rare peaks of demand • virtualisation • Now used for a new concept • Confused with – – GRIDs Autonomic computing Utility Computing Service-Oriented Architecture ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 33

Cloud Computing Hardware • A very large number of processors – Clustered in racks as blades • In one major computer centre – May be replicated for business continuity • With massive online storage – RAID for resilience • And excellent communications links Economies of scale – both purchasing and operation Energy economies in location Staffing economies in location – For access ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 34

Cloud Computing Customer View • Low cost of entry for customers • Device and location independence • Capacity at reasonable cost (performance, space) • Cloud Operator manages resource sharing balancing different peak loads • Elastically scalable as demand rises (or falls) from user • Security due to data centralisation and software centralisation • Sustainable and environmentally friendly – concentrated power • it is a service and the user does not know or care from where, by whom, and how it is provided • as long as the SLA (service level agreement) Qo. S (quality of service) is satisfied • it is a ‘computing utility’ (Iaa. S, Paa. S, Saa. S…. ’Xaa. S’) ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 35

Cloud Computing Ownership • Private Cloud: in-house cluster run using CLOUD middleware; • Public Cloud: outsourced computing to commercial provider – proprietary; • Hybrid Cloud: linked Private and Public CLOUDs ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 36

Cloud Computing Offerings Acknowledgements to U Southampton ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 37

Cloud Computing How does it work • Multitenancy: Cloud resources (hardware) shared dynamically between customers; • Each customer application in its own virtual machine • Isolation for security, privacy • Allows scheduling with respect to shared resources • Application in one VM multithreaded with user data / profiles etc in other VMs ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 38

Cloud Computing What is it? • Is cluster computing – with the advantages that brings • With GRIDs features – Scheduling / resource allocation – self-* • ASP (Application Service Provider) ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 39

Cloud Computing • Obtains from GRIDs work: – resource sharing/scheduling – virtualisation of hardware and low-level software (under middleware) – resilience – trust, security, privacy – (more or less) self-* Utility computing Autonomic computing ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 40

Cloud Computing Obtains from software/systems engineering: Service-Oriented Architecture with implications of interfaces, metadata, composition ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 41

Cloud Computing But the real novelty is… • Pay-As-You-Go (only for what you need); • Accounting for ICT used by departments in an organisation; • Private cloud • Public cloud • CAPEX to OPEX ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 42

Cloud Computing • Private Cloud Software : • Private/Hybrid Cloud software: Eucalyptus Open Nebula, Open Stack • Commercial examples of Public Clouds: • Amazon EC 2 Elastic Compute Cloud • Google (Engine for Apps; Connect for Office) • Microsoft Azure • IBM Smart. CLOUD • (note all needed massive resource for infrequent use so could sell of excess capacity) • Note Thomas J Watson in late fifties: “total number of computers required in the world is five” • are we reaching this goal? ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 43

Cloud Problems • Inefficient to move data to the cloud • Remember earlier comments about networking bandwidth • Hard to realise the technology - elasticity • Despite SLA/Qo. S guarantees some concerns: • Performance • Security/trust/privacy • Especially transnational ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 44

Cloud Studies Above the Clouds: A Berkeley View of Cloud Computing http: //www. eecs. berkeley. edu/Pubs/Tech Rpts/2009/EECS-2009 -28. pdf ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 45

Cloud Quotes The interesting thing about Cloud Computing is that we’ve redefined Cloud Computing to include everything that we already do. . I don’t understand what we would do differently in the light of Cloud Computing other than change the wording of some of our ads. Larry Ellison, quoted in the Wall Street Journal, September 26, 2008 It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign. Somebody is saying this is inevitable — and whenever you hear somebody saying that, it’s very likely to be a set of businesses campaigning to make it true. Richard Stallman, quoted in The Guardian, September 29, 2008 ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 46

Hype? • Architectural model: outsourcing – to large datacentres - has been around as long as computing • Business Model: So has ‘pay as you go’ • The difference now is • • Autonomicity for management (including elastic scalability) SOA (Service Oriented Architecture) ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 47

Characterisation Paa. S Iaa. S Saa. S Elasticity Reliability Virtualisation Cost Reduction … COMPARES TO … Local LOCALITY Remote Distributed STAKEHOLDERS Users Internet of Services ©STFC/Keith G Jeffery Hybrid MODES Cloud Systems BENEFITS Service-oriented Architecture Public FEATURES … Ease of use Private TYPES Adopters Resellers … Grid Providers The Challenges in ICT: Debunking the Hype 20130127 48

Terminology Types of Clouds – Iaa. S; Paa. S; Saa. S Deployment Types (Usage) – Private, Public and Hybrid Clouds – Community Cloud and Special Purpose Clouds Cloud Environment Roles – Cloud Providers: offer cloud systems – Cloud Resellers or Aggregators: aggregate platforms from cloud providers – Cloud Adopters or Software / Services Vendors: use cloud platforms to enhance their services – Cloud Consumers or Users: make direct use of the cloud capabilities – Cloud Tool Providers: provide supporting tools for using / improving cloud environments ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 49

Characteristics Non-Functional Economic Technological relate to qualities of cloud systems, rather than technological aspects. These include: key driver behind (commerical) cloud systems. Typical interests on: Arise from realising nonfunctional / economic concerns. Particular issues: • Elasticity • Reliability • Quality of service • Agility and adaptability • Availability ©STFC/Keith G Jeffery • Cost reduction • Pay per use • Improved time to market • Return of investment • CAPEX to OPEX • “Going green” The Challenges in ICT: Debunking the Hype • Virtualisation • Multi-tenancy • Security, privacy and compliance • Data management • APIs and / or programming enhancements • Metering • Tools in general 20130127 50

Related Areas Internet of Services – Cloud systems are “enablers” for Internet of Services Internet of Things – No direct relationship – Clouds may extend capabilities of the Io. T The Grid – Strong conceptual overlap – Services may move from grid to cloud Service Oriented Architectures – Clouds principally architecture-agnostic – Service offerings should follow the SOA model ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 51

State of the Art Commercial Efforts Research & Academic Efforts Manageability and Self-* Data Management Privacy & Security Federation & Interoperability Virtualisation, Elasticity and Adaptability • APIs, Programming Models & Resource Control • • • Technical Gaps ©STFC/Keith G Jeffery • Legislation, Government & Policies • Economic Concerns Non-Technical Gaps The Challenges in ICT: Debunking the Hype 20130127 52

Economic Aspects Cost Reduction Pay per Use Improved Time to Market Return on Investment CAPEX to OPEX Going Green ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 53

Legislation, Governance, Policies No standards – vendor lock in Legality of operating / using CLOUDs – International jurisdiction – Privacy and personal data Security of operating / using CLOUDs – International jurisdiction – Security and investigatory powers No free market in services across CLOUDs ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 54

Opportunities Enhanced Technical Capabilities Reference Implementations Easier Toolsets Policies & Governmental Issues Clear Legalistic Solutions Global Cloud Ecosystems major contributions to Increased Interoperability Higher Trust in Clouds ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype Tools & Service Market Cloud Knowledge & Business Expertise Clouds Provisioning & Usage 20130127 55

Analysis Cloud computing will play a relevant role in (at least) the next 10 years Current status insufficient for future business needs Þ Europe can contribute in particular with – R&D in the technological and non-technological areas – Legislatory & governmental support – Business & economical expertise Þ Relevant business opportunities – – Iaa. S Cloud Provisioning Paa. S Cloud Provisioning Cloud Adopters and Service Vendors (Enhanced Service (Saa. S) Provisioning) Cloud Consultancy ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 56

Research Issues Business opportunities not currently realisable Existing knowledge from preceding research and development can be harvested Technical Non-Technical • • Scale and elastic scalability • Economic aspects Trust, security and privacy • Legalistic issues Data handling Programming models and resource control • Systems development and management ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 57

Target Private CLOUD (inhouse, cluster) interface Common Service Environment with metadata and dynamic systems development and composition capability interface Public CLOUD ©STFC/Keith G Jeffery interface Public CLOUD interface Other Private CLOUD The Challenges in ICT: Debunking the Hype 20130127 58

What Stops us. . • I have listed a whole lot of CLOUD specific problems already such as: • • Interoperability / vendor lock-in Security, privacy Quality of service / service level agreements Legislation • But CLOUDs throws into sharp relief many underlying computer science / informatics problems ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 59

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 60

Research Challenges: 1 Metadata • the need for metadata related to services, data/information/knowledge, agents; • what is data, what is metadata? • kinds of metadata and their use; • representation and structure - syntax; • semantics (meaning); ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 61

The Vision: The Models Complete cohort of users User Model interaction with data, processing, persons Processing Model representing the world providing what the user requires Data Model representing ICT Resource Model Complete ICT environment ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 62

The Vision: Metadata for ENGAGE Data Model Linked open data DISCOVERY (DC, e. GMS…) Generate CONTEXT (CERIF) Formal Information Systems ©STFC/Keith G Jeffery Point to DETAIL (SUBJECT OR TOPIC SPECIFIC) The Challenges in ICT: Debunking the Hype 20130127 63

Research Challenges: 2 Management of state • detection of state across millions of individual nodes; • maintenance of state across many nodes; – transactions and locking; – roll-back and compensation; ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 64

Management of State • ACID, 2 PC, Locking, Rollback or compensation: – As number of tables increases – And the number of instructions to be executed increases – And the latency (due to distribution) increases • It becomes impossible to represent the real world with: – Integrity – Consistency – Accuracy ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 65

Research Challenges: 3 (1) Data representativity • data structures representing real-world interrelationships; – data attribute value encoding (character set, media encoding), types, lengths; – data attribute value language; – fully connected graphs – the death of the hierarchy; – the time-machine: temporal duration of the interrelationships; – certainty, probability of the inter-relationships – Incomplete and inconsistent information ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 66

Research Challenges: 3 (2) Data representativity • Interoperation – reconciliation of different data structures representing a similar real-world domain; • data location / locality and replication – for business continuity; ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 67

Data Representativity: Interoperation • Homogeneous view over heterogeneous sources – Character set, language, syntax, semantics • Schema reconciliation – Structural mapping – graph theory – Lexical mapping – domain ontologies – Need richer metadata ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 68

Research Challenges: 4 Data quality, veracity and permanency • detection of quality against metadata parameters e. g. precision, accuracy; • provenance; • temporal recording; • data curation across media and policy evolution; ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 69

Research Challenges: 5 Trust, security and privacy • policies declared, enforced and monitored through restrictive metadata; • policy reconciliation for interoperation; • Legalistics • Rights • Responsibilities ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 70

Research Challenges: 6 Management of service levels and quality of service • policies declared, enforced and monitored through restrictive metadata; • service level negotiation (e. g. lower price for lower performance); ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 71

Research Challenges: 7 Systems design, development, maintenance and decommissioning • based on strong separation of: – services (processes), – data, information and knowledge • assuming self-(re-)composition, self-managing and adjusting, self-maintaining properties ; • assuming mobile code properties ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 72

A final challenge? • Is the von Neumann architecture still valid? • Should we not optimise communications over other priorities? • Remember the transputer • Do we need to write programs? • Should we not just compose (dynamically – software ’robots’) from services as components (like other branches of engineering)? • Will social / legal changes ever catch up with technology? ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 73

STRUCTURE Introduction – Who? The Pervasiveness of ICT A Short History of ICT CLOUD Computing Challenges Conclusion ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 74

Conclusion Being ever optimistic, I believe these challenges will be met. But has some machine passed the Turing test and nobody noticed? (2012 was the year commemorating Alan Turing birth centenary) ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 75

Prof. Keith G Jeffery CEng, CITP, FBCS, FGS, HFICS Director, International IT Strategy STFC Rutherford Appleton Laboratory keith. jeffery@stfc. ac. uk Acknowledgements to Lutz Schubert, HLRS, Stuttgart; rapporteur EC CLOUDs expert Group ©STFC/Keith G Jeffery The Challenges in ICT: Debunking the Hype 20130127 76