HPC in the Cloud Impact on Future Enterprise

HPC in the Cloud Impact on Future Enterprise Architectures Dr. Tirumale K (TK ) Ramesh The Boeing Company NCOIC Cloud Computing Working Group tirumale. k. ramesh@boeing. com NCOIC Panel NAVAIR Military Aviation Architecture Conference September 22, 2010 Approved for Public Release Distribution Unlimited NCOIC-Def. Daily-TKR 20100917

Cloud Computing & High Performance Computing Definitions and Synergies q Cloud Computing q A paid-for service model for delivery and consumption of computing allowing organizations to access to applications, deployment and infrastructure platforms as a service provided on-demand q Infrastructure services which is more of a la carte with control for infrastructure provisioning is not what Cloud offers at least for now q What Cloud provides is an on-demand infrastructure as a whole (Infrastructure as a Service-Iaa. S) via set of pre-packaged menu options to choose the infrastructure from the providers q High Performance Computing (HPC) q A computational model in which high performance is delivered to support highly compute-intensive calculations mainly by using parallel algorithms with tightly-coupled, data-intensive inter-process communication, running on hundreds or even thousands of processors or cores. q HPC in the Cloud- Setting up a configuration of distributed cloud resources as a tightly coupled HPC engine to meet requirements of locality of reference, communication bandwidth and latency from node to node 2

Cloud Computing Impact on Future Enterprise Architectures (EA) q It is predicted that in next 10+ years as Cloud Computing(CC) grows and matures, it impacts and may redefine and advance the EA of an organization q There is even more interest of EA to global organizations to sustain global competition and support for interoperability in a global ecosystem environment q Users would be able to buy cost-effective mission-critical processes as cloud services and general buying habits of organizations may change from “buy all once” mindset to “buy breed of needed services” demands Cloud Services Future Enterprise Architectures Adaptable and scalable architecture to support changing buying habits and support for ecosystem of cloud computing Organizations to re-examine organizational security policies demands Evolvable Architectures Organizations to have high level control over data & network periodically evaluate vendor technologies and risk management including COTS technologies risks Organizations to engage pro-actively in architecture implementations in cloud. For example designs that support integrating multiple CC interfaces 3

Cloud Service & Delivery Model Trade-offs Basis Criteria's: Future Architectures, Security, Interoperability and Reliability User Responsible for Security Iaa. S better Interoperability Paa. S Reliability Saa. S better Adaptation to Future Architectures Vendor Responsible for Security better Trade-offs Trade-off from EA Centric Trade-off from Secured & global ops Centric Proprietary Specifications Private Interoperability Hybrid Reliability Public Adaptation to Future Architectures Open Standard Specifications security concerns better Trade-off from EA Centric Trade-off from Secured & global ops centric Paa. S (Iaa. S high cost) Paa. S Hybrid (Open arch) Hybrid better 4 With balanced options, Security is still an issue for global operations

Cloud in Open Architecture Global Environment § Most of cloud providers expose APIs but have specific implementations that do not permit interoperability Enables open architecture Public – Ope Cloud Virtualization Cloud Providers Service. Oriented Arch Routing Switching Firewalls Application Delivery Private Cloud § § Hybrid Cloud Open Standards Provides global interoperability and portability Some Open Standards under development including OGF’s Open Cloud Computing Interface. OCC (Open Cloud Consortium) is working on obtaining consensus on open standards and best practices The Cloud Security Alliance is a non-profit organization that is working on promoting the use of best practices for providing security assurance 5

High Performance Computing (HPC) in the Cloud- Are there Cost Savings? q HPC in the Cloud offers the possibility of creating applications that would otherwise be prohibitive in HPC infrastructure costs to many HPC application developers q Transforming HPC to Cloud via virtualization of cloud infrastructure and adapting to HPC usage patterns of cloud resources q Primary determinants of HPC performance are data locality and locality of reference, and communications latency and bandwidth q Currently most of the HPC providers cannot guarantee these but possible in the near future q Today, nobody would build an HPC cloud for great grand challenge scientific applications as the business is not profitable q HPC users are far too small and does not take full advantage of cloud scaling q As HPC usage moves more from scientific users to main stream enterprise users, HPC cloud will become more economical 6

High Performance Computing (HPC) in the Cloud for Future EA q HPC benefits in future enterprise global operations q As organizations in future become smarter in global collaborative work using 3 D web technologies and other collaborative computing technologies , it creates an opportunity for HPC to tap into much broader main stream enterprise user base and instantiate an architecture view in future enterprise architectures q HPC in the Cloud migration path for future EA q First migrate via Saa. S by tapping some acceleration HPC applications as a Cloud Service to use with traditional HPC data centers (some FPGA vendors are interested in this) q Better Security in Saa. S q Better Adaptation to Future Enterprise Architectures to support HPC for main stream users q Slowly migrate to Iaa. S (full provisioning of HPC resources in Cloud) q So, is it all good news to migrate HPC to Cloud to save cost? No, not yet. q More learning curve and initial efforts needed q Initial high cost leading to lot of wait and watch now 7

High Performance Computing (HPC) in the Cloud- What are the Gaps? Gaps for Migration of HPC to Cloud Dynamic and adaptive workload management to fully utilize cloud resources Cloud Fabric Scalability: Intelligent management of resources from network all the way up to processor/multi-core level( to gain fine-grain multithreading parallelism) Cloud Computing Open Architecture to leverage COTS HPC solutions and dramatically decrease the cost for doing HPC – a clear lift off opportunity for HPC to enter main stream Need virtualization of accelerators on the cloud to migrate any Hybrid HPC to cloud HPC (Traditional P & D computing with high throughput HW and optimized SW stack) Cloud Computing More compute intensive applications today demands HPC acceleration via CPU/GPU, CPU/FPGA q Some HPC in Cloud offerings today q. Penguin’s HPC-as-a Service. TM Open Question: Is the Cloud robust enough to handle the massive computing cycles and very high system throughput needed for an effective HPC environment? 8

Cloud in Open Architecture Environment Impact to HPC in the Cloud q Security and Legal/Legislation Issues Impact q Potential exposure of sensitive and valuable information processed in HPC in the Cloud q HPC have well developed security layers in a single HPC data center environment q Security providers are looking to enhance the security of cloud offerings such as cloud-based intrusion prevention which allows users to monitor network activity for malicious behavior q This requires integrating traditional HPC security layers with higher level cloud security layer for HPC offerings in cloud q There is no doubt that Cloud Computing is still in its infancy and many offerings remain insecure or poorly documented q Need to put more trust in security suppliers q Private cloud architectures may be holding well at a good level in the enterprise, but open standards to facilitate managing across global competing platforms is still far away for global operations q Currently opportunities are opening up for HPC in private Clouds 9

Secured Cloud Computing for Global Operations- Are We There Yet? q I say it is a Mixed Bag q Security is a major concern q Some cloud opponents argue that customer data is more secure when managed internally and not throw it on cloud q Some cloud supporters argue that Cloud Providers have a strong incentive to maintain trust so that they can provide a higher level of security in the near future q Expectations going forward q Most organizations experimenting with small implementations as Private and Hybrid Cloud architectures q Private Cloud may move into the mainstream well before truly global standards may even come into play q Eventually there will be one standard that some experts argue may even take longer for standardization (10+ years) 10

Final Thoughts People (Stakeholders) Processes Cloud (Virtual Connector al y hnic rabilit ec pe T ro inte Smart Network (technology enabler) Information/ Knowledge/ EA View as a Cloud Fabric Semantic interoperability Organizational interoperability q Delivering Secured Cloud Computing for Global Operations is not the responsibility of one single organization q We need a well established framework for collaborations of all stakeholders (Industries, Consumers, Government and Academia) q Fundamentally Cloud needs to be a “Virtual Connector” to achieve “Net centricity” q Future Enterprise divides into community of people, processes, information, knowledge and services interconnected by a smarter network to achieve maximum benefit of its resources with reduced cost 11

Final Thoughts q Commercial and Military Aviation Architecture need to align with the global enterprise architecture in order to support today’s global business using key technology pointers q Recommended migration plan is to first apply So. A in Aviation Architectures and transform them to cloud offerings q Take the architectural principle of SOA and applying it to an infrastructure offered by cloud q Finally integrate into HPC usage in the long run within a comprehensive enterprise architectural framework to support global interoperability q Network Technology is only an enabler for future aviation architectures implemented on Cloud but network itself is not a major problem q NCOIC can help in providing the necessary guidance in the proliferation of Cloud Computing in support of future enterprise open architectures that evolves and uses emerging technologies rendered on the cloud that can be accessed via cloud services Cloud computing can provide highly cost effective implementations of the architecture but Security is still the main issue 12

Thank You! 13