Скачать презентацию grid Mat rix Tec hno log y Grids Скачать презентацию grid Mat rix Tec hno log y Grids

e87ad6ba35ba6b1836e7c4ce6f8abc57.ppt

  • Количество слайдов: 19

grid Mat rix Tec hno log y Grids in EDA Software Development Tom Grotton grid Mat rix Tec hno log y Grids in EDA Software Development Tom Grotton Cadence Design Systems, Inc. Director, IT Server Farm Initiative CADENCE CONFIDENTIAL 1

Corporate Needs • Need more computing capacity – Products are becoming more complex and Corporate Needs • Need more computing capacity – Products are becoming more complex and taking additional cycles to build and test. – Products need to be supported on multiple platforms (OS and version) – Process and machine utilization measurements are inadequate to effectively plan and manage capital equipment requirements. We continue to add more and more servers to meet this unplanned demand. • Need a more reliable hardware, storage and network – Nightly runs of product builds and tests fail too often due to machine or infrastructure problems – To decrease the dependency on individual hardware resources • Need more cost-effective IT contribution – System administration costs are ever increasing as more and more hardware is added. According to Gartner and based on our experience, TCO costs are being primarily driven by support costs and not capital. For every 20 cents spent on equipment, 80 cents is spent supporting it. 2

How We Address these Needs • Manage Cost (Total Cost of Ownership) • Agility How We Address these Needs • Manage Cost (Total Cost of Ownership) • Agility (Adaptive IT infrastructure) • Increase Quality (Product and Service) • Virtualization (Computing, Networking, Storage, Security, File systems, Monitoring, and Common Process Architecture) • Federation (Brokering, Provisioning, Resource Sharing, Globalization) • Automation (Process, Workflow and Ease-of-use) 3

Capitalizing On Our Expertise • The key point of coordination, information exchange and collaboration Capitalizing On Our Expertise • The key point of coordination, information exchange and collaboration for those involved in largescale grid projects in the US, Europe, Canada and Asia-Pacific is the Global Grid Forum (GGF). GGF’s platinum and silver sponsors are Compaq, HP, IBM, Sun, Microsoft, Platform Computing, AVAKI and Entropia. • As a member of the GGF, Cadence chairs and leads the – GGF Common Use Case Model Architecture Working Group, formerly known as NPi –This Working Group is responsible for defining a particular lightweight, high-level architecture for distributed computing management. Creating an overall Reference Model for advanced distributed computing management – GGF Job Submission Description Language working group –This Working Group is to define a standard Job Submission Description Language (JSDL) for describing a computational 'job' and its required execution environment for submission to a Grid. This language would be used to construct a document that would encapsulate all of the information needed by a Distributed Resource Management (DRM) system or a job submission system on a Grid, such as a Grid scheduler, to place a job in its required environment for execution. – GGF Grid Policy Architecture working group –This Working Group is developing the requirements and architecture for interoperable policy management, and for how they are described, evaluated, stored, managed, distributed, and enforced. Since policies may also be associated with instrumentation to provide feedback into evaluating the effectiveness of the policies, the requirements and architecture will also provide a framework for associating the appropriate metrics and instrumentation with the policies. – Is a seated member of the GGF DRMAA (Distributed Resource Management Application Architecture) working group. Reference “The 451 Group” Grids 2004 4

Grids • Definition: Grid computing refers to a network architecture designed for large -scale Grids • Definition: Grid computing refers to a network architecture designed for large -scale dynamic sharing computing resources. A grid works by taking the responsibility for input/output requests for storage, memory, processing, and/or communications resources away from individual machines and instead moves that responsibility to a network grid that searches for available resources to handle resource requests. • Usage: A growing number of companies are considering deploying Grid Technology or have deployed it. These companies are finding the Total-Costof-Ownership to deployment, management and support Grid Technology to be overly costly. • EDA is the slowest industry to adopt massive parallel processing grids. – Partly because of the industry itself and partly because of the nature of the products. • Industries that are using Grid Technology are Life Science, Engineering, Commerce, Financial, Electronic Design Automation, Digital content creation, Testing, Server/Storage, Batch processing, Decision support/data mining. 5

Grid Industry • 2005 grid Hardware and Software solutions: $18 billion • 2005 grid Grid Industry • 2005 grid Hardware and Software solutions: $18 billion • 2005 grid Software solutions: $4 billion – There are several middleware DRM (Distributed Resource Management) companies: Platform, Sun, IBM, etc. – There is an increase in next-generation DRM products that are no-cost. – There also products that have their DRM layer, Grid. Iron, Data. Synapse. • “Enter Utility Computing - With the emergence of this universal computing grid, there will be a need for a new breed of utility provider. Although companies and individuals will retain some local computing capabilities, there will be a growing opportunity for trusted neutral parties to operate and manage shared resources within the grid. ” (ASPnews 9/02) • Projected growth of the software grid technology market is 25% until 2007, then it jumps up considerably due to the larger number of industries that will start to use grid technology. EDA is one of these. Data sources: http: //www. gridpartners. com, http: //www. idc. com, http: //www. econstrat. org/, http: //gridcomputingplanet. com, http: //www. bloor-research. com/, http: //www. gridforum. org/ 6

Grid Technology – Evolution • Compute Grids share excess PC compute cycles to provide Grid Technology – Evolution • Compute Grids share excess PC compute cycles to provide a highperformance, low-cost computing environment. Their application is scientific research and engineering. – Example, [email protected] – Market leaders are Entropia, Sun, Platform, and United Devices. – The EDA industry deploys this kind of grid, but in a UNIX environment and not across the internet. • Information Grids are distributed architecture designed for largescale dynamic sharing of commercial and technical applications, data, and compute power within the enterprise or across multiple external organizations. They are used in specialize industries, such as, pharmaceutical, biotech, medical, financial services, oil and gas exploration. – Example, IBM/Upenn Mammography research project. – Market leaders are AVAKI, Platform, IBM, Sun, and HP. Data source: The Grid Report, Bloor Research, November 2002 7

Grid Technology – Evolution • Service Grids make use of the intersection of grid Grid Technology – Evolution • Service Grids make use of the intersection of grid and web services technology concepts. As such, the service grid provides the underlying architecture for Utility Computing model. Sun describes a Service Grid as collections of services brought together from across the network. Service Grid are composed of services that you need at a particular time – mail, stock feeds, word-processing, or flight information, etc. The idea of the service grid is that each piece comes from a source that specializes in an external function. – Examples, Sun ONE and HP’s Utility Data Center – OGSA. – Market leaders are HP, IBM and Sun. • Intelligent Grids are self-managing utility architectures that go beyond the bounds of department and enterprise – massive extended enterprises – encompassing partners, suppliers, and customers. Currently, intelligent grids use homogeneous resources. – Examples of intelligent grids are IBM’s e. Liza – IBM Workload Manager, HP’s Planetary Computing project, and Sun’s Management Center, “N 1. ” – Market leaders are IBM, Sun, and HP. Data source: The Grid Report, Bloor Research, November 2002 8

Grid Technology – What to do? • Utility Grids are to some extent conceptual Grid Technology – What to do? • Utility Grids are to some extent conceptual to the grid industry. They use the power utility model, where there are main generation grids and peak request or overflow local grids. Utility grids takes the position that computing resources and the DRM layer are commodities, because these resources can be provisioned on demand. Utility grids are capable of managing higher-level functions, such as, security, data access and transformation in a massively parallel environment across the enterprise. • According to several industry sources, Compute, Information, and Service grids will likely converge into Intelligent grids in the future. The fifth grid type – Utility Grids or Corporate Utility Computing Grids – are evolving and will likely replace the Service Grid and Intelligent Grid in time. This grid is designed to make heterogeneous computing power and heterogeneous resources available on-demand. 9

What does the future look like? Utility Grids Peak Grid Server Server Workstation (Trusted What does the future look like? Utility Grids Peak Grid Server Server Workstation (Trusted by group) Data Grid (Trusted by group) Server Trusted Grid (Pooled resources, trusted by all) Server Workstation Data Grid Server Server Workstation Peak Grid (Trusted by group) 10

Cadence Grid • Inside Cadence we are deploying Grid Technology today! So far, it Cadence Grid • Inside Cadence we are deploying Grid Technology today! So far, it has resulted in an average of 67% reduction in process job run-time for R&D and Services business units worldwide. • We have around 3000 CPUs in our grid. Next year we are looking at doubling this! • R&D and Services business units require IT provide grid control, monitoring, statusing, error/alert reporting, performance reporting and job optimizing for massively parallel processes across hundreds and thousands of heterogeneous computer resources, ranging from Microsoft Windows, Linux and UNIX (AIX, HP-UX, Solaris). • Cadence has developed and deployed patented grid technology. • As most of you have done or are trying to do, we have developed an abstraction done layer that performs Grid Automated Process Control allowing a Developer, CM, PV, Flow, Support engineer to interface with complex job definitions in a natural manner. Today, LSF is the Distributed Resource Management (DRM) layer and networked to this are Data Grids, which make computer resources, networking and storage virtually invisible. • Results - Significant reduction of Total-Cost-of-Ownership by reducing cost of Results adoption and deployment, job run-time, costs in support, capital equipment, personnel and the technical knowledge. 11

Yesterday’s Process… Harness Scripts (fire up DRM - bsub) Run Job or Tool Hardware Yesterday’s Process… Harness Scripts (fire up DRM - bsub) Run Job or Tool Hardware (Servers/Workstations) 12 Logs & Email Reporting (Some Web Reporting)

Today’s Process… Engineer submits a grid-enabled job Intelligent Job Control including web-based monitoring and Today’s Process… Engineer submits a grid-enabled job Intelligent Job Control including web-based monitoring and job control DRM Hardware (Servers/Workstations) Resource Monitoring & Alert Handling (DRM, system and system configuration) 13 This is the only interface that is seen!

What is Needed… Standard means to grid-enable EDA Tool by thread Engineer submits a What is Needed… Standard means to grid-enable EDA Tool by thread Engineer submits a grid-enabled job Intelligent Job Control including web-based monitoring and job control DRM Hardware (Servers/Workstations) Resource Monitoring & Alert Handling (DRM, system and system configuration) 14

Where are we? Site Grid Health Local Compute Grid Health (Commission & Platform) t Where are we? Site Grid Health Local Compute Grid Health (Commission & Platform) t y t en s log rces en o m es n e m u ag roc Tech so ge n e P a e an Ma ous s. R id ag r u M u y ng nc gene ent G eneo ce e a La sp nd ero lig erog al l e ork atur ep Het nte Het I W D N Job Definition Utility Grid Process Web Server (Process Control) Multi-Site/Cluster Job Control Custom & Integrated Presentation Job Priorities Workspace Management (RCS, Clear. Case, CVS, VNC, Extendable) Metrics Self-Healing File Systems Workspaces Data Grids RCS/CVS Clear. Case Res. Control 15 Job Execution Platform Definition Dynamic Web Reporting (Customizable) Process Metrics Job Statistics Queue Definition Optimization Alert Handling (GRS) Job Monitoring Host Groups and Host Partitions Pool Configuration Job Distribution Scripts and Commands Commission and Monitoring DRM Grid Task Execution

Site Grid Status 16 Site Grid Status 16

Horizons Cycle-times reduced 67% Examples: - 4 Hrs to 17 Mins - 13 Hrs Horizons Cycle-times reduced 67% Examples: - 4 Hrs to 17 Mins - 13 Hrs to 2 Hrs - 6 Hrs to 30 Mins - 24 Hrs to 3. 5 Hrs Corporate Grid (2005) Site Grid (2003) Multiple site grids sharing resources and data intelligently across the WAN. - >50% utilization Compute Farms (2002) Enables shared system usage across departments Increase Utilization from >11% 17 Combines compute farms to facilitate the sharing of capital resources across multiple departments, projects, and groups Increase server utilization from >30%

Q&A 18 Q&A 18

19 19