Скачать презентацию status Emmanuel Cecchet c-jdbc objectweb org JOn AS developer Скачать презентацию status Emmanuel Cecchet c-jdbc objectweb org JOn AS developer

6405627b18cf76ed5e3bac7841ffb7dc.ppt

  • Количество слайдов: 50

status Emmanuel Cecchet c-jdbc@objectweb. org JOn. AS developer workshop – http: //www. objectweb. org status Emmanuel Cecchet [email protected] org JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 1 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 2 - 23/02/2004

Motivations J 2 EE performance scalability bounded by database performance Database tier must be Motivations J 2 EE performance scalability bounded by database performance Database tier must be – scalable – fault tolerant (high availability + failover) – without modifying the client application – using open source databases – on commodity hardware JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 3 - 23/02/2004

How do we want to use ? • end-to-end open source solution for large How do we want to use ? • end-to-end open source solution for large scale J 2 EE clustered application servers • cost-effective solution for both small and large configurations Apache Internet My. SQL JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 4 - 23/02/2004

Redundant Array of Inexepensive Databases RAIDb controller – gives the view of a single Redundant Array of Inexepensive Databases RAIDb controller – gives the view of a single database to the client – balance the load on the database backends RAIDb levels – – RAIDb-0: full partitioning RAIDb-1: full mirroring RAIDb-2: partial replication composition possible JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 5 - 23/02/2004

C-JDBC Middleware implementing RAIDb Two components – generic JDBC 2. 0 driver (C-JDBC driver) C-JDBC Middleware implementing RAIDb Two components – generic JDBC 2. 0 driver (C-JDBC driver) – C-JDBC Controller provides – – performance scalability high availability failover caching, logging, monitoring, … Supports heterogeneous databases JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 6 - 23/02/2004

Using JOn. AS with C-JDBC • Without C-JDBC Database JDBC Driver JOn. AS developer Using JOn. AS with C-JDBC • Without C-JDBC Database JDBC Driver JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 7 - 23/02/2004

Using JOn. AS with C-JDBC Update the datasource to use the C-JDBC driver JOn. Using JOn. AS with C-JDBC Update the datasource to use the C-JDBC driver JOn. AS Virtual Database Controller JDBC Driver Cache Pooling JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org Backend 8 - 23/02/2004

What C-JDBC offers No application modification No JOn. AS modification No database modification Load What C-JDBC offers No application modification No JOn. AS modification No database modification Load balancing Virtual Database Controller Driver JDBC Driver Cache Pooling JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org Backend recovery 9 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 10 - 23/02/2004

Inside the Controller Sockets JMX JOn. AS developer workshop – http: //www. objectweb. org Inside the Controller Sockets JMX JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 11 - 23/02/2004

Fault tolerant recovery log JOn. AS developer workshop – http: //www. objectweb. org - Fault tolerant recovery log JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 12 - 23/02/2004

Cascading controller JOn. AS developer workshop – http: //www. objectweb. org - c-jdbc@objectweb. org Cascading controller JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 13 - 23/02/2004

Heterogeneity support application already written for a specific [commercial] database user defined rules for Heterogeneity support application already written for a specific [commercial] database user defined rules for on-the-fly query rewriting to execute on heterogeneous backends JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 14 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 15 - 23/02/2004

Query caching status Cache contains a list of SQL->Result. Set Policy defined by query. Query caching status Cache contains a list of SQL->Result. Set Policy defined by query. Pattern->Policy 3 policies – Eager. Caching: variable granularities for invalidations – Relaxed. Caching: invalidations based on timeout – No. Caching: never cached RUBi. S bidding mix with 450 clients Throughput (rq/min) No cache Coherent cache Relaxed cache 3892 4184 4215 Avg response time 801 ms 284 ms 134 ms Database CPU load 100% 85% 20% C-JDBC CPU load - 15% 7% JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 16 - 23/02/2004

RUBi. S - Tomcat without C-JDBC caching Throughput: 3900 pages/min 1 Database 100% cpu RUBi. S - Tomcat without C-JDBC caching Throughput: 3900 pages/min 1 Database 100% cpu Tomcat ~50% cpu JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 17 - 23/02/2004

RUBi. S - Tomcat with C-JDBC relaxed caching Throughput: 4200 pages/min Tomcat ~55% cpu RUBi. S - Tomcat with C-JDBC relaxed caching Throughput: 4200 pages/min Tomcat ~55% cpu 1 Database ~20% cpu C-JDBC <10% cpu JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 18 - 23/02/2004

JOn. AS and C-JDBC query caching RUBi. S Throughput (req/min) No cache Response time JOn. AS and C-JDBC query caching RUBi. S Throughput (req/min) No cache Response time (ms) No cache Servlet 3890 4140 744 310 JOn. AS SF CMP 1. 1 2947 3418 2929 2045 JOn. AS CMP 2. 0 2178 2450 6372 4630 JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 19 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 20 - 23/02/2004

Horizontal scalability JGroups for controller synchronization Groups messages for writes only JOn. AS developer Horizontal scalability JGroups for controller synchronization Groups messages for writes only JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 21 - 23/02/2004

Horizontal scalability Centralized write approach issues Issues with transactions assigned to connections JOn. AS Horizontal scalability Centralized write approach issues Issues with transactions assigned to connections JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 22 - 23/02/2004

Horizontal scalability General case for a write query – 3 multicast + 2 n Horizontal scalability General case for a write query – 3 multicast + 2 n unicast JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 23 - 23/02/2004

Horizontal scalability Solution: No backend sharing – 1 multicast + n unicast [+ 1 Horizontal scalability Solution: No backend sharing – 1 multicast + n unicast [+ 1 multicast] JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 24 - 23/02/2004

Horizontal scalability Issues with JGroups – resources needed by a channel – instability of Horizontal scalability Issues with JGroups – resources needed by a channel – instability of throughput with UDP – performance scalability TCP better than UDP but – unable to disable reliability on top of TCP – unable to disable garbage collection – ordering implementation is sub-optimal Need for a new group communication layer optimized for cluster JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 25 - 23/02/2004

Horizontal scalability JGroups performance on UDP/Fast. Ethernet JOn. AS developer workshop – http: //www. Horizontal scalability JGroups performance on UDP/Fast. Ethernet JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 26 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 27 - 23/02/2004

Design choices JMX for both administration and monitoring – JMX server is MX 4 Design choices JMX for both administration and monitoring – JMX server is MX 4 J HTTP console for basic features but not suited for monitoring Swing console for monitoring – uses JMX/RMI connector Currently working on a scalable monitoring infrastructure JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 28 - 23/02/2004

Online Monitoring JOn. AS developer workshop – http: //www. objectweb. org - c-jdbc@objectweb. org Online Monitoring JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 29 - 23/02/2004

Viewing logs remotely JOn. AS developer workshop – http: //www. objectweb. org - c-jdbc@objectweb. Viewing logs remotely JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 30 - 23/02/2004

Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop Outline Overview Advanced concepts Query caching Horizontal scalability Monitoring Perspectives JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 31 - 23/02/2004

Current status C-JDBC 1. 0 rc 2 release Web site: >100. 000 hits per Current status C-JDBC 1. 0 rc 2 release Web site: >100. 000 hits per month Downloads: >1200 per month Mailing lists: 134 subscribers Ready for production environments JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 32 - 23/02/2004

Ongoing work Scalable group communication middleware for clusters – Needed for horizontal scalability – Ongoing work Scalable group communication middleware for clusters – Needed for horizontal scalability – Decouple application streams from network channels Monitoring – Continuous monitoring stored in repositories – Generic monitoring console for both online monitoring and monitoring history browsing Administration – Fully featured administration console – Graphical configuration and deployment of centralized/distributed backeds and controllers (offline/online) – Dynamic reconfiguration Misc. – Automated Load testing – RPM packaging (JPackage) JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 33 - 23/02/2004

Questions & Answers _____ Thanks to all users and contributors. . . http: //c-jdbc. Questions & Answers _____ Thanks to all users and contributors. . . http: //c-jdbc. objectweb. org JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 34 - 23/02/2004

Virtual Database gives the view of a single database establishes the mapping between the Virtual Database gives the view of a single database establishes the mapping between the database name used by the application and the backend specific settings backends can be added and removed dynamically configured using an XML configuration file JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 35 - 23/02/2004

TPC-W Performance (Amazon. com) JOn. AS developer workshop – http: //www. objectweb. org - TPC-W Performance (Amazon. com) JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 36 - 23/02/2004

Outline Overview Fault tolerance Performance New features Roadmap JOn. AS developer workshop – http: Outline Overview Fault tolerance Performance New features Roadmap JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 37 - 23/02/2004

Building the initial state Octopus is an ETL tool Use Octopus to store a Building the initial state Octopus is an ETL tool Use Octopus to store a dump of the initial database state JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 38 - 23/02/2004

Journaling Backend is enabled All database updates are logged (SQL statement, user, transaction, …) Journaling Backend is enabled All database updates are logged (SQL statement, user, transaction, …) JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 39 - 23/02/2004

Adding backend on the fly Add new backends while system online Restore dump corresponding Adding backend on the fly Add new backends while system online Restore dump corresponding to initial checkpoint with Octopus JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 40 - 23/02/2004

Synchronizing backends Replay updates from the log JOn. AS developer workshop – http: //www. Synchronizing backends Replay updates from the log JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 41 - 23/02/2004

Expanded Cluster Enable backends when done JOn. AS developer workshop – http: //www. objectweb. Expanded Cluster Enable backends when done JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 42 - 23/02/2004

Handling a backend failure A node fails! Automatically disabled but should be fixed or Handling a backend failure A node fails! Automatically disabled but should be fixed or changed by administrator JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 43 - 23/02/2004

Restoring a backend Restore latest dump with Octopus JOn. AS developer workshop – http: Restoring a backend Restore latest dump with Octopus JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 44 - 23/02/2004

Re-synchronization Replay missing updates from log JOn. AS developer workshop – http: //www. objectweb. Re-synchronization Replay missing updates from log JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 45 - 23/02/2004

Healed Cluster Re-enable backend when done JOn. AS developer workshop – http: //www. objectweb. Healed Cluster Re-enable backend when done JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 46 - 23/02/2004

Prototype C-JDBC Management Framework Shared design JOn. AS developer workshop – http: //www. objectweb. Prototype C-JDBC Management Framework Shared design JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 47 - 23/02/2004

Making new checkpoints Disable one backend to have a coherent snapshot Mark the new Making new checkpoints Disable one backend to have a coherent snapshot Mark the new checkpoint entry in the log Use Octopus to store the dump JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 48 - 23/02/2004

Making new checkpoints Replay missing updates from log JOn. AS developer workshop – http: Making new checkpoints Replay missing updates from log JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 49 - 23/02/2004

Making new checkpoints Re-enable backend when done JOn. AS developer workshop – http: //www. Making new checkpoints Re-enable backend when done JOn. AS developer workshop – http: //www. objectweb. org - [email protected] org 50 - 23/02/2004