Fermilab Run II Database Requirements f

Зарегистрируйтесь, чтобы просмотреть полный документ!

Fermilab Run II Database Requirements • • f Online databases are maintained at each experiment and are critical for data taking. Offline databases are maintained in the Feynman Computing Center and are critical for data processing and analysis. High Availability for both online and offline database systems is required. Database Applications Overview – – • Detector and physics data • Detector Calibration • Trigger lists • Data Luminosity • Detector Slow Controls • Run and Run Quality information Data Handling (The SAM Database) • Physics Metadata • File catalog • File replica management • Processing information Database storage growth is shown in the accompanying charts (D 0 left, CDF right). Fermilab Database Experience in Run II

Table Partitioning • • • f Partitioning has been implemented for very large table(s) in the database. D 0 uses a partitioned Events table with 50 M events in each partition. Each partition is stored in its own tablespace and corresponding indexes are also partitioned and stored in their own tablespaces. Partitioning improves Query Optimization and Backup Performance Over 1 billion events are distributed over 24 partitions and a new partition is started about once a month. Data Base Monitoring: • Oracle in Run II Replication • • Replication is used to share data in a large user. CDF has the same database structure for online and offline databases. Oracle’s asynchronous replication is used to refresh offline tables from online tables periodically. One replica is used by Farm Users and the other is used by CAF and other READ ONLY users. A key feature of CDF replication is Fail-Over from one replica to another for high reliability. CDF is planning to migrate to Oracle Streams replication available from version 9. 2. x release soon. • • • 9. 2. 0. 4 On-line DB (cdfonprd) – – – Node up and down, Database Listener down, Intelligent Agent Number of storage extents and space usage Database Alerts – Db down , file corruption Number of concurrent sessions, CPU usage, Memory usage Hit ratios for Library, Buffer Cache and other database resources. Basic 4 Applications Basic Monitoring is done using Oracle Enterprise Manager (OEM, by Oracle Corp) and TOOLMAN, an in-house developed tool. OEM monitors the following: 4 Applications On-line Users • DFC 8. 7. 1. 4 Off-line DB (cdfofprd) Farm Users Basic 9. 2. 0. 3 Off-line DB (cdfrep 01) Failover for Read Service ONLY CAF and Others TOOLMAN – Provides an alternative method to OEM for monitoring Oracle databases. – Can be customized in several ways for the machine and databases it monitors. Fermilab Database Experience in Run II

Run II Database Access • • f For D 0, only a subset of the online information was transferred to the offline database (Lower left). All access to the D 0 offline database was through the Calibration DB server (DAN, upper right) or Data Handling server (SAM). CDF employed Basic Oracle replication to transfer all online database information to offline databases (See poster ‘Oracle in Run II’). Fro. Ntier is a web-based, highly scalable, approach which is being developed for CDF to provide high performance database access to read-only information (Lower right). http: //whcdf 03. fnal. gov/ntier-wiki DØ Offline Caching Server: DAN (Database Access Network) • CORBA interface to Client apps • Memory (L 1) and Disk (L 2) caching • Connection management to Database • Server has common code base with SAM DB server DØ Online to Offline Database Copy ONLINE L 1 C++ Header and Stubs Level 3 Nodes -- NT MFC Entry Alarm GUI HDB SIG EVNT Datalogger , DLSAM PROCESS META DATA Alarm SRV mdata REPO LUM SERVER COOR TRIG LUM TRIG DL PROC CALIB PROCESS run ctl ONLINE TO OFFLINE CONNECTION Caching ENSTORE OFFLINE DB SAM DB ETC MON ON CAL Fro. Ntier Client API Library Local Host DATA FILE DL Client HTTP Offline Host -- Sun ONLINE DB FF Online Host -- DEC CR FE NODES Web Entry EXAMINE Front End -- 68 k L 2 TRIG CTL Fro. Ntier Overview OFFLINE DD LEVEL 3 FILTER NODES Read-only DB access OFF LINE CAL LUM run ctl CDF Persistent Object Templates (Java) Squid Proxy/Caching Server HTTP XML Server Descriptors Fro. Ntier Server Fro. Ntier Servlet running under Tomcat JDBC DDL for Table Descriptions Fermilab Database Experience in Run II Database (or other persistency service)

Run II Database Performance and Monitoring f Database Monitoring is a crucial component of our Database Operation. DBS Monitor • DBS Monitor is used for collecting information on database access and presenting it through a web interface • Project Goal: Common DB Monitor Overview tools for Application Monitoring DAN Server CDF Client CDF DAN Server • Information Generation Applications DAN Logging Server is Experiment Specific • The Collector gathers CDF or D 0 and parses data • The Archiver uses a My. SQL Repository • Plotting tools use Java. Free. Chart • Histogramming uses JAIDA • Admin and automation scripts are included. • http: //dbsmon. fnal. gov Number of connections per minute for CDF Top CPU users on CDF Database Applications over an 8 hour interval Average duration time for Database connections for CDF. D 0 Sam Servers query counts over 24 hours interval Number of queries per hour for D 0 Farm and Non-Farm servers Query counts per week for D 0 SAM station server Fermilab Database Experience in Run II

Скачать презентацию Fermilab Run II Database Requirements f

5d1cf0a290548bf0d2897c92a5ae4d69.ppt

Количество слайдов: 4