Belle Grid Computing in Korea University 1 Belle

Belle Grid Computing in Korea University 1 Belle Grid Computing in Korea University Soohyung Lee, Department of Physics, Korea University High Performance Computing Asia 2007 – Particle & Nuclear Physics Session Seoul, Korea September 12, 2007 High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 2 Contents 1. Introduction • Grid Computing • Elementary Particle Physics Laboratory • KEK and Belle Experiment 2. Architecture • Physical Site Overview • Logical Site Overview • Role of Each Node • World-wide Belle Grid System • Site Verification 4. Current Status • Current Problems • Reason of Problems 5. Future Applications • Belle Analysis Framework • New BASF 6. Summary 3. Construction and Verification • Pre-Steps for Installation • Installation Procedures • Certificates High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 3 Introduction • Grid Computing • Elementary Particle Physics Laboratory • KEK and Belle Experiment High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 4 • Grid Computing § One of the parallel computing system. § A Distributed System for heavy calculation. § Grid systems are already constructed and used in high energy physics experiments by CERN, KEK and FNAL. § Grid Computing is also hot issue in high performance computing. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 5 • Elementary Particle Physics Laboratory § We study about experiments for high energy physics. § Analysis, hardwares and softwares related to experiments. § Please refer to http: //hep. korea. ac. kr for detail information. Seoul, Korea Asan Science Bldg, Korea Univ. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 6 • KEK and Belle Experiment § KEK: High Energy Accelerator Research Organization in Japan. § Belle Experiment: Observation of CPV in B-meson Decays and better understanding of CPV Phenomena. B-meson e- e+ B-meson High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 7 Architecture • • Physical Site Overview Logical Site Overview Role of Each Node World-wide Belle Grid System High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 8 • Physical Site Overview kulcg 01. ku. koren 21. net ~ kulcg 05. ku. koren 21. net Intel Pentium 4 3. 0 Ghz * 2 Samsung DDR-2 512 MB * 2 Seagate SATA-II HDD 250 GB kulcg 01. ku. koren 21. net kulcg 06. ku. koren 21. net kulcg 02. ku. koren 21. net kulcg 06. ku. koren 21. net Intel Pentium 3 800 Mhz * 2 Samsung SDRAM 512 MB * 2 Maxtor SCSI HDD 160 GB kulcg 03. ku. koren 21. net kulcg 04. ku. koren 21. net kulcg 05. ku. koren 21. net High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 9 • Logical Site Overview MON UI CE SITE_BDII PX RB SE WN Firewall kulcg 06. ku. koren 21. net (KOrea advanced Research Network) Firewall kulcg 05. ku. koren 21. net HUB KOREN Firewall kulcg 04. ku. koren 21. net Firewall kulcg 03. ku. koren 21. net Firewall kulcg 02. ku. koren 21. net Internet Firewall kulcg 01. ku. koren 21. net High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 10 • Role of Each Node Name MON Monitoring UI User Interface CE Computing Element SITE_BDII Site Berkeley Database Information Index Description Site R-GMA server and fabric monitoring Submit job, cancel job, view status of job and get output Actual computing node for submitted jobs Contains information of grid site Contains information of grid system PX My. Proxy Server Managing proxy certificates RB Resource Broker Workload management SE Storage Element Interface to (semi-)permanent storage and MSS WN Worker Node Batch system client High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 11 • World-wide Belle Grid System KEK, Japan Korea Univ. Korea Nagoya Univ. , Japan Cyfronet AGH, Poland Academia Sinica, Taiwan National Central Univ. , Taiwan Univ. of Melbourne, Australia High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 12 Construction and Verification • • Pre-steps for Installation Procedures Certificates Site Verification High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 13 • Pre-steps for Installation § OS : Scientific Linux CERN 3. 0. 8 § Personal / Host certificates : Issued by KEKCA § Certification tool: NAREGI CA Toolkit (Provided by KEK – http: //gridca. kek. jp) § Installation and Configuration Tool: YAIM (Provided by CERN – http: //www. cern. ch/grid-deployment/gis/yaim) § JAVA SDK : Provided by SUN – http: //java. sun. com § NTP : Time synchronizing for all nodes. All Belle Grid sites uses UTC timezone. Using ntp server: ntp. xbsd. pe. kr High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 14 • Installation Procedures Host cert. / VOMS cert. # /opt/naregi/ca/bin/globus-host-req. sh (issue host certificates) # wget http: //voms. kek. jp. 10266 (get public key) Middleware Installation # /opt/glite/yaim/bin/yaim --install --siteinfo. /siteinfo. def --metapackage lcg-CE_torque (For CE node) Middleware Configuration # /opt/glite/yaim/bin/yaim --configure --siteinfo. /siteinfo. def --nodetype CE_torque (For CE node) Firewall Setting Open the appropriate ports for each node. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 15 • Certificates § Personal Certificates ü It is issued by KEKCA. ü Each user has to get personal certificate for using Grid system. ü It is protected by personal certificate key pair (usercert. pem / userkey. pem) and pass phrase password. § Host Certificates ü It is also issued by KEKCA. ü Each node has to get host certificate. ü Generated license codes are required for each node. § VOMS (Virtual Organization Membership Service) ü Provides information on the user’s relationship with his VO. ü KEKCA provides belle, ilc and calice VO. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 16 • Certificates (Cont’d) KEKCA User’s ID and Password License Code for nodes (by web page) Host Certificate (using globus toolkit) hostcert. pem hostkey. pem Personal Certificate (using globus toolkit) Proxy Certificate userkey. pem usercert. pem Convert pem to p 12 for web browser certificate VOMS High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 17 • Site Verification KU to Outside to KU KU Side Parasitic UI KEK Grid Site RB CE KU Grid Site UI CE RB Other Grid Sites CE Test command globus-job-run (simple command) / edg-job-submit (With JDL format) High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 18 Current Status • Current Problems • Reason for Problems High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 19 • Current Problems KU to Outside ; Work done Outside to KU ; Work Failed Using “globus-job-run <site> /usr/bin/whoami” KU to KU ; Work done KU Side Parasitic UI KEK Grid Site RB CE KU Grid Site UI CE RB Other Grid Sites CE High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 20 • Current Problems (Cont’d) KU to Outside to KU ; Work Failed KU to KU Using “edg-job-submit --vo belle --resource <site> testjob. jdl” ; Work Failed ; Work done KU Side Parasitic UI KEK Grid Site RB CE KU Grid Site UI CE RB Other Grid Sites CE High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 21 • Reason for Problems § DNS Problem • DNS can’t resolve IP address to domain address or vice versa. • We’re working to fix this problem. (Construct our own DNS) KU Grid Site KOREN Internet Provides domain name New DNS KU line (*. kugrid. korea. ac. kr)* * Not determined yet. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 22 Future Applications • Belle Analysi. S Framework (BASF) • New BASF High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 23 • Belle Analysi. S Framework (BASF) § Current KEK KU BASF Limited resources, slow connection and… § Future KEK KU Grid site 1 BASF Grid site 2 BASF Grid site 3 BASF …. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 24 Summary High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 25 • Summary § GRID is a new solution for high performance computing. § GRID computing is a new trend for analysis of high energy physics experiments. § KU Grid site will be completed until Oct. 2007 and will be contributed to analysis for Belle experiment. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University

Belle Grid Computing in Korea University 26 Thanks for your attention. High Performance Computing Asia 2007 – Particle and Nuclear Physics Session Soohyung Lee, Korea University