Enabling Grids for E-scienc E EGEODE usage overview

Enabling Grids for E-scienc. E EGEODE usage overview in EGEE infrastructure Gaël Youinou CGGVeritas 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy www. eu-egee. org EGEE-III INFSO-RI-222667 EGEE and g. Lite are registered trademarks

Contents Enabling Grids for E-scienc. E • EGEODE Virtual Organisation – Applications – Global Workflow • EGEODE Statistics and Usage – – – ISTEP - Laboratoire de Tectonique Paris 6 UMR – Géosciences Azur UPMC - Sisyphe IPG/EOST - Strasbourg IPGP Jussieu Paris ENS Géologie Paris • EGEE-III Earth Science Cluster – KEREON: Grid file explorer – Automatic application test • Conclusion EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 2

EGEODE Virtual Organisation Enabling Grids for E-scienc. E • EGEODE Virtual Organization « Expanding GEosciences On DEmand » – Virtual Organization to share IT resources and best-practices – Opened to all Research centers in geosciences, initially geophysics – Close collaboration with ESR « Earth Sciences Research » VO (Earth Observation, Climate, Hydrology, Solid Earth Geophysics) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 3

EGEODE Application Enabling Grids for E-scienc. E • Seismic processing generic platform for research and education – Based on Geocluster (GCT), an industrial application, used in production at CGGVeritas – Key technology for earth sciences in Oil&Gas and Environment – Ported to EGEE infrastructure for Academia (2005) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 4

EGEODE Application Enabling Grids for E-scienc. E • Seismic processing generic platform for research and education (cont’d) – Include several standard tools for signal processing, simulation and inversion (model optimization) – Open – Any user can write new algorithms in new modules – Free access for academic research – Controlled by license keys • Added value – Capability to solve complex problems and to validate innovative algorithms on real size data sets – Close the gap between Research and Industrial environment – Attract and keep brightest researchers – Framework for Industry/Research collaboration EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 5

EGEODE Global workflow Enabling Grids for E-scienc. E • Pre-processing • (1 D, 2 D) • Velocity • Analysis • (1 D, 2 D) • Time • Imaging • (3 D) • Depth • Imaging • (3 D) Run on EGEE • 3 D seismic EGEE-III INFSO-RI-222667 • Velocity model • Reservoir • Geophysics • Rock properties • Structural model 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 6

EGEODE VO Statistics and Usage Enabling Grids for E-scienc. E • EGEODE VO statistics – 6 CNRS laboratories ISTEP - Laboratoire de Tectonique Paris 6 UMR – Géosciences Azur UPMC - Sisyphe IPG/EOST - Strasbourg IPGP Jussieu Paris ENS Géologie Paris – UNIL – Université de Lausane – ~30 users – 2000+ Geocluster batch jobs in 2008 – 200 Gb of seismic data EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 7

ISTEP - Laboratoire de Tectonique Paris 6 Enabling Grids for E-scienc. E • Hardware – Local UI (Jason) – GCT 4100 &GCT 3100 versions on local workstations • Grid usage – Geocluster – Joint inversion of geophysical data (gravity, seismic) • EGEODE – 4 users (permanent members and students) – 2 D Multichannel seismic (360 channels), 5500 km of lines – Sequence: time variant amplitude adjustment, spherical divergence correction, spiking deconvolution, Radon (τ, p) domain antimultiple, controlled frequency-wave number (F, k) domain antimultiple and (F, k) domain filtering, dynamic correction, stack, migration EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 8

ISTEP - Laboratoire de Tectonique Paris 6 Enabling Grids for E-scienc. E • EGEODE (cont’d) – TIKIM (Kirchhoff prestack time migration) was tested by a student – No use of Geovel (velocity picking) yet • Dissemination – Autin Julia, Ph. D Thesis dec. 2008 – Basuyau Clémence, Ph. D Thesis, 2011. Articles expected • SWOT Analysis – Advantages: Almost no installation and software management is necessary; High level of computation capacity – Drawbacks: Input and output data management; For short job (2 D seismic) grid is not necessary; The module KIRCH (poststack Kirchhoff Migration) and a similar interface as Geopad to manage the project, data, … are lacking EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 9

ISTEP - Laboratoire de Tectonique Paris 6 Enabling Grids for E-scienc. E • Training – Training on the grid • Collaborations – With CNRS labs who are providing hardware or expertise – IPSL’s UI ? EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 10

UMR - Geoazur Enabling Grids for E-scienc. E • Hardware deployments – 1 local UI Sun Fire X 2200 Opteron dual core 2, 2 GHz, 4 Go Ram, SL 4. 4) – 1 CE and 1 WN virtualized with VMware ESXi on a Sun Fire X 2200 (Opteron dual core 2, 66 GHz, 32 Go Ram, SL 4. 7) – Short term planned installations 2 WN and 1 SE – NX clients are used for remote access to our local UI • Grid usage – Seismic data processing on Geocluster – Others software deployed on the Grid are used Seismic Unix and GMT (Generic Mapping Tools) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 11

UMR - Geoazur Enabling Grids for E-scienc. E • EGEODE – Users into the laboratory 3 permanent members, students – Processed studies 2 D marine multichannel seismic Around 3000 km of seismic lines are recorded during a survey – Seismic processing workflow – Minimum phase conversion (WAPCO) – Static application (HISTA) – High frequency filtering - time gain function , spherical divergence compensation (REFOR, SDICO) – corrections of amplitude anomalies (EDITE, EXPEN, HARMO, SPARN, DYNQ) – Deconvolution (TRITA, (DECON)) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 12

UMR - Geoazur Enabling Grids for E-scienc. E • EGEODE (cont’d) – Velocity analysis VESPA, VELCOM – NMO correction (FANMO) - trace muting (MUTES) – F-K multiple attenuation (FKMUL, FKFIL)- 2 D Kirchhoff DMO (QRDMO, KIDMO) – Trace stacking (STACK)- frequency filtering (FILTR) – 2 D post stack migration (KIRCH, FKMIG)- amplitude equalization - Plots (DYNQ, AMCO, FKFIL, PLOTX) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 13

UMR - Geoazur Enabling Grids for E-scienc. E • EGEODE (cont’d) – Feedback information on Geocluster utilization Few difficulties have been encountered when transferring older Geovecteur and local Geocluster sequences to the Grid The Kirchhoff prestack time migration (TIKIM) is still in phase of test Interactive applications on the Grid offer the same performances as working on a local version of the software • Dissemination – Ph. D students were used to working on Geovecteur for their processing workflow This practice tends now to disappear to the benefit of Geocluster usage A Master student is actually working on Geocluster and will transfer very soon her job flow to the Grid EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 14

UMR - Geoazur Enabling Grids for E-scienc. E • SWOT Analysis – In practice the installation and the setup of a EGEE UI is very simple compared to a local installation of Geocluster, however the installation of CE and WN requires good knowledge in system and network administration – Requests for GCT module: a module that takes seismic section as input and generates images in Postscript format file EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 15

UMR - Geoazur Enabling Grids for E-scienc. E • Training – Feedback information on CGGVU training courses Very good dedicated program, well adapted to our 2 D marine processing sequences and data – Futures needs Deconvolution and multiple attenuation • Conclusion – The execution time of a job on the Grid indicates good performances, however when large files are transferred to the Grid, this has direct impact to the execution time of a job In this case, it’s more comfortable to work on a local version of Geocluster – We plan ton install a SE on the laboratory to improve access to the data EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 16

UPMC - Sisyphe Enabling Grids for E-scienc. E • Hardware – Inside laboratory Small cluster of 9 nodes (bi-processor 32 bit) Three years old Access only from local network Study for a computing server gathering several labs of Earth Siences inside UPMC, including Sisyphe. – Nx server Only UI 2 from CGGVeritas • Grid usage – Geocluster – Geovel – Other in-house codes (Fortran 90) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 17

UPMC - Sisyphe Enabling Grids for E-scienc. E • EGEODE – Number of users For now 3 – Type of studies Seismic processing (Geocluster, Geovel), development of a modeling code in CSEM (Controlled Source Electro. Magnetic) – Type of jobs Sequential for Geocluster, MPI for CSEM modeling – Geocluster feedback Simplicity of utilization, but lack of staff for solving technical problems quickly on the site (sometimes long waiting time) – Geovel feed back Simplicity of utilization, but random bugs EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 18

UPMC - Sisyphe Enabling Grids for E-scienc. E • Dissemination (Ph. D thesis) – “Contrôle de la qualité du traitement des données sismiques du bassin sédimentaire de la Côte d’Ivoire – Apport de la géostatistique – Tentative de retraitement” (Marie Oussou) – “Etude par simulation de méthodes électromagnétiques employées au suivi d’un réservoir en production” (Cyril Schamper) • SWOT Analysis – Pros/Cons in comparison to a local use of Geocluster: access from everywhere via internet – Needs for Geocluster Nothing particular, exhaustive information EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 19

UPMC - Sisyphe Enabling Grids for E-scienc. E • Training – No training Learning by experiencing • Collaborations – Collaboration with other VO With ESR if needed – Collaboration with other CNRS labs (material and expertise) Occasionally with researchers from IPGP who administrate ESR VO EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 20

IPG - Strasbourg Enabling Grids for E-scienc. E • Hardware – Local UI bi-pro Xeon E 5345 quad-core 2, 33 GHz, RAM 8 Gb) – Nx server usage (local server, server installed on UI 2 based in CGGVeritas) • Grid usage – Only Geocluster – Local cluster available, access to distant academic clusters • EGEODE – – 3 (1 permanent member, 2 students) Type of studies: PSV-3 C and conventional seismic processing PSV-3 C tests, fluvial seismic Geometry, anti-multiples, migration EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 21

IPG - Strasbourg Enabling Grids for E-scienc. E • EGEODE (cont’d) – License sharing (SDI Montage) – Specific software (XPS, Geoland) installed on UI 2 located in CGGVeritas • Dissemination – Internship reports, current work • SWOT Analysis – Advantages: Access to the state of the art software including last versions of new applications (Kereon, Jxjob); Shared resources (CPU’s, disks, …) – Drawbacks: Tapes, cartridges or file management; Delays (execution, end of jobs – listings); No easy procedure for UI installation EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 22

IPG - Strasbourg Enabling Grids for E-scienc. E • Training – CGGVeritas University training Useful and necessary – Tutorial or How. To on complex applications (geoland, XPS) would be nedeed • Collaborations – Collaborations with EGEODE users EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 23

IPG - Paris Enabling Grids for E-scienc. E • Hardware – No hardware installed locally • Grid usage – Only Geoclustrer • EGEODE – Pre-stack migration jobs will be ported ASAP on the grid • Training – Well adapted training on TEAMVIEW, GEOVEL and TIKIM – No more need today • Collaborations – With IPSL or the “Laboratoire de Tectonique“ EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 24

ENS Géologie Paris Enabling Grids for E-scienc. E • User report on Geocluster and EGEODE VO in academic research EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 25

ENS Géologie Paris Enabling Grids for E-scienc. E • Marine geophysics in the Geol Lab, geodynamics team – Episodic cruises, episodic processing needs – Basic seismic processing lessons for students using CGG software – Long periods without activity, as marine geophysics is a component amongst wider topics in geodynamics – Software used before Geocluster/EGEODE Geovector 6100! Updates are time consuming for researchers when compared to the episodic use As a result we cannot follow state of the art processing methods EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 26

ENS Géologie Paris Enabling Grids for E-scienc. E • Problems solved by Geocluster & EGEODE VO in such an environment – Updates are progressive through the CGGVeritas servers – Stability of the UI platform – No need to invest in a Cluster for just an episodic use The laboratory can just share a few nodes on the grid 24/7 and use this shared capacity in a short time on more nodes when needed – New tools (Teamview, Chronovista. . . ) – Prestack time migration (TIKIM) now possible EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 27

EGEE-III Earth Science Cluster Enabling Grids for E-scienc. E • KEREON: Explorer for local and lfc grid files – – – Browse directories Upload and download files Rename files and directories Delete files and directories Replicate lfc files Read the properties of files and directories – A GUI to manage data on the grid – User can browse the grid storage services and catalogs as easier as for a local file system. – Allow the user to exchange data between the local UI and the grid storage services very easily – Without typing any command line, only by mouse click EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 28

EGEE-III Earth Science Cluster Enabling Grids for E-scienc. E • Automatic application test – Create a tool (like SAM) who Check all sites automatically for an application Ban some of them if they are not appropriate for the application This information could be used before sending a new job EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 29

Conclusion Enabling Grids for E-scienc. E • EGEODE Virtual Organisation – Advantages Almost no installation and software management is necessary High level of computation capacity Shared resources (CPU’s, disks, …); No need to invest in a Cluster for just an episodic use Access from everywhere via internet Access to the state of the art software including last versions of new applications – Drawbacks Input and output data management For short job (2 D seismic) grid is not necessary UI installation very simple however the installation of CE and WN requires good knowledge in system and network administration Good performances, however when large files are transferred to the Grid, this has direct impact to the execution time of a job (more comfortable to work on a local version) EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 30

References Enabling Grids for E-scienc. E • EGEE project: http: //www. eu-egee. org • EGEE Earth Science Cluster Portal: http: //esc. wdcb. ru/ ES Workplan: https: //edms. cern. ch/file/937512/1/work-plan-ES-v 4. 0. doc • BEin. GRID project: http: //www. beingrid. eu • DEGREE project: http: //www. eu-degree. eu • Geocluster: http: //www. cggveritas. com/default. aspx? cid=4 -13 -1925 • EGEODE: http: //www. egeode. org • Kereon Online demonstration: http: //www. egeode. org/egeode/packages/KEREON/1. 4. 6/kereon. html Installation document: http: //www. egeode. org/egeode/packages/KEREON/1. 4. 6/kereon-installation. pdf EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 31

Enabling Grids for E-scienc. E CGGVeritas is the world’s leading international pure-play geophysical company delivering a wide range of technologies, services and equipment to its broad base of customers mainly throughout the global oil and gas industry Focused on Performance. Passion for Innovation. Powered by People. Delivered with Integrity We look forward, as CGGVeritas, to serving you in the future. EGEE-III INFSO-RI-222667 4 th EGEE User Forum 2 -6 March 2009 Le Ciminiere, Catania, Sicily, Italy 32