HPS Web 2 0 OR Web Apps and

HPS Web 2. 0 OR Web Apps and Databases (Oh My!) Jeremy Mc. Cormick (SLAC) Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC)

OVERVIEW • Database Server • Conditions Database • Run Database • Web Server • Run Web • Data Catalog • Online DQM Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 2

WHY? • Databases • • Central facility for accessing shared information Functional replacement for text files Provides backend for other applications like recon and web apps Change management and authentication (user accounts) • Web applications • View from “anywhere” without needing any special applications (such as ROOT) • Easily share important information such as DQM plots • See what’s inside data sources (databases, files, etc. ) • Because Javascript is magic! Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 3

DATABASE SERVER • Dedicated HPS MYSQL server • hpsdb. jlab. org • Hosts conditions, run and datacat databases • Full access from inside the JLAB firewall from ‘jlab’ hosts • Grants for write operations depend on the account • Several users have admin privileges (myself and Maurik) • Read only access via an external replica • External non-jlab hosts cannot see the primary db directly. • Non-production copy at SLAC for Hudson builds • Accessible in hps-java through APIs built on JDBC Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 4

CONDITIONS DATABASE • Contains detector conditions data from Test Run to present. • Each type of condition has its own table with sets of rows assigned to a collection. • Collections are assigned valid run number ranges. • Sets of collections can be tagged for reproducibility. • Data is accessible in reconstruction via an extensive Java API. • The ‘hpsuser’ account has a public password and is used for read-only mode. • The ‘hpscalibrations’ account can be used to insert, update or delete existing data. Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 5

RUN DATABASE • Database of run metadata extracted from crawling the EVIO data • Run start and end times • Number of events • etc. • EPICS data • Scaler data • TI Trigger Time Offset • Java API through Run. Manager • Can easily additional information to DB and API! Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 6

RUN LOG • Run database currently != run log • Run database is currently populated from crawling the data after it is copied to MSS and extracting metadata from EVIO. • Run log would be updated via hooks in the DAQ so that it happens concurrently with END and PRESTART events. • Implementing requires some coordination with Hall B staff. • Need to add C code into CODA • Determine which values to include (have gotten some good feedback here) • Some values also could be filled interactively by pop-ups, GUI interfaces, etc. Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 7

WEB SERVER • http: //hpsweb. jlab. org • RHEL 7, dual core, 4 GB VM • Tomcat 7 • Standard Java Servlet container • Admin manager only accessible behind JLAB firewall • Apache Proxy • Specific application URLs mapped to Tomcat • Managed by IT Division / CNI Group • Terminal access only allowed with crytocard Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 8

RUN WEB • https: //hpsweb. jlab. org/hps-run-webapp/ • Shows information from run database on the web. • Sortable run table • Run page • • • Oct 27 2015 Summary EPICS data Scaler data EVIO datacat link DQM datacat link HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 9

DATACAT • • • https: //hpsweb. jlab. org/datacat/display/browser/HPS New data catalog server and REST API from SLAC Query and update via HTTP requests Built-in browser interface My. SQL backend (runs at JLAB) Java client API • Wrapper for URL based requests • Add, update, delete, search • Crawler for finding data, extracting meta data and adding to the catalog Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 10

EXAMPLE DATACAT PATHS • Display Browser • /display/browser/HPS • Show a path • /r/path/HPS • Show a dataset • /r/path. json/HPS/test/dataset 01; s=SLAC • Filter • /r/search. json/HPS/dataset 01? filter=run. Min+>+1000 • Recursive Search • /r/search. json/HPS/**? filter=run. Min+eq+5772 Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 11

DQM ONLINE • Example DQM Plots (not production yet!) • AIDA TLD library of JSP tags (SLAC) • Allows easily showing plot files in a web page • Can read from AIDA or ROOT files on the file system • At JLAB /work can be mounted by server read-only. • Export individual plots to graphics files (PNG, etc. ) Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 12

TODO • Finish up JLAB web config • Enable authentication via scicomp certificate • Get DQM server working • Run database • Consolidate multiple sources of run data into database • Code for extraction of run (meta)data should be integrated into the crawler module • Data catalog • Command line interface • Integration with reconstruction and Auger jobs Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 13

NOW FOR SOMETHING DIFFERENT! • What’s new in SLIC these days? • Current versions of Geant 4 and other dependencies • Support for 3 D magnetic field maps • Displaced vertices in Std. Hep files now supported (and maybe even tested) • Conversion from Std. Hep event records into LCIO MCParticle collection uses a standard/shared LCIO class • Code for translation of Geant 4 tracks into MCParticle output collection rewritten Oct 27 2015 HPS Collaboration Meeting Jeremy Mc. Cormick (SLAC) 14