Скачать презентацию National Energy Research Scientific Computing Center NERSC First Скачать презентацию National Energy Research Scientific Computing Center NERSC First

5910b302b0fb9d0a9c35e92d14314ad4.ppt

  • Количество слайдов: 15

National Energy Research Scientific Computing Center (NERSC) First Look At ROOT I/O Igor A. National Energy Research Scientific Computing Center (NERSC) First Look At ROOT I/O Igor A. Gaponenko NERSC Center Division, LBNL September 26, 2003

The talk’s goals • An overview of ROOT I/O as a persistency mechanism – The talk’s goals • An overview of ROOT I/O as a persistency mechanism – Comparison with OODBMS • xrootd – Ba. Bar’s extension to rootd • Conclusions and thoughts

Ideal Persistency (Rene B. ) No constraints on object model Automatic converters transient persistent Ideal Persistency (Rene B. ) No constraints on object model Automatic converters transient persistent Efficient storage compression Remote access Automatic schema evolution obj 1; 1, obj 1; 2, obj 1; 3 obj 2; 1, obj 2; 2 LAN WAN Granularity matching access patterns Machine independent format

Main features • • It’s not a database – it’s just a specific file Main features • • It’s not a database – it’s just a specific file format “Streamer”-s approach to transient<->persistent transformation • • Supports reflection (aka RTTI) • • Within a file: very limited Across files: awkward Automatic schema evolution – – • Version numbers for classes (stored persistently and used by “Streamers”) May be customized by users (his/her own “Streamer”-s) Integrated with CINT (dynamic loading) • • Requires “rootcint” preprocessor to produce a shared library Explicit optimization for data clustering • – – • Requires “rootcint” preprocessing to extend a class’s contract Inter-objects references support – – • Requires “rootcint” preprocessor to make a class persistent capable ( Rene’s slides) Tree, Branch, Leaf, Chain Some control (veto) during preprocessing by “rootcint” to prevent “branching” Local, Shared Memory, Remote (rootd) and Web access • (Rene’s slides)

. . . just a file format • Platform independent • Support compression (gzip) . . . just a file format • Platform independent • Support compression (gzip) • 10 levels optimized for better performance or space • Is self sufficient (schema + data) • Logical organization • Supports directory-like organization of data inside a file • Provides both sequential (iterators) and direct (through a map) access to keys • Limited support for recovery in the design of the format • No provisions for parallel access – all should be done by the corresponding FS or OS

Comparison with OODBMS • Look at OODBMS manifesto: – http: //www 2. cs. cmu. Comparison with OODBMS • Look at OODBMS manifesto: – http: //www 2. cs. cmu. edu/People/clamen/OODBMS/Manifesto/ht. Mani festo/Manifesto. html • Not an OODBMS at all, also (Surprise!) checking against the “mandatory list” in OODBMS manifesto would produce all “Yes”, but: • Objects identity – No “good” inter-objects references • Concurrency • Recovery • Transactions

Bringing persistency into a class My. Class. H My. Class. C My. Class. Sys. Bringing persistency into a class My. Class. H My. Class. C My. Class. Sys. Def. H My. Class. H* rootcint lib Compile & link with My. Class. C* Compile & build shared library & load dynamically (CINT) Link with My. Application rootcint Objects & Schema

ROOTCINT preprocessor class Event : public TObject { private: TDirectory *f. Transient; Float_t f. ROOTCINT preprocessor class Event : public TObject { private: TDirectory *f. Transient; Float_t f. Pt; char f. Type[20]; Int_t f. Ntrack; Int_t f. Nseg; Int_t f. Nvertex; UInt_t f. Flag; Float_t f. Temperature; Event. Header f. Evt. Hdr; TClones. Array *f. Tracks; TH 1 F *f. H; Int_t f. Measures[10]; Float_t f. Matrix[4][4]; Float_t *f. Closest. Distance; . . . //! current directory //! transient value //|| don't split //-> //[f. Nvertex]

Streamers // in header file My. Class. h class My. Class : public TObject Streamers // in header file My. Class. h class My. Class : public TObject { public: . . . Class. Def(My. Class, 1) }; rootcint // in implementation file My. Class. cc. . . Class. Impl(my. Class). . . void My. Class: : Streamer(TBuffer &R__b) { // Stream an object of class Event. if (R__b. Is. Reading()) { My. Class: : Class()->Read. Buffer(R__b, this); } else { My. Class: : Class()->Write. Buffer(R__b, this); } }

Streamers (1) // The Streamer method is invoked by the “Write” // method derived Streamers (1) // The Streamer method is invoked by the “Write” // method derived from TObject (very) base class My. Class obj(); Int_i status = obj. Write(. . . );

RTTI-like support • Classes have to derive from TObject – Extremely “fat” class – RTTI-like support • Classes have to derive from TObject – Extremely “fat” class – “mother of all ROOT classes”! – Include graphics, etc. • In addition classes have to be run through “rootcint” preprocessor to extend a class’s contract with RTTI-like methods • A special class to access type information: TClass

Adding RTTI to a class // in header file My. Class. h class My. Adding RTTI to a class // in header file My. Class. h class My. Class : public TObject { public: . . . Class. Def(My. Class, 1) }; rootcint // in implementation file My. Class. cc. . . Class. Impl(my. Class). . . // in a client code My. Class* obj =. . . TClass* c = obj->Is. A(); Const char* obj->Class. Name(); Bool_t b = obj->Inherits. From("TLine"); Bool_t b = obj->Inherits. From(TLine: : Class());

xrootd – Ba. Bar’s extension to rootd • See Andy Hanushevsky’s slides at: – xrootd – Ba. Bar’s extension to rootd • See Andy Hanushevsky’s slides at: – http: //www. slac. stanford. edu/~abh/xrootd/XRootd. ppt • Extras (not in these slides yet): – – Asynchronous read/write “Look ahead” within a file (pre-fetch data by an offset) “Look ahead” for whole files (preorder files) “Limited server-controlled clients” • To suspend a client till some event will happen • To switch a client to another daemon (“hot-swap” reconfiguration of servers) • Grid-like access to remote files (see next slide) – Under discussion

Grid-like access to remote files An automatic file migration across sites invisibly to a Grid-like access to remote files An automatic file migration across sites invisibly to a client. SLAC xrootd Master xrootd client IN 2 P 3 xrootd

Conclusions and thoughts • ROOT I/O – Pros • More or less adequate (for Conclusions and thoughts • ROOT I/O – Pros • More or less adequate (for HEP) data model • Performance and efficiency (compression, trees and branches) • Open source, free, supported – Cons • (Terribly) bad documentation • (Terribly) bad API (not OO at all) • Not a database (need an external support for data catalogs, metadata, etc. ) • Limited inter-objects support • (Perhaps) HEP needs a better persistency mechanism – Capitalize on experience gained with Objy & ROOT – Reuse recent developments in efficient file serving (xrootd)